2012 · 79 citations · 20 references
ComputingComputer ScienceComputer SystemsData ManagementData MiningScientific ComputingData IntegrationData ParallelismData ReductionData Storage SystemsDistributed Data ProcessingFile SystemsData-intensive ComputingDistributed File System ArchitectureLarge-scale Data ProcessingMapreduceMapreduce AlgorithmsMulti-file SystemsScientific Data AnalysisScientific Data Handling
Despite the popularity of MapReduce, there are several obstacles to applying it for developing scientific data analysis applications. Current MapReduce implementations require that data be loaded into specialized file systems, like the Hadoop Distributed File System (HDFS), whereas with rapidly growing size of scientific datasets, reloading data in another file system or format is not feasible. We present a framework that allows scientific data in different formats to be processed with a MapReduce like API. Our system is referred to as SciMATE, and is based on the MATE system developed at Ohio State. SciMATE is developed as a customizable system, which can be adapted to support processing on any of the scientific data formats. We have demonstrated the functionality of our system by creating instances that can be processing NetCDF and HDF5 formats as well as flat-files. We have also implemented three popular data mining applications and have evaluated their execution with each of the three instances of our system.
20
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
Instance-Based Learning Algorithms
David W. Aha, Dennis Kibler, Marc K. Albert · Machine Learning · 1991 · 4.1K citations · Full text
Instance-based learning algorithms
David W. Aha, Dennis Kibler, Marc K. Albert · Machine Learning · 1991 · 2.9K citations · Full text
Jaliya Ekanayake, Hui Li, Bingjing Zhang et al. · 2010 · 777 citations