ComputingComputer EngineeringComputer ScienceDistributed SystemsHeterogeneous ComputingParallel ComputingData ParallelismDistributed ComputingDistributed Data ProcessingHardware AccelerationParallel ArchitecturesParallel Computing SystemsData Parallel ExecutionGpu ArchitectureHeterogeneous Multicore ArchitectureLarge-scale Data ProcessingMapreduceMapreduce AlgorithmsMulticore SystemsParallel Data Processing
MapReduce is an efficient distributed computing model for large-scale data processing. However, single-node performance is gradually to be the bottleneck in compute-intensive jobs. This paper presents an approach of MapReduce improvement with GPU acceleration, which is implemented by Hadoop and OpenCL. Different from other implementations, it targets at general and inexpensive hardware platform, and it is seamless-integrated with Apache Hadoop, a most widely used MapReduce framework. As a heterogeneous multi-machine and multicore architecture, it aims at both data- and compute-intensive applications. An almost 2 times performance improvement has been validated, without any farther optimization.
12
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
Evaluating MapReduce for Multi-core and Multiprocessor Systems
Colby Ranger, R. Raghuraman, Arun Penmetsa et al. · 2007 · 968 citations
Bingsheng He, Wenbin Fang, Qiong Luo et al. · 2008 · 718 citations
Clifford A. Lynch · Nature · 2008 · 705 citations