2007 · 968 citations · 24 references
ComputingComputer EngineeringComputer ScienceDistributed SystemsFault ToleranceParallel ComputingDistributed ComputingDistributed Data ProcessingParallel Computing SystemsParallel Runtime SystemsShared-memory MultiprocessingShared-memory ParallelismDynamic Task SchedulingMapreduceParallel SchedulingRuntime System ArchitectureScheduling In Multiprocessor SystemsShared MemoryShared Memory SystemsThread Creation
MapReduce, developed by Google for large‑scale data‑center applications, lets programmers write functional‑style code that is automatically parallelized and scheduled across distributed systems. The paper evaluates the suitability of MapReduce for multi‑core and multi‑processor systems, studying Phoenix’s performance potential and error‑recovery features. Phoenix implements MapReduce for shared‑memory systems, providing a programming API and a runtime that automatically manages thread creation, dynamic task scheduling, data partitioning, fault tolerance, and compares its code to lower‑level P‑threads APIs. The study finds that, with careful implementation, MapReduce is a promising model for scalable performance on shared‑memory systems using simple parallel code.
This paper evaluates the suitability of the MapReduce model for multi-core and multi-processor systems. MapReduce was created by Google for application development on data-centers with thousands of servers. It allows programmers to write functional-style code that is automatically parallelized and scheduled in a distributed system. We describe Phoenix, an implementation of MapReduce for shared-memory systems that includes a programming API and an efficient runtime system. The Phoenix runtime automatically manages thread creation, dynamic task scheduling, data partitioning, and fault tolerance across processor nodes. We study Phoenix with multi-core and symmetric multiprocessor systems and evaluate its performance potential and error recovery features. We also compare MapReduce code to code written in lower-level APIs such as P-threads. Overall, we establish that, given a careful implementation, MapReduce is a promising model for scalable performance on shared-memory systems with simple parallel code
24
Sanjay Ghemawat, Howard Gobioff, Shun-Tak A. Leung · 2003 · 1.3K citations
Richard E. Ladner, Michael J. Fischer · Journal of the ACM · 1980 · 1.3K citations · Full text
Philippe Charles, Christian Grothoff, Vijay Saraswat et al. · 2005 · 1.3K citations