2009 · 58 citations · 8 references
ComputingCloud ComputingComputer ScienceData ManagementDistributed SystemsStorage SystemsCloud Data ManagementData Storage SystemsDistributed StorageDistributed Storage SystemsStorage ManagementStorage ScalabilityData AvailabilityDistributed Data StorageDistributed Storage ArchitecturesInterferenceScalable Data ManagementStorage System ReliabilityTransport Layer ProtocolsTransport Protocols
This paper takes a renewed look at the problem of managing intermediate data that is generated during dataflow computations (e.g., MapReduce, Pig, Dryad, etc.) within clouds. We discuss salient features of this intermediate data and outline requirements for a solution. Our experiments show that existing local writeremote read solutions, traditional distributed file systems (e.g., HDFS), and support from transport protocols (e.g., TCP-Nice) cannot guarantee both data availability and minimal interference, which are our key requirements. We present design ideas for a new intermediate data storage system. 1
8
Sanjay Ghemawat · Communications of the ACM · 2008 · 18.5K citations · Full text
Michael Isard, Mihai Budiu, Yuan Yu et al. · 2007 · 2.5K citations
Christopher Olston, Benjamin Reed, Utkarsh Srivastava et al. · 2008 · 1.7K citations
Improving MapReduce performance in heterogeneous environments
Matei Zaharia, Andy Konwinski, Anthony D. Joseph et al. · UC Berkeley · 2008 · 1.6K citations
Measurements of a distributed file system
Mary G. Baker, John H. Hartman, Michael D. Kupfer et al. · 1991 · 535 citations