Concurrency and Computation Practice and Experience · 2012 · 50 citations · 16 references
ComputingComputer ScienceComputer SystemsData EngineeringData ManagementDistributed SystemsData CompressionData ParallelismData RedundancyDatabase ManagementDistributed Data ProcessingDistributed DatabasesCompressionCompression TechniquesData-intensive ComputingMapreduceMapreduce AlgorithmsResource Description FrameworkResource Description LanguagesSemantic Web
SUMMARY The Semantic Web contains many billions of statements, which are released using the resource description framework (RDF) data model. To better handle these large amounts of data, high performance RDF applications must apply a compression technique. Unfortunately, because of the large input size, even this compression is challenging. In this paper, we propose a set of distributed MapReduce algorithms to efficiently compress and decompress a large amount of RDF data. Our approach uses a dictionary encoding technique that maintains the structure of the data. We highlight the problems of distributed data compression and describe the solutions that we propose. We have implemented a prototype using the Hadoop framework, and evaluate its performance. We show that our approach is able to efficiently compress a large amount of data and scales linearly on both input size and number of nodes. Copyright © 2012 John Wiley & Sons, Ltd.
16
Tim Berners‐Lee, James Hendler, Ora Lassila · Scientific American · 2001 · 8.5K citations
Gary Stix · Scientific American · 2001 · 1.8K citations
Life Sciences, Biology, Zoology +4
Christopher Olston, Benjamin Reed, Utkarsh Srivastava et al. · 2008 · 1.7K citations
Ashish Thusoo, Joydeep Sen Sarma, Namit Jain et al. · Proceedings of the VLDB Endowment · 2009 · 1.5K citations