Proceedings of the VLDB Endowment · 2012 · 138 citations · 5 references
ComputingComputer ScienceComputer SystemsCluster ComputingData MiningParallel ComputingData ParallelismData PipelinesDistributed ComputingDistributed Data ProcessingParallel DatabasesTask ParallelismData-intensive ComputingEntity ResolutionHigh PerformanceLarge-scale Data ProcessingMapreduceMapreduce AlgorithmsParallel Task ExecutionParallel Workflow Execution
We demonstrate a powerful and easy-to-use tool called Dedoop (<u>De</u>duplication with Ha<u>doop</u>) for MapReduce-based entity resolution (ER) of large datasets. Dedoop supports a browser-based specification of complex ER workflows including blocking and matching steps as well as the optional use of machine learning for the automatic generation of match classifiers. Specified workflows are automatically translated into MapReduce jobs for parallel execution on different Hadoop clusters. To achieve high performance Dedoop supports several advanced load balancing strategies.
5
Frameworks for entity matching: A comparison
Hanna Köpcke, Erhard Rahm · Data & Knowledge Engineering · 2009 · 371 citations · Full text
Load Balancing for MapReduce-based Entity Resolution
Lars Kolb, Andreas Thor, Erhard Rahm · 2012 · 155 citations