Hadoop framework: impact of data organization on performance

Abstract

SUMMARY Hadoop, based on the popular MapReduce framework, is an open‐source distributed computing framework that has been gaining much popularity and usage. It aims to allow programmers to focus on building applications that deals with processing large amount of data, without having to handle other issues when performing parallel computations. However, tuning the performance of Hadoop applications is not an easy task due to the level of abstraction of the framework. In this paper, we present three case studies and some of the challenges and issues that are to be considered in performance tuning when running applications in Hadoop. The focus is mainly on the impact of input data on Hadoop's performance and how they can be tuned. Copyright © 2011 John Wiley & Sons, Ltd.