Big Data Processing in Cloud Computing Environments

TLDR

The rapid growth of applications such as social network, semantic web, and bioinformatics has led to a surge in large‑scale data, posing significant management and analysis challenges that have attracted academic, industry, and governmental interest. The authors introduce several big data processing techniques from system and application perspectives. They present key issues of big data processing in cloud environments—cloud platforms, architectures, databases, storage schemes—and discuss MapReduce optimization strategies and applications. The paper concludes by outlining open challenges and proposing future research directions for big data processing in cloud computing environments.

Abstract

With the rapid growth of emerging applications like social network analysis, semantic Web analysis and bioinformatics network analysis, a variety of data to be processed continues to witness a quick increase. Effective management and analysis of large-scale data poses an interesting but critical challenge. Recently, big data has attracted a lot of attention from academia, industry as well as government. This paper introduces several big data processing technics from system and application aspects. First, from the view of cloud data management and big data processing mechanisms, we present the key issues of big data processing, including cloud computing platform, cloud architecture, cloud database and data storage scheme. Following the Map Reduce parallel processing framework, we then introduce Map Reduce optimization strategies and applications reported in the literature. Finally, we discuss the open issues and challenges, and deeply explore the research directions in the future on big data processing in cloud computing environments.