Modeling and Simulation in Performance Optimization of Big Data Processing Frameworks

Abstract

The enormous increase in data has led to the next grand challenge in computing: the big data problem-that is, the practice of collecting complex datasets so large that they're difficult to store, process, and interpret manually or using traditional data management applications. This column discusses the role of modeling and simulation science in the era of big data applications. Modeling and simulation can empower practitioners and academics in conducting "what-if" analyses for scheduling policies under variable cloud resource configurations, big data processing framework configurations, and workload.