A survey into performance and energy efficiency in HPC, cloud and big data environments

TLDR

High‑performance computing, cloud, and big‑data environments are driven by growing performance demands, yet energy consumption limits further gains, prompting widespread use of workload characterisation to model system behaviour. The paper surveys workload characterisation methods aimed at improving performance and energy efficiency in HPC, cloud, and big‑data settings. The study systematically reviews and classifies existing workload characterisation techniques across HPC, cloud, and big‑data platforms. The review shows that 56.4 % of examined papers contribute to performance and energy‑efficiency gains, with interest growing at 7.86 % annually.

Abstract

The growing demand for performance observed in many organisations can still be considered the main motivator for the evolution of high performance computing and, more recently, cloud environments. Part of this demand regards the need to deal with large and complex datasets, called big data . Performance improvement in such environments begins to be limited by energy consumption. Workload characterisation is a well-known approach to reproducing systems’ behaviour. However, there are several methodologies, techniques and parameters that can be considered for a workload characterisation. As a result, we present a differentiated survey on workload characterisation focusing on performance and energy efficiency improvement on HPC, cloud and big data environments. After an extensive review and classification of research works, our study indicates that around 56.4% of the papers reviewed offer contributions to performance and energy efficiency improvement, and the growing interest in this subject has a rate of 7.86% per year.