Debunking the 100X GPU vs. CPU myth

TLDR

Recent advances in computing have generated massive data, making throughput computing essential for emerging applications, yet many studies claim GPUs deliver 10X–1000X speedups over multi‑core CPUs for these kernels. To understand where such large performance differences arise, we performed a rigorous performance analysis and found that after applying optimizations appropriate for both CPUs and GPUs the gap between an Nvidia GTX280 and an Intel Core i7‑960 narrows to only 2.5× on average. We discuss optimization techniques for both CPU and GPU, analyze architecture features contributing to performance differences, and recommend architectural features that improve efficiency for throughput kernels. Our analysis shows that throughput kernels have ample parallelism suitable for both CPUs and GPUs, and after applying appropriate optimizations the performance gap between an Nvidia GTX280 and an Intel Core i7‑960 narrows to only 2.5× on average.

Abstract

Recent advances in computing have led to an explosion in the amount of data being generated. Processing the ever-growing data in a timely manner has made throughput computing an important aspect for emerging applications. Our analysis of a set of important throughput computing kernels shows that there is an ample amount of parallelism in these kernels which makes them suitable for today's multi-core CPUs and GPUs. In the past few years there have been many studies claiming GPUs deliver substantial speedups (between 10X and 1000X) over multi-core CPUs on these kernels. To understand where such large performance difference comes from, we perform a rigorous performance analysis and find that after applying optimizations appropriate for both CPUs and GPUs the performance gap between an Nvidia GTX280 processor and the Intel Core i7-960 processor narrows to only 2.5x on average. In this paper, we discuss optimization techniques for both CPU and GPU, analyze what architecture features contributed to performance differences between the two architectures, and recommend a set of architectural features which provide significant improvement in architectural efficiency for throughput kernels.