Analyzing CUDA workloads using a detailed GPU simulator

TLDR

Modern GPUs offer massively parallel, SIMD cores that enable flexible programming models, yet many data‑parallel applications fail to reach peak performance, making them valuable for studying tradeoffs among memory, data, and thread‑level parallelism. The study characterizes non‑graphics CUDA applications by executing them on a detailed GPU microarchitecture simulator that implements NVIDIA’s PTX instruction set. Twelve CUDA applications were simulated on a high‑end GPU model to evaluate how microarchitectural choices—interconnect topology, cache usage, memory controller design, workload distribution, and request coalescing—affect performance. The results show that interconnect bisection bandwidth is more critical than latency, and that reducing concurrent thread count can improve performance by alleviating memory‑system contention for some applications.

Abstract

Modern Graphic Processing Units (GPUs) provide sufficiently flexible programming models that understanding their performance can provide insight in designing tomorrow's manycore processors, whether those are GPUs or otherwise. The combination of multiple, multithreaded, SIMD cores makes studying these GPUs useful in understanding tradeoffs among memory, data, and thread level parallelism. While modern GPUs offer orders of magnitude more raw computing power than contemporary CPUs, many important applications, even those with abundant data level parallelism, do not achieve peak performance. This paper characterizes several non-graphics applications written in NVIDIA's CUDA programming model by running them on a novel detailed microarchitecture performance simulator that runs NVIDIA's parallel thread execution (PTX) virtual instruction set. For this study, we selected twelve non-trivial CUDA applications demonstrating varying levels of performance improvement on GPU hardware (versus a CPU-only sequential version of the application). We study the performance of these applications on our GPU performance simulator with configurations comparable to contemporary high-end graphics cards. We characterize the performance impact of several microarchitecture design choices including choice of interconnect topology, use of caches, design of memory controller, parallel workload distribution mechanisms, and memory request coalescing hardware. Two observations we make are (1) that for the applications we study, performance is more sensitive to interconnect bisection bandwidth rather than latency, and (2) that, for some applications, running fewer threads concurrently than on-chip resources might otherwise allow can improve performance by reducing contention in the memory system.