Designing efficient sorting algorithms for manycore GPUs

TLDR

These sorting algorithms, though designed for GPUs, are also suitable for other manycore processors. The paper presents high‑performance parallel radix and merge sort routines for manycore GPUs using CUDA. The algorithms expose fine‑grained parallelism, decompose work into independent tasks with minimal global communication, and leverage NVIDIA GPU on‑chip shared memory and parallel scan primitives. The radix sort achieves the fastest GPU sort, outperforming GPUSort by up to four times, other CUDA radix sorts by more than twice, and a highly optimized multicore CPU routine by 23% on average, while the merge sort is the fastest comparison‑based GPU sort reported.

Abstract

We describe the design of high-performance parallel radix sort and merge sort routines for manycore GPUs, taking advantage of the full programmability offered by CUDA. Our radix sort is the fastest GPU sort and our merge sort is the fastest comparison-based sort reported in the literature. Our radix sort is up to 4 times faster than the graphics-based GPUSort and greater than 2 times faster than other CUDA-based radix sorts. It is also 23% faster, on average, than even a very carefully optimized multicore CPU sorting routine. To achieve this performance, we carefully design our algorithms to expose substantial fine-grained parallelism and decompose the computation into independent tasks that perform minimal global communication. We exploit the high-speed onchip shared memory provided by NVIDIA's GPU architecture and efficient data-parallel primitives, particularly parallel scan. While targeted at GPUs, these algorithms should also be well-suited for other manycore processors.