IEEE Transactions on Parallel and Distributed Systems · 2022 · 13 citations · 58 references
EngineeringComputer EngineeringComputer ScienceMemoryParallel ComputingSystem-on-chip DesignCachingGraphics Processing Unit ArchitectureHardware AccelerationMemory Hierarchy DesignShared-memory MultiprocessingShared-memory ParallelismCache DesignMemory UtilizationShared Cache DesignShared MemoryShared-memory ArchitectureThread ParallelismThread-based ParallelismThread-level Parallelism
Graphics Processing Units (GPUs) employ a shared memory, a software-managed cache for programmers, in each streaming multiprocessor to accelerate data sharing among the threads in a thread block. Although 60% of the shared memory space is underutilized, on average, there are some workloads that demand higher shared memory capacities. Therefore, improving shared memory utilization while satisfying the needs of shared memory intensive workloads is challenging. We make a key observation that the lifetime of each shared memory address is significantly shorter than the execution time of a thread block. In this paper, we first propose Off-Chip Shared Memory (OSM) that allocates shared memory space in the off-chip memory, and accelerates accesses to it via a small on-chip cache. Using an 8 KB cache for shared memory addresses, OSM provides almost the same performance as the baseline GPU that uses 96 KB on-chip shared memory. OSM improves GPU performance in two ways. First, it allocates higher shared memory capacities in the off-chip memory, and improves thread-level parallelism (TLP). Second, it designs a unified cache for shared memory and global address spaces, providing more caching space for global memory address space even for the workloads with high shared memory utilization. Our experimental results show an average 21% and 18% IPC improvement compared to the baseline and the state-of-the-art architectures.
58
Rodinia: A benchmark suite for heterogeneous computing
Shuai Che, Michael Boyer, Jiayuan Meng et al. · 2009 · 3K citations
Analyzing CUDA workloads using a detailed GPU simulator
Ali Bakhoda, George L. Yuan, Wilson Wai Lun Fung et al. · 2009 · 1.7K citations
Bingsheng He, Wenbin Fang, Qiong Luo et al. · 2008 · 718 citations
GPUs and the Future of Parallel Computing
Stephen W. Keckler, William J. Dally, Brucek Khailany et al. · IEEE Micro · 2011 · 664 citations