Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology : Industrial Product

TLDR

Emerging deep‑learning workloads demand high off‑chip memory bandwidth, yet increasing it is costly under chip‑package constraints and data movement consumes a growing share of system energy, and no silicon proof‑of‑concept exists because prior PIM designs require host‑processor or application changes. This work proposes an innovative yet practical PIM architecture that addresses these challenges. The authors implement the design using 20 nm DRAM, integrate it with an unmodified commercial processor, develop a software stack, and run existing applications without source‑code modifications. System‑level evaluation shows the PIM boosts memory‑bound neural‑network kernel performance by 11.2× and application performance by 3.5×, while cutting energy per bit transfer by 3.5× and overall system energy efficiency by 3.2×.

Abstract

Emerging applications such as deep neural network demand high off-chip memory bandwidth. However, under stringent physical constraints of chip packages and system boards, it becomes very expensive to further increase the bandwidth of off-chip memory. Besides, transferring data across the memory hierarchy constitutes a large fraction of total energy consumption of systems, and the fraction has steadily increased with the stagnant technology scaling and poor data reuse characteristics of such emerging applications. To cost-effectively increase the bandwidth and energy efficiency, researchers began to reconsider the past processing-in-memory (PIM) architectures and advance them further, especially exploiting recent integration technologies such as 2.5D/3D stacking. Albeit the recent advances, no major memory manufacturer has developed even a proof-of-concept silicon yet, not to mention a product. This is because the past PIM architectures often require changes in host processors and/or application code which memory manufacturers cannot easily govern. In this paper, elegantly tackling the aforementioned challenges, we propose an innovative yet practical PIM architecture. To demonstrate its practicality and effectiveness at the system level, we implement it with a 20nm DRAM technology, integrate it with an unmodified commercial processor, develop the necessary software stack, and run existing applications without changing their source code. Our evaluation at the system level shows that our PIM improves the performance of memory-bound neural network kernels and applications by 11.2× and 3.5×, respectively. Atop the performance improvement, PIM also reduces the energy per bit transfer by 3.5×, and the overall energy efficiency of the system running the applications by 3.2×.