Eyeriss

TLDR

Deep CNNs deliver high accuracy but incur high computational complexity and energy consumption due to extensive data movement across many filters and channels. The study aims to design a dataflow that enables parallel processing while minimizing data movement to achieve energy‑efficient CNN execution without sacrificing accuracy. The authors introduce a row‑stationary dataflow that maximizes local reuse of filter weights and activations, reduces partial‑sum movement, adapts to various CNN shapes, and leverages local PE storage, inter‑PE communication, and spatial parallelism, evaluated via an energy‑analysis framework under fixed area and parallelism. Experiments on AlexNet demonstrate that the row‑stationary dataflow achieves 1.4×–2.5× lower energy in convolutional layers and at least 1.3× in fully connected layers for batch sizes > 16, and its effectiveness is confirmed on a fabricated chip.

Abstract

Deep convolutional neural networks (CNNs) are widely used in modern AI systems for their superior accuracy but at the cost of high computational complexity. The complexity comes from the need to simultaneously process hundreds of filters and channels in the high-dimensional convolutions, which involve a significant amount of data movement. Although highly-parallel compute paradigms, such as SIMD/SIMT, effectively address the computation requirement to achieve high throughput, energy consumption still remains high as data movement can be more expensive than computation. Accordingly, finding a dataflow that supports parallel processing with minimal data movement cost is crucial to achieving energy-efficient CNN processing without compromising accuracy. In this paper, we present a novel dataflow, called row-stationary (RS), that minimizes data movement energy consumption on a spatial architecture. This is realized by exploiting local data reuse of filter weights and feature map pixels, i.e., activations, in the high-dimensional convolutions, and minimizing data movement of partial sum accumulations. Unlike dataflows used in existing designs, which only reduce certain types of data movement, the proposed RS dataflow can adapt to different CNN shape configurations and reduces all types of data movement through maximally utilizing the processing engine (PE) local storage, direct inter-PE communication and spatial parallelism. To evaluate the energy efficiency of the different dataflows, we propose an analysis framework that compares energy cost under the same hardware area and processing parallelism constraints. Experiments using the CNN configurations of AlexNet show that the proposed RS dataflow is more energy efficient than existing dataflows in both convolutional (1.4× to 2.5×) and fully-connected layers (at least 1.3× for batch size larger than 16). The RS dataflow has also been demonstrated on a fabricated chip, which verifies our energy analysis.