Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices

TLDR

Recent deep neural network development targets mobile devices, producing compact and sparse models that vary widely in layer shapes and sizes, which existing large‑model accelerators cannot efficiently handle. Eyeriss v2 is introduced as a DNN accelerator architecture specifically designed to run these compact and sparse networks. It employs a highly flexible hierarchical mesh on‑chip network that adapts to diverse data reuse and bandwidth needs, and processes weights and activations directly in their compressed form to boost speed and energy efficiency. On a 65‑nm CMOS chip, Eyeriss v2 achieves 1470.6 inferences per second and 2560.3 inferences per joule with sparse MobileNet, outperforming the original Eyeriss by 12.6× in speed and 2.5× in energy efficiency.

Abstract

A recent trend in deep neural network (DNN) development is to extend the reach of deep learning applications to platforms that are more resource and energy-constrained, e.g., mobile devices. These endeavors aim to reduce the DNN model size and improve the hardware processing efficiency and have resulted in DNNs that are much more compact in their structures and/or have high data sparsity. These compact or sparse models are different from the traditional large ones in that there is much more variation in their layer shapes and sizes and often require specialized hardware to exploit sparsity for performance improvement. Therefore, many DNN accelerators designed for large DNNs do not perform well on these models. In this paper, we present Eyeriss v2, a DNN accelerator architecture designed for running compact and sparse DNNs. To deal with the widely varying layer shapes and sizes, it introduces a highly flexible on-chip network, called hierarchical mesh, that can adapt to the different amounts of data reuse and bandwidth requirements of different data types, which improves the utilization of the computation resources. Furthermore, Eyeriss v2 can process sparse data directly in the compressed domain for both weights and activations and therefore is able to improve both processing speed and energy efficiency with sparse models. Overall, with sparse MobileNet, Eyeriss v2 in a 65-nm CMOS process achieves a throughput of 1470.6 inferences/s and 2560.3 inferences/J at a batch size of 1, which is 12.6× faster and 2.5× more energyefficient than the original Eyeriss running MobileNet.