IEEE Journal on Emerging and Selected Topics in Circuits and Systems · 2022 · 39 citations · 42 references
ComputingComputer EngineeringComputer ScienceComputer ArchitectureHeterogeneous ComputingLow-power DesignAccelerator ArchitectureAccelerator IntegrationDeep LearningEnergy-efficient Embedded SystemsHardware AccelerationHeterogeneous System ArchitectureAcceleratorsArea EfficiencyHeterogeneous Cluster ArchitectureHeterogeneous Computing ArchitectureHeterogeneous Computing SystemsNon-volatile MemoryRisc ArchitecturesRisc-v Core Design
Deployment of modern TinyML tasks on small battery-constrained IoT devices requires high computational energy efficiency. Analog In-Memory Computing (IMC) using non-volatile memory (NVM) promises major efficiency improvements in deep neural network (DNN) inference and serves as on-chip memory storage for DNN weights. However, IMC’s functional flexibility limitations and their impact on performance, energy, and area efficiency are not yet fully understood at the system level. To target practical end-to-end IoT applications, IMC arrays must be enclosed in heterogeneous programmable systems, introducing new system-level challenges which we aim at addressing in this work. We present a heterogeneous tightly-coupled clustered architecture integrating 8 RISC-V cores, an in-memory computing accelerator (IMA), and digital accelerators. We benchmark the system on a highly heterogeneous workload such as the Bottleneck layer from a MobileNetV2, showing <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$11.5\times $ </tex-math></inline-formula> performance and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$9.5\times $ </tex-math></inline-formula> energy efficiency improvements, compared to highly optimized parallel execution on the cores. Furthermore, we explore the requirements for end-to-end inference of a full mobile-grade DNN (MobileNetV2) in terms of IMC array resources, by scaling up our heterogeneous architecture to a multi-array accelerator. Our results show that our solution, on the end-to-end inference of the MobileNetV2, is one order of magnitude better in terms of execution latency than existing programmable architectures and two orders of magnitude better than state-of-the-art heterogeneous solutions integrating in-memory computing analog cores.
42
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu et al. · 2018 · 25.4K citations
Samuel Williams, Andrew Waterman, David A. Patterson · Communications of the ACM · 2009 · 2.4K citations · Full text
Ali Shafiee, Anirban Nag, Naveen Muralimanohar et al. · ACM SIGARCH Computer Architecture News · 2016 · 1.5K citations