IEEE Transactions on Computers · 2020 · 88 citations · 45 references
EngineeringComputer EngineeringComputer ScienceComputer ArchitectureDesign OptimizationHardware-software Co-designAccelerator ArchitectureDeep LearningHardware AccelerationNeural CircuitsArchitectural ExplorationArchitecture ExplorationArchitecture SearchCircuit TopologyNeural Architecture SearchNeural Circuit ArchitectureNeuromorphic HardwareVon Neumann Architecture
Co-exploration of neural architectures and hardware design is promising due to its capability to simultaneously optimize network accuracy and hardware efficiency. However, state-of-the-art neural architecture search algorithms for the co-exploration are dedicated for the conventional von-Neumann computing architecture, whose performance is heavily limited by the well-known memory wall. In this article, we are the first to bring the computing-in-memory architecture, which can easily transcend the memory wall, to interplay with the neural architecture search, aiming to find the most efficient neural architectures with high network accuracy and maximized hardware efficiency. Such a novel combination makes opportunities to boost performance, but also brings a bunch of challenges: The optimization space spans across multiple design layers from device type and circuit topology to neural architecture; and the presence of device variation may drastically degrade the neural network performance. To address these challenges, we propose a cross-layer exploration framework, namely NACIM, which jointly explores device, circuit and architecture design space and takes device variation into consideration to find the most robust neural architectures, coupled with the most efficient hardware design. Experimental results demonstrate that NACIM can find the robust neural network with 0.45 percent accuracy loss in the presence of device variation, compared with a 76.44 percent loss from the state-of-the-art NAS without consideration of variation; in addition, NACIM achieves an energy efficiency up to 16.3 TOPs/W, 3.17x higher than the state-of-the-art NAS.
45
YOLOv3: An Incremental Improvement
Joseph Redmon, Ali Farhadi · arXiv (Cornell University) · 2018 · 5.9K citations · Full text
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Vivienne Sze, Yu‐Hsin Chen, Tien-Ju Yang et al. · Proceedings of the IEEE · 2017 · 4K citations