2024 · 12 citations · 34 references
EngineeringComputer EngineeringComputer ScienceComputer VisionField-programmable Gate ArraysReconfigurable ComputingAccelerator ArchitectureDeep LearningEvent-driven ArchitectureFpga ArchitectureHardware AccelerationReconfigurable AcceleratorsDeep Learning AcceleratorsEdge IntelligenceEvent ProcessingEvent-based VisionEvent-driven SensingGpu AcceleratorsReal-time ProcessingSparse Feature Representation
Event-based vision represents a paradigm shift in how vision information is captured and processed. By only responding to dynamic intensity changes in the scene, event-based sensing produces far less data than conventional frame-based cameras, promising to springboard a new generation of high-speed, low-power machines for edge intelligence. However, processing such dynamically sparse input originated from event cameras efficiently in real time, particularly with complex deep neural networks (DNN), remains a formidable challenge. Existing solutions that employ GPUs and other frame-based DNN accelerators often struggle to efficiently process the dynamically sparse event data, missing the opportunities to improve processing efficiency with sparse data. To address this, we propose ESDA, a composable dynamic sparse dataflow architecture that allows customized DNN accelerators to be constructed rapidly on FPGAs for event-based vision tasks. ESDA is a modular system that is composed of a set of parametrizable modules for each network layer type. These modules share a uniform sparse token-feature interface and can be connected easily to compose an all-on-chip dataflow accelerator on FPGA for each network model. To fully exploit the intrinsic sparsity in event data, ESDA incorporates the use of submanifold sparse convolutions that largely enhance the activation sparsity throughout the layers while simplifying hardware implementation. Finally, a network architecture and hardware implementation co-optimizing framework that allows tradeoffs between accuracy and performance is also presented. Experimental results demonstrate that when compared with existing GPU and hardware-accelerated solutions, ESDA achieves substantial speedup and improvement in energy efficiency across different applications, and it allows much wider design space for real-world deployments.
34
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu et al. · 2018 · 25.4K citations
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin et al. · 2018 · 9K citations
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Mike Davies, Narayan Srinivasa, Tsung-Han Lin et al. · IEEE Micro · 2018 · 3.7K citations
Engineering, Computer Engineering, Computational Neuroscience +16