[DL] A Survey of FPGA-based Neural Network Inference Accelerators

TLDR

Neural networks dominate image, speech, and video recognition, yet their high compute and storage demands limit CPU deployment; GPUs are the current standard, but FPGA-based accelerators are emerging as a promising alternative for speed and energy efficiency. This review surveys prior FPGA-based neural network inference accelerator work and summarizes the main techniques employed. The authors analyze designs from software to hardware, from circuit to system level, to provide a comprehensive framework for future development. The resulting analysis offers a detailed guide to FPGA accelerator design, highlighting performance and energy efficiency trade‑offs across all system levels.

Abstract

Recent research on neural networks has shown a significant advantage in machine learning over traditional algorithms based on handcrafted features and models. Neural networks are now widely adopted in regions like image, speech, and video recognition. But the high computation and storage complexity of neural network inference poses great difficulty on its application. It is difficult for CPU platforms to offer enough computation capacity. GPU platforms are the first choice for neural network processes because of its high computation capacity and easy-to-use development frameworks. However, FPGA-based neural network inference accelerator is becoming a research topic. With specifically designed hardware, FPGA is the next possible solution to surpass GPU in speed and energy efficiency. Various FPGA-based accelerator designs have been proposed with software and hardware optimization techniques to achieve high speed and energy efficiency. In this article, we give an overview of previous work on neural network inference accelerators based on FPGA and summarize the main techniques used. An investigation from software to hardware, from circuit level to system level is carried out to complete analysis of FPGA-based neural network inference accelerator design and serves as a guide to future work.