2020 · 38 citations · 11 references
EngineeringComputer EngineeringComputer ScienceComputer ArchitectureEmbedded SystemsField-programmable Gate ArraysAccelerator ArchitectureDeep LearningInstruction Set ArchitectureMicroarchitecture DesignProcessor Core DesignProcessor MicroarchitectureMemory Footprint ReductionMicroarchitectural OptimizationNeural Network QuantizationPrecisionQuantizationResource-constrained DevicesRisc-v Core DesignSimd Architectures
Low bit-width Quantized Neural Networks (QNNs) enable deployment of complex machine learning models on constrained devices such as microcontrollers (MCUs) by reducing their memory footprint. Fine-grained asymmetric quantization (i.e., different bit-widths assigned to weights and activations on a tensor-by-tensor basis) is a particularly interesting scheme to maximize accuracy under a tight memory constraint. However, the lack of sub-byte instruction set architecture (ISA) support in SoA microprocessors makes it hard to fully exploit this extreme quantization paradigm in embedded MCUs. Support for sub-byte and asymmetric QNNs would require many precision formats and an exorbitant amount of opcode space. In this work, we attack this problem with status-based SIMD instructions: rather than encoding precision explicitly, each operand's precision is set dynamically in a core status register. We propose a novel RISC-V ISA core MPIC (Mixed Precision Inference Core) based on the open-source RI5CY core. Our approach enables full support for mixed-precision QNN inference with 292 different combinations of operands at 16-, 8-, 4- and 2-bit precision, without adding any extra opcode or increasing the complexity of the decode stage. Our results show that MPIC improves both performance and energy efficiency by a factor of 1.1-4.9x when compared to software-based mixed-precision on RI5CY; with respect to commercially available Cortex-M4 and M7 microcontrollers, it delivers 3.6-11.7x better performance and 41-155x higher efficiency.
11
Going Deeper with Embedded FPGA Platform for Convolutional Neural Network
Jiantao Qiu, Jie Wang, Song Yao et al. · 2016 · 1.3K citations
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Jungwook Choi, Zhuo Wang, Swagath Venkataramani et al. · arXiv (Cornell University) · 2018 · 720 citations · Full text