2021 · 29 citations · 19 references
ComputingComputer EngineeringComputer ScienceArtificial IntelligenceComputer VisionMachine LearningDeep LearningHardware AccelerationNeural NetworksPattern RecognitionRepresentation LearningSemantic SegmentationDigital Image EnhancementNeural Network QuantizationPrecisionQuantizationTraining
Quantization plays an important role in deep neural network (DNN) hardware. In particular, logarithmic quantization has multiple advantages for DNN hardware implementations, and its weakness in terms of lower performance at high precision compared with linear quantization has been recently remedied by what we call selective two-word logarithmic quantization (STLQ). However, there is a lack of training methods designed for STLQ or even logarithmic quantization in general. In this paper we propose a novel STLQ-aware training method, which significantly out-performs the previous state-of-the-art training method for STLQ. Moreover, our training results demonstrate that with our new training method, STLQ applied to weight parameters of ResNet-18 can achieve the same level of performance as state-of-the-art quantization method, APoT, at 3-bit precision. We also apply our method to various DNNs in image enhancement and semantic segmentation, showing competitive results.
19
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Xception: Deep Learning with Depthwise Separable Convolutions
François Chollet · 2017 · 19K citations
Rethinking Atrous Convolution for Semantic Image Segmentation
Liang-Chieh Chen, George Papandreou, Florian Schroff et al. · arXiv (Cornell University) · 2017 · 7.5K citations · Full text