arXiv (Cornell University) · 2017 · 28 citations · 0 references
ComputingComputer EngineeringComputer ScienceArtificial IntelligenceMachine LearningMobile ComputingDeep LearningDigital Image ProcessingHardware AccelerationImage ProcessingPattern RecognitionAccuracy-computational Cost TradeoffConvolutional Neural NetworksNeural Network QuantizationOn-device InferenceOn-device Machine LearningQuantization
The rising popularity of intelligent mobile devices and the daunting\ncomputational cost of deep learning-based models call for efficient and\naccurate on-device inference schemes. We propose a quantization scheme that\nallows inference to be carried out using integer-only arithmetic, which can be\nimplemented more efficiently than floating point inference on commonly\navailable integer-only hardware. We also co-design a training procedure to\npreserve end-to-end model accuracy post quantization. As a result, the proposed\nquantization scheme improves the tradeoff between accuracy and on-device\nlatency. The improvements are significant even on MobileNets, a model family\nknown for run-time efficiency, and are demonstrated in ImageNet classification\nand COCO detection on popular CPUs.\n