Quantization and Training of Neural Networks for Efficient\n Integer-Arithmetic-Only Inference

Benoît Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew F. Tang, Andrew W. Howard, Hartwig Adam, Dmitry Kalenichenko

arXiv (Cornell University) · 2017 · 28 citations · 0 references

DOIFull text

Open access

Concepts

Abstract

The rising popularity of intelligent mobile devices and the daunting\ncomputational cost of deep learning-based models call for efficient and\naccurate on-device inference schemes. We propose a quantization scheme that\nallows inference to be carried out using integer-only arithmetic, which can be\nimplemented more efficiently than floating point inference on commonly\navailable integer-only hardware. We also co-design a training procedure to\npreserve end-to-end model accuracy post quantization. As a result, the proposed\nquantization scheme improves the tradeoff between accuracy and on-device\nlatency. The improvements are significant even on MobileNets, a model family\nknown for run-time efficiency, and are demonstrated in ImageNet classification\nand COCO detection on popular CPUs.\n