Once-for-All: Train One Network and Specialize it for Efficient\n Deployment

Abstract

We address the challenging problem of efficient inference across many devices\nand resource constraints, especially on edge devices. Conventional approaches\neither manually design or use neural architecture search (NAS) to find a\nspecialized neural network and train it from scratch for each case, which is\ncomputationally prohibitive (causing $CO_2$ emission as much as 5 cars'\nlifetime) thus unscalable. In this work, we propose to train a once-for-all\n(OFA) network that supports diverse architectural settings by decoupling\ntraining and search, to reduce the cost. We can quickly get a specialized\nsub-network by selecting from the OFA network without additional training. To\nefficiently train OFA networks, we also propose a novel progressive shrinking\nalgorithm, a generalized pruning method that reduces the model size across many\nmore dimensions than pruning (depth, width, kernel size, and resolution). It\ncan obtain a surprisingly large number of sub-networks ($> 10^{19}$) that can\nfit different hardware platforms and latency constraints while maintaining the\nsame level of accuracy as training independently. On diverse edge devices, OFA\nconsistently outperforms state-of-the-art (SOTA) NAS methods (up to 4.0%\nImageNet top1 accuracy improvement over MobileNetV3, or same accuracy but 1.5x\nfaster than MobileNetV3, 2.6x faster than EfficientNet w.r.t measured latency)\nwhile reducing many orders of magnitude GPU hours and $CO_2$ emission. In\nparticular, OFA achieves a new SOTA 80.0% ImageNet top-1 accuracy under the\nmobile setting ($<$600M MACs). OFA is the winning solution for the 3rd Low\nPower Computer Vision Challenge (LPCVC), DSP classification track and the 4th\nLPCVC, both classification track and detection track. Code and 50 pre-trained\nmodels (for many devices & many latency constraints) are released at\nhttps://github.com/mit-han-lab/once-for-all.\n