2023 · 16 citations · 16 references
ComputingComputer EngineeringComputer ScienceArtificial IntelligenceDesign AutomationMachine LearningDeep LearningEdge Artificial IntelligenceGradient-based OptimizationNeural NetworksOptimization For Machine LearningPattern RecognitionBaseline ArchitectureGradient Method OptimizationMobile Edge DevicesOptimization AlgorithmsPipelines
Accurate yet efficient Deep Neural Networks (DNNs) are in high demand, especially for applications that require their execution on constrained edge devices. Finding such DNNs in a reasonable time for new applications requires automated optimization pipelines since the huge space of hyper-parameter combinations is impossible to explore extensively by hand. In this work, we propose PLiNIO, an open-source library implementing a comprehensive set of state-of-the-art DNN design automation techniques, all based on lightweight gradient-based optimization, under a unified and user-friendly interface. With experiments on several edge-relevant tasks, we show that combining the various optimizations available in PLiNIO leads to rich sets of solutions that Pareto-dominate the considered baselines in terms of accuracy vs model size. Noteworthy, PLiNIO achieves up to 94.34% memory reduction for a <1% accuracy drop compared to a baseline architecture.
16
AutoML: A survey of the state-of-the-art
Xin He, Kaiyong Zhao, Xiaowen Chu · Knowledge-Based Systems · 2020 · 1.7K citations · Full text
HAQ: Hardware-Aware Automated Quantization With Mixed Precision
Kuan Wang, Zhijian Liu, Yujun Lin et al. · 2019 · 940 citations
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Jungwook Choi, Zhuo Wang, Swagath Venkataramani et al. · arXiv (Cornell University) · 2018 · 720 citations · Full text
HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision
Zhen Dong, Zhewei Yao, Amir Gholami et al. · 2019 · 458 citations