OpenVINO Deep Learning Workbench: Comprehensive Analysis and Tuning of Neural Networks Inference

Abstract

A task of maximizing deep learning neural networks performance is a challenging and actual goal of modern hardware and software development. Regardless the huge variety of optimization techniques and emerging dedicated hardware platforms, the process of tuning the performance of the neural network is hard. It requires configuring dozens of hyper parameters of optimization algorithms, selecting appropriate metrics, benchmarking the intermediate solutions to choose the best method, platform etc. Moreover, it is required to setup the hardware for the specific inference target. This paper introduces a sophisticated software solution (Deep Learning Workbench) that provides interactive user interface, simplified process of 8-bit quantization, speeding up convolutional operations using the Winograds minimal filtering algorithms, measuring accuracy of the resulting model. The proposed software is built over the open source OpenVINO framework and supports huge range of modern deep learning models.