daBNN: A Super Fast Inference Framework for Binary Neural Networks on\n ARM devices

Abstract

It is always well believed that Binary Neural Networks (BNNs) could\ndrastically accelerate the inference efficiency by replacing the arithmetic\noperations in float-valued Deep Neural Networks (DNNs) with bit-wise\noperations. Nevertheless, there has not been open-source implementation in\nsupport of this idea on low-end ARM devices (e.g., mobile phones and embedded\ndevices). In this work, we propose daBNN --- a super fast inference framework\nthat implements BNNs on ARM devices. Several speed-up and memory refinement\nstrategies for bit-packing, binarized convolution, and memory layout are\nuniquely devised to enhance inference efficiency. Compared to the recent\nopen-source BNN inference framework, BMXNet, our daBNN is\n$7\\times$$\\sim$$23\\times$ faster on a single binary convolution, and about\n$6\\times$ faster on Bi-Real Net 18 (a BNN variant of ResNet-18). The daBNN is a\nBSD-licensed inference framework, and its source code, sample projects and\npre-trained models are available on-line: https://github.com/JDAI-CV/dabnn.\n