DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices

TLDR

Deep learning breakthroughs are transforming sensor data interpretation to provide high‑level information for mobile apps. The authors aim to embed deep learning inference accuracy gains into future mobile apps by developing DeepX, a software accelerator. DeepX employs resource‑control algorithms that decompose deep models into unit‑blocks for efficient execution on heterogeneous processors and scale resources to reduce overhead. DeepX lowers memory, computation, and energy demands, enabling large models to run efficiently on modern mobile processors and outperforming cloud‑based offloading.

Abstract

Breakthroughs from the field of deep learning are radically changing how sensor data are interpreted to extract the high-level information needed by mobile apps. It is critical that the gains in inference accuracy that deep models afford become embedded in future generations of mobile apps. In this work, we present the design and implementation of DeepX, a software accelerator for deep learning execution. DeepX signif- icantly lowers the device resources (viz. memory, computation, energy) required by deep learning that currently act as a severe bottleneck to mobile adoption. The foundation of DeepX is a pair of resource control algorithms, designed for the inference stage of deep learning, that: (1) decompose monolithic deep model network architectures into unit- blocks of various types, that are then more efficiently executed by heterogeneous local device processors (e.g., GPUs, CPUs); and (2), perform principled resource scaling that adjusts the architecture of deep models to shape the overhead each unit-blocks introduces. Experiments show, DeepX can allow even large-scale deep learning models to execute efficently on modern mobile processors and significantly outperform existing solutions, such as cloud-based offloading.