Accelerating real-time embedded scene labeling with convolutional networks

TLDR

Deploying advanced computer vision systems in real‑time, power‑constrained scenarios is increasingly common, and brain‑inspired algorithms offer high accuracy and flexibility for embedded vision. The paper aims to develop an optimized convolutional network for real‑time scene labeling on embedded platforms. The implementation uses a convolutional network architecture tailored for embedded deployment, achieving high throughput on the Nvidia Tegra K1. The algorithm achieves up to 96 GOp/s on the Tegra K1 and delivers a 1.5× throughput improvement over a modern desktop CPU at only 11 W.

Abstract

Today there is a clear trend towards deploying advanced computer vision (CV) systems in a growing number of application scenarios with strong real-time and power constraints. Brain-inspired algorithms capable of achieving record-breaking results combined with embedded vision systems are the best candidate for the future of CV and video systems due to their flexibility and high accuracy in the area of image understanding. In this paper, we present an optimized convolutional network implementation suitable for real-time scene labeling on embedded platforms. We show that our algorithm can achieve up to 96GOp/s, running on the Nvidia Tegra K1 embedded SoC. We present experimental results, compare them to the state-of-the-art, and demonstrate that for scene labeling our approach achieves a 1.5x improvement in throughput when compared to a modern desktop CPU at a power budget of only 11 W.