Neural Acceleration for General-Purpose Approximate Programs

TLDR

The paper proposes a learning‑based method to accelerate approximate programs. The authors introduce the Parrot transformation, which trains a neural network to emulate a code region and replaces it with a low‑power neural processing unit (NPU) tightly integrated into the processor pipeline, enabling programmers to mark approximable regions for efficient execution. Experiments show that NPU acceleration yields on average 2.3× speedup and 3.0× energy savings with less than 9.6% quality loss, outperforming the original code.

Abstract

This paper describes a learning-based approach to the acceleration of approximate programs. We describe the \emph{Parrot transformation}, a program transformation that selects and trains a neural network to mimic a region of imperative code. After the learning phase, the compiler replaces the original code with an invocation of a low-power accelerator called a \emph{neural processing unit} (NPU). The NPU is tightly coupled to the processor pipeline to accelerate small code regions. Since neural networks produce inherently approximate results, we define a programming model that allows programmers to identify approximable code regions -- code that can produce imprecise but acceptable results. Offloading approximable code regions to NPUs is faster and more energy efficient than executing the original code. For a set of diverse applications, NPU acceleration provides whole-application speedup of 2.3× and energy savings of 3.0× on average with quality loss of at most 9.6%.