Power Efficient Machine Learning Models Deployment on Edge IoT Devices

TLDR

Ubiquitous computing has evolved to rely on small, networked edge devices that enable AI inference, but their limited computational and power resources constrain machine‑learning deployment. This study evaluates the power efficiency of applying established optimization techniques to common machine‑learning models on edge IoT devices. The authors benchmarked several standard optimization methods on a selection of popular ML models across two distinct edge systems with varying architectures. The experiments show that optimized models consume significantly less power than idle baselines, revealing that architecture choice strongly influences energy savings on edge devices.

Abstract

Computing has undergone a significant transformation over the past two decades, shifting from a machine-based approach to a human-centric, virtually invisible service known as ubiquitous or pervasive computing. This change has been achieved by incorporating small embedded devices into a larger computational system, connected through networking and referred to as edge devices. When these devices are also connected to the Internet, they are generally named Internet-of-Thing (IoT) devices. Developing Machine Learning (ML) algorithms on these types of devices allows them to provide Artificial Intelligence (AI) inference functions such as computer vision, pattern recognition, etc. However, this capability is severely limited by the device’s resource scarcity. Embedded devices have limited computational and power resources available while they must maintain a high degree of autonomy. While there are several published studies that address the computational weakness of these small systems-mostly through optimization and compression of neural networks- they often neglect the power consumption and efficiency implications of these techniques. This study presents power efficiency experimental results from the application of well-known and proven optimization methods using a set of well-known ML models. The results are presented in a meaningful manner considering the “real world” functionality of devices and the provided results are compared with the basic “idle” power consumption of each of the selected systems. Two different systems with completely different architectures and capabilities were used providing us with results that led to interesting conclusions related to the power efficiency of each architecture.