Performance evaluation in non‐intrusive load monitoring: Datasets, metrics, and tools—A review

TLDR

Non‑intrusive load monitoring estimates individual appliance energy use from limited power measurements, using machine learning to reduce sensing costs, but its generalization depends on standardized datasets and agreed performance metrics. The paper reviews datasets, metrics, and tools for evaluating NILM performance. The review examines publicly available datasets, performance metrics, and evaluation frameworks and toolkits. It recommends future research on cross‑dataset studies, standardized metrics, and generalizable benchmarking frameworks. Article categorized under Application Areas > Science and Technology, Data Mining Software Tools, Computational Intelligence, and Machine Learning.

Abstract

Non‐intrusive load monitoring (also known as NILM or energy disaggregation) is the process of estimating the energy consumption of individual appliances from electric power measurements taken at a limited number of locations in the electric distribution of a building. This approach reduces sensing infrastructure costs by relying on machine learning techniques to monitor electric loads. However, the ability to evaluate and benchmark the proposed approaches across different datasets is key for enabling the generalization of research findings and consequently contributes to the large‐scale adoption of this technology. Still, only recently researchers have focused on creating and standardizing the existing datasets in order to deliver a single interface to run NILM evaluations. Furthermore, there is still no consensus regarding, which performance metrics should be used to measure and report the performance of NILM systems and their underlying algorithms. This paper provides a review of the main datasets, metrics, and tools for evaluating the performance of NILM systems and technologies. Specifically, we review three main topics: (a) publicly available datasets, (b) performance metrics, and (c) frameworks and toolkits. The review suggests future research directions in NILM systems and technologies, including cross‐datasets, performance metrics for evaluation and generalizable frameworks for benchmarking NILM technology. This article is categorized under: Application Areas > Science and Technology Application Areas > Data Mining Software Tools Technologies > Computational Intelligence Technologies > Machine Learning