Handling concept drift in deep learning applications for process monitoring

Abstract

As machine learning applications transcend from laboratories to real-world operations in manufacturing use cases such as intelligent maintenance and quality control, questions regarding their continuous reliability and robustness arise. Static datasets used to develop machine learning models can only capture a subset of possible real-world conditions. Instances of concept drift, such as changing environmental-, equipment- and operating conditions may, over time, significantly degrade the performance of machine learning models, compromising safety, acceptance, and economics if not properly addressed. It is thus necessary to (1) detect drifts and (2) efficiently adapt the model to dynamically changing conditions. In this work, we propose a framework to tackle the issue of drift detection by utilizing the underlying neural network's uncertainty. We demonstrate the framework's effectiveness in a case study involving process monitoring of a real-world CNC milling process. The dataset includes accelerometer data and multiple types of realistic concept drifts. In addition, we introduce a methodology for inducing controllable, synthetic concept drift through a simulated sensor rotation. Our experiments show that the introduced drifts lead to a strong performance degradation of the model, which highlights the relevance for practitioners. All drifts are detected using the proposed framework, enabling continuously reliable applications.