Introducing Hybrid Modeling with Time-Series-Transformers: A Comparative Study of Series and Parallel Approach in Batch Crystallization

Abstract

Given the hesitance surrounding the direct implementation of black-box tools due to safety and operational concerns, fully data-driven deep-neural-network (DNN)-based digital twins are facing an implementation hurdle. To address this, hybrid models combining physics-based, first-principles with machine learning models have gained traction. These models are perceived as the “best of both worlds” solution. However, existing simplistic DNN models fall short of predicting the long-term evolution of process data. Recently, time-series transformers (TSTs), which utilize a multiheaded attention mechanism to capture both long and short-term process dynamics, have demonstrated superior performance. Consequently, a first-of-a-kind, TST-based hybrid modeling framework for batch crystallization has been developed, offering improved accuracy and interpretability when compared to conventional black-box models. Particularly, two different configurations, series and parallel, of TST-based hybrid models were constructed and compared. They demonstrated a normalized-mean-square-error within the range of [10, 50] × 10–4 and an R2 value over 0.99.