Robust Model-Based Reinforcement Learning Control of a Batch Crystallization Process

Abstract

The development of robust and effective control strategies for crystallization processes is challenging due to the complexity of the underlying phenomena. The main objective is to control crystal mean size, crystal size distribution, purity and crystal shape. A novel trajectory tracking control strategy based on Reinforcement Learning (RL) was developed to control the batch cooling crystallization of paracetamol in water, used here as a case study. More specifically, a model-based reinforcement learning technique was designed to achieve effective tracking of three reference trajectories namely: process temperature, supersaturation and mean particle size to ensure the attainment of the targeted final product quality and dynamic process performance. An off-policy RL method based on a deep deterministic policy gradient (DDPG) algorithm was used. The performance of the RL control was compared and validated against MPC. To address the tracking control problem in the case of noisy measurements and model uncertainties, the RL approach was combined with a Kalman filter. Finally, Transfer Learning (TL) was implemented to investigate optimal and cost-effective training strategies of the RL agent.