Application-Driven Learning: A Closed-Loop Prediction and Optimization Approach Applied to Dynamic Reserves and Demand Forecasting

Abstract

Application-Driven Learning: Closing the Loop Between the Application and the Estimation of Forecast Models This paper introduces a closed-loop framework called application-driven learning, where the best forecast model is tailored to the application cost structure. Our methodology employs two-stage optimization schemes to derive multivariate point forecasts. The estimation problem is conceived as a bilevel model, and we propose two solution methodologies: an exact one using KKT conditions and a scalable decomposition heuristic. This approach offers a scientifically grounded alternative to ad hoc demand biasing approaches and reserve requirement rules currently adopted by power system operators worldwide. Testing with real data and large-scale systems demonstrates that our methodology consistently outperforms traditional open-loop methods, providing significant potential benefits for energy system operations.