A Policy-Based Reinforcement Learning Approach for High-Speed Railway Timetable Rescheduling

Abstract

In the daily management of high-speed railway systems, the train timetable rescheduling problem with unpredictable disturbances is a challenging task. The large number of stations and trains leads to a long-time consumption to solve the rescheduling problem, making it difficult to meet the realtime requirements in real-world railway networks. This paper proposes a policy-based reinforcement learning approach to address the high-speed railway timetable rescheduling problem, in which the agent minimizes the total delay by adjusting the departure sequence of all trains along the railway line. A two-stage Markov Decision Process model is established to model the environment where states, actions, and reward functions are designed. The proposed method contains an offline learning process and an online application process, which can give the optimal rescheduling schedule based on the current state immediately. Numerical experiments are performed over two different delay scenarios on the Beijing-Shanghai high-speed railway line. The simulation results show that our approach can find a high-quality rescheduling strategy within one second, which is superior to the First-Come-First-Served (FCFS) and First-Scheduled-First-Served (FSFS) methods.