International Journal of Production Research · 2025 · 15 citations · 57 references
ManagementIndustrial EngineeringManufacturing EngineeringMarkov Decision ProcessesOptimizationSchedulingDynamic SchedulingFlexible Manufacturing SchedulingJob Shop SchedulingMachine SchedulingProduction SchedulingReinforcement LearningDeep Reinforcement LearningDispatching RulesDynamic Scheduling AlgorithmsFlexible Job Shop SchedulingManufacturingMarkov Decision Process SchedulingReconfigurable Manufacturing SystemsReinforcement Learning For Scheduling
In smart manufacturing, uncertainties in discrete manufacturing processes are increasing, making efficient dynamic scheduling a critical challenge. This paper addresses the Dynamic Flexible Job Shop Scheduling Problem with Reconfigurable Manufacturing Cells (DFJSP-RMC), considering dynamic job arrivals and aiming to minimise makespan, tardiness, and operation instability. To tackle this problem, a Noisy Dueling Double DQN with Prioritized Experience Replay (ND3QNP) algorithm is proposed. First, the DFJSP-RMC is formulated as a mixed-integer programming problem and transformed into a Markov decision process. Next, 28 state features are designed based on job attributes, reconfigurable manufacturing cell characteristics, and dynamic factors, while 33 actions are derived from heuristic dispatching rules. A reward function is then constructed by integrating makespan, tardiness, and instability. To mitigate overestimation in the original DQN, double Q-learning, a duelling network, and a noisy network are incorporated. Additionally, prioritised experience replay is employed to enhance learning efficiency. Furthermore, an operation insertion strategy is introduced, significantly improving the algorithm's optimisation performance. Finally, experimental results demonstrate that the proposed ND3QNP algorithm outperforms classical dispatching rules and state-of-the-art deep reinforcement learning approaches in terms of makespan, total tardiness, and operation instability, showcasing its robustness in dynamic and reconfigurable manufacturing environments.
57
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Random search for hyper-parameter optimization
James Bergstra, Yoshua Bengio · 2012 · 7.9K citations
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2015 · 5.4K citations · Full text
Computing, Computer Science, Physics +17