Computers & Industrial Engineering · 2024 · 15 citations · 50 references
EngineeringIndustrial EngineeringManufacturing EngineeringMaintenance EngineeringProduction PlanningSchedulingDynamic SchedulingIndustrial MaintenanceMaintenance ManagementMaintenance OptimizationMaintenance SchedulingReinforcement LearningDeep Reinforcement LearningDigital Twin-based MaintenanceFlexible Job Shop SchedulingJoint Decision-makingMaintenance Cost OptimizationMaintenance PlanningProduction ReliabilityReal-time Decision Making
Maintenance and production scheduling are intertwined activities that should be addressed simultaneously to uphold production systems’ reliability and production efficiency. The digital twin is an emerging technology that advances industrial digitalization, as it embeds a “virtual” image of reality that runs in line with the real system, enabling evaluation, optimization, and prediction of the physical system’ state. On the other hand, deep reinforcement learning (DRL) can provide real-time decision-making by leveraging real-time data from the digital twin such as condition monitoring and production progress data. This research addresses the joint flexible job shop scheduling and maintenance planning problem, considering new job insertions and multi-component machines with economic dependencies among components. The objective is to minimize both the expected total tardiness and maintenance cost, considering opportunistic grouping of maintenance activities on components and breakdown costs associated with failure risk. To achieve this joint decision-making, we develop a hierarchical architecture with two interconnected hierarchies. The upper-level hierarchy determines whether to switch the decision-making process between production scheduling and maintenance planning. Subsequently, the lower-level hierarchy selects an action through the corresponding agent: a multi-head Deep Q-Network (DQN) if the maintenance option is chosen, and a DQN with seven dispatching rules for the production option. The computational experiment results reveal that the proposed scheduling method can learn a high-quality dispatching policy, outperforming the non-hierarchical DRL approach and individual dispatching rules in solution quality, CPLEX in runtime efficiency, and the genetic algorithm and particle swarm optimization in both solution quality and runtime efficiency.
50
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison et al. · Nature · 2016 · 15.8K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Deep Reinforcement Learning with Double Q-Learning
Hado van Hasselt, Arthur Guez, David Silver · 2016 · 6K citations · Full text