Multi-resource constrained dynamic workshop scheduling based on proximal policy optimisation

Abstract

Multi-resource constrained dynamic workshop scheduling is a complex and challenging task in discrete manufacturing. In this paper, to obtain a high-performance scheduling in limited time, this problem is modelled into a Markov decision process, and solved by proximal policy optimisation algorithm, which can learn from the simulated workshop environment directly. A multi-modal hybrid neural network is used in the model to make good use of numerical state features representing workshop environment information and graphical state features representing constraint information during the learning process. Multi-label technique is used in this paper to decouple the output acts of jobs, machines, tools, and workers. Action mask technique coding the constraints is also used to prune invalid exploration. The experimental results show that compared with heuristic rules such as weighted shortest processing time, weighted modified due date, weighted cost over time, apparent tardiness cost and other reinforcement learning methods such as DeepRM and DeepRM2, the performance of the proposed method is at least 1.138% better in scheduling penalty.