Multi-Objective Workflow Scheduling With Deep-Q-Network-Based Multi-Agent Reinforcement Learning

TLDR

Cloud computing enables large‑scale workflow execution, yet optimal scheduling for multiple conflicting objectives remains inadequately addressed, with existing methods limited by expert‑dependent encoding that hampers performance. This study applies a deep‑Q‑network in a multi‑agent reinforcement learning framework to schedule multiple workflows on IaaS clouds. The authors formulate scheduling as a Markov game whose state encodes workflow counts and heterogeneous VMs, rewards combine makespan and cost, and the game seeks a correlated equilibrium without prior knowledge, converging in dynamic real‑time and validated on scientific workflow templates and Amazon EC2. Experimental results show the proposed approach outperforms traditional multi‑objective scheduling algorithms such as NSGA‑II, MOPSO, and game‑theoretic greedy methods in generating more optimal plans.

Abstract

Cloud Computing provides an effective platform for executing large-scale and complex workflow applications with a pay-as-you-go model. Nevertheless, various challenges, especially its optimal scheduling for multiple conflicting objectives, are yet to be addressed properly. The existing multi-objective workflow scheduling approaches are still limited in many ways, e.g., encoding is restricted by prior experts' knowledge when handling a dynamic real-time problem, which strongly influences the performance of scheduling. In this paper, we apply a deep-Q-network model in a multi-agent reinforcement learning setting to guide the scheduling of multi-workflows over infrastructure-as-a-service clouds. To optimize multi-workflow completion time and user's cost, we consider a Markov game model, which takes the number of workflow applications and heterogeneous virtual machines as state input and the maximum completion time and cost as rewards. The game model is capable of seeking for correlated equilibrium between make-span and cost criteria without prior experts' knowledge and converges to the correlated equilibrium policy in a dynamic real-time environment. To validate our proposed approach, we conduct extensive case studies based on multiple well-known scientific workflow templates and Amazon EC2 cloud. The experimental results clearly suggest that our proposed approach outperforms traditional ones, e.g., non-dominated sorting genetic algorithm-II, multi-objective particle swarm optimization, and game-theoretic-based greedy algorithms, in terms of optimality of scheduling plans generated.