2024 · 15 citations · 36 references
ComputingComputer SystemsEdge ComputingMobile ComputingSchedulingMobile Edge ComputingResource SchedulingWorkload SchedulingAdaptive SchedulingDeep Reinforcement LearningHeterogeneous SchedulingHeterogeneous Task SchedulingMobile Edge Computing ArchitectureMulti-agent Decision MakingReliability-aware SchedulingSafe Multi-agent Learning
In mobile edge computing (MEC), resource scheduling is crucial to task requests’ performance and service providers’ cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice.We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9:1 more profit than existing approaches without sacrificing the delay performance.
36
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
A Survey on Mobile Edge Computing: The Communication Perspective
Yuyi Mao, Changsheng You, Jun Zhang et al. · IEEE Communications Surveys & Tutorials · 2017 · 5.4K citations
Charging and rate control for elastic traffic
F. P. Kelly · European Transactions on Telecommunications · 1997 · 3K citations · Full text
High-Dimensional Continuous Control Using Generalized Advantage Estimation
John Schulman, Philipp Moritz, Sergey Levine et al. · arXiv (Cornell University) · 2015 · 1.8K citations · Full text