IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · 2022 · 35 citations · 34 references
ComputingComputer SystemsMarkov Decision ProcessesOptimizationSchedulingMobile Edge ComputingReinforcement LearningConstrained Partially Observable Markov Decision ProcessesHeuristic SchedulingMarkov Decision Process SchedulingMinimizationMobile Edge Computing ArchitectureReal-time Monitoring
With the development of Mobile Edge Computing (MEC) and Internet of Things (IoT) technology, various real-time monitoring and control applications are deployed to benefit people’s daily life. The performance of these applications relies heavily on the timeliness of collected environmental information, which can be effectively quantified by the recently introduced metric named age of information (AoI). Although extensive researches have been conducted to optimize AoI under various circumstances, these works commonly require a priori information about the system dynamics that is usually unknown in realistic situations. To design a more practical scheduling algorithm, in this paper, we formulate the AoI minimization problem as a Constrained Markov Decision Process (CMDP) which can be solved by Reinforcement Learning (RL) algorithms without prior knowledge. To improve the running efficiency, we (1) introduce post-decision states (PDSs) to exploit the partial knowledge of the system’s dynamics, (2) perform a batch update in every learning step, (3) decompose the system-level value function into multiple device-level value functions, and (4) propose a heuristic algorithm to find the greedy action. Numerical results demonstrate that our algorithm is highly efficient and outperforms the benchmarks under various scenarios.
34
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
A Survey on Mobile Edge Computing: The Communication Perspective
Yuyi Mao, Changsheng You, Jun Zhang et al. · IEEE Communications Surveys & Tutorials · 2017 · 5.4K citations
Real-time status: How often should one update?
Sanjit K. Kaul, Roy D. Yates, Marco Gruteser · 2012 · 2.4K citations