2020 · 18 citations · 13 references
ComputingComputer ScienceComputer SystemsInternet Of ThingsMachine LearningSchedulingCachingCaching StrategiesDynamic SchedulingQueueing TheoryReinforcement LearningResource SchedulingAdaptive CachingCache OptimizationCache-aware SchedulingData CachingDeep Reinforcement LearningMarkov Decision ModelsMarkov Decision Process SchedulingMultimedia Caching
The recent development in Internet of Things necessitates caching of dynamic contents, where new versions of contents become available around-the-clock and thus timely update is required to ensure their relevance. The age of information (AoI) is a performance metric that evaluates the freshness of contents. Existing works on AoI-optimization of cache content update algorithms focus on minimizing the long-term average AoI of all cached contents. Sometimes user requests that need to be served in the future are known in advance and can be stored in user request queues. In this paper, we propose dynamic cache content update scheduling algorithms that exploit the user request queues. We consider a special use case where the trained neural networks (NNs) from deep learning models are being cached in a heterogeneous network. A queue-aware cache content update scheduling algorithm based on Markov decision process (MDP) is developed to minimize the average AoI of the NNs delivered to the users plus the cost related to content updating. By using deep reinforcement learning (DRL), we propose a low complexity suboptimal scheduling algorithm. Simulation results show that, under the same update frequency, our proposed algorithms outperform the periodic cache content update scheme and reduce the average AoI by up to 35%.
13
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver et al. · Nature · 2015 · 30K citations
Reinforcement Learning: An Introduction
Jeffrey D. Johnson, Jinghong Li, Zengshi Chen · Neurocomputing · 2000 · 8.7K citations
Real-time status: How often should one update?
Sanjit K. Kaul, Roy D. Yates, Marco Gruteser · 2012 · 2.4K citations
Mohammad Ali Maddah-Ali, Urs Niesen · IEEE Transactions on Information Theory · 2014 · 1.8K citations