A $Q$ -Learning-Based Proactive Caching Strategy for Non-Safety Related Services in Vehicular Networks

Abstract

Content caching has brought huge potential for the provisioning of non-safety related infotainment services in future vehicular networks. Assisted by multiaccess edge computing, roadside units (RSUs) could become cache-capable and offer fast caching services to moving vehicles for content providers. On the other hand, deep learning makes it possible to accurately estimate the behavior of vehicles, which enables effective proactive caching strategies. However, caching services considering both the mobility of vehicles and storage could incur increased latency and considerable cost due to the cache size needed in RSUs. In this paper, we model such a problem using Markov decision processes, and propose a heuristic Q-learning solution together with vehicle movement predictions based on a long short-term memory network. The optimal caching strategy which minimizes the latency of caching services can be derived by our heuristic ε <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> -greedy training processes. Numerical results demonstrate that our proposed strategy can achieve better performance compared with several baselines under different prediction accuracies.