Learning-based Cache Placement and Content Delivery for Satellite-Terrestrial Integrated Networks

Abstract

To support the explosive content demands from multifarious services and applications, cache-enabled satellite-terrestrial integrated networks (STINs) are envisioned as a key enabler to reduce the content delivery delay and alleviate the backhaul pressure. In this paper, we investigate the joint optimization of cache placement and content delivery in the STIN to minimize the long-term overall content delivery delay. Considering that cache placement and content delivery are interrelated and affected by network dynamics in terms of satellite movement and random content requests, the joint optimization problem is formulated as a sequential decision making problem by leveraging a Markov decision process. We propose a hierarchical deep Q learning (HDQL) algorithm by leveraging two independent deep neural networks to learn the cache placement and content delivery policies with small action space and low time complexity. Simulation results demonstrate that the proposed HDQL algorithm outperforms the benchmark algorithms in terms of content delivery delay in the STINs.