Adaptive Video Streaming With Edge Caching and Video Transcoding Over Software-Defined Mobile Networks: A Deep Reinforcement Learning Approach

Abstract

Both mobile edge cloud (MEC) and software-defined networking (SDN) are technologies for next generation mobile networks. In this paper, we propose to simultaneously optimize energy consumption and quality of experience (QoE) metrics in video streaming over software-defined mobile networks (SDMN) combined with MEC. Specifically, we propose a novel mechanism to jointly consider buffer dynamics, video quality adaption, edge caching, video transcoding and transmission. First, we assume that the time-varying channel is a discrete-time Markov chain (DTMC). Then, based on this assumption, we formulate two optimization problems which can be depicted as a constrained Markov decision process (CMDP) and a Markov decision process (MDP). Then, we transform the CMDP problem into regular MDP by deploying Lyapunov technique. We utilize asynchronous advantage actor-critic (A3C) algorithm, one of the model-free deep reinforcement learning (DRL) methods, to solve the corresponding MDP issues. Simulation results are presented to show that the proposed scheme can achieve the goal of energy saving and QoE enhancement with the corresponding constraints satisfied.