Plato: Learning-based Adaptive Streaming of 360-Degree Videos

Abstract

Interactive applications that come along with 360- degree (or 360) videos have brought immersive experiences to users thanks to the elevated machine computability. In fact, the provision of such high quality of experience (QoE) hinges on the persistent delivery of 360 videos, potentially consuming an excessive need of network bandwidth. To prevent the delivery of entire 360 videos from adversely affecting QoE, tile-based viewport adaptive streaming that divides 360 video chunks into tiles and conveys streams with differentiated quality levels to viewport and non-viewport areas has been regarded as a promising solution. Existing works have been devoted to the design of 1) viewport prediction (VPP) to predict users' viewport orientation due to head movements, and 2) tile bitrate selection (TBS) to determine tile-based bitrates for viewport and non-viewport areas. Despite the heuristic solutions proposed by the existing works, there is lack of knowledge of whether QoE can be enhanced by learning from historical data. In this paper, we propose the system-Plato, to leverage machine learning to tile-based viewport adaptive streaming for 360 videos. In particular, Plato applies long short term memory (LSTM) model to VPP, and uses part of non-viewport areas to help resist prediction errors. In addition, Plato uses real-world traces to train a TBS agent based on reinforcement learning to determine tile bitrates for both viewport and non-viewport areas. Our simulation results show that Plato outperforms existing schemes in various QoE metrics.