Streaming 360-Degree Videos Using Super-Resolution

TLDR

360° videos offer immersive experiences but demand far more bandwidth than regular videos, and existing viewport‑prediction methods are error‑prone, degrading user QoE. This work introduces PARSEC, a 360° streaming system that simultaneously cuts bandwidth usage and boosts video quality. PARSEC achieves this by heavily compressing video on the server and applying client‑side deep‑learning super‑resolution, using lightweight micro‑models trained on short segments to overcome model size, inference speed, and quality variance, and combining conventional encoding with SR, evaluated on WiFi, FCC broadband, and 4G traces. PARSEC outperforms state‑of‑the‑art 360° streaming systems while significantly reducing bandwidth requirements.

Abstract

360° videos provide an immersive experience to users, but require considerably more bandwidth to stream compared to regular videos. State-of-the-art 360° video streaming systems use viewport prediction to reduce bandwidth requirement, that involves predicting which part of the video the user will view and only fetching that content. However, viewport prediction is error prone resulting in poor user Quality of Experience (QoE). We design PARSEC, a 360° video streaming system that reduces bandwidth requirement while improving video quality. PARSEC trades off bandwidth for additional client-side computation to achieve its goals. PARSEC uses an approach based on super-resolution, where the video is significantly compressed at the server and the client runs a deep learning model to enhance the video to a much higher quality. PARSEC addresses a set of challenges associated with using super-resolution for 360° video streaming: large deep learning models, slow inference rate, and variance in the quality of the enhanced videos. To this end, PAR-SEC trains small micro-models over shorter video segments, and then combines traditional video encoding with super-resolution techniques to overcome the challenges. We evaluate PARSEC on a real WiFi network, over a broadband network trace released by FCC, and over a 4G/LTE network trace. PARSEC significantly outperforms the state-of-art 360° video streaming systems while reducing the bandwidth requirement.