A Quality-of-Experience Database for Adaptive Video Streaming

Abstract

The dynamic adaptive streaming over HTTP provides an inter-operable solution to overcome the volatile network conditions, but its complex characteristic brings new challenges to objective video quality-of-experience (QoE) measurement. To test the generalizability and to facilitate the wide usage of QoE measurement techniques in real-world applications, we establish a new database named Waterloo Streaming QoE Database III (SQoE-III). Unlike existing databases constructed with hand-crafted test sequences, the SQoE-III database, so far the largest and most realistic of its kind, consists of a total of 450 streaming videos created from diverse source content and diverse distortion patterns, with six adaptation algorithms of diverse characteristics under 13 representative network conditions. All streaming videos are assessed by 34 subjects, and a comprehensive evaluation is conducted on the performance of 15 objective QoE models from four categories with regards to their efficacy in predicting subjective QoE. Detailed correlation analysis and statistical hypothesis testing are carried out. The results of this paper shed light on the future development of adaptive bitrate streaming algorithm and video QoE monitoring system. The subjective database is available online at https://ece.uwaterloo.ca/~zduanmu/tbc2018qoe/.