Human perception of jitter and media synchronization

TLDR

Multimedia synchronization involves defining and establishing temporal relationships among media types, and presenting data streams in sync is essential for a natural impression, while out‑of‑sync data feels artificial or annoying, and achieving this requires detailed knowledge of user‑interface synchronization requirements. The study aims to enable multimedia systems to present data with minimal synchronization errors and to provide human‑perception‑based quality‑of‑service guidelines through a series of experiments. The authors conducted experiments on human media perception and used the results to devise a scheme for processing nontrivial synchronization skew among more than two data streams. The experiments reveal that certain skews between related data streams can still be perceived as in sync, defining constraints under which jitter is tolerable, and these insights inform a scheme for handling nontr.

Abstract

Multimedia synchronization comprises both the definition and the establishment of temporal relationships among media types. The presentation of 'in sync' data streams is essential to achieve a natural impression, data that is 'out of sync' is perceived as being somewhat artificial, strange, or even annoying. Therefore, the goal of any multimedia system is to enable an application to present data without no or little synchronization errors. The achievement of this goal requires a detailed knowledge of the synchronization requirements at the user interface. The paper presents the results of a series of experiments about human media perception that may be used as 'quality of service' guidelines. The results show that skews between related data streams may still give the effect that the data is 'in sync' and gives some constraints under which jitter may be tolerated. The author uses the findings to develop a scheme for the processing of nontrivial synchronization skew between more than two data streams.