Temporal consistency enhancement on depth sequences

Abstract

Currently, depth sequences generated by automatic depth estimation suffer from the temporal inconsistency problem. Estimated depth values of some objects vary in adjacent frames, whereas the objects actually remain on the same depth planes. These temporal depth errors significantly impair the visual quality of the synthesized virtual view as well as the coding efficiency of the depth sequences. Since depth sequences correspond to texture sequences, some erroneous temporal depth variations can be detected by analyzing temporal variations of the texture sequences. Utilizing this property, we propose a novel solution to enhance the temporal consistency of depth sequences by applying adaptive temporal filtering on them. Experiments demonstrate that the proposed depth filtering algorithm can effectively suppress transient depth errors and generate more stable depth sequences, resulting in notable temporal quality improvement of the synthesized views and higher coding efficiency on the depth sequences.