View-Consistent MeshFlow for Stereoscopic Video Stabilization

Abstract

This paper presents a method to stabilize shaky stereoscopic videos captured by hand-held stereo cameras. It is often problematic to apply a traditional monocular video stabilization techniques directly to the stereoscopic views independently. This is mainly because some undesirable vertical disparities and inaccurate horizontal disparities are produced, which violates the original stereoscopic disparity constraint, leading to erroneous depth perceptions. In this paper, we show that the MeshFlow stabilization method for monocular videos can be extended for stereoscopic videos by taking additional disparity constraints during the stabilization. In particular, we first estimate disparities between two views. Then, we compute camera motions by the MeshFlow motion model, in which the camera paths can be extracted from the meshes at each view. Next, we smooth these paths of two views separately. After path optimization, we adjust the meshes of one view by our proposed joint the disparity and stability mesh warp, so that the temporal stability and the correct depth perception can be achieved simultaneously. We evaluate our method on various challenging stereoscopic videos with different camera motions and scene types. In the experiment, we adapt the objective quality assessment of single videos to evaluate our stereoscopic video stabilization. We further propose an objective evaluation method to assess the quality of the disparities in terms of the spatial and temporal coherence after the stabilization. The experimental results demonstrate the effectiveness of our method both quantitatively and qualitatively.