Refined TV-<i>L</i> ¹ Optical Flow Estimation Using Joint Filtering

Abstract

Though the accuracy and robustness of optical flow has been dramatically enhanced over the past few years, the issue of edge-blurring near the image and motion boundaries has remained a challenge in flow field estimation. In this paper, we propose a refined total variation with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> norm (TV- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) optical flow estimation approach using joint filtering, named JOF. First, we divide the image into three categorized regions: mutual-structure regions, inconsistent structure regions, and smooth regions. The mutual-structure guided filter for optical flow estimation is constructed by extracting the mutual-structure regions of the flow field. Second, the refined TV- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> optical flow model is proposed by incorporating the non-local term and mutual-structure guided filter objective function into the classical TV- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> energy function. Furthermore, the novel TV- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> optical flow objective function is minimized using a joint filtering program composed of a weighted median filter and a mutual-structure guided filter to optimize the estimated flow field during the coarse-to-fine optical flow computation scheme. Finally, we compare the proposed JOF method with several state-of-the-art approaches including variational and deep learning based optical flow models using the Middlebury, MPI-Sintel, and UCF101 test databases. The evaluation results indicate that the proposed method has high accuracy and good robustness for flow field computation and, especially, the significant benefit of edge-preserving.