Saliency-Aware Video Object Segmentation

TLDR

Video saliency estimates a dominant object in a sequence, providing strong cues for unsupervised video object segmentation. The paper proposes a geodesic‑distance technique that yields reliable, temporally consistent saliency of superpixels to guide pixel‑wise labeling. The method builds undirected intra‑ and inter‑frame graphs from spatiotemporal edges, appearance, and motion, applies a skeleton abstraction to enhance saliency, and formulates pixel‑wise segmentation as an energy minimization with unary foreground/background, dynamic location, and pairwise smoothness terms. Our method outperforms state‑of‑the‑art in accuracy and speed on benchmark datasets.

Abstract

Video saliency, aiming for estimation of a single dominant object in a sequence, offers strong object-level cues for unsupervised video object segmentation. In this paper, we present a geodesic distance based technique that provides reliable and temporally consistent saliency measurement of superpixels as a prior for pixel-wise labeling. Using undirected intra-frame and inter-frame graphs constructed from spatiotemporal edges or appearance and motion, and a skeleton abstraction step to further enhance saliency estimates, our method formulates the pixel-wise segmentation task as an energy minimization problem on a function that consists of unary terms of global foreground and background models, dynamic location models, and pairwise terms of label smoothness potentials. We perform extensive quantitative and qualitative experiments on benchmark datasets. Our method achieves superior performance in comparison to the current state-of-the-art in terms of accuracy and speed.