Performance Measures for Video Object Segmentation and Tracking

TLDR

The authors propose quantitative performance measures for video object segmentation and tracking that do not require ground‑truth segmentation maps. These measures compute spatial differences of color and motion along the estimated object boundary and temporal differences of color histograms between successive frames, enabling localization of good or bad segmentation regions and the aggregation into a single numerical score for boundary segmentation and tracking quality over a sequence. The measures were validated by canonical correlation analysis against ground‑truth based metrics and experimentally evaluated on multiple sequences using various segmentation approaches.

Abstract

We propose measures to evaluate quantitatively the performance of video object segmentation and tracking methods without ground-truth (GT) segmentation maps. The proposed measures are based on spatial differences of color and motion along the boundary of the estimated video object plane and temporal differences between the color histogram of the current object plane and its predecessors. They can be used to localize (spatially and/or temporally) regions where segmentation results are good or bad; and/or they can be combined to yield a single numerical measure to indicate the goodness of the boundary segmentation and tracking results over a sequence. The validity of the proposed performance measures without GT have been demonstrated by canonical correlation analysis with another set of measures with GT on a set of sequences (where GT information is available). Experimental results are presented to evaluate the segmentation maps obtained from various sequences using different segmentation approaches.