YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark

TLDR

Learning long‑term spatial‑temporal features is essential for video analysis, yet current methods rely on static image segmentation or pretrained optical flow and are limited by small datasets of only a few dozen clips. The authors create the YouTube‑VOS dataset to address the lack of large‑scale video segmentation data. YouTube‑VOS comprises 4,453 YouTube clips across 94 object categories, and the authors benchmark multiple state‑of‑the‑art video segmentation methods on it. YouTube‑VOS is the largest video object segmentation dataset to date and is publicly available at youtube‑vos.org.

Abstract

Learning long-term spatial-temporal features are critical for many video analysis tasks. However, existing video segmentation methods predominantly rely on static image segmentation techniques, and methods capturing temporal dependency for segmentation have to depend on pretrained optical flow models, leading to suboptimal solutions for the problem. End-to-end sequential learning to explore spatialtemporal features for video segmentation is largely limited by the scale of available video segmentation datasets, i.e., even the largest video segmentation dataset only contains 90 short video clips. To solve this problem, we build a new large-scale video object segmentation dataset called YouTube Video Object Segmentation dataset (YouTube-VOS). Our dataset contains 4,453 YouTube video clips and 94 object categories. This is by far the largest video object segmentation dataset to our knowledge and has been released at http://youtube-vos.org. We further evaluate several existing state-of-the-art video object segmentation algorithms on this dataset which aims to establish baselines for the development of new algorithms in the future.