State-Aware Tracker for Real-Time Video Object Segmentation

TLDR

Semi‑supervised video object segmentation (VOS) requires leveraging limited labeled frames while exploiting video properties to maintain accuracy. This study introduces the State‑Aware Tracker (SAT), a pipeline designed to deliver accurate VOS results at real‑time speed. SAT achieves this by treating each target as a tracklet, exploiting inter‑frame consistency, and employing two feedback loops that stabilize tracklets and build a robust holistic representation. On the DAVIS 2017‑Val dataset, SAT attains a 72.3 % J&F mean at 39 FPS, demonstrating a favorable balance between speed and accuracy.

Abstract

In this work, we address the task of semi-supervised video object segmentation (VOS) and explore how to make efficient use of video property to tackle the challenge of semi-supervision. We propose a novel pipeline called State-Aware Tracker (SAT), which can produce accurate segmentation results with real-time speed. For higher efficiency, SAT takes advantage of the inter-frame consistency and deals with each target object as a tracklet. For more stable and robust performance over video sequences, SAT gets awareness for each state and makes self-adaptation via two feedback loops. One loop assists SAT in generating more stable tracklets. The other loop helps to construct a more robust and holistic target representation. SAT achieves a promising result of 72.3% J&F mean with 39 FPS on DAVIS 2017-Val dataset, which shows a decent trade-off between efficiency and accuracy.