Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation

TLDR

Optical flow estimation traditionally requires dense temporal sampling, but coarse‑to‑fine warping only partially relaxes this, creating a scale–velocity dependency that hampers accurate tracking of fast‑moving small body parts. The study proposes integrating rich descriptors into variational optical flow to address this limitation. The authors incorporate descriptor matching into the variational framework to enhance motion estimation. The approach yields dense optical flow with accuracy comparable to traditional variational methods while enabling analysis in domains lacking dense temporal sampling.

Abstract

Optical flow estimation is classically marked by the requirement of dense sampling in time. While coarse-to-fine warping schemes have somehow relaxed this constraint, there is an inherent dependency between the scale of structures and the velocity that can be estimated. This particularly renders the estimation of detailed human motion problematic, as small body parts can move very fast. In this paper, we present a way to approach this problem by integrating rich descriptors into the variational optical flow setting. This way we can estimate a dense optical flow field with almost the same high accuracy as known from variational optical flow, while reaching out to new domains of motion analysis where the requirement of dense sampling in time is no longer satisfied.