2023 · 90 citations · 43 references
ComputingComputer ScienceComputer VisionMachine VisionRobot VisionMotion AnalysisMotion EstimationOptical FlowVisual TrackingLearning-based Optical FlowMotion Blur RemovalMulti-frame Optical FlowOptical Flow EstimationSpatial Attention NetworksTransformer ModelsVideo Object Detection
Optical flow is an indispensable building block for various important computer vision tasks, including motion estimation, object tracking, and disparity measurement. In this work, we propose TransFlow, a pure transformer architecture for optical flow estimation. Compared to dominant CNN-based methods, TransFlow demonstrates three advantages. First, it provides more accurate correlation and trustworthy matching in flow estimation by utilizing spatial self-attention and crossattention mechanisms between adjacent frames to effectively capture global dependencies; Second, it recovers more compromised information (e.g., occlusion and motion blur) in flow estimation through long-range temporal association in dynamic scenes; Third, it enables a concise self-learning paradigm and effectively eliminate the complex and laborious multi-stage pre-training procedures. We achieve the state-of-the-art results on the Sintel, KITTI-15, as well as several downstream tasks, including video object detection, interpolation and stabilization. For its efficacy, we hope TransFlow could serve as a flexible baseline for optical flow estimation.
43
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
FlowNet: Learning Optical Flow with Convolutional Networks
Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg et al. · 2015 · 4.2K citations
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg, N. Michael Mayer, Tonmoy Saikia et al. · 2017 · 3.3K citations