2023 · 13 citations · 25 references
ComputingComputer ScienceComputer VisionMachine VisionRobot VisionOptical FlowScene UnderstandingSelf-supervised LearningConsistency RegularizationData AugmentationDeep Optical FlowDistractionsOptical Flow EstimationOptical Flow RegularizationRegularizationSelf-supervised Pretraining
We propose a novel data augmentation approach, DistractFlow, for training optical flow estimation models by introducing realistic distractions to the input frames. Based on a mixing ratio, we combine one of the frames in the pair with a distractor image depicting a similar domain, which allows for inducing visual perturbations congruent with natural objects and scenes. We refer to such pairs as distracted pairs. Our intuition is that using semantically meaningful distractors enables the model to learn related variations and attain robustness against challenging deviations, compared to conventional augmentation schemes focusing only on low-level aspects and modifications. More specifically, in addition to the supervised loss computed between the estimated flow for the original pair and its ground-truth flow, we include a second supervised loss defined between the distracted pair's flow and the original pair's ground-truth flow, weighted with the same mixing ratio. Furthermore, when unlabeled data is available, we extend our augmentation approach to self-supervised settings through pseudo-labeling and cross-consistency regularization. Given an original pair and its distracted version, we enforce the estimated flow on the distracted pair to agree with the flow of the original pair. Our approach allows increasing the number of available training pairs significantly without requiring additional annotations. It is agnostic to the model architecture and can be applied to training any optical flow estimation models. Our extensive evaluations on multiple benchmarks, including Sintel, KITTI, and SlowFlow, show that DistractFlow improves existing models consistently, outperforming the latest state of the art.
25
Vision meets robotics: The KITTI dataset
Andreas Geiger, Philip Lenz, Christoph Stiller et al. · The International Journal of Robotics Research · 2013 · 9.7K citations
mixup: Beyond Empirical Risk Minimization
Hongyi Zhang, Moustapha Cissé, Yann Dauphin et al. · arXiv (Cornell University) · 2017 · 4.8K citations · Full text
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Sangdoo Yun, Dongyoon Han, Sanghyuk Chun et al. · 2019 · 4.7K citations
FlowNet: Learning Optical Flow with Convolutional Networks
Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg et al. · 2015 · 4.2K citations
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg, N. Michael Mayer, Tonmoy Saikia et al. · 2017 · 3.3K citations