2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · 2022 · 84 citations · 32 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingLanguage ModelingMultimodal CorporaAnnotation Of Video DataBenchmark BiasBiasesData Collection StrategyData-intensive ComputingGroundingMomentsVisual Grounding
The recent and increasing interest in video-language research has driven the development of large-scale datasets that enable data-intensive machine learning techniques. In comparison, limited effort has been made at assessing the fitness of these datasets for the video-language grounding task. Recent works have begun to discover significant limitations in these datasets, suggesting that state-of-the-art techniques commonly overfit to hidden dataset biases. In this work, we present MAD (Movie Audio Descriptions), a novel benchmark that departs from the paradigm of augmenting existing video datasets with text annotations and focuses on crawling and aligning available audio descriptions of mainstream movies. MAD contains over 384, 000 natural language sentences grounded in over 1, 200 hours of videos and exhibits a significant reduction in the currently diagnosed biases for video-language grounding datasets. MAD's collection strategy enables a novel and more challenging version of video-language grounding, where short temporal moments (typically seconds long) must be accurately grounded in diverse long-form videos that can last up to three hours. We have released MAD's data and baselines code at https://github.com/Soldelli/MAD.
32
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
Video Google: a text retrieval approach to object matching in videos
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford, Jong Wook Kim, Chris Hallacy et al. · arXiv (Cornell University) · 2021 · 5.3K citations · Full text
ActivityNet: A large-scale video benchmark for human activity understanding
Fabian Caba Heilbron, Víctor Escorcia, Bernard Ghanem et al. · 2015 · 2.6K citations
DeepGCNs: Can GCNs Go As Deep As CNNs?
Guohao Li, Matthias Müller, Ali Thabet et al. · 2019 · 1.3K citations