arXiv (Cornell University) · 2019 · 11 citations · 36 references
ComputingComputer ScienceComputer VisionMachine LearningVisual RepresentationAction RecognitionDeep LearningObject RecognitionRepresentation LearningScene UnderstandingVideo AnalysisAblation StudiesEmbedding LearningLearned Feature EmbeddingsObject CategorizationUnsupervised Action RecognitionVideo Action RecognitionVideo Representation Learning
Because of the rich dynamical structure of videos and their ubiquity in everyday life, it is a natural idea that video data could serve as a powerful unsupervised learning signal for training visual representations in deep neural networks. However, instantiating this idea, especially at large scale, has remained a significant artificial intelligence challenge. Here we present the Video Instance Embedding (VIE) framework, which extends powerful recent unsupervised loss functions for learning deep nonlinear embeddings to multi-stream temporal processing architectures on large-scale video datasets. We show that VIE-trained networks substantially advance the state of the art in unsupervised learning from video datastreams, both for action recognition in the Kinetics dataset, and object recognition in the ImageNet dataset. We show that a hybrid model with both static and dynamic processing pathways is optimal for both transfer tasks, and provide analyses indicating how the pathways differ. Taken in context, our results suggest that deep neural embeddings are a promising approach to unsupervised visual learning across a wide variety of domains.
36
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Automatic differentiation in PyTorch
Adam Paszke, Sam Gross, Soumith Chintala et al. · 2017 · 11.1K citations
HMDB: A large video database for human motion recognition
Hilde Kuehne, Hueihan Jhuang, Estíbaliz Garrote et al. · 2011 · 3.9K citations · Full text