Proceedings of the AAAI Conference on Artificial Intelligence · 2017 · 41 citations · 18 references
ComputingComputer ScienceComputer VisionMachine LearningTime Series AnalysisAction RecognitionAttention NetworksDeep LearningTemporal AttentionTemporal StructureVideo AnalysisActivitiesActivity RecognitionHuman Activity RecognitionLong Short-term Memory NetworksSequence Representation LearningSpatiotemporal Feature ExtractionTemporal Convolutional NetworksTemporal Structure Analysis
In this paper, we newly introduce the concept of temporal attention filters, and describe how they can be used for human activity recognition from videos. Many high-level activities are often composed of multiple temporal parts (e.g., sub-events) with different duration/speed, and our objective is to make the model explicitly learn such temporal structure using multiple attention filters and benefit from them. Our temporal filters are designed to be fully differentiable, allowing end-of-end training of the temporal filters together with the underlying frame-based or segment-based convolutional neural network architectures. This paper presents an approach of learning a set of optimal static temporal attention filters to be shared across different videos, and extends this approach to dynamically adjust attention filters per testing video using recurrent long short-term memory networks (LSTMs). This allows our temporal attention filters to learn latent sub-events specific to each activity. We experimentally confirm that the proposed concept of temporal attention filters benefits the activity recognition, and we visualize the learned latent sub-events.
18
Learning Spatiotemporal Features with 3D Convolutional Networks
Du Tran, Lubomir Bourdev, Rob Fergus et al. · 2015 · 9.7K citations
Large-Scale Video Classification with Convolutional Neural Networks
Andrej Karpathy, George Toderici, Sanketh Shetty et al. · 2014 · 6.3K citations
HMDB: A large video database for human motion recognition
Hilde Kuehne, Hueihan Jhuang, Estíbaliz Garrote et al. · 2011 · 3.9K citations · Full text