2019 · 110 citations · 30 references
ComputingAutonomous SystemsComputer ScienceAutonomous VehiclesComputer VisionMachine LearningAction RecognitionAction RepresentationReinforcement LearningSequence LearningTemporal AttentionTime Series PredictionHuman ActionSpatiotemporal Feature LearningTemporal AnticipationTemporal Prediction
The goal of human action anticipation is to predict future actions. Ideally, in real-world applications such as video surveillance and self-driving systems, future actions should not only be predicted with high accuracy but also at arbitrary and variable time-horizons ranging from short- to long-term predictions. Current work mostly focuses on predicting the next action and thus long-term prediction is achieved by recursive prediction of each next action, which is both inefficient and accumulates errors. In this paper, we propose a novel time-conditioned method for efficient and effective long-term action anticipation. There are two key ingredients to our approach. First, by explicitly conditioning our anticipation network on time allows to efficiently anticipate also long-term actions. And second, we propose an attended temporal feature and a time-conditioned skip connection to extract relevant and useful information from observations for effective anticipation. We conduct extensive experiments on the large-scale Epic-Kitchen and the 50Salads Datasets. Experimental results show that the proposed method is capable of anticipating future actions at both short-term and long-term, and achieves state-of-the-art performance.
30
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Kelvin Xu, Jimmy Ba, Ryan Kiros et al. · arXiv (Cornell University) · 2015 · 7.5K citations · Full text
Long-term recurrent convolutional networks for visual recognition and description
Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama et al. · 2015 · 5.3K citations
Temporal Convolutional Networks for Action Segmentation and Detection
Colin Lea, M. D. Flynn, René Vidal et al. · 2017 · 2K citations