arXiv (Cornell University) · 2019 · 84 citations · 0 references
ComputingComputer ScienceComputer VisionMachine LearningNatural Language ProcessingAction RecognitionAttention NetworksDeep LearningMeta-learningMetric LearningRepresentation LearningAttentionAttention MechanismsAttention-based Action RecognitionFew-frame Action RecognitionFew-shot Action RecognitionFew-shot LearningTemporal AlignmentZero-shot Action Recognition
In this paper we propose a novel Temporal Attentive Relation Network (TARN)\nfor the problems of few-shot and zero-shot action recognition. At the heart of\nour network is a meta-learning approach that learns to compare representations\nof variable temporal length, that is, either two videos of different length (in\nthe case of few-shot action recognition) or a video and a semantic\nrepresentation such as word vector (in the case of zero-shot action\nrecognition). By contrast to other works in few-shot and zero-shot action\nrecognition, we a) utilise attention mechanisms so as to perform temporal\nalignment, and b) learn a deep-distance measure on the aligned representations\nat video segment level. We adopt an episode-based training scheme and train our\nnetwork in an end-to-end manner. The proposed method does not require any\nfine-tuning in the target domain or maintaining additional representations as\nis the case of memory networks. Experimental results show that the proposed\narchitecture outperforms the state of the art in few-shot action recognition,\nand achieves competitive results in zero-shot action recognition.\n