C3D: Generic Features for Video Analysis.

Abstract

Videos have become ubiquitous due to the ease of capturing and sharing via social platforms like Youtube, Facebook, Instagram, and others. The computer vision community has tried to tackle various video analysis problems independently. As a consequence, even though some really good hand-crafted features have been proposed there is a lack of generic feature for video analysis. On the other hand, the image domain has progressed rapidly by using features from deep convolutional networks. These deep features are proving to be generic and perform well on variety of image tasks. In this work we propose Convolution 3D(C3D) feature, a generic spatio-temporal feature obtained by training a deep 3-dimensional convolutional network on a large annotated video dataset comprising objects, scenes, actions, and other frequently occurring concepts. We show that by using spatio-temporal convolutions the trained features encapsulate appearance and motion cues and perform well on different discriminative video classification tasks. C3D has three main advantages. First, it is generic: achieving state of the art results on object recognition, scene classification, and action similarity labeling in videos. Second, it is compact: obtaining better accuracies than best hand-crafted features and best deep image features with a lower dimensional feature descriptor. Third, it is efficient to compute: 91 times faster than current hand-crafted features, and two orders of magnitude faster than current deep-learning based video classification methods.