2011 · 234 citations · 21 references
ComputingComputer ScienceComputer VisionHuman BehaviorMachine LearningAction RecognitionAction RepresentationRepresentation LearningScene UnderstandingSequence LearningVideo AnalysisAction DetectionActivitiesActivity RecognitionGraph-based Action RecognitionSpatiotemporal Feature ExtractionSpatiotemporal Feature LearningTemporal RelationsVideo Representation Learning
Complex human activities occurring in videos can be defined in terms of temporal configurations of primitive actions. Prior work typically hand-picks the primitives, their total number, and temporal relations (e.g., allow only followed-by), and then only estimates their relative significance for activity recognition. We advance prior work by learning what activity parts and their spatiotemporal relations should be captured to represent the activity, and how relevant they are for enabling efficient inference in realistic videos. We represent videos by spatiotemporal graphs, where nodes correspond to multiscale video segments, and edges capture their hierarchical, temporal, and spatial relationships. Access to video segments is provided by our new, multiscale segmenter. Given a set of training spatiotemporal graphs, we learn their archetype graph, and pdf's associated with model nodes and edges. The model adaptively learns from data relevant video segments and their relations, addressing the “what” and “how.” Inference and learning are formulated within the same framework - that of a robust, least-squares optimization - which is invariant to arbitrary permutations of nodes in spatiotemporal graphs. The model is used for parsing new videos in terms of detecting and localizing relevant activity parts. We out-perform the state of the art on benchmark Olympic and UT human-interaction datasets, under a favorable complexity-vs.-accuracy trade-off.
21
Learning realistic human actions from movies
Ivan Laptev, Marcin Marszałek, Cordelia Schmid et al. · 2008 · 3.5K citations · Full text
Lena Gorelick, Miri Blank, Eli Shechtman et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2007 · 1.4K citations
Spectral grouping using the nystrom method
Charless C. Fowlkes, Serge Belongie, Fan Chung et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2004 · 1.3K citations · Full text