Highly Efficient Multimedia Event Recounting from User Semantic Preferences

Abstract

We present the design of a video event recounting system that takes YouTube-like videos, and identifies a minimal set of video segments and textual keyword descriptions in order to convince a user, in a time efficient manner, that the video contains an instance of a user-specificed human activity. The system is based on extensive user studies that have lead to nine design principles about human preferences and limits in semantic understanding. The processing pipeline locates the presence of user query keywords within the video, segments the video according to a model of human short-term memory for semantic similarities, selects those segments that best contain query terms, and abbreviates both the video and textual presentation. Speed-ups of a factor of 6 over simple video viewing time are achievable, without loss of semantic accuracy. In the 2013 Trecvid Multimedia Event Recounting competition, this system placed first in time efficiency, while remaining above average in description accuracy.