Towards intelligent recognition of multimedia episodes in real-time applications

Abstract

The ability to automatically capture and index multimedia information for later perusal and review is critical to the success of future multimedia services. In this paper, we describe how to automatically generate indexes of real-time streams without requiring deep content analysis. Our techniques involve segmenting continuous audio and video into natural units, and relating these to discrete events from the multimedia application, such as user interactions, control events, and data content. In addition, we describe how to search within multimedia streams using query-based retrieval and visual and auditory retrieval modes. This multimodal retrieval allows for quick browsing and visual comprehension of multimedia streams. Finally we show how our techniques apply to the area of multimedia conference recording.