Video Skimming for Quick Browsing based on Audio and Image Characterization

TLDR

Digital video is increasingly vital for information and entertainment, yet the rapid growth of collections to thousands of hours demands efficient browsing tools that preserve content. The study proposes a method to extract key audio and video cues and produce a concise skim video summarizing the original content. The method extracts salient objects, audio keywords, and video structure using integrated image and language understanding techniques. The skim video is substantially smaller while preserving the essential content of the original segment. This research was funded by the National Science Foundation (grant IRI9411299), the National Space and Aeronautics Administration, and the Advanced Research Projects Agency, with the authors’ views not representing official U.S.

Abstract

Digital video is rapidly becoming an important source for information, entertainment and a host of multimedia applications. With the size of these collections growing to thousands of hours, technology is needed to effectively browse segments in a short time without losing the content of the video. We propose a method to extract the significant audio and video information and create a “skim” video which represents a short synopsis of the original. The extraction of significant information, such as specific objects, audio keywords and relevant video structure, is made possible through the integration of techniques in image and language understanding. The resulting skim is much smaller, and retains the essential content of the original segment. This research is sponsored by the National Science Foundation under grant no. IRI9411299, the National Space and Aeronautics Administration, and the Advanced Research Projects Agency. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing official policies or endorsements, either expressed or implied, of the United States Government.