The fusion of audio-visual features and external knowledge for event detection in team sports video

Abstract

Most existing systems detect events in broadcast team sports video using only internal audio-visual (AV) features with limited success. We found that there are many widely available external knowledge sources - such as match reports and real-time game logs in newspapers and on the Web - that can help in detecting events. This paper proposes a scalable framework that utilizes both internal AV features and external knowledge sources to detect events and identify their boundaries in full-length match videos. Besides detecting events, the framework has the potential of discovering detailed semantics and performing question answering regarding these semantics. We demonstrate the effectiveness of the framework using three full-length soccer matches