Toward semantic retrieval of pictures and video

Abstract

Existing techniques for content-based retrieval of pictures and video are based either on physical characteristics of the data like color and texture, or on a representation of the objects and actions depicted by the data. But the fact that there are many equivalent representations of a given picture or video clip poses a significant problem for object-based techniques in matching users' queries to the represented collection. This paper proposes a possible solution to this problem through the use of semantic knowledge from multiple large-scale sources. Several rules are presented to illustrate the proper incorporation of semantic relations into the process of matching queries to object-based representations of pictures and video.