Distributional semantics from text and images

Abstract

We present a distributional semantic model combining text- and image-based features. We evaluate this multimodal semantic model on simulating similarity judgments, concept clus-tering and the BLESS benchmark. When inte-grated with the same core text-based model, image-based features are at least as good as further text-based features, and they capture different qualitative aspects of the tasks, sug-gesting that the two sources of information are complementary. 1