VQA: Visual Question Answering

TLDR

Visual Question Answering (VQA) involves open‑ended questions and answers that mirror real‑world scenarios such as assisting the visually impaired, requiring detailed image understanding and complex reasoning beyond generic captions. The authors propose the task of free‑form, open‑ended Visual Question Answering (VQA). They define VQA as answering a natural‑language question about an image, targeting specific image regions, and provide a large dataset (~0.25 M images, ~0.76 M questions, ~10 M answers) with baselines and human comparisons, noting that many answers are short or from a closed set enabling automatic evaluation. The paper releases this dataset, offers multiple baseline methods compared to human performance, and provides a CloudCV demo for public use.

Abstract

We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and answers are open-ended. Visual questions selectively target different areas of an image, including background details and underlying context. As a result, a system that succeeds at VQA typically needs a more detailed understanding of the image and complex reasoning than a system producing generic image captions. Moreover, VQA is amenable to automatic evaluation, since many open-ended answers contain only a few words or a closed set of answers that can be provided in a multiple-choice format. We provide a dataset containing ~0.25M images, ~0.76M questions, and ~10M answers (this http URL), and discuss the information it provides. Numerous baselines and methods for VQA are provided and compared with human performance. Our VQA demo is available on CloudCV (this http URL).