2018 · 233 citations · 46 references
ComputingComputer ScienceComputer VisionKnowledge RepresentationVisual AttentionDeep LearningGraph-based LearningObject RecognitionRepresentation LearningScene UnderstandingVisual SemanticsAttentionGraph-based ReasoningIterative ReasoningKnowledge Graph ReasoningSemantic RelationshipsSpatial MemorySpatial Visual MemoryVisual Reasoning
We present a novel framework for iterative visual reasoning. Our framework goes beyond current recognition systems that lack the capability to reason beyond stack of convolutions. The framework consists of two core modules: a local module that uses spatial memory [4] to store previous beliefs with parallel updates; and a global graph-reasoning module. Our graph module has three components: a) a knowledge graph where we represent classes as nodes and build edges to encode different types of semantic relationships between them; b) a region graph of the current image where regions in the image are nodes and spatial relationships between these regions are edges; c) an assignment graph that assigns regions to classes. Both the local module and the global module roll-out iteratively and cross-feed predictions to each other to refine estimates. The final predictions are made by combining the best of both modules with an attention mechanism. We show strong performance over plain ConvNets, e.g. achieving an 8.4% absolute improvement on ADE [55] measured by per-class average precision. Analysis also shows that the framework is resilient to missing regions for reasoning.
46
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
ImageNet Large Scale Visual Recognition Challenge
Olga Russakovsky, Jia Deng, Hao Su et al. · International Journal of Computer Vision · 2015 · 40.2K citations
Feature Pyramid Networks for Object Detection
Tsung-Yi Lin, Piotr Dollár, Ross Girshick et al. · 2017 · 28.7K citations
The Pascal Visual Object Classes (VOC) Challenge
Mark Everingham, Luc Van Gool, Christopher K. I. Williams et al. · International Journal of Computer Vision · 2009 · 19.5K citations · Full text