IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023 · 211 citations · 64 references
ComputingComputer ScienceComputer VisionNatural Language ProcessingVisual AttentionDeep LearningGraph-based LearningObject DetectionScene UnderstandingAttentionAttention MechanismsEncoder-decoder ArchitecturesGraph GenerationScene Graph GenerationScene Graph ModelingScene Graph RepresentationScene Graphs
Different objects in the same scene are more or less related to each other, but only a limited number of these relationships are noteworthy. Inspired by Detection Transformer, which excels in object detection, we view scene graph generation as a set prediction problem. In this article, we propose an end-to-end scene graph generation model Relation Transformer (RelTR), which has an encoder-decoder architecture. The encoder reasons about the visual feature context while the decoder infers a fixed-size set of triplets subject-predicate-object using different types of attention mechanisms with coupled subject and object queries. We design a set prediction loss performing the matching between the ground truth and predicted triplets for the end-to-end training. In contrast to most existing scene graph generation methods, RelTR is a one-stage method that predicts sparse scene graphs directly only using visual appearance without combining entities and labeling all possible predicates. Extensive experiments on the Visual Genome, Open Images V6, and VRD datasets demonstrate the superior performance and fast inference of our model.
64
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Decoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter · arXiv (Cornell University) · 2017 · 9.1K citations · Full text
FCOS: Fully Convolutional One-Stage Object Detection
Zhi Tian, Chunhua Shen, Hao Chen et al. · 2019 · 6.1K citations