Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 56 citations · 23 references
ComputingComputer ScienceComputer VisionRobot VisionVisual Attention3D ReconstructionDepth EstimationRobotic VisionScene ReconstructionScene UnderstandingSemantic Segmentation3D Scene Understanding3D Semantic SegmentationAttentionAttention-based FusionDepth CuesModal FusionScene CompletionSemantic Features
This paper presents an end-to-end 3D convolutional network named attention-based multi-modal fusion network (AMFNet) for the semantic scene completion (SSC) task of inferring the occupancy and semantic labels of a volumetric 3D scene from single-view RGB-D images. Compared with previous methods which use only the semantic features extracted from RGB-D images, the proposed AMFNet learns to perform effective 3D scene completion and semantic segmentation simultaneously via leveraging the experience of inferring 2D semantic segmentation from RGB-D images as well as the reliable depth cues in spatial dimension. It is achieved by employing a multi-modal fusion architecture boosted from 2D semantic segmentation and a 3D semantic completion network empowered by residual attention blocks. We validate our method on both the synthetic SUNCG-RGBD dataset and the real NYUv2 dataset and the results show that our method respectively achieves the gains of 2.5% and 2.6% on the synthetic SUNCG-RGBD dataset and the real NYUv2 dataset against the state-of-the-art method.
23
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2017 · 22.1K citations
Residual Attention Network for Image Classification
Fei Wang, Mengqing Jiang, Chen Qian et al. · 2017 · 3.8K citations
Squeeze-and-Excitation Networks
Jie Hu, Li Shen, Samuel Albanie et al. · arXiv (Cornell University) · 2017 · 2.2K citations · Full text