2017 · 62 citations · 39 references
ComputingComputer ScienceComputer VisionMachine LearningMachine VisionDeep LearningDepth EstimationNetwork ModularityScene UnderstandingSemantic SegmentationConvolutional Neural NetworksDeep Learning FusionDepth Map RefinementEncoder-decoder ArchitecturesModular ArchitectureRefinementSemantic LabelingSemantic Prediction
This paper addresses the task of designing a modular neural network architecture that jointly solves different tasks. As an example we use the tasks of depth estimation and semantic segmentation given a single RGB image. The main focus of this work is to analyze the cross-modality influence between depth and semantic prediction maps on their joint refinement. While most of the previous works solely focus on measuring improvements in accuracy, we propose a way to quantify the cross-modality influence. We show that there is a relationship between final accuracy and cross-modality influence, although not a simple linear one. Hence a larger cross-modality influence does not necessarily translate into an improved accuracy. We find that a beneficial balance between the cross-modality influences can be achieved by network architecture and conjecture that this relationship can be utilized to understand different network design choices. Towards this end we propose a Convolutional Neural Network (CNN) architecture that fuses the state-of-the-art results for depth estimation and semantic labeling. By balancing the cross-modality influences between depth and semantic prediction, we achieve improved results for both tasks using the NYU-Depth v2 benchmark.
39
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Ross Girshick, Jeff Donahue, Trevor Darrell et al. · 2014 · 31.8K citations
Yangqing Jia, Evan Shelhamer, Jeff Donahue et al. · 2014 · 11.2K citations
RELATIONS BETWEEN TWO SETS OF VARIATES
Harold Hotelling · Biometrika · 1936 · 5.4K citations