ComputingComputer ScienceComputer VisionMachine LearningMachine VisionDeep LearningImage ClassificationImage Feature EngineeringMulti-task LearningScene UnderstandingTransfer LearningFeature TransferMulti-task Learning For Computer VisionMulti-task Neural NetworksSurface Normal EstimationTask TransferTransformer ModelsTransformer TheoryVisual Transfer Learning
Multi-task pixel perception is one of the most important topics in the field of machine intelligence. Inspired by the observation of cross-task interdependencies of visual patterns, we propose a multi-task vision pattern transformation (VPT) method to adaptively correlate and transfer cross-task visual patterns by leveraging the powerful transformer mechanism. To better transfer visual patterns, specifically, we build two types of pattern transformation based on the statistic prior that the affinity relations across tasks are correlated. One aims to transfer feature patterns for the integration of different task features; the other aims to exchange structure patterns for mining and leveraging the latent interaction cues. These two types of transformations are encapsulated into two VPT units, which provide universal matching interfaces for multi-task learning, complement each other to guide the transmission of feature/structure patterns, and finally realize an adaptive selection of important patterns across tasks. Extensive experiments on the joint learning of semantic segmentation, depth prediction and surface normal estimation demonstrate that our proposed method is more effective than those baselines and achieve the state-of-that-art performance in three pixel-level visual tasks.
42
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Guosheng Lin, Anton Milan, Chunhua Shen et al. · 2017 · 3.2K citations
CCNet: Criss-Cross Attention for Semantic Segmentation
Zilong Huang, Xinggang Wang, Lichao Huang et al. · 2019 · 2.9K citations