IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024 · 11 citations · 54 references
ComputingComputer ScienceComputer VisionMachine VisionRemote SensingScene UnderstandingSemantic SegmentationDomain AdaptationDomain Adaptation For Semantic SegmentationDomain Knowledge ExtractionEquirectangular ProjectionField Of ViewPanoramic ImagingProjectionsUnsupervised Semantic Segmentation
In this paper, we address the challenging source-free unsupervised domain adaptation (SFUDA) for pinhole-to-panoramic semantic segmentation, given only a pinhole image pre-trained model (i.e., source) and unlabeled panoramic images (i.e., target). Tackling this problem is non-trivial due to three critical challenges: 1) semantic mismatches from the distinct Field-of-View (FoV) between domains, 2) style discrepancies inherent in the UDA problem, and 3) inevitable distortion of the panoramic images. To tackle these problems, we propose 360SFUDA++ that effectively extracts knowledge from the source pinhole model with only unlabeled panoramic images and transfers the reliable knowledge to the target panoramic domain. Specifically, we first utilize Tangent Projection (TP) as it has less distortion and meanwhile slits the equirectangular projection (ERP) to patches with fixed FoV projection (FFP) to mimic the pinhole images. Both projections are shown effective in extracting knowledge from the source model. However, as the distinct projections make it less possible to directly transfer knowledge between domains, we then propose Reliable Panoramic Prototype Adaptation Module (RPAM) to transfer knowledge at both prediction and prototype levels. RPAM selects the confident knowledge and integrates panoramic prototypes for reliable knowledge adaptation. Moreover, we introduce Cross-projection Dual Attention Module (CDAM), which better aligns the spatial and channel characteristics across projections at the feature level between domains. Both knowledge extraction and transfer processes are synchronously updated to reach the best performance. Extensive experiments on the synthetic and real-world benchmarks, including outdoor and indoor scenarios, demonstrate that our 360SFUDA++ achieves significantly better performance than prior SFUDA methods.
54
The Cityscapes Dataset for Semantic Urban Scene Understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos et al. · 2016 · 11.9K citations
Alexander M. Kirillov, Eric Mintun, Nikhila Ravi et al. · 2023 · 9.1K citations
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
Judy Hoffman, Dequan Wang, Fisher Yu et al. · arXiv (Cornell University) · 2016 · 695 citations · Full text