FDA: Fourier Domain Adaptation for Semantic Segmentation

TLDR

Current state‑of‑the‑art domain adaptation methods for semantic segmentation are complex, often relying on adversarial training to make neural backbones invariant to domain labels. This work proposes a simple unsupervised domain adaptation technique that reduces source–target distribution discrepancy by swapping low‑frequency Fourier spectra between domains. The technique is applied to semantic segmentation by exchanging low‑frequency components of synthetic and real images using a Fourier transform and its inverse, without any training. When integrated into a standard segmentation model, the method achieves state‑of‑the‑art benchmark performance and shows that simple Fourier‑based alignment can effectively suppress nuisance variability that more sophisticated methods struggle to eliminate.

Abstract

We describe a simple method for unsupervised domain adaptation, whereby the discrepancy between the source and target distributions is reduced by swapping the low-frequency spectrum of one with the other. We illustrate the method in semantic segmentation, where densely annotated images are aplenty in one domain (synthetic data), but difficult to obtain in another (real images). Current state-of-the-art methods are complex, some requiring adversarial optimization to render the backbone of a neural network invariant to the discrete domain selection variable. Our method does not require any training to perform the domain alignment, just a simple Fourier Transform and its inverse. Despite its simplicity, it achieves state-of-the-art performance in the current benchmarks, when integrated into a relatively standard semantic segmentation model. Our results indicate that even simple procedures can discount nuisance variability in the data that more sophisticated methods struggle to learn away.