FF-Former: Swin Fourier Transformer for Nighttime Flare Removal

Abstract

In the process of removing nighttime flare, it is crucial to have a large receptive field due to the fact that flare can occupy a substantial portion of an image, even potentially the entire image. However, the conventional window-based Transformer approaches restrict the receptive field within the window, limiting its ability to capture global features. And the flare can cause the dark regions to become brighter and result in a loss of contrast and alteration of the frequency characteristics of the image. To address these challenges, we introduce FF-Former, which is based on Fast Fourier Convolution (FFC) and is designed to extract global frequency features for enhancing nighttime flare removal. To achieve this, we incorporate a Spatial Frequency Block (SFB) after the Swin Transformer, which forms the Swin Fourier Transformer Block (SFTB). This configuration enables the establishment of long dependencies and the extraction of global features. Unlike the traditional Transformer, which relies on global self-attention, the SFB module only performs convolution computation, making it both effective and efficient. Additionally, during the training phase, we optimize the loss function to preserve the light source points after nighttime flare removal. Experimental results on both real-world and synthetic benchmarks demonstrate that the proposed FF-Former significantly improves the performance of nighttime flare removal.