Physics-Based Generative Adversarial Models for Image Restoration and Beyond

TLDR

Image restoration tasks such as deblurring, dehazing, and deraining are ill‑posed and traditionally rely on heuristic priors, yet simple GAN formulations often fail to preserve image structure. The authors aim to solve these tasks by guiding GAN‑based generative models with physics‑consistent constraints. They develop a physics‑guided GAN that trains end‑to‑end to enforce consistency between estimated images and observed inputs, applicable to many low‑level vision tasks. Experiments show the method outperforms state‑of‑the‑art baselines on various restoration benchmarks.

Abstract

We present an algorithm to directly solve numerous image restoration problems (e.g., image deblurring, image dehazing, and image deraining). These problems are ill-posed, and the common assumptions for existing methods are usually based on heuristic image priors. In this paper, we show that these problems can be solved by generative models with adversarial learning. However, a straightforward formulation based on a straightforward generative adversarial network (GAN) does not perform well in these tasks, and some structures of the estimated images are usually not preserved well. Motivated by an interesting observation that the estimated results should be consistent with the observed inputs under the physics models, we propose an algorithm that guides the estimation process of a specific task within the GAN framework. The proposed model is trained in an end-to-end fashion and can be applied to a variety of image restoration and low-level vision problems. Extensive experiments demonstrate that the proposed method performs favorably against state-of-the-art algorithms.