Structure-Aware Deep Networks and Pixel-Level Generative Adversarial Training for Single Image Super-Resolution

Abstract

The resolution of current display devices is getting higher and higher, and 4K/8K display devices have become popular, which require image super-resolution technologies to enlarge and restore the input low-resolution images into high-resolution ones. In addition, image super-resolution technologies can facilitate various vision-based measurement applications. Recently, super-resolution methods based on generative adversarial networks (GANs) have become mainstream. However, some recent studies have shown that GAN-based image super-resolution methods will cause structural distortion. Existing methods alleviate the problem of structural distortion by enhancing structure generation. But this type of method cannot essentially solve the problem of structural distortion caused by adversarial training. In contrast, this article proposes pixel-level generative adversarial training to solve the structural distortion problem, which finely constrains the structure of images during the adversarial training process. In addition, to better generate the structure and details of images and make full use of similar texture details within images, we build a structure-aware image super-resolution network, which not only enhances the structure generation through gradient guidance but also effectively integrates non-local self-similarity modules in a multi-level manner. Experimental results show that the proposed method achieves better quantitative and qualitative results than the state-of-the-art methods. The ablation experiments show that the proposed structure-aware deep network and pixel-level adversarial training can improve the performance of image super-resolution.