Single image super-resolution from transformed self-exemplars

TLDR

Self‑similarity based super‑resolution relies on internal patch dictionaries, but these may lack sufficient expressiveness to capture texture variations within a single image. This work extends self‑similarity SR to address the limited expressiveness of internal dictionaries. The authors enlarge the patch search space by exploiting geometric cues—local plane detection, perspective geometry, and affine transformations—within a compositional model, and evaluate the method on urban and natural scenes. The proposed approach surpasses state‑of‑the‑art SR algorithms on urban images while matching their performance on natural scenes, all without external training data.

Abstract

Self-similarity based super-resolution (SR) algorithms are able to produce visually pleasing results without extensive training on external databases. Such algorithms exploit the statistical prior that patches in a natural image tend to recur within and across scales of the same image. However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene. In this paper, we extend self-similarity based SR to overcome this drawback. We expand the internal patch search space by allowing geometric variations. We do so by explicitly localizing planes in the scene and using the detected perspective geometry to guide the patch search process. We also incorporate additional affine transformations to accommodate local shape variations. We propose a compositional model to simultaneously handle both types of transformations. We extensively evaluate the performance in both urban and natural scenes. Even without using any external training databases, we achieve significantly superior results on urban scenes, while maintaining comparable performance on natural scenes as other state-of-the-art SR algorithms.