2023 · 49 citations · 22 references
ArtsVision ScienceComputer VisionDigital ImagingMachine VisionDigital Image ProcessingImage ProcessingImage RestorationError PropagationError Propagation AnalysisImage InpaintingInpaintingMask PredictionMasked Image Modeling
A novel continuous-mask-aware transformer for image inpainting, called CMT, is proposed in this paper, which uses a continuous mask to represent the amounts of errors in tokens. First, we initialize a mask and use it during the self-attention. To facilitate the masked self-attention, we also introduce the notion of overlapping tokens. Second, we update the mask by modeling the error propagation during the masked self-attention. Through several masked self-attention and mask update (MSAU) layers, we predict initial inpainting results. Finally, we refine the initial results to reconstruct a more faithful image. Experimental results on multiple datasets show that the proposed CMT algorithm outperforms existing algorithms significantly. The source codes are available at https://github.com/keunsoo-ko/CMT.
22
Generative Image Inpainting with Contextual Attention
Jiahui Yu, Zhe Lin, Xiaohui Shen et al. · 2018 · 2.5K citations
Describing Textures in the Wild
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos et al. · 2014 · 2.2K citations · Full text