2024 · 85 citations · 29 references
ComputingComputer ScienceComputer VisionMachine LearningPaintingDeep LearningGenerative ModelingDiffusion ModelsImage InpaintingInpaintingLatent Space ExplorationLatent Space InterpretationState-of-the-art ReviewTraining
Image inpainting using diffusion models is generally done using either preconditioned models, i.e. image conditioned models fine-tuned for the painting task, or postconditioned models, i.e. unconditioned models repurposed for the painting task at inference time. Preconditioned models are fast at inference time but extremely costly to train. Postconditioned models do not require any training but are slow during inference, requiring multiple forward and backward passes to converge to a desirable solution. Here, we derive an approach that does not require expensive training, yet is fast at inference time. To solve the costly inference computational time, we perform the forward-backward fusion step on a latent space rather than the image space. This is solved with a newly proposed propagation module in the diffusion process. Experiments on a number of domains demonstrate our approach attains or improves state-of-the-art results with the advantages of preconditioned and postconditioned models and none of their disadvantages.
29
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang, Phillip Isola, Alexei A. Efros et al. · 2018 · 12.4K citations
Marcelo Bertalmı́o, Guillermo Sapiro, V. Caselles et al. · 2000 · 3.6K citations · Full text
Generative Image Inpainting with Contextual Attention
Jiahui Yu, Zhe Lin, Xiaohui Shen et al. · 2018 · 2.5K citations