Intermediate Layer Optimization for Inverse Problems using Deep\n Generative Models

Abstract

We propose Intermediate Layer Optimization (ILO), a novel optimization\nalgorithm for solving inverse problems with deep generative models. Instead of\noptimizing only over the initial latent code, we progressively change the input\nlayer obtaining successively more expressive generators. To explore the higher\ndimensional spaces, our method searches for latent codes that lie within a\nsmall $l_1$ ball around the manifold induced by the previous layer. Our\ntheoretical analysis shows that by keeping the radius of the ball relatively\nsmall, we can improve the established error bound for compressed sensing with\ndeep generative models. We empirically show that our approach outperforms\nstate-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of\ninverse problems including inpainting, denoising, super-resolution and\ncompressed sensing.\n