Image Denoising Using A Generative Adversarial Network

TLDR

Path tracing renders 3D scenes by shooting thousands of Monte Carlo rays per pixel and averaging their colors, a process that can take 8–16 hours per frame due to its computational complexity. The study aims to reduce rendering time to under one second by training a generative adversarial network to denoise low‑sample images. The approach first renders images with only 1–8 samples per pixel, then feeds the resulting noisy images into a GAN that generates high‑quality photorealistic outputs.

Abstract

Animation studios render 3D scenes using a technique called path tracing which enables them to create high quality photorealistic frames. Path tracing involves shooting 1000’s of rays into a pixel randomly (Monte Carlo) which will then hit the objects in the scene and, based on the reflective property of the object, these rays reflect or refract or get absorbed. The colors returned by these rays are averaged to determine the color of the pixel. This process is repeated for all the pixels. Due to the computational complexity it might take 8–16 hours to render a single frame. We implemented a neural network-based solution to reduce the time it takes to render a frame to less than a second using a generative adversarial network (GAN), once the network is trained. The main idea behind this proposed method is to render the image using a much smaller number of samples per pixel than is normal for path tracing (e.g., 1, 4, or 8 samples instead of, say, 32,000 samples) and then pass the noisy, incompletely rendered image to our network, which is capable of generating a high-quality photorealistic image.