A Deep Motion Deblurring Network Based on Per-Pixel Adaptive Kernels With Residual Down-Up and Up-Down Modules

Abstract

Due to the object motion during the camera exposure time, latent pixel information appears scattered in a blurred image. A large dataset of dynamic motion blur and blur-free frame pairs enables deep neural networks to learn deblurring operations directly in end-to-end manners. In this paper, we propose a novel motion deblurring kernel learning network that predicts the per-pixel deblur kernel and a residual image. The learned deblur kernel filters and linearly combines neighboring pixels to restore the clean pixels in its corresponding location. The per-pixel adaptive convolution with the learned deblur kernel can effectively handle non-uniform blur. At the same time, the generated residual image is added to the adaptive convolution result to compensate for the limited receptive field of the learned deblur kernel. That is, the adaptive convolution and the residual image play different but complementary roles each other to reconstruct the latent clean images in a collaborative manner. We also propose residual down-up (RDU) and residual up-down (RUD) blocks that help improve the motion deblurring performance. The RDU and RUD blocks are designed to adjust the spatial size and the number of channels of the intermediate feature within the blocks. We demonstrate the effectiveness of our motion deblurring kernel learning network by showing intensive experimental results compared to those of the state-of-the-art methods.