ADMM-ADAM: A New Inverse Imaging Framework Blending the Advantages of Convex Optimization and Deep Learning

Abstract

Alternating direction method of multipliers (ADMM) and adaptive moment estimation (ADAM) are two optimizers of paramount importance in convex optimization (CO) and deep learning (DL), respectively. Numerous state-of-the-art algorithms for solving inverse problems are achieved by carefully designing a convex criterion, typically composed of a data-fitting term and a regularizer. Even when the regularizer is convex, its mathematical form is often sophisticated, hence inducing a math-heavy optimization procedure and making the algorithm design a daunting task for software engineers. Probably for this reason, people turn to solve the inverse problems via DL, but this requires big data collection, quite time-consuming if not impossible. Motivated by these facts, we propose a new framework, termed as ADMM-ADAM, for solving inverse problems. As the key contribution, even just with small/single data, the proposed ADMM-ADAM is able to exploit DL to obtain a convex regularizer of very simple math form, followed by solving the regularized criterion using simple CO algorithm. As a side contribution, a state-of-the-art hyperspectral inpainting algorithm is designed under ADMM-ADAM, demonstrating its superiority even without the aid of big data or sophisticated mathematical regularization.