Spectral reflectance recovery using optimal illuminations

Abstract

The spectral reflectance of objects provides intrinsic information on material properties that have been proven beneficial in a diverse range of applications, e.g., remote sensing, agriculture and diagnostic medicine, to name a few. Existing methods for the spectral reflectance recovery from RGB or monochromatic images either ignore the effect from the illumination or implement/optimize the illumination under the linear representation assumption of the spectral reflectance. In this paper, we present a simple and efficient convolutional neural network (CNN)-based spectral reflectance recovery method with optimal illuminations. Specifically, we design illumination optimization layer to optimally multiplex illumination spectra in a given dataset or to design the optimal one under physical restrictions. Meanwhile, we develop the nonlinear representation for spectral reflectance in a data-driven way and jointly optimize illuminations under this representation in a CNN-based end-to-end architecture. Experimental results on both synthetic and real data show that our method outperforms the state-of-the-arts and verifies the advantages of deeply optimal illumination and nonlinear representation of the spectral reflectance.