Single-pixel imaging using physics enhanced deep learning

TLDR

Single‑pixel imaging reconstructs 2‑D/3‑D images from a one‑dimensional bucket signal, is valuable for low‑light and hard‑to‑camera spectral regimes, but its resolution is limited by the number of temporal measurements. The authors aim to develop a physics‑enhanced deep‑learning framework for single‑pixel imaging. They integrate a physics‑informed layer with a model‑driven fine‑tuning process to reconstruct undersampled bucket signals, and implement this in both an in‑house SPI system and an outdoor LiDAR, achieving higher robustness and fidelity than existing algorithms. The method generalizes across platforms, outperforms conventional SPI algorithms, and bridges data‑driven and model‑driven approaches, enabling the use of both data and physics priors for inverse problems in computational imaging from remote sensing to microscopy.

Abstract

Single-pixel imaging (SPI) is a typical computational imaging modality that allows two- and three-dimensional image reconstruction from a one-dimensional bucket signal acquired under structured illumination. It is in particular of interest for imaging under low light conditions and in spectral regions where good cameras are unavailable. However, the resolution of the reconstructed image in SPI is strongly dependent on the number of measurements in the temporal domain. Data-driven deep learning has been proposed for high-quality image reconstruction from a undersampled bucket signal. But the generalization issue prohibits its practical application. Here we propose a physics-enhanced deep learning approach for SPI. By blending a physics-informed layer and a model-driven fine-tuning process, we show that the proposed approach is generalizable for image reconstruction. We implement the proposed method in an in-house SPI system and an outdoor single-pixel LiDAR system, and demonstrate that it outperforms some other widespread SPI algorithms in terms of both robustness and fidelity. The proposed method establishes a bridge between data-driven and model-driven algorithms, allowing one to impose both data and physics priors for inverse problem solvers in computational imaging, ranging from remote sensing to microscopy.