Fusion of multi-sensor imagery for night vision: color visualization, target learning and search

Abstract

We present methods and results for fusion of imagery from multiple sensors to create a color night vision capability. The fusion system architectures are based on biological models of the spatial and opponent-color processes in the human retina and visual cortex, implemented as shunting center-surround feed-forward neural networks. Real-time implementation of the dual-sensor fusion system combines imagery from either a low-light CCD camera or a short-wave infrared camera, with thermal long-wave infrared imagery. Results are also shown for extensions of this fusion architecture to include imagery from all three of these sensors, Visible/SWIR/LWIR, as well as a four sensor system using Visible/SWIR/MWIR/LWIR cameras. We also demonstrate how results from these multi-sensor fusion systems are used as inputs to an interactive tool for target designation, learning, and search based on a fuzzy ARTMAP neural network.