Vertically integrated spiking cone photoreceptor arrays for color perception

TLDR

Cone photoreceptors convert natural light into spiking signals, enabling energy‑efficient color vision, but creating a device that mimics this color‑selective, spike‑encoding function is challenging. The study proposes a metal‑oxide based vertically integrated spiking cone photoreceptor array that directly transduces persistent light into wavelength‑dependent spike trains. The array employs three wavelengths as pseudo‑primary colors to generate spike‑based images for recognition, achieving improved color discrimination accuracy. The device consumes less than 400 pW per spike, close to biological cones, and its ability to discriminate mixed colors yields higher recognition accuracy, enabling hardware spiking neural networks for biologically plausible visual perception and dynamic vision sensors.

Abstract

The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form 'colorful' images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.