Parallel perception of visual motion using light-tunable memory matrix

TLDR

Parallel perception of visual motion is crucial for intelligent machine vision, yet conventional CMOS sensors cannot simultaneously encode temporal and spatial motion information. We aim to demonstrate in‑sensor parallel perception of diverse motion modes using a light‑tunable memory matrix in a van der Waals heterostructure array. The array exploits gate‑tunable photoconductivity and light‑tunable memory to encode and process spatiotemporal light patterns simultaneously. The resulting visual motion perceptron can decode direction, velocity, acceleration, and angular velocity in parallel, paving the way for in‑sensor machine vision.

Abstract

Parallel perception of visual motion is of crucial significance to the development of an intelligent machine vision system. However, implementing in-sensor parallel visual motion perception using conventional complementary metal-oxide semiconductor technology is challenging, because the temporal and spatial information embedded in motion cannot be simultaneously encoded and perceived at the sensory level. Here, we demonstrate the parallel perception of diverse motion modes at the sensor level by exploiting light-tunable memory matrix in a van der Waals (vdW) heterostructure array. The optoelectronic characteristics of gate-tunable photoconductivity and light-tunable memory matrix enable devices in the array to realize simultaneous encoding and processing of incoming spatiotemporal light pattern. Furthermore, we implement a visual motion perceptron with the array capable of deciphering multiple motion parameters in parallel, including direction, velocity, acceleration, and angular velocity. Our work opens up a promising venue for the realization of an intelligent machine vision system based on in-sensor motion perception.