Optoelectronic Devices for In‐Sensor Computing

TLDR

The growing need for accurate perception has increased the number of sensory nodes, but transmitting massive, unstructured data to centralized units is power‑inefficient and bandwidth‑heavy, prompting in‑sensor computing that integrates perception, memory, and processing to compress and structure data at the source. The study aims to enable efficient processing of massive sensory data by compressing and structuring it directly at the sensor terminals. Using vision sensors as a case study, the authors examine electronic, optical, and optoelectronic hardware that employs volatile and nonvolatile resistive switching to compress and structure multidimensional visual data within the sensor. The authors conclude by outlining future directions for optoelectronic devices in in‑sensor computing.

Abstract

Abstract The demand for accurate perception of the physical world leads to a dramatic increase in sensory nodes. However, the transmission of massive and unstructured sensory data from sensors to computing units poses great challenges in terms of power‐efficiency, transmission bandwidth, data storage, time latency, and security. To efficiently process massive sensory data, it is crucial to achieve data compression and structuring at the sensory terminals. In‐sensor computing integrates perception, memory, and processing functions within sensors, enabling sensory terminals to perform data compression and data structuring. Here, vision sensors are adopted as an example and discuss the functions of electronic, optical, and optoelectronic hardware for visual processing. Particularly, hardware implementations of optoelectronic devices for in‐sensor visual processing that can compress and structure multidimensional vision information are examined. The underlying resistive switching mechanisms of volatile/nonvolatile optoelectronic devices and their processing operations are explored. Finally, a perspective on the future development of optoelectronic devices for in‐sensor computing is provided.