Learned Integrated Sensing Pipeline: Reconfigurable Metasurface Transceivers as Trainable Physical Layer in an Artificial Neural Network

TLDR

Intelligent systems rely on low‑latency sensors, yet existing sensing systems ignore a priori knowledge at the hardware level, limiting task‑relevant information extraction. The study proposes a learned integrated sensing pipeline (LISP) that jointly learns optimal measurement strategies and processing algorithms by incorporating task, scene, and measurement constraints. LISP integrates a reconfigurable metasurface transceiver with a neural network, enabling end‑to‑end training of programmable microwave patterns and downstream processing. Numerical experiments show that LISP yields about 15% higher object‑recognition accuracy with few measurements, and the learned microwave patterns are nonintuitive, highlighting the paradigm’s value.

Abstract

Abstract The rapid proliferation of intelligent systems (e.g., fully autonomous vehicles) in today's society relies on sensors with low latency and computational effort. Yet current sensing systems ignore most available a priori knowledge, notably in the design of the hardware level, such that they fail to extract as much task‐relevant information per measurement as possible. Here, a “learned integrated sensing pipeline” (LISP), including in an end‐to‐end fashion both physical and processing layers, is shown to enable joint learning of optimal measurement strategies and a matching processing algorithm, making use of a priori knowledge on task, scene, and measurement constraints. Numerical results demonstrate accuracy improvements around 15% for object recognition tasks with limited numbers of measurements, using dynamic metasurface apertures capable of transceiving programmable microwave patterns. Moreover, it is concluded that the optimal learned microwave patterns are nonintuitive, underlining the importance of the LISP paradigm in current sensorization trends.