Multimodal Artificial Neurological Sensory–Memory System Based on Flexible Carbon Nanotube Synaptic Transistor

TLDR

The sensory–memory system is essential for human intelligence, and its electronic implementation can enable AI to learn and interact with diverse external information. The study introduces a multimodal artificial sensory–memory system that combines visual, auditory, and tactile sensors with a flexible carbon‑nanotube synaptic transistor. The system converts physical stimuli into presynaptic action potentials and the carbon‑nanotube transistor demonstrates synaptic plasticity and memory behaviors in response to both single and prolonged excitations. Characterization of signal transduction, synaptic plasticity, and memory behaviors, along with electronic implementations of multistore memory and Pavlov conditioning, demonstrate the system’s biomimetic intelligence and its promise for multimodal, user‑environment interactive AI.

Abstract

As the initial stage in the formation of human intelligence, the sensory–memory system plays a critical role for human being to perceive, interact, and evolve with the environment. Electronic implementation of such biological sensory–memory system empowers the development of environment-interactive artificial intelligence (AI) that can learn and evolve with diversified external information, which could potentially broaden the application of the AI technology in the field of human–computer interaction. Here, we report a multimodal artificial sensory–memory system consisting of sensors for generating biomimetic visual, auditory, tactile inputs, and flexible carbon nanotube synaptic transistor that possesses synapse-like signal processing and memorizing behaviors. The transduction of physical signals into information-containing, presynaptic action potentials and the synaptic plasticity of the transistor in response to single and long-term action potential excitations have been systematically characterized. The bioreceptor-like sensing and synapse-like memorizing behaviors have also been demonstrated. On the basis of the memory and learning characteristics of the sensory–memory system, the well-known psychological model describing human memory, the "multistore memory" model, and the classical conditioning experiment that demonstrates the associative learning of brain, "Pavlov's dog's experiment", have both been implemented electronically using actual physical input signals as the sources of the stimuli. The biomimetic intelligence demonstrated in this neurological sensory–memory system shows its potential in promoting the advancement in multimodal, user-environment interactive AI.