Neurolight: A Deep Learning Neural Interface for Cortical Visual Prostheses

TLDR

Visual neuroprostheses that electrically stimulate multiple sites in the visual system offer a promising avenue for restoring vision, and recent advances in neural engineering and artificial intelligence provide a fertile framework for developing such neurotechnology. The study aims to create a computationally powerful, flexible cortical visual neuroprosthesis—CORTIVIS—to deliver functional vision to profoundly blind individuals. We built Neurolight, a modular framework that models retinal ganglion cell firing, applies deep‑learning image preprocessing (semantic segmentation, object detection), and translates these outputs into electrical stimulation patterns delivered via intracortical microelectrodes in the visual cortex. This is the first deep‑learning‑based system that directly interfaces with the visual brain through an intracortical microelectrode array.

Abstract

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.