bioRxiv (Cold Spring Harbor Laboratory) · 2024 · 21 citations · 34 references
EngineeringBiomedical EngineeringNeuroscienceComputational NeuroscienceNeural EngineeringSystems NeuroscienceBrain-computer InterfacesNeural DecodingNeural InterfacesNeural ProstheticsSpeech SynthesisVoice GenerationIntracortical Brain-computer InterfacesIntracortical InterfacesIntracortical MicrostimulationNeural Speech SynthesisParalinguistic DecodingReal-time Brain-computer Interface SystemsSpeech DecodingSpeech Neuroprostheses
Abstract Brain computer interfaces (BCIs) have the potential to restore communication to people who have lost the ability to speak due to neurological disease or injury. BCIs have been used to translate the neural correlates of attempted speech into text 1–3 . However, text communication fails to capture the nuances of human speech such as prosody, intonation and immediately hearing one’s own voice. Here, we demonstrate a “brain-to-voice” neuroprosthesis that instantaneously synthesizes voice with closed-loop audio feedback by decoding neural activity from 256 microelectrodes implanted into the ventral precentral gyrus of a man with amyotrophic lateral sclerosis and severe dysarthria. We overcame the challenge of lacking ground-truth speech for training the neural decoder and were able to accurately synthesize his voice. Along with phonemic content, we were also able to decode paralinguistic features from intracortical activity, enabling the participant to modulate his BCI-synthesized voice in real-time to change intonation, emphasize words, and sing short melodies. These results demonstrate the feasibility of enabling people with paralysis to speak intelligibly and expressively through a BCI.
34
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
A multi-modal parcellation of human cerebral cortex
Matthew F. Glasser, Timothy S. Coalson, Emma C. Robinson et al. · Nature · 2016 · 5.6K citations · Full text
Cortical activity in the null space: permitting preparation without movement
Matthew T. Kaufman, Mark M. Churchland, Stephen I. Ryu et al. · Nature Neuroscience · 2014 · 924 citations · Full text
Speech synthesis from neural decoding of spoken sentences
Gopala K. Anumanchipalli, Josh Chartier, Edward F. Chang · Nature · 2019 · 899 citations · Full text
Functional organization of human sensorimotor cortex for speech articulation
Kristofer E. Bouchard, Nima Mesgarani, Keith Johnson et al. · Nature · 2013 · 712 citations · Full text