Frontiers in Neuroscience · 2019 · 19 citations · 23 references
Life SciencesNeuroscienceAcousticsSensory NeuroscienceSystems NeuroscienceAuditory DiscriminationSpeech AcousticsSpeech PerceptionSpeech RecognitionAcoustic Vowel AnalysisKeyword SpottingNeural Signal ClassificationNeural Signal RecordingNon-speech InformationSpeech Activity DetectionSpeech Signal AnalysisVowel Perception
Neural keyword spotting could form the basis of a speech brain-computer-interface for menu-navigation if it can be done with low latency and high specificity comparable to the "wake-word" functionality of modern voice-activated AI assistant technologies. This study investigated neural keyword spotting using motor representations of speech via invasively-recorded electrocorticographic signals as a proof-of-concept. Neural matched filters were created from monosyllabic consonant-vowel utterances: one keyword utterance, and 11 similar non-keyword utterances. These filters were used in an analog to the acoustic keyword spotting problem, applied for the first time to neural data. The filter templates were cross-correlated with the neural signal, capturing temporal dynamics of neural activation across cortical sites. Neural vocal activity detection (VAD) was used to identify utterance times and a discriminative classifier was used to determine if these utterances were the keyword or non-keyword speech. Model performance appeared to be highly related to electrode placement and spatial density. Vowel height (/a/ vs /i/) was poorly discriminated in recordings from sensorimotor cortex, but was highly discriminable using neural features from superior temporal gyrus during self-monitoring. The best performing neural keyword detection (5 keyword detections with two false-positives across 60 utterances) and neural VAD (100% sensitivity, ~1 false detection per 10 utterances) came from high-density (2 mm electrode diameter and 5 mm pitch) recordings from ventral sensorimotor cortex, suggesting the spatial fidelity and extent of high-density ECoG arrays may be sufficient for the purpose of speech brain-computer-interfaces.
23
Phonetic Feature Encoding in Human Superior Temporal Gyrus
Nima Mesgarani, Connie Cheung, Keith Johnson et al. · Science · 2014 · 1.1K citations
Supratim Ray, Nathan E. Crone, Ernst Niebur et al. · Journal of Neuroscience · 2008 · 776 citations · Full text
Functional organization of human sensorimotor cortex for speech articulation
Kristofer E. Bouchard, Nima Mesgarani, Keith Johnson et al. · Nature · 2013 · 712 citations · Full text
BioImage Suite: An integrated medical image analysis suite: An update
Xenophon Papademetris, Marcel P. Jackowski, Nallkkandi Rajeevan et al. · The Insight Journal · 2006 · 331 citations · Full text
Computing, Radiology, Visualization +11