EEG-Based Discrimination of Imagined Speech Phonemes

Abstract

This paper reports positive results for classifying imagined phonemes on the basis of EEG signals. Subjects generated in imagination five types of phonemes that differ in their primary manner of vocal articulation during overt speech production (jaw, tongue, nasal, lips and fricative). Naive Bayes and linear discriminant analysis classification methods were applied to EEG signals that were recorded during imagined phoneme production. Results show that signals from these classes can be differentiated from those generated during periods of no imagined speech and that the signals among the classes are discriminable, particularly in data collected on a single day. The simple linear classification methods are suited well to online use in BCI applications. Using imagined speech in an EEG-based BCI potentially offers a natural means of expression consistent with mobility. During the production of imagined speech, one might expect to find in EEG traces of brain activity related to auditory imagery (the voice in one's head), motor imagery (imagined vocal articulation), and other aspects of speech production. Yet positive results are few. Suppes and colleagues reported over a decade ago some success in using EEG signals to discriminate among imagined sentences(Suppes et al, 1997, 1998), yet the result has not been replicated. Better substantiated are MEG and EEG results for heard speech, which show that traces of the acoustic speech waveform envelope can be extracted(Ahissar et al., 2001; Luo & Poeppel, 2007; Deng and Srinivasan, 2010). This has recently been shown true also for imagined speech; the presumptive loudness envelopes of auditory imagery generated--the rhythm or pattern of stress--are discriminable(Deng et al., 2010). The present study focuses more squarely on motor imagery; it seeks to determine whether phonemes that differ in pattern of vocal articulation may be discriminated in EEG. Simple classification methods applied to spectrograms of EEG activity recorded during production of the imagined phonemes provide discrimination performance of high statistical significance. The simplicity of the analysis lends itself well to future online use in BCI applications. 2. Experimental Methods An experimental session included twelve types of trial, ten of which involved the production of a phoneme in imagination. The subjects' task was to generate in imagination the phoneme cued at trial onset. Each trial started with an auditory cue presented through a loudspeaker that either stated the phoneme to be imagined or instructed the subject to relax (see Figure 1). The cue was followed by two audible clicks, separated by an interval of duration 1 sec. The purpose of the clicks was to indicate, on imagined phoneme trials, the time at which the phoneme was to be produced in imagination. Subjects were instructed to generate as best as possible both clear auditory imagery and motor imagery while remaining completely still during imagined phoneme production. A camera was used to capture video of the subject's face, neck and shoulders during experimental sessions to help monitor muscular activity, which was limited almost exclusively to blinks and eye movements and was absent during the intermittent periods during which imagined speech was produced. Five articulation classes of phonemes were used in the experiment; each class was represented by a pair of phonemes. The ten English-language phonemes used include two examples of sounds that involve notable movements of the jaw (-aa and -ae), of the tongue (-l and -r), of the velum (nasal -m and -n) and of the lips (rounded for -uu and -ow) as well as two fricatives (-s and -z); see Table 1. Two further types of trial appeared during which the subject was instructed to relax. Twelve instances