You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

Poster D78, Thursday, November 9, 6:15 – 7:30 pm, Harborview and Loch Raven Ballrooms

Articulatory gesture encoding in human sensorimotor cortex during continuous speech production

Josh Chartier1, Gopala K. Anumanchipalli1, Edward F. Chang1;1University of California, San Francisco

To speak, we must coordinate over 100 muscles to precisely actuate our lips, jaw, tongue, larynx, and other vocal tract articulators. It is an extraordinary motor control feat, yet nearly all of us produce fluent speech. Our previous work focused on short consonant-vowel syllable production, and demonstrated that the human ventral sensorimotor cortex (vSMC) is functionally activated along somatotopic representations of articulators. However, natural continuous speech is far more complex and dynamic than single syllables because of co-articulation between adjacent segments and execution of motor plans over longer duration. To address this, we studied the encoding of kinematic properties in the human vSMC during natural sentence production. We recorded high-density intracranial electrocorticography signals, while speakers produced a set of sentences designed to cover all phonetic contexts in American English (MOCHA-TIMIT). We first developed a method to estimate vocal tract kinematic parameters from phonetic transcriptions and produced acoustics (acoustic-to-articulatory inversion). We then fit linear kinematic-trajectory models to each electrode using kinematic parameters to predict neural activity. We found single electrode encoding of dynamical representations of highly specific, coordinated out-and-back trajectories of articulators (e.g. tongue protrusion, lip closure, etc). Kinematic trajectories of electrodes clustered into four main categories differentiated by place of vocal tract constriction. Furthermore, electrodes in each trajectory category showed activations during the production of phonemes with similar places of articulation. Each trajectory category appeared to be spatially localized in the sensorimotor cortex. Lastly, the kinematic-trajectory model better explained electrode activity when compared against phoneme and single articulator representations. We have used natural continuous speech to demonstrate the neural representation of articulatory gestures in speech production.

Topic Area: Speech Motor Control and Sensorimotor Integration

Back to Poster Schedule