Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Phase-Amplitude Coupling for the Integration of Predictive and Structural Cues during Language Comprehension
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster B78 in Poster Session B, Tuesday, October 24, 3:30 - 5:15 pm CEST, Espace Vieux-Port
Hugo Weissbart1, Andrea A. Martin; 1Donders Centre for Cognitive Neuroimaging, Nijmegen, NL, 2Max Planck Institute for Psycholinguistics, Nijmegen, NL
Humans excel at extracting meaning from speech despite the inherent physical variability of spoken language. Despite the presence of masking background noise or while encountering new accents, speech is almost effortlessly comprehended. One mechanism underlying such robustness to noise and variability is for the brain to predict its sensory input and, to some extent, the linguistic content conveyed. Although prediction over a sequence of words is non-trivial, regularities exist. Word-level predictions can be computed from those regularities, over sequences, and over syntactic structures and categories, and also within words at the phonemic level and so forth down to acoustic information. But to extract meaning from a sentence, the brain must also interpret and process hierarchically organised phrases. The syntax must be jointly processed as the utterances are predicted for the listener to access the intended meaning. This study considered syntactic features reflecting operations working on nested phrases together with surface statistics based on word sequential predictability in order to describe cortical responses. We analysed magnetoencephalogram (MEG) signals, emphasising the role of cortical oscillations, particularly on phase and amplitude dynamics. Participants' MEG was recorded while they were asked to listen to audiobook stories in their native language. We used linear forward encoding models to model the brain response to different word features comprising rule-based -such as depth of syntactic trees and the number of closing branches- and statistical features -such as entropy and surprisal. The first result presents the joint contribution of statistical and syntactical word-level features. They demonstrate synergic and overlapping roles during speech processing, although with distinct temporal dynamics. But furthermore, by estimating temporal response functions in the complex domain, we could directly compute a metric for phase consistency and phase-amplitude coupling from continuous recordings while disentangling the contributions of different features. We found above chance delta-beta coupling linked to each linguistic word-level feature, with only a few eliciting a theta-gamma coupling after word onset. The entropy of a word, and the number of syntactic structures being integrated at that word (equivalently, the number of branches in a bottom-up constituency parser), resulted in a synchronous increase in the phase-amplitude relationship. This study offers new perspectives for analysing continuous signals in response to speech while providing theoretical insights into the mechanisms at play during listening.
Topic Areas: Speech Perception, Methods