Slide Slam M3
Cortical dynamics supporting the early integration of sound and meaning in human speech interpretation
Yuxing Fang1, Bingjiang Lyu2, Benedict Vassileiou3, Kamen Tsvetanov4, Lorraine Tyler5, William Marslen-Wilson6; 1University of Cambridge
Human speech comprehension depends on the dynamic integration of speech inputs with contextual constraints to enable the rapid and robust incremental interpretation of the current utterance. Here we focus on the earliest stages of this process, to determine how contextual constraints (typically generic in nature rather than lexically specific) interact with very early cues in the speech input to constrain word choice within the first 150 ms after word onset. To map out the cortical dynamics of the neurocomputational infrastructure that supports these core integrative processes, we combine MEG and EEG measures of real-time brain activity with ASR and NLP computational models of linguistic form and content and multivariate imaging analysis methods (Representational Similarity Analysis [RSA], Grainger Causal analysis [GCA]), and apply these to data-driven whole-brain procedures (ICA) for segmenting brain activity into potential component networks. We uniquely identify a distributed LH fronto-temporal network that integrates acoustic-phonetic cues and contextual constraints to support the early identification of lexical form and meaning. This integration process is not visible at word-onset, suggesting that bottom-up constraints are necessary to set the representational geometry of an analysis space with which semantic constraints can interface. A parallel RH fronto-temporal network shows strong sensitivity to acoustic-phonetic and phonological models, but no systematic response to semantic models. Using data-driven methods to decompose the LH network into temporal and frontal centres of gravity, we combine GCA measures of connectivity with RSA analyses of the representational content of the patterns of connectivity within the network. We find a continuous interchange of semantic and phonological constraints between regions, with some evidence of a stronger role for BA47 in semantic processing and BA45 in phonological processing. Overall, these results open a novel scientific window on the mesoscale cortical dynamics underpinning incremental speech interpretation.