Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
The representational dynamics of speech prosody: An MEG study
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster C36 in Poster Session C, Wednesday, October 25, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
Seung-Cheol BAEK1, Seung-Goo Kim1, Burkhard Maess2, Maren Grigutsch2, Daniela Sammler1,2; 1Max Planck Institute for Empirical Aesthetics, 2Max Planck Institute for Human Cognitive and Brain Sciences
Speech prosody plays a critical role in communication, particularly in conveying the speaker’s intention through intonation. Previous fMRI research in our lab demonstrated a network predominantly in the right hemisphere for processing speech prosody. This network consisted of an auditory ventral pathway connecting the posterior (pSTS) and anterior superior temporal sulcus (aSTS), and auditory-motor dorsal pathways connecting pSTS and the inferior frontal gyrus (IFG) and premotor cortex (PMC). Existing models suggest that prosody is acoustically analyzed in the bilateral posterior temporal regions and is abstracted and evaluated in the right anterior temporal and frontal regions. However, it remains to be shown how these areas represent prosodic information over time. To this end, we collected magnetoencephalography data from 34 native German listeners while they listened to single words (“Bar” [bar], “Paar” [pair]; on average, 480ms) that gradually varied in prosody (statement – question) and word-initial phoneme (/b/ – /p/) along orthogonal continua generated by audio morphing. Participants categorized these words in terms of either prosody or phoneme in alternating blocks. We analyzed the data in two major steps in source space (eLORETA). Firstly, we conducted a whole-brain searchlight multivariate pattern analysis (MVPA) using a linear classifier to identify the brain regions differentially involved in the phoneme and prosody tasks and their temporal dynamics. Secondly, a representational similarity analysis (RSA) was carried out in these areas to determine whether and when the multivariate differences between the tasks represent the acoustics of the prosodic contours or the abstract categories of statement and question. Specifically, the neural activity patterns across stimuli were compared with modeled patterns based on either the acoustic or categorical (behavioural) dissimilarity of the stimuli. The RSA was repeated with a whole-brain searchlight approach to confirm these results and screen for potential contributions of further brain regions. The MVPA revealed a distributed set of bilateral frontotemporal areas, replicating and extending our previous fMRI findings. Notably, the right PMC and pSTS were involved earlier (around 400 ms after word onset) than the right insula/IFG (around 550 ms), suggesting different processing stages of speech prosody. The region-based RSA showed that bilateral temporal (around 400ms) and right frontal regions (around 550ms) represented the prosody acoustics, while only the right pSTS (around 450 ms) showed categorical representations of question/statement. Similar representational dynamics of prosodic information were found in the searchlight RSA, but within broader frontotemporal regions, with a right-hemispheric predominance. These preliminary findings indicate that both acoustics and abstract categories of speech prosody are primarily represented in temporal areas. While the acoustics are represented bilaterally, the categorical representations of speech prosody are lateralized to the right hemisphere. The additional acoustic representations that emerged later in the right frontal regions could reflect the evaluative process to sharpen the categorical representations for final decision-making. The representational transfer between the involved regions will be further investigated with a directed information transfer analysis. Overall, our multivariate approach draws a comprehensive spatiotemporal picture of the cascade of perceptual and evaluative processes in the brain which underlie the comprehension of prosody.
Topic Areas: Prosody, Speech Perception