Poster C77, Wednesday, August 21, 2019, 10:45 am – 12:30 pm, Restaurant Hall
Task-related effects during natural audio-visual dialogues in fMRI
Patrik Wikman1, Artturi Ylinen1, Alina Leminen1, Miika Leminen2, Kimmo Alho1,3;1Department of Psychology and Logopedics, University of Helsinki, 2Department of Phoniatrics, Helsinki University Hospital, 3AMI Centre, Aalto University
The ability to attend to a single speech stream in the presence of irrelevant speech is fundamental to human audition. Effects of different speech related task-manipulations on these functions have, however, not been studied using naturalistic audiovisual dialogues. In our recent study (Leminen et al., in preparation), we used videos of dialogues between two speakers during functional magnetic resonance imaging (fMRI). The visual and auditory quality of dialogues was modulated from high (i.e., fully comprehensible) to low (i.e., virtually incomprehensible) with masking and noise-vocoding, respectively. The participants either listened to dialogues delivered with a concurrent irrelevant voice in the background, and answered questions about the dialogue afterwards, or ignored the dialogues and performed a visual control task. In Leminen et al., listening to the dialogues increased activation in inferior parietal lobule and inferior frontal gyrus (IFG), previously associated with top-down cognitive control. Additionally, activations were stronger in medial parietal and frontal regions, previously associated with socio-emotional and semantic processing. Here we used the same paradigm as in Leminen et al., but in addition to the listening task (L task) and visual control task (V task), participants also performed a phonological detection task (P task) where they were to report the number of occurrences of the phoneme [r] in the dialogue. We expected that as the P task requires thorough processing of the acoustic signal it would be associated with stronger activation in the auditory cortex than the L task. Also, as the P task is a much more novel than the L task, the P task might be associated with stronger activations in regions associated with executive control. As expected, our results showed that the P task activated more strongly regions associated with executive control and phonological processing than either the L or V task (IFG, premotor and dorsolateral frontal cortex; dLFC). The L task, in turn, activated regions associated with semantic and socio-emotional processing more strongly than the other tasks (anterior temporal, inferior parietal, posterior cingulate and orbitofrontal cortices). However, contrary to previous studies, which have shown that motor regions are recruited when fine phonological discrimination is needed, activations in the motor cortex were not modulated by auditory quality during the P task. Yet, unexpectedly, activations were stronger in the left IFG, dLFC and dMFC during the clear speech condition than the poor speech condition in the P task, but not in the L task. Our results indicate that although both the P and L task demanded attentive listening to the dialogues, they were associated with distinct activation patterns. That is, presumably because the L task demanded semantic and social processing it activated regions previously associated with these functions. The phonetic detection task, in turn, activated regions in the premotor cortex, supporting the idea that the motor areas are recruited when attention to phonetic details in speech is required. However, contrary to a prominent hypothesis activations in regions associated with speech production such as the left IFG were actually stronger during the clear speech than the distorted speech condition.
Themes: Perception: Speech Perception and Audiovisual Integration, Control, Selection, and Executive Processes
Method: Functional Imaging