Poster D73, Wednesday, August 21, 2019, 5:15 – 7:00 pm, Restaurant Hall
Resolving language processing from decision making: a multimodal imaging approach
Mia Liljeström1,2, Annika Hulten1,2, Jan Kujala1,3, Riitta Salmelin1,2;1Aalto University, 2Aalto NeuroImaging, 3University of Jyväskylä
Language use is intimately connected to cognitive systems such as attention, decision making, and memory. Yet, how language-specific vs. domain general processes contribute to performing language tasks is still an open question in studying the organization of language. We examined how task (semantic vs. perceptual decisions on the same written words) and choice difficulty affect the neural response by utilizing the complementary views provided by MEG and fMRI. To disentangle processes related to the underlying decision making, we model the data as a drift diffusion process. 14 healthy volunteers participated in three sessions, performing the tasks (300 trials per task) either overtly (reaction time measurements) or covertly (MEG and fMRI measurements). The task was either a semantic (size) judgment task (e.g. “Is this item smaller or larger than a rubber boot?”), or a perceptual (color) judgment task (“Is this font written in blue or green?”). Within each category, the choice difficulty was varied (small/medium/large size; blue/turquoise/green color). A hierarchical Bayesian drift diffusion model of decision making was used to estimate the non-decision time and the drift-rate of the two tasks and the different difficulty levels (HDDM, Wiecki et al., 2013, Front. Neuroinform. 7:14). MEG evoked responses were analyzed using cortically constrained minimum norm estimates in MNE-Python. fMRI data analysis was performed in SPM12. An effect of choice difficulty was determined using a parametric model with reaction times for each item as a regressor. The reaction times for the semantic decision task were significantly slower (951ms +- 286ms) than for the perceptual decision task 727ms +- 270ms (one-way ANOVA on ranks p<0.001). Drift-diffusion modeling of semantic vs. perceptual decision making revealed a distinction in non-decision time between the two tasks, suggesting that slower reaction times for the semantic vs. perceptual decisions are due to additional processing stages in extracting semantic information to inform choices regarding item size. In addition, significantly slower drift-rates as a function of choice difficulty indicated that the slower reaction times for difficult choices were linked to the decision process. FMRI results (FDR, p<0.05) showed that attention to words and performing the semantic judgment task increased activation within the left lateral prefrontal cortex, inferior frontal gyrus, and inferior occipito-temporal cortex, and the bilateral ACC. Attention to colors and perceptual judgment increased the activation in the ventromedial prefrontal cortex and the precuneus bilaterally. Choice difficulty modulated activation within the left lateral prefrontal cortex and bilateral ACC in the semantic judgment task, and in the ventromedial prefrontal cortex, precuneus and posterior parietal cortices in the perceptual judgment task. The results suggest that the two judgment tasks draw partially on different neural substrates underlying decision making. MEG showed the time course of the decision process with early activation of the visual cortex, followed by activation in the superior/middle temporal gyrus and middle/inferior frontal cortex. The results highlighted processing within the left middle temporal cortex around 400ms, similarly in both tasks, possibly reflecting highly automatized lexical processing, irrespective of task demands.
Themes: Reading, Control, Selection, and Executive Processes
Method: Electrophysiology (MEG/EEG/ECOG)