Slide Slam B10
The language network reliably tracks non-linguistic meaningful stimuli in a naturalistic setting
Yotaro Sueoka1, Alexander Paunov1, Anna Ivanova1, Idan Blank2, Olessia Jouravlev3, Zachary Mineroff4, Jeanne Gallee5, Evelina Fedorenko1,6; 1MIT, 2UCLA, 3Carleton University, 4Carnegie Mellon University, 5Harvard University, 6Massachusetts General Hospital
The language network, comprised of brain regions in the left frontal and temporal cortex, responds robustly and reliably during language comprehension, but not during many non-linguistic cognitive tasks, including arithmetic, music perception, logical reasoning, executive function tasks, action/gesture observation, mentalizing, and even the processing of computer code (e.g., Fedorenko & Blank, 2020). However, one domain whose relationship with language remains debated is abstract semantics: our conceptual knowledge of the world. Given that the language network responds robustly to meaningful linguistic stimuli, could some of this response be driven by the presence of rich conceptual representations encoded in linguistic inputs? Past studies have reported responses in the language regions to pictures of objects and events (e.g., Devereaux et al., 2013; Handjaras et al., 2017; Visser et al., 2012; Ivanova et al., 2021), albeit lower than the response to linguistic stimuli. However, in static images, the possibility of linguistic re-coding of visual semantic information is difficult to rule out. In this study, we used a naturalistic experimental paradigm to ask whether the cognitive and neural resources responsible for language processing are also recruited for processing semantically rich audiovisual inputs that do not contain language. Here, we adopted the inter-subject correlation (ISC) approach (Hasson et al., 2008) to examine the tracking of non-linguistic meaningful naturalistic stimuli by the language network (which was defined in each individual using a localizer task; Fedorenko et al., 2010). The use of rich naturalistic stimuli should minimize the probability of re-coding the information into a linguistic format. If the language network represents/processes abstract conceptual information, then we should observe correlated patterns of BOLD signal fluctuations across participants during the presentation of naturalistic stimuli. In fMRI, forty-six participants were presented with 10 naturalistic stimuli (each ~5 minutes long): 4 critical, meaningful non-linguistic, stimuli (silent films and animations, and a “story” made up of sound effects with no linguistic information); 4 linguistic stimuli (stories and dialogs); and 2 non-linguistic stimuli with no/minimal propositional meaning (a musical piece and a movie of changing kaleidoscope images). Across all the regions of the language network, non-linguistic meaningful stimuli elicited reliable ISCs. These ISCs were substantially lower in magnitude than the ISCs elicited by linguistic stimuli, but stronger than the ISCs elicited by non-linguistic stimuli without propositional content. These results suggest that the language network encodes abstract semantic content even from entirely non-linguistic—visual and auditory—stimuli.