Slide Slam B7
Searchlight RSA using multiple representational models reveals the detailed structure of the semantic system
Leonardo Fernandino1, Jia-Qing Tong1, Colin Humphries1, Lisa Conant1, Jeffrey Binder1; 1Medical College of Wisconsin
Functional neuroimaging studies of semantic language processing have implicated a set of heteromodal cortical regions in the inferior parietal lobule, lateral and ventral temporal cortex, medial parietal cortex, and medial and lateral prefrontal cortex (the “semantic network”). Here we used a surface-based representational similarity analysis (RSA) searchlight approach with high-resolution fMRI to investigate the fine-grained structure of this network. The aim was to identify the precise cortical locations of the constituent functional nodes and to get insight into possible differences in their representational structures. Our approach combined RSA results for 6 different models of word semantics: 2 experience-based models (Exp48 and SM8), 2 taxonomic models (Categorical and Wordnet), and 2 distributional models (word2vec and GloVe). We expected that the commonalities and differences between the representational structures encoded in these models would reveal fine-grained distinctions between functional areas making up the semantic network. Methods: Thirty-six adult participants performed a semantic judgment task on individual words while undergoing fMRI. The stimulus set consisted of 320 English nouns, half denoting objects (e.g., “fork”, “motorcycle”, “snake”) and half denoting events (e.g., “laughter”, “convention”, “flood”). The task consisted of rating each object or event according to how often the participant encountered them in their daily lives on a 1-3 scale, responding via key press. The entire stimulus set was presented 6 times over the course of 3 scanning sessions, in randomized order within each presentation. MR imaging was performed with a 3T scanner. Each session included 8 functional scans (4x multiband, TR = 1500 ms, TE = 23 ms, 512 volumes, voxel size = 2 x 2 x 2 mm). A general linear model was used to generate activation (beta) maps for each noun relative to the mean signal across all other nouns. Response time, response key pressed, number of letters for each word, and head motion parameters were included as regressors of no interest. Searchlight RSA was performed using 2-dimensional patches (5-mm radius) on the reconstructed cortical surface. Vertices in the patch were used to select cortical voxels in the participant’s native volume space, and the RSA score was mapped to the central vertex. Group-level analysis was performed on the RSA correlation score maps after aligning each individual map to a common surface template. Permutation testing was used to determine cluster-level statistical significance. Maps were thresholded at p < 0.001 at vertex level and cluster-corrected p < 0.01. All models predicted representational similarity structure in the angular gyrus (AG), anterior superior temporal sulcus (aSTS), inferior frontal gyrus (IFG), ventral premotor cortex (area 6r), superior frontal sulcus (SFS) and gyrus (SFG), posterior cingulate gyrus (PCG), precuneus (PreCun), and collateral sulcus (CS), all left-lateralized. Only taxonomic and experiential models predicted activity in frontal areas 55b and 10 and in the posterior STS, indicating a difference in representational structure. Averaging the t statistic across the 6 models at each vertex revealed over two dozen distinct RSA peaks in those areas. These results provide the most detailed characterization to date of the functional neuroanatomy of the semantic system.