Poster Slam Session C, Wednesday, August 21, 2019, 10:30 - 10:45 am, Finlandia Hall, James Magnuson
fMRI Representational Similarity Analysis Reveals the Information Structure Underlying Word Semantics
Leonardo Fernandino1, Jia-Qing Tong1, Colin Humphries1, Lisa Conant1, Jeffrey Binder1;1Medical College of Wisconsin
The question of how word meaning is encoded in the brain remains a central problem in the neurobiology of language. Most prominent models favor one of three organizational principles: experiential/embodied features, taxonomic relationships, or word co-occurrence statistics. Here, we used representational similarity analyses (RSA) of fMRI data to evaluate several models in terms of their predictions for the degree of semantic similarity between words. Given a distributed pattern of fMRI activation for each word in the stimulus set, the degree of correspondence between the pairwise similarities computed from the fMRI activation patterns and the pairwise similarities predicted by a given model is a measure of the extent to which neural activity patterns encode the type of information on which that model is based. This approach allowed us to directly compare, for the first time, models of semantic representation based on different types of information in terms of their predictions for neural activation patterns elicited by words. We evaluated a model based on experiential feature ratings (CREA; Binder et al., 2016), a model based on taxonomic relationships (WordNet), and 3 models based on word co-occurrence statistics (LSA, HAL, and Word2Vec). Methods: Participants were 7 right-handed, native speakers of English. They performed a semantic task (familiarity rating) on a set of 300 English nouns including concrete and abstract concepts. Words were presented on a computer screen, one at a time, in pseudo-random order. The entire stimulus set was presented 6 times over the course of 3 scanning sessions. Preprocessing and GLM analyses were conducted in each participant’s original coordinate space. A GLM was used to generate a whole-brain activation map for each word. Nuisance regressors included word length and RT for each trial. The resulting maps were subsequently masked to include only cortical regions involved in semantic word processing. We used two independent semantic masks: one encompassing all voxels that were modulated by word concreteness in a separate GLM analysis, and the other defined by an ALE meta-analysis of 120 fMRI studies of word semantics (Binder et al, 2009). Both masks consisted mainly of heteromodal cortical regions in temporal, parietal, and frontal lobes. From each semantic model, we computed the dissimilarity matrix (DSM) for the 300 words included in the task. We then computed the Pearson correlation between each model-based DSM and the two fMRI-based DSMs (one for each semantic mask). Statistical significance was estimated via the Mantel test with 10,000 permutations. Results: The two semantic masks produced a similar pattern of results, with the concreteness-defined mask generating higher correlation coefficients for all models. All models predicted fMRI activation patterns in this mask (CREA: r=.24; WordNet: r=.16; HAL: r=.14; LSA: r=.12; Word2Vec: r=.11; all p < .0001). In the ALE-defined mask, the RSA was significant for all models except LSA. Importantly, the experiential CREA model was superior to all other models in both masks (all p < .001). These results strongly suggest that the neural representation of word meaning primarily encodes information about experiential features, rather than taxonomic or word co-occurrence information.
Themes: Meaning: Lexical Semantics, Methods
Method: Functional Imaging
Poster C30