Poster E81, Thursday, August 22, 2019, 3:45 – 5:30 pm, Restaurant Hall
Mapping visual symbols onto spoken language along the ventral visual stream
J S H Taylor1,2, Matthew H Davis3, Kathleen Rastle4;1Aston University, 2University College London, 3MRC Cognition and Brain Sciences Unit, 4Royal Holloway University of London
Introduction: Reading acquisition requires the brain to map arbitrary visual symbols onto their sounds and meanings. This requires a degree of invariance to information such as case, font, and size, and position (e.g., the b in Cab is the same as the B in Bad). Dehaene and colleagues (2005; 2011) proposed that such invariance is achieved by left ventral occipitotemporal cortex (vOT), with neural representations becoming increasingly tolerant to location-shifts and encoding increasingly complex orthographic information, along a posterior-to-anterior hierarchy. However, others argue that the primary characteristic of this posterior-to-anterior vOT hierarchy is that representations become increasingly sensitive to higher-level information about word sounds and meanings (Price & Devlin, 2011). The current study used Representational Similarity Analysis (RSA) of brain responses measured with fMRI to reveal how representations along the vOT hierarchy transform visual information in the service of reading. In particular, we sought to uncover how vOT represents letter identity and position, and the extent to which representations in this region come to capture word sounds and meanings. Method: 24 adults learned to read two sets of 24 novel words that shared phonemes and semantic categories but were written in different artificial orthographies. This allowed us to independently manipulate the similarity between words with respect to both their letters (within and across positions) and their linguistic associations. Following two-weeks of training, participants recalled the meanings of the newly learned written words whilst neural activity was measured with fMRI. Using RSA we examined the extent to which the multivoxel patterns of fMRI responses along the vOT hierarchy encoded 1) basic visual form (simple cell representations from the HMAX model, Serre et al., 2007), 2) letter identity and position (position-specific and spatial coding models, Davis, 2010), 3) phonemes and 4) semantic category. Results: RSA on item-pairs from the same orthography revealed that right and more posterior vOT regions were sensitive to basic visual similarity whereas mid-anterior left vOT was sensitive to letter identity. These representations of letter identity became progressively more position-invariant from posterior to anterior vOT. In more anterior regions of the left vOT, item-pairs that shared sounds or meanings, but were represented by different orthographies with no letters in common, had similar neural patterns. Conclusions: These results reveal a hierarchical, posterior-to-anterior gradient in vOT regions, in which representations of letters become increasingly invariant to position and are transformed to convey their phonological and semantic attributes. This demonstrates the critical role of the vOT processing stream in transforming written words into their spoken forms and meanings.
Themes: Reading, Development
Method: Functional Imaging