Slide Slam C8 Sandbox Series
Using EEG and RSA to test neural computations underlying phonological analysis in spoken language comprehension
Victoria R. Poulton1, Máté Aller1, Lucy J. MacGregor1, Matthew H. Davis1; 1MRC Cognition and Brain Sciences Unit, University of Cambridge
EEG has been used extensively to study the time course of sentence comprehension, with different computations proposed to underlie prominent ERP effects. For example, it has been argued that the increased magnitude of earlier neural responses (~100ms post-stimulus) reflects the magnitude of phonological prediction error (Sohoglu & Davis, eLife, 2020), while later N400 responses reflect the magnitude of semantic prediction error (e.g., Rabovsky, Neuropsychologia, 2020). However, changes in response magnitude are also consistent with other computations, such as those which integrate current words with contexts to sharpen predicted features of the input (Aitchison & Lengyel, Curr Opin Neurobiol, 2017). Therefore, changes in ERP magnitude alone are not sufficient to determine which computations are performed by the brain during language comprehension. In the present study, we are interested in the time course of computations underlying phonological analysis during sentence comprehension. Using representational similarity analysis (RSA; Kriegeskorte et al., Front Syst Neurosci, 2008) and EEG, we will compare patterns of neural activity elicited by predictable and unpredictable word-initial phonemes to investigate different neurocomputational proposals. While sharpening proposals suggest the predictable speech sounds will have an enhanced or sharpened representation, prediction error proposals suggest that the brain computes the difference between the prediction and the input, thereby representing unpredictable information. Since the representational output of these computations should contain qualitatively different information, we expect that different patterns of neural activity should emerge corresponding to which computations are present. EEG data were collected from 70 Dutch participants at the Max Planck Institute for Psycholinguistics. Participants listened to highly predictive sentences (mean cloze = 0.92) that ended in either a predictable or unpredictable word. Unpredictable words mismatched with predicted words in both semantics and phonology. For example, predictable ‘boeket’ (bouquet) was replaced with unpredictable ‘tonnetje’ (barrel). Our analyses will investigate the correlation between observed representational dissimilarity matrices (RDMs), determined from neural responses to predictable and unpredictable word-initial speech sounds, and theoretically-motivated model RDMs, which describe the hypothetical representations proposed for various neural computations. We will use four model RDMs, corresponding to (1) representations of the heard phonemes; (2) representations of the predicted phonemes (e.g., Wang et al., eLife, 2018, observe anticipatory representations of predicted lexico-semantic information); (3) sharpening representations (i.e., a blend of the features of the predicted and heard phonemes); and (4) prediction error representations (i.e., the difference between the features of the predicted and heard phonemes). The temporal precision of EEG data allows us to test for different computations as speech unfolds. The performance of each model RDM over time will be determined by the strength of the correlation between the model RDMs and observed RDMs. Since previous studies using fMRI report findings that are consistent with either sharpening (e.g., Heilbron et al., Nat Commun, 2020) or prediction error (e.g., Blank & Davis, PloS Biol, 2016), we expect that we may observe neural responses consistent with all four model RDMs, but perhaps with different time courses.