My Account

Poster E68, Thursday, August 22, 2019, 3:45 – 5:30 pm, Restaurant Hall

Language ERPs reflect learning through prediction error propagation

Hartmut Fitz1,2, Franklin Chang3,4;1Donders Centre for Cognitive Neuroimaging, Radboud University, 2Neurobiology of Language Department, Max Planck Institute for Psycholinguistics, 3Kobe City University of Foreign Studies, 4ESRC International Centre for Language and Communicative Development

Event-related potentials (ERPs) in language processing have been linked, variously, to word retrieval or integration (N400) and syntactic repair or unification (P600), but it has been difficult to reach consensus on their functional interpretation. Here, we propose a novel theory arguing that ERPs in comprehension arise as side effects of an error-based learning mechanism which is driven by prediction in the production system. On this view, ERPs reflect learning signals that play an important role in language acquisition and adult linguistic adaptation. We instantiated this theory in a recurrent neural network model with interacting processing pathways for syntax and semantics. The model learns language in production by mapping meaning representations onto appropriate sentences that convey the intended message. In comprehension, the model covertly pre-activates the most likely continuation at each sentence position while incrementally processing an overheard utterance. Words that are unexpected within a given context trigger prediction error which generates a learning signal that is propagated through the network. Thus, the model continuously adapts its representations in response to linguistic experience in order to make future predictions more accurate. Since these representations are distributed across the network, distinct areas will be differentially sensitive to semantic and syntactic expectation mismatch. When these learning signals are interpreted as ERPs, the model is able to simulate data from three studies on the N400 where amplitude is modulated by expectancy (Kutas & Hillyard, 1984) and sentence position (Van Petten & Kutas, 1991), but insensitive to the strength of contextual constraints (Federmeier, Wlotko, De Ochoa-Dewald & Kutas, 2007). We also simulated effects from five studies on the P600 in response to violations of number agreement (Hagoort, Brown & Groothusen, 1993), tense inflection (Allen, Badecker & Osterhout, 2003), word category (Hagoort, Wassenaar & Brown, 2003), verb subcategorization (Osterhout & Holcomb, 1992), as well as temporary ambiguity in garden-path sentences (Osterhout, Holcomb & Swinney, 1994). Furthermore, the model displays the semantic P600 when processing role reversal anomalies (Kim & Osterhout, 2005). A unique prediction of this approach is that error-based learning will lead to adaptation of ERPs. Support for this account comes from experimentally observed developmental changes in ERPs (Clahsen, Lück, & Hahne, 2007), the adaptation of ERP amplitude to frequency manipulations within a block of trials (Coulson, King & Kutas, 1998), and the sensitivity of ERPs to word predictability in previous prime sentences (Rommers & Federmeier, 2018). The model can account for these findings because ERPs as error propagation signals are not merely indices of comprehension processing but have a emph{functional role} in learning and adaptation. This computational approach provides a unified account of the sensitivity of ERPs to expectation mismatch, the relative timing of the N400 and P600, the semantic nature of the N400, the syntactic nature of the P600, and learning-based changes in ERP components and amplitude. It demonstrates that prediction error propagation is not only a useful way to explain how humans learn and adapt their language representations, but also sheds light on the neural signatures measured in EEG during language processing.

Themes: Computational Approaches, Speech Perception
Method: Computational Modeling

Back