Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Memory-based processing models predict neural tracking during comprehension of complex sentences
Poster A48 in Poster Session A, Tuesday, October 24, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
M. Blake Rafferty1,2, David Jenson3, Tanzida Zaman2, Devin M. Casenhiser2; 1New Mexico State University, 2University of Tennessee Health Science Center, 3Washington State University
Introduction Models of sentence processing seek to explain the relationship between linguistic complexity and processing difficulty. Towards this end, many studies have investigated the processing asymmetry that arises between (simpler) subject-relative clauses (SRCs) and (more complex) object-relative clauses (ORCs). Memory-based models—such as Dependency Locality Theory (DLT; Gibson, 2000) and Cue-Based Parsing Theories (CBP; Lewis & Vasishth, 2005)—make distinct predictions as to the locus of difficulty in ORCs. This has been investigated extensively, using online behavioral measures, eye-tracking, computational modeling, and functional neuroimaging. The current study evaluates whether the predictions of DLT or CBP are borne out in electrophysiological responses as individuals listen to sentences with SRCs and ORCs. Methods We recorded EEG as participants (n = 13) listened to sentences with embedded ORCs and SRCs and completed a sentence-picture matching task. For each sentence, we constructed linear kernels depicting the predicted moment-by-moment processing costs for each model in its base form, as well as for several alternative implementations(for a similar approach using fMRI, see Shain et al., 2022). We additionally coded models for constituent closure and bracket count (Nelson et al., 2017; Brennan & Hale, 2016). We then separately convolved each kernel with the narrow-band filtered, Hilbert-transformed EEG data over an integration window of -100 to 400ms in time-steps of 10ms (Brodbeck, 2018). We quantified the strength of the dependency between the two signals at each step using Mutual Information (MI; Ince et al., 2017). Following this we assessed each model using linear mixed-effects models with fixed effects for construction and model-type, the interaction between construction and model-type, and by-subject random intercepts. Pairwise comparisons were conducted using estimated marginal means. Results Results revealed significant main effects for model (F(6) = 35.62, p < .001; Eta2 (partial) = 0.58, 95% CI [0.49, 1.00]) , construction, (F(1) = 20.74, p < .001; Eta2 (partial) = 0.12, 95% CI [0.05, 1.00]), and a significant interaction between model and construction (F(6) = 24.30, p < .001; Eta2(partial) = 0.48, 95% CI [0.38, 1.00]). The best fitting model was a modified DLT model that calculates processing costs as instances of memory retrieval and instances of integration over long-distance dependencies based on the number of intervening nouns and verbs, with additional weight given to intervening verbs (Shain et al., 2016). Conclusion These results are consistent with recent findings from functional imaging studies suggesting that processes related to memory encoding and retrieval are central to language comprehension in the human cortex (Shain et al., 2022). However, future work seeking to co-localize hemodynamic and electrophysiological responses to memory cost would be necessary to fully evaluate the relationship between these two effects.
Topic Areas: Syntax and Combinatorial Semantics, Computational Approaches