Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
A new eye on semantics: Coregistering eye-movements and EEG+MEG to study semantics in context
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster B23 in Poster Session B, Tuesday, October 24, 3:30 - 5:15 pm CEST, Espace Vieux-Port
FEDERICA MAGNABOSCO1, Olaf Hauk1; 1MRC Cognition and Brain Sciences Unit
Semantic cognition is the representation of the acquired knowledge about the world and its controlled and task-oriented use. Two neural systems make flexible semantic cognition possible: a representation system that is the collection of information on features and concepts, and a control system, for the flexible retrieval of information based on task and context. In this study, we aim to better characterise the interaction between these systems during natural reading. We used co-registered eye-tracking and EEG+MEG from 27 participants to explore dynamic brain activity and behaviour in the same paradigm. Each sentence contained a target word that was manipulated for predictability (reflecting semantic control demands) and concreteness (semantic representation resources). This resulted in a 2x2 factorial design, with 80 stimuli per cell. Here, we present preliminary results and a detailed description of our pre-processing methods. We compared different approaches for eye movements artefact correction: we contrasted different high-pass filtering settings (0.1, 0.5, 1, 2 Hz), different methods for selecting ICA components (based on electrooculogram (EOG) correlation, variance ratio during fixations and saccades, or both), and ICA overweighting procedures (no overweighting, saccade onset overweighting, saccade overweighting). We found that a 0.5 Hz high-pass filter decreased the prominence of slow drifts caused by following fixations with minimal loss of neural signal. Removing ICA components that either correlated with EOG or had a variance ratio greater than 1 yielded the best results across participants. For the overweighting procedure, there was no absolute best setting, so we applied the method with the highest SNR across multiple aspects for each participant. The data was then used for source estimation at a whole-brain level and in regions of interest within the semantic brain network (left and right anterior temporal lobe, inferior frontal gyrus, posterior temporal cortex, angular gyrus, and primary visual areas). We applied a conventional evoked as well as a multivariate decoding analysis. Whole brain analysis using cluster-based permutation tests of evoked data did not reveal any effect of concreteness or predictability. We then fitted mixed effects models for each ROI across conditions. While we found some effects of concreteness and predictability in the hypothesised areas within the semantic network, the effects did not survive correction for multiple comparisons. Source space decoding results showed predictability effects in a subset of regions across the semantic network and in visual areas, while concreteness was decodable only in the right ATL. Overall, our results confirm that co-registration is a promising technique for the study of language in naturalistic paradigms, but also highlights the need for further technical and theoretical improvements of these methods. Deconvolution techniques and more advanced data-cleaning procedures might be necessary for reliably detecting effects previously observed in traditional serial visual presentation paradigms. We hope to encourage more researchers to use and develop methods to study the neural correlates of naturalistic language processing. In particular, the co-registration of eye movements and EEG/MEG has the unique benefit of allowing for the simultaneous exploration of dynamic brain activity and behaviour.
Topic Areas: Meaning: Lexical Semantics, Reading