You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

Poster E52, Friday, November 10, 10:00 – 11:15 am, Harborview and Loch Raven Ballrooms

Investigating brain mechanisms of natural reading by combining EEG, MEG and eye-tracking

Olaf Hauk1, Felix Dreyer1,3, Maarten van Casteren1, Caroline Coutout1, Elisabeth Fonteneau4, Béla Weiss1,2;1MRC Cognition and Brain Sciences Unit, Cambridge, 22Brain Imaging Centre, Research Centre for Natural Sciences, Hungarian Academy of Sciences, Budapest, 33Brain Language Laboratory, Free University of Berlin, Berlin, 4Department of Psychology, University of Cambridge

Our knowledge about the neural correlates of natural reading is still very limited, because most previous neuroimaging studies have used fixed-gaze word-by-word paradigms. Eye movements are usually minimized during standard neuroimaging experiments to minimize artifacts. However, natural reading is a complex active process employing series of saccades and fixations for sampling of written text. Thus, there is a need for the investigation of neural mechanisms of reading under ecologically valid conditions. Recently, different research groups have developed novel methods for research into natural reading by combining eye tracking (ET) and EEG (Dimigen, 2011), and by applying single-trial modelling of fixation-related brain activity (Weiss, 2016). In order to enhance the spatial resolution and provide source estimation of brain correlates of natural reading, we will present spatio-temporal dynamics of EEG/MEG brain responses related to plausible and implausible visual words in minimal context, supposed to elicit the well-known N400 effect (Kutas, 1980). Participants (N=18) were instructed to read 248 four-word English sentences at their own pace and to make a judgement whether the last (target) word was plausible or implausible. ET (SMI, 250 Hz) and EEG/MEG (Neuromag Vectorview) data were recorded simultaneously. ET data were processed by an adaptive algorithm (Nyström, 2010). Behavior of subjects was characterized by their reading speed (RS), saccade amplitude (SA), fixation duration (FD), total number of saccades (TNS) and percentage of regressive saccades (PRS). EEG/MEG recordings were cleaned using independent component analysis (ICA). Source localization of cleaned EEG/MEG activity was carried out by using individual MRI anatomies, creating a 3-layer BEM forward solution and calculating the inverse solution using L2 minimum-norm estimates. Statistical differences between brain activity related to plausible and implausible target words were assessed in source space by spatio-temporal cluster-based permutation testing. Subjects read plausible sentences significantly faster (P=0.0009) compared to implausible sentences, and significantly larger TNS (P= 0.0018), PRS (P= 0.0033) and SA (P= 0.0057) were found for implausible sentences. The latter trend was also observed for FD, but the difference did not reach the level of significance (P=0.2953). Visual inspection of fixation-related EEG/MEG data revealed prominent deflections peaking around 90 and 250 ms in occipital and occipito-temporal sensors as well as a later component more extended in time (270-460 ms) in centro-parietal sensors. The latter component showed more negative-going deflection for the implausible words (N400). Cluster-based permutation testing resulted in a significant cluster (P=0.005) comprising medial temporal, inferior frontal and supplementary motor brain regions in the left hemisphere, with stronger activations for plausible compared to implausible words. To our knowledge, this is the first study applying the combination of ET and EEG/MEG for the investigation of neural correlates of reading in an ecologically valid condition. We could demonstrate that our approach produces clean EEG/MEG data for source estimation, that the well-known N400 effect can be replicated, and that plausible spatio-temporal dynamics can be revealed during natural reading. Our results and methodological framework may contribute to the deeper understanding of the neural mechanisms of natural reading as well as on active vision in general.

Topic Area: Methods

Back to Poster Schedule