You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

Poster E73, Friday, November 10, 10:00 – 11:15 am, Harborview and Loch Raven Ballrooms

Reading at the speed of speech: Convergence between visual and auditory language perception at 5 Hz

Benjamin Gagl1,2, Julius Golch1, Stefan Hawelka3, Jona Sassenhagen1, David Poeppel4,5, Christian J. Fiebach1,2;1Department of Psychology, Goethe University Frankfurt, Frankfurt am Main, Germany, 2Center for Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt am Main, Germany, 3Centre for Cognitive Neuroscience, University of Salzburg, Salzburg, Austria, 4Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany, 5Department of Psychology, New York University, USA

Across languages, the speech signal is characterized by a ~4-5 Hz rhythm in the amplitude modulation spectrum (Ding et al., 2017). It is suggested that during comprehension, this temporal structure drives brain activity in the language system, reflecting the processing of linguistic information chunks every 200 ms. Interestingly, this is the typical eye-fixation duration in reading. To investigate this observation systematically, we first estimated the individual sampling frequency of 50 German-native speakers during reading sentences and scanning of z-strings (as a non-linguistic control task; note that several empirical investigations showed that the eye movement scan path, i.e. where the eyes fixate, is very similar in both tasks, z-string scanning and reading). For sentence reading, the individual sampling rate varied around 5 Hz, i.e., in the same frequency range as spoken language, with a low standard deviation of 0.6 Hz. In z-string-scanning, in contrast, the sampling frequency was significantly lower, i.e., 4 Hz, with a higher variance (0.8 Hz). This result suggests a remarkable temporal alignment of the information uptake in reading and speech processing. The eye movement-sampling rate in the non-linguistic task more likely reflects a mean rate at which humans reorient attention, which was identified in previous research at around 4 Hz. To substantiate and further elaborate this empirical finding, we then conducted a meta-analysis of eye-tracking studies of natural reading (83 studies between 2006 and 2016; N=772 fixation durations; FD). This analysis converges with our experimental finding, by demonstrating that the predominant fixation-based sampling frequency for German is 5.1 Hz (12 studies; 36 fixation durations). Across different languages the frequency varied between 4-5.3 Hz, with systematic differences between languages arguably reflecting the difficulty of the writing systems. For example, Chinese, a character-based writing system for which sufficiently many original studies were available (12 studies; 112 fixation durations) showed significantly lower sampling rates (i.e., 4.0 Hz) then alphabetic writing systems (Italian, English, Dutch, Spanish, Hebrew, German, Finnish, Japanese, French and Thai; average sampling frequency: ~4.5 Hz). Within alphabetic writing systems, the letter-to-sound transparency of the writing system significantly influenced the sampling frequency of reading, such that transparent languages (e.g., Dutch, German) showed sampling frequencies around 5 Hz, while opaque orthographies like English had sampling frequencies as low as 4.5 Hz. Combined, this set of results indicates that the speed of speech is reflected in the rate at which our eyes sample visual information in reading in transparent writing systems. In contrast, when encoding of the writing system becomes more difficult, the sampling rate is reduced. The present results thus invite the hypothesis that a language system, relevant for speech perception, drives voluntary eye-movements in reading, presumably to supply linguistic information in chunks at an optimal rate, ~5 Hz, reflecting a common ‘uptake rate’ for linguistic information.

Topic Area: Perception: Speech Perception and Audiovisual Integration

Back to Poster Schedule