You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

Poster B79, Wednesday, November 8, 3:00 – 4:15 pm, Harborview and Loch Raven Ballrooms

Electrophysiological correlates of statistical features of word sequences in natural spoken language

Hugo Weissbart1, Katerina D. Kandylaki1, Tobias Reichenbach1;1Imperial College London

Research on electrophysiological correlates of language processing often employs simplified stimuli such as single words or short sentences. However, such an approach cannot assess neural responses to statistical features of word sequences in longer narratives. To overcome this limitation, we measured cortical responses to natural spoken language. Continuous speech has recently been shown to yield cortical responses that partly reflects acoustic processing since they occur for incomprehensible reversed speech as well. Here we show how a hierarchical model of acoustic and linguistic features of spoken language can disentangle neural responses to the acoustic and linguistic aspects of a continuous spoken language. We employed electroencephalography (EEG) to measure neural responses of native English speakers to continuous English speech. Linguistic features were extracted from the text that corresponded to the speech signals and were aligned to the acoustic signal through forced alignment. As a first feature, the frequency of each word in large corpus was obtained using Google Ngrams. This feature is independent of the word sequence. Second, we employed recurrent neural networks for language modelling to obtain two statistical features of word sequences. In particular, we obtained the probability for each word in a sequence conditioned on the previous words in the sequence. The negative logarithm of that probability is the surprisal of that word in its context (Nelson et al. 2017). The posterior estimate of all probabilities yields a measure of the entropy at each word location. To control for neural responses to the acoustic properties of speech, we determined the onset of each word and used that feature as a control variable. We then employed linear regression with regularization to correlate the EEG responses to the linguistic and acoustic features. As an additional control, we performed the same analysis for EEG responses to a foreign language, namely Dutch, which has similar acoustic properties as English but was incomprehensible to the participants. Results: We found that both the acoustic feature as well as the linguistic features elicited distinct neural responses. In particular, we obtained specific electrophysiological correlates of the surprisal of a word in its sequence as well as of the associated entropy. Both neural responses could not be explained by the acoustic properties or by word frequency. The neural responses to the linguistic features were absent when participants listened to the foreign language. Our study reveals electrophysiological correlates of statistical features of word sequences that emerged when analyzing neural responses to long sequences of natural spoken language. In particular, we show how statistical features of natural language that were extracted through applying recurrent neural networks to text have corresponding neural responses. The cortical response to the surprisal of a word may support the predictive coding hypothesis where a sequence of words leads to the prediction of the next word. As this analysis discriminates conditions (English or Dutch), we can assess comprehension such that our findings may be applied to better understand and characterize types of aphasia that yield a difficulty with speech comprehension.

Topic Area: Computational Approaches

Back to Poster Schedule