You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

Poster E28, Friday, November 10, 10:00 – 11:15 am, Harborview and Loch Raven Ballrooms

Robust Electrophysiological Indices of Semantic Surprisal during Natural, Ongoing Speech Processing.

Michael Broderick1, Andrew James Anderson2, Giovanni M. Di Liberto1, Edmund C. Lalor1,2;1School of Engineering, Trinity Centre for Bioengineering, and Trinity College Institute of Neuroscience, Trinity College Dublin, Dublin, Ireland, 2Department of Biomedical Engineering and Department of Neuroscience, University of Rochester, Rochester, New York, 14627

Studies of natural language processing using EEG have typically measured neural activation for short snippets of language, and contrasted EEG responses to subtle variations in linguistic stimuli. For instance, the well known N400 effect can be revealed by contrasting the time-aligned EEG response to sentences such as the "the dentist told me to brush my teeth" with "the dentist told me to brush my tree". Although such approaches have been the foundation of an extensive body of research, they tend to be grounded on artificial modulations of a small stimulus set that is constrained to be amenable to conventional analyses. How much the results generalize to natural language is unclear. Consequently, there has been a recent move toward EEG-based analyses of more natural linguistic stimuli. We here build on related work on natural speech (audio-book) comprehension that used time-stamped models of the acoustic and phonemic properties of speech to predict and disentangle associated EEG signal. This demonstrated that EEG signal exclusively associated with phonemic properties of speech could be extracted, thus supporting the inference that the acoustic stimulus had been first decoded into speech units by the experimentee’s brain. We here go beyond this, and build a measure that enables the further inference that the experimentee also processed words’ meanings. We do this by adding in an additional “semantic” layer to the acoustic and phonemic features of the earlier predictive model. We exploit the recently popular “word2vec” computational model of words’ meanings as the basis for semantic prediction. By computing the semantic difference between a word and the words in the previous phrase we build a predictive measure of “semantic surprisal”: if a new word’s meaning is not correlated with the previous words’ meanings then it is a surprise! Here, using a regression analysis, we demonstrate that EEG activity over centro-parietal scalp reflects the magnitude of semantic surprisal in ongoing, natural speech at a latency of 250-400 ms. Furthermore, we show that this semantic effect disappears in reversed and unattended speech, despite robust EEG tracking of the acoustics of those stimuli. This work provides a new index of semantic comprehension in natural speech, which has implications for both cognitive and clinical neuroscience.

Topic Area: Meaning: Combinatorial Semantics

Back to Poster Schedule