Poster A63, Tuesday, August 20, 2019, 10:15 am – 12:00 pm, Restaurant Hall
Decoding of second-language proficiency level from multivariate neural data
Giovanni Di Liberto1, Jingping Nie2, Jeremy Yeaton1, Bahar Khalighinejad2, Shihab Shamma1,3, Nima Mesgarani2;1Laboratoire des Systèmes Perceptifs, UMR 8248, CNRS, Ecole Normale Supérieure, PSL University, 2Department of Electrical Engineering, Columbia University, 3Institute for Systems Research, Electrical and Computer Engineering, University of Maryland
Speech comprehension requires our brain to process a variety of acoustic and linguistic properties. Recent research has found detailed insights on how, where, and when some of those properties are processed. However, there remains considerable uncertainty on how these mechanisms differ in the case of second-language listening. In this work, we recorded electroencephalography (EEG) signals from native English (L1 group; N = 22) and native Chinese speakers (L2 group; N = 50) as they listened to English sentences. Behavioural measures of proficiency were derived by means of a standardised language test, which assigned each participant to a language level according to the Common European Framework of Reference for Languages (CEFR). Multivariate linear regression was used to quantify the coupling between EEG signals from each participant and the corresponding speech stimulus properties at the level of acoustics, phonemes, and semantics. This coupling allowed us to investigate the effect of language skills on the brain responses to speech at various processing levels. We found that cortical responses to speech differ between L1 and L2 listeners and change with the English proficiency level within the L2 group. The similarity of high-level cortical responses (semantic level) between L2 and L1 participants increased with language proficiency, while significant but more complex effects emerged for responses to lower-level properties of speech. Crucially, classifiers fitted on both low- and high-level cortical responses could correctly identify the proficiency of an L2 participant (low vs. high) with over 80% accuracy. We contend that the present finding provides a novel perspective on the cortical processing of a second-language and sets the basis for a novel procedure to decode language proficiency from multivariate brain data.
Themes: Speech Perception, Multilingualism
Method: Electrophysiology (MEG/EEG/ECOG)