You are viewing the SNL 2017 Archive Website. For the latest information, see the Current Website.

 
Poster B78, Wednesday, November 8, 3:00 – 4:15 pm, Harborview and Loch Raven Ballrooms

Trial-by-trial N400 variability reflects temporal gesture-speech integration

Laura Morett1,2, Nicole Landi1,3, Julia Irwin3,4, James McPartland1;1Yale Child Study Center, 2University of Alabama, 3Haskins Laboratories, 4Southern Connecticut State University

In natural conversation, beat gesture (i.e., simple rhythmic manual gesture) and pitch accent (i.e., speech intonation) are tightly linked. Previous research indicates that the N400 event-related potential (ERP), which reflects prediction, shows sensitivity to contingencies in presence between beat gesture and pitch accent (Wang & Chu, 2013). Currently, it is unknown whether the N400 ERP shows sensitivity to contingencies in timing between beat gesture and pitch accent. Because the N400 ERP is typically measured as an average response, and trial-by-trial variability contains information concerning response stability (Dinstein, Heeger, & Behrmann, 2015), N400 variability has the potential to be particularly informative about temporal beat gesture-pitch accent integration. In light of this, the current research examined N400 response variability to determine how it reflects temporal beat gesture-pitch accent integration. In this study, the N400 response was recorded in neurotypical adults (n=24) as they watched videos featuring the torso of a speaker producing sentences in which a word in a specific grammatical role (patient) was pitch accented. In one block, the speaker did not produce a beat gesture (no beat condition) in half of the sentences and produced a beat gesture concurrently with the pitch accented word (synchronous beat condition) in the other half of the sentences. In the other block, the speaker produced a beat gesture 500 ms after the pitch accented word (asynchronous beat condition) in half of the sentences and produced a beat gesture concurrently with the pitch accented word (synchronous beat condition) in the other half of the sentences. No behavioral response was required during the task, and the N400 response was measured in relation to the onset of pitch accented words. In the channel showing the largest N400 response, trial-by-trial variability in amplitude was compared between the no beat and temporally asynchronous beat conditions to examine the impact of temporal asynchrony on the N400 response. As in prior work, variability of N400 responses was computed using two measures: median amplitude of deviation (MAD) and inter-trial phase coherence (ITPC) in the alpha range (11 Hz). Across trials, MAD of N400 response amplitude was greater in the temporally asynchronous beat condition than the no beat condition (t=2.88, p=.004). In addition, across time points, ITPC at 11 Hz was greater in the no beat condition than in the asynchronous beat condition, (t=53.81, p<.001). These results were consistent with comparisons of N400 response variability in the asynchronous beat and no beat conditions with the synchronous beat condition. Together, these results indicate that trial-by-trial variability in the N400 response reflects temporal synchrony of beat gesture and pitch accenting. In particular, they provide evidence of greater variability in N400 responses to beat gestures temporally asynchronous with pitch accents than to beat gestures absent from pitch accents. These results show that N400 response variability is a potential indicator of temporal processing in language comprehension that may be able to serve as a biomarker of impairment in communication disorders in which cross-modal integration is abnormal, such as autism spectrum disorders.

Topic Area: Signed Language and Gesture

Back to Poster Schedule