Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Do listeners use speakers’ iconic hand gestures to predict upcoming words?
There is a Poster PDF for this presentation, but you must be a current member or registered to attend SNL 2023 to view it. Please go to your Account Home page to register.
Poster D14 in Poster Session D, Wednesday, October 25, 4:45 - 6:30 pm CEST, Espace Vieux-Port
Marlijn ter Bekke1,2, Linda Drijvers1,2, Judith Holler1,2; 1Donders Institute for Brain, Cognition and Behaviour, 2Max Planck Institute for Psycholinguistics
During face-to-face conversation, people rapidly take turns talking and use hand gestures that depict semantic meaning in addition to speech. In this pre-registered EEG study, we investigated whether listeners use speakers’ hand gestures to predict upcoming words. Participants listened to questions asked by a virtual avatar. Each question was accompanied by an iconic gesture (or control self-adaptor movement) that preceded a short silent pause and a target word. During the pause, participants showed stronger alpha and beta desynchronization in the Gesture versus the Adaptor condition, which have been reported as markers of anticipation. Moreover, gestures facilitated semantic processing of target words, as shown by less negative N400 amplitudes. A Cloze test with separate participants showed that seeing the gestures improved explicit predictions of the target words. However, how much each gesture improved predictions in the Cloze test was not related to the alpha and beta desynchronization in the EEG experiment. Altogether, these results suggest that listeners can indeed use speakers’ iconic gesture to predict upcoming words, which may facilitate coordination during conversational turn-taking by enabling earlier response planning. However, the results also raise interesting questions about the extent to which pre-stimulus alpha and beta desynchronization reflect predictive processing, thus feeding into a topical debate in the field of language and cognitive neuroscience.
Topic Areas: Meaning: Discourse and Pragmatics,