Presentation
Search Abstracts | Symposia | Slide Sessions | Poster Sessions | Lightning Talks
Addressee gaze direction and response timing signal upcoming response preference: evidence from behavioral and EEG experiments
Poster C10 in Poster Session C, Wednesday, October 25, 10:15 am - 12:00 pm CEST, Espace Vieux-Port
This poster is part of the Sandbox Series.
Alexandra Emmendorfer1,2, Judith Holler1,2; 1Donders Institute for Brain, Cognition, and Behaviour, 2Max Planck Institute for Psycholinguistics
Speaker gaze is a prominent visual signal in face-to-face communication. Data from corpus analyses suggests that an addressee’s gaze direction may also be an early signal foreshadowing the extent to which a response aligns with the content of the question. Conversation analysis has shown that interactants prefer aligned over non-aligned responses, such as accepting an offer. A recent corpus study showed that dispreferred responses are more often preceded by gaze aversions (Kendrick & Holler, 2017), possibly to allow the questioner to reformulate the phrasing of their question in order to conform to the norms of preference organization in conversation. Psycholinguistic research has examined response timing as an additional early indicator of the upcoming response, with “yes” responses being associated with shorter gap times compared to “no” responses. Evidence from EEG indicates that interlocuters may be sensitive to these differences in gap times, with dispreferred (no) responses eliciting an N400 after short gaps but not long gaps (Bögels et al., 2015). The current research combines these two factors to examine how they influence perception of the upcoming responses in two experiments (online behavioral, EEG). Participants will be presented with 240 animated videos that represent short conversation fragments between two (photorealistic) avatars. The participants view the conversation from an over-the-shoulder perspective, as if they are standing behind one avatar (Questioner), facing the other (Responder). Participants are instructed to listen carefully to the conversation while observing the Responder at the center of the screen. At the end of each fragment, the Questioner asks a polar question. The question is followed by a pause (Short: 333 ms, Long: 1000 ms), during which the Responder either averts their gaze, or keeps it directed at the Questioner. After the pause, the Responder responds to the question with either “yes” or “no”. In the online behavioral experiment, the video ends before a response is given, and participants are asked to indicate on a scale from 1 to 6 what response they expect, where 1 indicates most likely No, and 6 indicates most likely Yes. We expect questions followed by long gaps and gaze aversions to receive lower scores compared to questions where the avatars maintain direct gaze, indicating that participants anticipate a “no” response. In the EEG experiment, participants will passively watch the conversations while their brain activity is measured with EEG. To monitor attention, 20% of clips will be followed by a comprehension question. Our analyses will focus on the N400 amplitude, measured as the difference between No and Yes responses. We hypothesize that both gap duration and gaze will modulate N400 response, with larger N400 amplitudes to responses with short gaps and direct gaze. We additionally explore whether gap duration and gaze direction influence the N400 interactively or additively. Data collection for the behavioral experiment is set to launch in June, for the EEG experiment in September. We anticipate to present results from the behavioral study as well has preliminary findings from the first batch of EEG participants at the 2023 meeting.
Topic Areas: Meaning: Discourse and Pragmatics, Multisensory or Sensorimotor Integration