Presentation

Athena Willis¹, Lorna Quandt¹; ¹Gallaudet University

Recent research shows that deaf signers with early exposure to a signed language (SL) show increased behavioral and neural sensitivity to certain types of movement, such as biological motion, human actions, and signing avatars. Their early exposure to SL experience also interacts with the experience of being deaf, which may itself lead to improved understanding and perception of human movements. However, the neural mechanism behind these within- and cross-domain enhancements to human movement perception and understanding in deaf signers remain unclear. One possibility is that deaf signers’ lifelong experience with an SL leads to changes in their ongoing and automatic predictive processing of movements during perception. The perception and understanding of human movement have been found to be mediated in part by the mirror mechanism, a neural function that recruits the sensorimotor cortices to understand others’ actions. Research has shown that the experience of being deaf along with and early SL exposure lead to more negative behavioral responses toward uncanny signing avatars’ movements. However, we do not yet known which neural mechanism leads to this increased sensitivity to uncanny signing movements. One possibility is the mirror mechanism, a sensorimotor mechanism that has been previously implied in the uncanny valley, perception, and understanding of human movement. We hypothesize that in deaf signers who had early exposure to SL, their perception of uncanny signing avatar’s movement will lead to weaker automatic predictive processing by their mirror mechanism. To understand the effect of age of ASL exposure on deaf signers’ perception of signed word movement, we created stimuli that vary in two ways. Four signers differ in their movements (familiar/unfamiliar signing movements) and form (human/avatar). Those four signers are ASL (American Sign Language) Human, German Sign Language Human, Motion Capture (Mocap) Avatar, and Computer Synthesized (CS) Avatar. Mocap Avatar and CS Avatar both are signing ASL words, but the familiarity of their signed word movement differs. Mocap Avatar is animated with familiar movement of a human signing, while the CS avatar produces unfamiliar signing movements through synthetic language production. Both Humans provide with a control for the appearance of human/ avatar form on the perception of familiar and unfamiliar movement, along with any possible non-signed differences such as body movements during signing. We plan to collect behavioral and EEG data from signers with an early exposure (N = ~25) and late exposure to ASL (N = ~25) as they watch individual signed words. To measure participants’ familiarity with the signing movements, we will collect their rating of each signers’ familiarity, comprehension, naturalness, and appearance before and after the EEG experiment. Prior research has shown that mu rhythm desynchronization is modulated by the sensorimotor activity and also familiarity during perception. We will conduct time-frequency analysis on theta (4 - 7 Hz), alpha (8 - 12 Hz), and beta (13 - 30 Hz) EEG oscillations within the first 2500 ms of stimuli. This pre-registered EEG study will examine whether early exposure to SL lead to delayed sensorimotor EEG rhythms during perception of uncanny signing avatars.

Topic Areas: Signed Language and Gesture, Perception: Speech Perception and Audiovisual Integration