Slide Slam E2
Tracking the time-course of sign recognition using ERP repetition priming
Karen Emmorey1, Katherine Midgley1, Phillip Holcomb1; 1San Diego State University
An important distinction between signs and spoken words is that the linguistic articulators for speech are largely hidden from view, but are directly observable for sign language. This difference has direct consequences for how individual words and signs are perceived when presented in isolation. For example, the movement of the tongue in the word “gold” is not visible and not perceptible auditorily before voicing begins. Thus, the onset of an audio clip of the word “gold” and the onset of the stimulus word are identical. In contrast, the onset of a video clip of the sign GOLD from American Sign Language (ASL) typically begins with the sign model’s hands at rest and then her hand transitions to the target location of the sign at the cheek. Thus, the onset of the stimulus video and the onset of the sign are not identical. Further, this transitional movement may contain useful information about sign identity (e.g., early cues to handedness and hand configuration). The visibility and dynamic nature of the sign articulators creates a linguistic signal that is distinct from both auditorily and visually presented words. In this study, we used repetition priming and event-related potentials (ERPs) to investigate the time course of sign recognition in deaf ASL signers. Signers performed a go/no-go semantic categorization task to rare probe signs referring to people; critical target items were repeated and unrelated signs. In Experiment 1, ERPs were time-locked either to the onset of the video or to sign onset within the video; in Experiment 2, the same full videos were clipped so that video and sign onset were aligned (removing transitional movements), and ERPs were time-locked to video/sign onset. All analyses revealed an N400 repetition priming effect (less negativity for repeated than unrelated signs), but differed in the timing and/or duration of the N400 effect. Results from Experiment 1 revealed that repetition priming effects began before sign onset within a video, suggesting that signers are sensitive to linguistic information within the transitional movement to sign onset. The timing and duration of the N400 for clipped videos was more parallel to that observed previously for auditorily-presented words and was 200 ms shorter than either time-locking analysis from Experiment 1. We conclude that time-locking to full video onset is optimal when early ERP components or sensitivity to transitional movements are of interest and that time-locking to the onset of clipped videos is optimal for priming studies with fluent signers.