Keynote Lecture: Asli Özyürek
Asli Özyürek
Multimodality as a design feature of human language: Insights from brain, behavior and diversity
Thursday, October 26, 2023, 9:00 – 10:00 am, Auditorium
Log In to play session recordingChair: Andrea E. Martin, Radboud University
Speaker: Asli Özyürek, Max Planck Institute for Psycholinguistics
Understanding how brain processes language requires certain assumptions about how we define language. Most theories and models about the nature of human language have been based on aspects of language that can be either spoken or written. For example, traditionally language has been defined uniquely as a system composed of uni-modal combinatorial units with arbitrary form-meaning mappings, and sequential organization, etc. These assumptions still drive LLMs (e.g., ChatGPT), many neuro and psycholinguistic studies, and proposed processing mechanisms in brain and cognition (e.g., prediction of upcoming words etc).
However a mostly ignored but universal aspect of language is that all languages of the world embody “visible” components; spoken languages are accompanied by manual and facial gestures and language can be expressed by signed languages as well as spoken languages. I will argue that taking into account such universally shared visible components and their modality specific characteristics such as visible iconicity and indexicality and simultaneity (e.g., saying “play” in speech and performing a piano playing gesture at the same time) question some of our assumptions about language structure, its use in interaction, (neural) processing, acquisition and evolution. To do so I will present evidence from interdisciplinary multimodal research (e.g., language production, eye tracking, imaging) conducted in diverse spoken and signed languages and with individuals who lack accessibility to different sensory experiences (e.g, deaf, blind). I will argue that a multimodal but not a unimodal view of language can explain its adaptive structure, resiliency and learnability in different contexts and cultures with individual variations.
About Asli Özyürek
Asli Özyürek, received a joint PhD in Linguistics and Psychology from the University of Chicago. She is currently the Director of Multimodal Language Department at the Max Planck Institute for Psycholinguistics and is a Professor at Radboud University. She is also a PI at Donders Institute for Brain Cognition and Behavior and an elected member of Academia Europea. She has received many prestigious grants from NSF, NIH, Dutch Science Foundation, ERC and Turkish Science Foundation. Özyürek investigates the inherently and universal multimodal nature of human language capacity as one of its adaptive design features. To do so she studies how brain supports multimodal language, how typologically different spoken and signed languages pattern their structures given their multimodal diversity, and how the learning constraints and communicative pressures of interaction shape multimodal language, its acquisition and evolution as an adaptive system. She uses a variety of methodologies such as behavioral and kinematic analyses of multimodal linguistic structures, eye tracking, machine learning and brain imaging to understand the complex multimodal nature of human language capacity.
