My Account

Poster C3, Wednesday, August 21, 2019, 10:45 am – 12:30 pm, Restaurant Hall

Gesture-Speech Integration in the Adolescent Brain

Salomi Asaridou1, Özlem Ece Demir-Lira2, Susan Goldin-Meadow3, Steven L. Small4;1University of California, Irvine, 2The University of Iowa, 3The University of Chicago, 4The University of Texas at Dallas

Speakers frequently use gesture to disambiguate speech in everyday communication. In adult listeners, the left inferior frontal gyrus (IFG) and the left posterior middle temporal gyrus (pMTG) respond more strongly when co-speech gesture provides information missing in speech as opposed to redundant information already present in speech. We have shown in previous work that preadolescent children activate a wider network of areas than adults for the same contrast, including the bilateral IFG, the right MTG and left superior temporal gyrus (Demir-Lira et al., 2018). Based on these findings, we hypothesized that developmental changes in fronto-temporal white matter networks will lead to increasingly adult‐like specialized and lateralized activation during gesture–speech integration. In the present longitudinal study we set out to test this hypothesis by looking at gesture-speech integration in adolescent children using the same paradigm used in adults (Dick et al., 2014) and preadolescent children. We collected fMRI data from 29 adolescents (mean age=13y;6m ±10m) while they watched videos of an actor narrating short stories. The stories could vary as a function of semantic ambiguity in speech (AMBIGUOUS, UNAMBIGUOUS) and use of gesture (GESTURE, NoGESTURE) resulting in four conditions: 1) UNAMBIGUOUS+GESTURE, in which both speech and gesture contained specific information relevant to the story (e.g., a story about a pet bird accompanied by a flapping gesture); 2) UNAMBIGUOUS+NoGESTURE, in which the speech contained specific information (e.g., a story about a pet bird); 3) AMBIGUOUS+GESTURE, in which the gesture contained specific information that was missing in speech (e.g. a story about a pet accompanied by a flapping gesture); 4) AMBIGUOUS+NoGESTURE, in which speech was non-specific with no disambiguating gesture present (e.g., a story about a pet). We assessed participants’ attention to the task with a post-scan recognition test. We also collected diffusion MRI data and performed tractography to identify white matter fronto-temporal connections. For the crucial comparison, we found that in adolescent children, the left IFG (pars triangularis) was significantly more active when gesture added information missing in speech than when it provided redundant information (AMBIGUOUS+GESTURE > UNAMBIGUOUS+GESTURE). Activity in this area significantly correlated with post-scan recognition performance. Processing co-speech gestures (AMBIGUOUS+GESTURE > UNAMBIGUOUS+NoGESTURE and AMBIGUOUS+GESTURE > AMBIGUOUS+NoGESTURE) elicited significantly higher activity in the pMTG bilaterally. Interestingly, fractional anisotropy along the tract connecting the left IFG with the left pMTG positively correlated with post-scan accuracy. Our results are in line with our developmental predictions: Gesture-speech integration in adolescents engages a tighter network of areas, which includes the left IFG and the bilateral pMTG. With respect to frontal contributions to co-speech gesture processing, adolescents show increased activity in the left IFG when gesture disambiguated speech, resembling the pattern seen in adults. With respect to temporal contributions, unlike left lateralized activity in adults and right lateralized activity in children, adolescents show higher bilateral pMTG activation when processing gesture that disambiguates speech compared to speech without gesture. We posit that the strengthening of white matter connectivity between the left IFG and pMTG with development is partly responsible for the more adult-like activation patterns in adolescents.

Themes: Development, Signed Language and Gesture
Method: Functional Imaging

Back