Poster B22, Thursday, August 16, 3:05 – 4:50 pm, Room 2000AB
fMRI evidence for binding theory during anaphora resolution in naturalistic listening
Jixing Li1, Murielle Fabre1, Wen-Ming Luh1, John Hale1;1Cornell University
INTRODUCTION Within generative grammar, syntactic constraints have been identified that restrict possible pronoun-antecedent relationships. For instance in Chomsky’s (1981) Binding Theory, personal pronouns must not be bound by an antecedent within the same clause. However, debates persist over the precise role that binding theory play in the cognitive process of pronoun resolution. The current study compared brain activity associated with a syntax-sensitive computational model and a neural network model for pronoun resolution while participants listened to an audiobook during fMRI recording. The syntax-sensitive Hobbs algorithm (Hobbs, 1977) follows binding theory and implements gender/number matching, whereas the neural network model (Clark & Manning, 2016) encodes a set of semantic and discourse-level features with no explicit use of syntactic structures. The results revealed larger clusters associated with the Hobbs algorithm in the left Broca’s area and the bilateral Angular Gyrus--a network that has been reported in the neuroimaging literature for morpho-syntactic processing during anaphora resolution (e.g., Hammer et al., 2007; Matchin et al., 2014; Santi & Grodzinsky, 2012). METHODS 49 English speakers (30 female, mean age=21.3) listened to an audiobook version of "The Little Prince" for about 100 minutes. BOLD functional scans were acquired using a multi-echo planar imaging (ME-EPI) sequence with online reconstruction (TR=2000 ms; TE’s=12.8, 27.5, 43 ms; FA=77; matrix size=72x72; FOV=240.0x240.0 mm; 2 x image acceleration; 33 axial slices, voxel size=3.75x3.75x3.8 mm). Preprocessing was carried out with AFNI and ME-ICA (Kundu et al., 2011). The audiobook contains 277 third person pronouns that are correctly predicted by the Hobbs algorithm. For each of the 277 third person pronouns, we computed the Hobbs distance, namely the number of noun phrases that the algorithm skips while searching for the antecedent. Our hypothesis is that a higher Hobbs distance yields a processing effort due to syntactic and morphological constraints. We also computed the coreference score between the pronoun and its antecedent using the neural network model trained on the CoNLL-2012 Shared Task data (Pradhan et al., 2012). The negative of this score was taken to index processing difficulty due to semantic and discoursal factors. The observed BOLD signal was modeled by the two difficulty measures for pronoun resolution time-locked at the offsets of 277 third person pronouns in the audiobook. We also included a binary regressor which marks the presence of the third person pronouns, and three control regressors: RMS intensity at every 10 ms of the audio; word rate at the offset of each word, and log-frequency of each word in the unigram set from Google ngrams. RESULTS The Hobbs algorithm significantly correlates with activity in the left Precuneus, the bilateral Angular Gyrus, the left Inferior Frontal Gyrus and the left Inferior Temporal Gyrus. The neural network model only shows marginally significant activation in the right Superior Temporal Gyrus (p<0.05 FWE, k>=50). CONCLUSION Anaphora resolution modeled by the Hobbs algorithm is supported at the brain level in a network including the left Broca’s area, thus suggesting that the human parser attributes a central role to syntactic and morphological information during pronoun resolution.
Topic Area: Grammar: Syntax