Poster Slam Session B, Tuesday, August 20, 2019, 3:00 – 3:15 pm, Finlandia Hall, Angela Grant
Constraint Asyntactic Structure of Ancient Chinese Poetry Facilitates Content Parsing: A study combining MEG, RNN, and crowdsourcing
Xiangbin Teng1, Min Ma2, Jinbiao Yang3,4,5, Stefan Blohm1, Qing Cai4,5, Xing Tian3,4,5;1Max Planck Institute for Empirical Aesthetics, Frankfurt, 2Google, Inc, 3Division of Arts and Sciences, New York University Shanghai, 4East China Normal University, Shanghai, 5NYU-ECNU Institute of Brain and Cognitive Science at New York University Shanghai
Poetic forms and genres are uniquely structured and only allow limited variations within a genre. This strict formal and thematic structure distinguishes poetry from everyday speech and raises an interesting question: how does this structural constraint affect poem appreciation? To comprehend continuous speech streams, listeners need to establish hierarchies of perceptual, linguistic and conceptual chunks online (Ding et al., 2015). Hence, the appreciation of poetry presupposes that listeners correctly segregate the poetic stream and construct linguistic and conceptual structures of increasing complexity. Apart from prosodic segmentation cues imposed by human speakers, parsing poems may be guided by the regularities of poetic structure - listeners’ prior knowledge of the formal and thematic constraints may function as a cognitive ‘template’ to actively group words and lines and to predict the unfolding of poems. Testing this hypothesis can reveal how listeners appreciate poetry and illuminate why poetic genres are stringently structured. More broadly, it can deepen our understanding of speech perception by testing whether listeners can capitalize on asyntactic structures to parse speech streams. Methods: We chose Jueju, a genre of Chinese ancient poems with the most stringent form, and used a recurrent neural network to create thousands of artificial poems. This procedure controlled the language complexity of the poems and listeners’ familiarity with the materials. We selected 150 artificial poems and collected 55982 behavioral ratings on those poems regarding linguistic and aesthetic aspects of poem appreciation through online crowdsourcing. Next, we presented those poems to 13 native Chinese speakers while conducting magnetoencephalography (MEG) recording. Each poem was presented twice as a sequence of isomorphous syllables with each syllable individually synthesized, so that acoustic indicators of poetic structures, such as prosodic cues and pauses, were excluded from the auditory stimuli. The syllabic rate was 3.33 Hz. During MEG data analysis, we conducted both frequency-tagging spectral analyses and temporal analyses of phase precession in both the sensor and source spaces of MEG recording. Results: We found that the participants could rely on their knowledge of Jueju to correctly segment each sentence in the novel artificial poems, which was reflected by the salient MEG frequency component around 0.64 Hz corresponding to the sentential rate of the poems. The 0.64 Hz component was localised primarily on the auditory and motor areas of the left hemisphere and positively correlated with the language complexity. When the participants heard the same poem the second time, such auditory-motor component decreased in magnitude, but a sustained temporal response emerged in the left pars triangularis. The phase series of MEG signals during the second presentation of the poems advanced faster than the first time – a phenomenon of phase precession (Lisman, 2005). Moreover, the behavioural ratings correlated negatively with the language complexity but positively with MEG signals between 4 to 5 Hz. Conclusion: Listeners employ an auditory-motor circuit to actively parse incoming unfamiliar poem/speech streams based on their knowledge of the poetic structure, followed by the engagement of language areas to establish linguistic hierarchies to better predict the incoming speech contents.
Themes: Speech Perception, Syntax
Method: Electrophysiology (MEG/EEG/ECOG)
Poster B71