My Account

Poster D48, Wednesday, August 21, 2019, 5:15 – 7:00 pm, Restaurant Hall

Anaphoric distance dependencies in the sequential structure of wordless visual narratives

Neil Cohn1;1Tilburg University

Language has long been characterized as a “unique” facet of human cognition, particularly because of complex characteristics like anaphoric relations and distance dependencies. However, recent work has argued that visual narratives of sequential images, like those in comics, use sequencing mechanisms analogous to syntax. Within this structure, visual narratives use “refiner” panels that “zoom in” on the contents of another panel in a metonymic relationship (part-whole), such as one panel showing a character (“A”) extending their hand, with a subsequent panel zooming-in on that hand (“a”). Similar to anaphora in language, refiners connect referential information in one unit (refiner, pronoun) with that of another unit, the “referring expression.” Also like in syntax, refiners can follow their referring expressions (anaphor) or precede them (cataphor). In addition, refiners can be presented both locally, or separated at a distance with intervening panels showing other characters. Crossing these traits creates four sequencing patterns, with anaphoric refiners following their antecedents either locally (AaB, where “A” and “B” represent images of single characters, and lowercase “a” represents a refiner of character A) or at a distance (ABa), and cataphoric refiners either local (aAB) or at a distance (aBA). We thus presented participants with these patterns (24 of each type) embedded within 6-panel long wordless visual narrative sequences, with the manipulation always occurring at positions 2, 3, and 4, depicting the “rising action” of the narrative. We measured event-related brain potentials (ERPs) to panels presented one at a time (1350ms duration, 300ms ISI) using a 32 channel BrainVision ActiChamp. We found that, at the final position of the 3-panel manipulation, distance dependencies evoked a late frontal negativity (ABa, aBA) compared to local dependencies (AaB, aAB), 400-1100ms, suggesting a cost for the distance connection, regardless of the information content (zoom “a”, and non-zoom “A”). This is consistent with late frontal negativities (Nref) to anaphoric relations in language processing. When looking at refiner panels alone (i.e., “a” panels across all patterns), cataphoric refiners (aAB, aBA) evoked larger widespread fronto-central N400s from 200-500ms than anaphoric refiners (AaB, ABa). While this attenuation by anaphoric refiners could have been caused by sequence position, with cataphoric refiners preceding the anaphoric ones, distant anaphoric refiners (ABa) also had a larger N400 than local anaphoric refiners (AaB). This suggested that accessing the meaning of zoomed-in information benefited from the repetition of immediately following its referring expression (local anaphoric: AaB) more than occurring before it (cataphoric: aAB, aBA) or separated at a distance (distant anaphoric: ABa). These findings demonstrate that phenomena characteristic of sentence structure—anaphora and distance dependencies—also manifest in visual narrative sequencing, and evoke similar neurocognitive responses as in language (Nref, N400). Such work raises questions about the domain-specificity of anaphora and distance-dependencies as linguistic representations, and the neurocognition of their concomitant processing.

Themes: Meaning: Discourse and Pragmatics, Meaning: Combinatorial Semantics
Method: Electrophysiology (MEG/EEG/ECOG)

Back