Coreference as an indicator of context scope in multimodal narrative

Nikolai Ilinykh, Shalom Lappin, Asad Sayeed, Sharid Loáiciga
3/7/2025
cs.CL

Official code: GU-CLASP/coreference-context-scope (0 stars, MIT, last commit 2025-09-19).

Abstract

We demonstrate that large multimodal language models differ substantially from humans in the distribution of coreferential expressions in a visual storytelling task. We introduce a number of metrics to quantify the characteristics of coreferential patterns in both human- and machine-written texts. Humans distribute coreferential expressions in a way that maintains consistency across texts and images, interleaving references to different entities in a highly varied way. Machines are less able to track mixed references, despite achieving perceived improvements in generation quality. Materials, metrics, and code for our study are available at https://github.com/GU-CLASP/coreference-context-scope.

View on arXivView PDF

Code Implementations(1)

GU-CLASP/coreference-context-scopeOfficial100%
MIT

Code for the paper "Coreference as an indicator of context scope in multimodal narrative" (GEM2 @ ACL 2025).

00PythonJun 19, 20251 years agoMIT

Similar Papers

Semi-supervised multimodal coreference resolution in image narrations

84% match1 repo62023

Humans vs Vision-Language Models: A Unified Measure of Narrative Coherence

80% match1 repo2026

Enhancing Coreference Resolution with Pretrained Language Models: Bridging the Gap Between Syntax and Semantics

80% match2025

Are Large Language Models Robust Coreference Resolvers?

80% match1 repo72023

Disambiguating Reference in Visually Grounded Dialogues through Joint Modeling of Textual and Multimodal Semantic Structures

80% match1 repo32025

Multimodal Coreference Resolution for Chinese Social Media Dialogues: Dataset and Benchmark Approach

80% match2025

Cite this paper

@article{ilinykh2025coreference,
  title  = {Coreference as an indicator of context scope in multimodal narrative},
  author = {Nikolai Ilinykh and Shalom Lappin and Asad Sayeed and Sharid Loáiciga},
  year   = {2025},
  eprint = {2503.05298},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2503.05298v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2503.05298.svg)](https://codeofpaper.com/paper/2503.05298)

Discussion