Computational Sentence-level Metrics Predicting Human Sentence Comprehension

Kun Sun, Rong Wang
3/23/2024
cs.CLstat.ML

No official code found. Best implementation: fivehills/Sentence-level-metrics (0 stars, last commit 2024-06-28).

Abstract

The majority of research in computational psycholinguistics has concentrated on the processing of words. This study introduces innovative methods for computing sentence-level metrics using multilingual large language models. The metrics developed sentence surprisal and sentence relevance and then are tested and compared to validate whether they can predict how humans comprehend sentences as a whole across languages. These metrics offer significant interpretability and achieve high accuracy in predicting human sentence reading speeds. Our results indicate that these computational sentence-level metrics are exceptionally effective at predicting and elucidating the processing difficulties encountered by readers in comprehending sentences as a whole across a variety of languages. Their impressive performance and generalization capabilities provide a promising avenue for future research in integrating LLMs and cognitive science.

View on arXivView PDF

Code Implementations(1)

fivehills/Sentence-level-metricsCommunity100%

LLM-based sentence-level metrics predicting human sentence processing

01Jan 29, 20242 years ago

Similar Papers

To Words and Beyond: Probing Large Language Models for Sentence-Level Psycholinguistic Norms of Memorability and Reading Times

87% match2026

Vectors from Larger Language Models Predict Human Reading Time and fMRI Data More Poorly when Dimensionality Expansion is Controlled

86% match2025

Comparing Human and Language Models Sentence Processing Difficulties on Complex Structures

85% match2025

To model human linguistic prediction, make LLMs less superhuman

85% match2025

Multilingual Large Language Models do not comprehend all natural languages to equal degrees

83% match2026

Beyond Accuracy: Characterizing Code Comprehension Capabilities in (Large) Language Models

83% match2026

Cite this paper

@article{sun2024computational,
  title  = {Computational Sentence-level Metrics Predicting Human Sentence Comprehension},
  author = {Kun Sun and Rong Wang},
  year   = {2024},
  eprint = {2403.15822},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.15822v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.15822.svg)](https://codeofpaper.com/paper/2403.15822)

Discussion