Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground

Adil Soubki, John Murzaku, Arash Yousefi Jordehi, Peter Zeng, Magdalena Markowska +2 more
3/4/2024
cs.CL

No confident code implementation found for this paper.

Abstract

Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting experiments with human behavior. We introduce the first ToM dataset based on naturally occurring spoken dialogs, Common-ToM, and show that LMs struggle to demonstrate ToM. We then show that integrating a simple, explicit representation of beliefs improves LM performance on Common-ToM.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models

89% match2024

Re-evaluating Theory of Mind evaluation in large language models

89% match2025

ToMBench: Benchmarking Theory of Mind in Large Language Models

88% match1 repo692024

GPT-4o Lacks Core Features of Theory of Mind

88% match2026

Theory of Mind in Large Language Models: Assessment and Enhancement

88% match2025

Do Theory of Mind Benchmarks Need Explicit Human-like Reasoning in Language Models?

88% match1 repo112025

Cite this paper

@article{soubki2024views,
  title  = {Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground},
  author = {Adil Soubki and John Murzaku and Arash Yousefi Jordehi and Peter Zeng and Magdalena Markowska and Seyed Abolghasem Mirroshandel and Owen Rambow},
  year   = {2024},
  eprint = {2403.02451},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.02451v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.02451.svg)](https://codeofpaper.com/paper/2403.02451)

Discussion