Are LLMs Good Cryptic Crossword Solvers?

Abdelrahman Sadallah, Daria Kotova, Ekaterina Kochmar
3/15/2024
cs.AIcs.CLcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Cryptic crosswords are puzzles that rely not only on general knowledge but also on the solver's ability to manipulate language on different levels and deal with various types of wordplay. Previous research suggests that solving such puzzles is a challenge even for modern NLP models. However, the abilities of large language models (LLMs) have not yet been tested on this task. In this paper, we establish the benchmark results for three popular LLMs -- LLaMA2, Mistral, and ChatGPT -- showing that their performance on this task is still far from that of humans.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

What Makes Cryptic Crosswords Challenging for LLMs?

96% match1 repo62024

Language Models are Crossword Solvers

91% match1 repo22024

UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?

83% match1 repo2025

Puzzle Solving using Reasoning of Large Language Models: A Survey

82% match2024

Large Language Models: A Survey

81% match2024

Linguistic Intelligence in Large Language Models for Telecommunications

81% match2024

Cite this paper

@article{sadallah2024llms,
  title  = {Are LLMs Good Cryptic Crossword Solvers?},
  author = {Abdelrahman Sadallah and Daria Kotova and Ekaterina Kochmar},
  year   = {2024},
  eprint = {2403.12094},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.12094v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.12094.svg)](https://codeofpaper.com/paper/2403.12094)

Discussion