What Makes Math Word Problems Challenging for LLMs?

KV Aditya Srivatsa, Ekaterina Kochmar
3/17/2024
cs.CL

No official code found. Best implementation: kvadityasrivatsa/analyzing-llms-for-mwps (0 stars, MIT, last commit 2024-12-25).

Abstract

This paper investigates the question of what makes math word problems (MWPs) in English challenging for large language models (LLMs). We conduct an in-depth analysis of the key linguistic and mathematical characteristics of MWPs. In addition, we train feature-based classifiers to better understand the impact of each feature on the overall difficulty of MWPs for prominent LLMs and investigate whether this helps predict how well LLMs fare against specific categories of MWPs.

View on arXivView PDF

Code Implementations(1)

kvadityasrivatsa/analyzing-llms-for-mwpsCommunity100%
MIT

This repo contains the data, code, and analysis pertaining to our paper: What Makes Math Word Problems Challenging for LLMs?

00Python, Jupyter NotebookMar 17, 20241 years agoMIT

Similar Papers

Large Language Models Don't Make Sense of Word Problems. A Scoping Review from a Mathematics Education Perspective

88% match2025

Cutting Through the Noise: Boosting LLM Performance on Math Word Problems

87% match1 repo52024

Improving Math Problem Solving in Large Language Models Through Categorization and Strategy Tailoring

87% match2024

Elementary Math Word Problem Generation using Large Language Models

87% match2025

Can LLMs Solve longer Math Word Problems Better?

87% match1 repo22026

Can LLMs Solve longer Math Word Problems Better?

87% match1 repo22024

Cite this paper

@article{srivatsa2024what,
  title  = {What Makes Math Word Problems Challenging for LLMs?},
  author = {KV Aditya Srivatsa and Ekaterina Kochmar},
  year   = {2024},
  eprint = {2403.11369},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.11369v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.11369.svg)](https://codeofpaper.com/paper/2403.11369)

Discussion