GPT Can Solve Mathematical Problems Without a Calculator

Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He +3 more
9/6/2023
cs.LGcs.AIcs.CL

Official code: THUDM/MathGLM (328 stars, last commit 2023-11-20).

Abstract

Previous studies have typically assumed that large language models are unable to accurately perform arithmetic operations, particularly multiplication of >8 digits, and operations involving decimals and fractions, without the use of calculator tools. This paper aims to challenge this misconception. With sufficient training data, a 2 billion-parameter language model can accurately perform multi-digit arithmetic operations with almost 100% accuracy without data leakage, significantly surpassing GPT-4 (whose multi-digit multiplication accuracy is only 4.3%). We also demonstrate that our MathGLM, fine-tuned from GLM-10B on a dataset with additional multi-step arithmetic operations and math problems described in text, achieves similar performance to GPT-4 on a 5,000-samples Chinese math problem test set. Our code and data are public at https://github.com/THUDM/MathGLM.

View on arXivView PDF

Code Implementations(1)

THUDM/MathGLMOfficial100%

Official Pytorch Implementation for MathGLM

32827Shell, PythonSep 11, 20232 years ago

Similar Papers

Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks

88% match2024

How well do Large Language Models perform in Arithmetic tasks?

87% match1 repo572023

TinyGSM: achieving >80% on GSM8k with small language models

85% match2023

Calc-X and Calcformers: Empowering Arithmetical Chain-of-Thought through Interaction with Symbolic Systems

85% match1 repo92023

Exposing Numeracy Gaps: A Benchmark to Evaluate Fundamental Numerical Abilities in Large Language Models

84% match1 repo302025

LibAMM: Empirical Insights into Approximate Computing for Accelerating Matrix Multiplication

84% match2026

Cite this paper

@article{yang2023solve,
  title  = {GPT Can Solve Mathematical Problems Without a Calculator},
  author = {Zhen Yang and Ming Ding and Qingsong Lv and Zhihuan Jiang and Zehai He and Yuyi Guo and Jinfeng Bai and Jie Tang},
  year   = {2023},
  eprint = {2309.03241},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.03241v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.03241.svg)](https://codeofpaper.com/paper/2309.03241)

Discussion