Cross-Language Assessment of Mathematical Capability of ChatGPT

Gargi Sathe, Aneesh Shamraj, Aditya Surve, Nahush Patil, Kumkum Saxena
5/18/2024
cs.CLcs.LG

No confident code implementation found for this paper.

Abstract

This paper presents an evaluation of the mathematical capability of ChatGPT across diverse languages like Hindi, Gujarati, and Marathi. ChatGPT, based on GPT-3.5 by OpenAI, has garnered significant attention for its natural language understanding and generation abilities. However, its performance in solving mathematical problems across multiple natural languages remains a comparatively unexplored area, especially in regional Indian languages. In this paper, we explore those capabilities as well as using chain-of-thought prompting to figure out if it increases the accuracy of responses as much as it does in the English language and provide insights into the current limitations.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Investigating the Effectiveness of ChatGPT in Mathematical Reasoning and Problem Solving: Evidence from the Vietnamese National High School Graduation Examination

83% match2023

Mathematical Capabilities of ChatGPT

83% match1 repo392023

Evaluating Telugu Proficiency in Large Language Models_ A Comparative Analysis of ChatGPT and Gemini

83% match2024

Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English

82% match1 repo22024

Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT

80% match1 repo222024

ChatGPT Prompting Cannot Estimate Predictive Uncertainty in High-Resource Languages

80% match2023

Cite this paper

@article{sathe2024crosslanguage,
  title  = {Cross-Language Assessment of Mathematical Capability of ChatGPT},
  author = {Gargi Sathe and Aneesh Shamraj and Aditya Surve and Nahush Patil and Kumkum Saxena},
  year   = {2024},
  eprint = {2405.11264},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2405.11264v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2405.11264.svg)](https://codeofpaper.com/paper/2405.11264)

Discussion