On the robustness of ChatGPT in teaching Korean Mathematics

Phuong-Nam Nguyen, Quang Nguyen-The, An Vu-Minh, Diep-Anh Nguyen, Xuan-Lam Pham
2/17/2025
cs.AImath.HO

No confident code implementation found for this paper.

Abstract

ChatGPT, an Artificial Intelligence model, has the potential to revolutionize education. However, its effectiveness in solving non-English questions remains uncertain. This study evaluates ChatGPT's robustness using 586 Korean mathematics questions. ChatGPT achieves 66.72% accuracy, correctly answering 391 out of 586 questions. We also assess its ability to rate mathematics questions based on eleven criteria and perform a topic analysis. Our findings show that ChatGPT's ratings align with educational theory and test-taker perspectives. While ChatGPT performs well in question classification, it struggles with non-English contexts, highlighting areas for improvement. Future research should address linguistic biases and enhance accuracy across diverse languages. Domain-specific optimizations and multilingual training could improve ChatGPT's role in personalized education.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Can LLMs Master Math? Investigating Large Language Models on Math Stack Exchange

85% match1 repo42024

Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy

85% match2024

MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning

84% match2025

Is Mathematical Problem-Solving Expertise in Large Language Models Associated with Assessment Performance?

83% match2026

Predicting ChatGPT Use in Assignments: Implications for AI-Aware Assessment Design

83% match2025

Large Language Models for Education and Research: An Empirical and User Survey-based Analysis

83% match2025

Cite this paper

@article{nguyen2025robustness,
  title  = {On the robustness of ChatGPT in teaching Korean Mathematics},
  author = {Phuong-Nam Nguyen and Quang Nguyen-The and An Vu-Minh and Diep-Anh Nguyen and Xuan-Lam Pham},
  year   = {2025},
  eprint = {2502.11915},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.11915v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.11915.svg)](https://codeofpaper.com/paper/2502.11915)

Discussion