ChatGPT as a Translation Engine: A Case Study on Japanese-English

Vincent Michael Sutanto, Giovanni Gatti De Giacomo, Toshiaki Nakazawa, Masaru Yamada
10/9/2025
cs.CL

No confident code implementation found for this paper.

Abstract

This study investigates ChatGPT for Japanese-English translation, exploring simple and enhanced prompts and comparing against commercially available translation engines. Performing both automatic and MQM-based human evaluations, we found that document-level translation outperforms sentence-level translation for ChatGPT. On the other hand, we were not able to determine if enhanced prompts performed better than simple prompts in our experiments. We also discovered that ChatGPT-3.5 was preferred by automatic evaluation, but a tradeoff exists between accuracy (ChatGPT-3.5) and fluency (ChatGPT-4). Lastly, ChatGPT yields competitive results against two widely-known translation systems.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish

88% match2025

What is the Best Way for ChatGPT to Translate Poetry?

87% match2024

Prompting ChatGPT for Translation: A Comparative Analysis of Translation Brief and Persona Prompts

87% match2024

Gradable ChatGPT Translation Evaluation

86% match2024

ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions

84% match2024

ChatGPT vs Human-authored Text: Insights into Controllable Text Summarization and Sentence Style Transfer

83% match2023

Cite this paper

@article{sutanto2025chatgpt,
  title  = {ChatGPT as a Translation Engine: A Case Study on Japanese-English},
  author = {Vincent Michael Sutanto and Giovanni Gatti De Giacomo and Toshiaki Nakazawa and Masaru Yamada},
  year   = {2025},
  eprint = {2510.08042},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.08042v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.08042.svg)](https://codeofpaper.com/paper/2510.08042)

Discussion