Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction

Hayeong Ryu, Sunhee Jo, Seunguk Yu, YoungBin Kim
9/14/2026
cs.CLcs.AI

Official code: hayeonggg/SURE (0 stars, last commit 2026-08-22).

Abstract

Grammatical error correction (GEC) evaluation has traditionally relied on reference or edit overlap, which can penalize valid rewrites that differ from gold corrections. Reference-free metrics reduce this dependence, but evaluating whether a fluent output is a valid correction of the source remains challenging. We propose SURE, a source-conditioned reward evaluator trained on within-source preferences spanning minimal-edit and rewrite-oriented corrections. SURE jointly learns an overall reward with criteria-level supervision for grammaticality, faithfulness, and fluency, together with span-level grounding for source-side error resolution. Experiments on SEEDA show that SURE performs competitively against strong baselines, with particular gains on rewrite-style corrections and more disentangled criteria-level diagnostics. Our code is available at https://github.com/hayeonggg/SURE.

View on arXivView PDF

Code Implementations(1)

hayeonggg/SUREOfficial100%

[EMNLP'26 Findings] Official Repository of "Don't Count the Edits, Judge by the Outcome Alone:Reward-Based Evaluation for Grammatical Error Correction"

00Aug 22, 20261 months ago

Similar Papers

CheckDistill: Adaptive Checklist Distillation for Efficient Distributional Reward Modeling in Image Editing

82% match2026

How Much Evidence Should a Coding Agent's Self-Correction Carry? Adaptive Dirichlet Evidence for Self-Distillation

82% match2026

Scalable Minimal-Change Learning for Controllable Image Editing

82% match1 repo2026

Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR

82% match1 repo2026

Evaluation of really good grammatical error correction

81% match1 repo22023

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

81% match2026

Cite this paper

@article{ryu2026dont,
  title  = {Don't Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction},
  author = {Hayeong Ryu and Sunhee Jo and Seunguk Yu and YoungBin Kim},
  year   = {2026},
  eprint = {2609.15559},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2609.15559v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2609.15559.svg)](https://codeofpaper.com/paper/2609.15559)

Discussion