GECToR -- Grammatical Error Correction: Tag, Not Rewrite

Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, Oleksandr Skurzhanskyi
5/26/2020
cs.CLcs.LG

Official code: grammarly/gector (975 stars, Apache-2.0, last commit 2024-05-21).

Abstract

In this paper, we present a simple and efficient GEC sequence tagger using a Transformer encoder. Our system is pre-trained on synthetic data and then fine-tuned in two stages: first on errorful corpora, and second on a combination of errorful and error-free parallel corpora. We design custom token-level transformations to map input tokens to target corrections. Our best single-model/ensemble GEC tagger achieves an $F_{0.5}$ of 65.3/66.5 on CoNLL-2014 (test) and $F_{0.5}$ of 72.4/73.6 on BEA-2019 (test). Its inference speed is up to 10 times as fast as a Transformer-based seq2seq GEC system. The code and trained models are publicly available.

View on arXivView PDF

Code Implementations(1)

grammarly/gectorOfficial100%
Apache-2.0

Official implementation of the papers "GECToR – Grammatical Error Correction: Tag, Not Rewrite" (BEA-20) and "Text Simplification by Tagging" (BEA-21)

975220PythonMay 21, 20202 years agoApache-2.0
bertgrammatical-error-correctionnatural-language-processingnlproberta+4 more

Similar Papers

Encode, Tag, Realize: High-Precision Text Editing

85% match1 repo6022019

An Extended Sequence Tagging Vocabulary for Grammatical Error Correction

82% match2 repos3.5k2023

Neural POS Tagging for the Low-Resource Saraiki

82% match2026

FNet: Mixing Tokens with Fourier Transforms

81% match1 repo39.0k2021

Felix: Flexible Text Editing Through Tagging and Insertion

81% match1 repo39.0k2020

Multi-head Sequence Tagging Model for Grammatical Error Correction

80% match2024

Cite this paper

@article{omelianchuk2020gector,
  title  = {GECToR -- Grammatical Error Correction: Tag, Not Rewrite},
  author = {Kostiantyn Omelianchuk and Vitaliy Atrasevych and Artem Chernodub and Oleksandr Skurzhanskyi},
  year   = {2020},
  eprint = {2005.12592},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2005.12592}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2005.12592.svg)](https://codeofpaper.com/paper/2005.12592)

Discussion