Confidence through Attention

Matīss Rikters, Mark Fishel
10/10/2017
cs.CL

Official code: M4t1ss/ConfidenceThroughAttention (5 stars, last commit 2017-12-25).

Abstract

Attention distributions of the generated translations are a useful bi-product of attention-based recurrent neural network translation models and can be treated as soft alignments between the input and output tokens. In this work, we use attention distributions as a confidence metric for output translations. We present two strategies of using the attention distributions: filtering out bad translations from a large back-translated corpus, and selecting the best translation in a hybrid setup of two different translation systems. While manual evaluation indicated only a weak correlation between our confidence score and human judgments, the use-cases showed improvements of up to 2.22 BLEU points for filtering and 0.99 points for hybrid translation, tested on English<->German and English<->Latvian translation.

View on arXivView PDF

Code Implementations(1)

M4t1ss/ConfidenceThroughAttentionOfficial100%

Confidence Through Attention

51PHP, PythonJun 2, 20178 years ago
attentionattention-mechanismmachine-translationneural-machine-translationneural-networks

Similar Papers

Advancing Explainability in Neural Machine Translation: Analytical Metrics for Attention and Alignment Consistency

86% match1 repo2024

Attention Sinks in Massively Multilingual Neural Machine Translation:Discovery, Analysis, and Mitigation

83% match2026

BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation

83% match1 repo212023

Syntax-Aware Complex-Valued Neural Machine Translation

83% match2023

Rethinking Translation Memory Augmented Neural Machine Translation

82% match1 repo42023

Calibrating Translation Decoding with Quality Estimation on LLMs

82% match1 repo72025

Cite this paper

@article{rikters2017confidence,
  title  = {Confidence through Attention},
  author = {Matīss Rikters and Mark Fishel},
  year   = {2017},
  eprint = {1710.03743},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1710.03743}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1710.03743.svg)](https://codeofpaper.com/paper/1710.03743)

Discussion