Document-Level Neural Machine Translation with Hierarchical Attention Networks

Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, James Henderson
9/5/2018
cs.CL

Official code: idiap/HAN_NMT (67 stars, GPL-3.0, last commit 2022-05-09).

Abstract

Neural Machine Translation (NMT) can be improved by including document-level contextual information. For this purpose, we propose a hierarchical attention model to capture the context in a structured and dynamic manner. The model is integrated in the original NMT architecture as another level of abstraction, conditioning on the NMT model's own previous hidden states. Experiments show that hierarchical attention significantly improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods, and that both the encoder and decoder benefit from context in complementary ways.

View on arXivView PDF

Code Implementations(1)

idiap/HAN_NMTOfficial100%
GPL-3.0

Document-Level Neural Machine Translation with Hierarchical Attention Networks

6719Shell, PythonSep 21, 20184 years agoGPL-3.0

Similar Papers

Bidirectional Attention Flow for Machine Comprehension

83% match1 repo1.5k2016

Brain-Inspired Stepwise Patch Merging for Vision Transformers

82% match1 repo42024

Towards In-context Scene Understanding

81% match2023

Memory Attention

81% match2026

Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use

81% match1 repo82023

Scaling Parameter and Context in Attention: Native Sparse Attention from Mixture-of-Head

81% match2026

Cite this paper

@article{miculicich2018documentlevel,
  title  = {Document-Level Neural Machine Translation with Hierarchical Attention Networks},
  author = {Lesly Miculicich and Dhananjay Ram and Nikolaos Pappas and James Henderson},
  year   = {2018},
  eprint = {1809.01576},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1809.01576}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1809.01576.svg)](https://codeofpaper.com/paper/1809.01576)

Discussion