Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

Jason Lee, Elman Mansimov, Kyunghyun Cho
2/19/2018
cs.LGcs.CLstat.ML

Official code: nyu-dl/dl4mt-nonauto (120 stars, BSD-3-Clause, last commit 2019-02-20).

Abstract

We propose a conditional non-autoregressive neural sequence model based on iterative refinement. The proposed model is designed based on the principles of latent variable models and denoising autoencoders, and is generally applicable to any sequence generation task. We extensively evaluate the proposed model on machine translation (En-De and En-Ro) and image caption generation, and observe that it significantly speeds up decoding while maintaining the generation quality comparable to the autoregressive counterpart.

View on arXivView PDF

Code Implementations(1)

nyu-dl/dl4mt-nonautoOfficial100%
BSD-3-Clause
12017PythonFeb 19, 20187 years agoBSD-3-Clause

Similar Papers

Levenshtein Transformer

86% match1 repo32.2k2019

FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow

86% match1 repo2432019

AMOM: Adaptive Masking over Masking for Conditional Masked Language Model

83% match1 repo42023

RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformer

82% match2023

Non-Autoregressive Neural Machine Translation

82% match1 repo2712017

Non-autoregressive Machine Translation with Probabilistic Context-free Grammar

81% match1 repo122023

Cite this paper

@article{lee2018deterministic,
  title  = {Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement},
  author = {Jason Lee and Elman Mansimov and Kyunghyun Cho},
  year   = {2018},
  eprint = {1802.06901},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1802.06901}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1802.06901.svg)](https://codeofpaper.com/paper/1802.06901)

Discussion