Generating Wikipedia by Summarizing Long Sequences

Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi +2 more
1/30/2018
cs.CL

Official code: tensorflow/tensor2tensor (17,475 stars, Apache-2.0, last commit 2023-06-02).

Abstract

We show that generating English Wikipedia articles can be approached as a multi- document summarization of source documents. We use extractive summarization to coarsely identify salient information and a neural abstractive model to generate the article. For the abstractive model, we introduce a decoder-only architecture that can scalably attend to very long sequences, much longer than typical encoder- decoder architectures used in sequence transduction. We show that this model can generate fluent, coherent multi-sentence paragraphs and even whole Wikipedia articles. When given reference documents, we show it can extract relevant factual information as reflected in perplexity, ROUGE scores and human evaluations.

View on arXivView PDF

Code Implementations(1)

tensorflow/tensor2tensorOfficial100%
Apache-2.0

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

17,4753,800C++, HTMLJun 15, 20173 years agoApache-2.0
deep-learningmachine-learningmachine-translationreinforcement-learningtpu

Similar Papers

Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization

88% match1 repo3782018

Data-driven Summarization of Scientific Articles

87% match1 repo102018

Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting

87% match1 repo6242018

MeanSum: A Neural Model for Unsupervised Multi-document Abstractive Summarization

87% match1 repo1142018

SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders

86% match1 repo39.0k2019

Text Summarization with Pretrained Encoders

86% match1 repo1.3k2019

Cite this paper

@article{liu2018generating,
  title  = {Generating Wikipedia by Summarizing Long Sequences},
  author = {Peter J. Liu and Mohammad Saleh and Etienne Pot and Ben Goodrich and Ryan Sepassi and Lukasz Kaiser and Noam Shazeer},
  year   = {2018},
  eprint = {1801.10198},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1801.10198}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1801.10198.svg)](https://codeofpaper.com/paper/1801.10198)

Discussion