Constituency Parsing with a Self-Attentive Encoder

Nikita Kitaev, Dan Klein
5/2/2018
cs.CL

Official code: nikitakit/self-attentive-parser (913 stars, MIT, last commit 2022-01-10).

Abstract

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser. The use of attention makes explicit the manner in which information is propagated between different locations in the sentence, which we use to both analyze our model and propose potential improvements. For example, we find that separating positional and content information in the encoder can lead to improved parsing accuracy. Additionally, we evaluate different approaches for lexical representation. Our parser achieves new state-of-the-art results for single models trained on the Penn Treebank: 93.55 F1 without the use of any external data, and 95.13 F1 when using pre-trained word representations. Our parser also outperforms the previous best-published accuracy figures on 8 of the 9 languages in the SPMRL dataset.

View on arXivView PDF

Code Implementations(1)

nikitakit/self-attentive-parserOfficial100%
MIT

High-accuracy NLP parser with models for 11 languages.

913155C, PerlApr 27, 20184 years agoMIT
aimachine-learningnatural-language-processingnlpparser+1 more

Similar Papers

Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

83% match1 repo142019

Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

81% match1 repo892019

Multilingual Constituency Parsing with Self-Attention and Pre-Training

81% match1 repo9132018

Maximum Bayes Smatch Ensemble Distillation for AMR Parsing

81% match2 repos2782021

Rethinking Self-Attention: Towards Interpretability in Neural Parsing

80% match1 repo1452019

Constituency Parsing using LLMs

80% match2023

Cite this paper

@article{kitaev2018constituency,
  title  = {Constituency Parsing with a Self-Attentive Encoder},
  author = {Nikita Kitaev and Dan Klein},
  year   = {2018},
  eprint = {1805.01052},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1805.01052}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1805.01052.svg)](https://codeofpaper.com/paper/1805.01052)

Discussion