Span-based Joint Entity and Relation Extraction with Transformer Pre-training

Markus Eberts, Adrian Ulges
9/17/2019
cs.CLcs.LG

Official code: lavis-nlp/spert (713 stars, MIT, last commit 2024-02-01).

Abstract

We introduce SpERT, an attention model for span-based joint entity and relation extraction. Our key contribution is a light-weight reasoning on BERT embeddings, which features entity recognition and filtering, as well as relation classification with a localized, marker-free context representation. The model is trained using strong within-sentence negative samples, which are efficiently extracted in a single BERT pass. These aspects facilitate a search over all spans in the sentence. In ablation studies, we demonstrate the benefits of pre-training, strong negative sampling and localized context. Our model outperforms prior work by up to 2.6% F1 score on several datasets for joint entity and relation extraction.

View on arXivView PDF

Code Implementations(1)

lavis-nlp/spertOfficial100%
MIT

PyTorch code for SpERT: Span-based Entity and Relation Transformer

713149HTML, ShellSep 6, 20192 years agoMIT
joint-learningnamed-entity-recognitionrelation-extraction

Similar Papers

Dealing with negative samples with multi-task learning on span-based joint entity-relation extraction

88% match2023

SpanRE: Entities and Overlapping Relations Extraction Based on Spans and Entity Attention

86% match2023

Entity, Relation, and Event Extraction with Contextualized Span Representations

86% match1 repo5932019

CARE: Co-Attention Network for Joint Entity and Relation Extraction

85% match2023

Entity or Relation Embeddings? An Analysis of Encoding Strategies for Relation Extraction

84% match1 repo32023

Sentence-Level Relation Extraction via Contrastive Learning with Descriptive Relation Prompts

84% match2023

Cite this paper

@article{eberts2019spanbased,
  title  = {Span-based Joint Entity and Relation Extraction with Transformer Pre-training},
  author = {Markus Eberts and Adrian Ulges},
  year   = {2019},
  eprint = {1909.07755},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1909.07755}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1909.07755.svg)](https://codeofpaper.com/paper/1909.07755)

Discussion