Supervised Learning of Universal Sentence Representations from Natural Language Inference Data

Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, Antoine Bordes
5/5/2017
cs.CL

Official code: facebookresearch/InferSent (2,281 stars, last commit 2021-08-30). 1 more implementation below.

Abstract

Many modern NLP systems rely on word embeddings, previously trained in an unsupervised manner on large corpora, as base features. Efforts to obtain embeddings for larger chunks of text, such as sentences, have however not been so successful. Several attempts at learning unsupervised representations of sentences have not reached satisfactory enough performance to be widely adopted. In this paper, we show how universal sentence representations trained using the supervised data of the Stanford Natural Language Inference datasets can consistently outperform unsupervised methods like SkipThought vectors on a wide range of transfer tasks. Much like how computer vision uses ImageNet to obtain features, which can then be transferred to other tasks, our work tends to indicate the suitability of natural language inference for transfer learning to other NLP tasks. Our encoder is publicly available.

View on arXivView PDF

Code Implementations(2)

facebookresearch/InferSentOfficial100%

InferSent sentence embeddings

2,281464Python, Jupyter NotebookMay 18, 20175 years agoNOASSERTION
facebookresearch/SentEvalOfficial100%

A python tool for evaluating the quality of sentence embeddings.

2,111306sed, ShellMay 18, 20172 years agoNOASSERTION

Similar Papers

Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

88% match2 repos2.1k2018

Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features

85% match1 repo1.2k2017

Extracting Text Representations for Terms and Phrases in Technical Domains

85% match2023

On the Dimensionality of Sentence Embeddings

84% match2023

What you can cram into a single vector: Probing sentence embeddings for linguistic properties

84% match1 repo2.1k2018

JCSE: Contrastive Learning of Japanese Sentence Embeddings and Its Applications

84% match1 repo82023

Cite this paper

@article{conneau2017supervised,
  title  = {Supervised Learning of Universal Sentence Representations from Natural Language Inference Data},
  author = {Alexis Conneau and Douwe Kiela and Holger Schwenk and Loic Barrault and Antoine Bordes},
  year   = {2017},
  eprint = {1705.02364},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1705.02364}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1705.02364.svg)](https://codeofpaper.com/paper/1705.02364)

Discussion