Dynamic Meta-Embeddings for Improved Sentence Representations

Douwe Kiela, Changhan Wang, Kyunghyun Cho
4/21/2018
cs.CL

Official code: facebookresearch/DME (333 stars, last commit 2020-09-25).

Abstract

While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves. To that end, we introduce dynamic meta-embeddings, a simple yet effective method for the supervised learning of embedding ensembles, which leads to state-of-the-art performance within the same model class on a variety of tasks. We subsequently show how the technique can be used to shed new light on the usage of word embeddings in NLP systems.

View on arXivView PDF

Code Implementations(1)

facebookresearch/DMEOfficial100%

Dynamic Meta-Embeddings for Improved Sentence Representations

33346PythonSep 18, 20186 years agoNOASSERTION

Similar Papers

How to Generate a Good Word Embedding?

84% match1 repo2382015

ITEm: Unsupervised Image-Text Embedding Learning for eCommerce

84% match2023

Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit

84% match2022

Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddings

83% match1 repo42023

Sentiment analysis in Tourism: Fine-tuning BERT or sentence embeddings concatenation?

83% match2023

Mimicking Word Embeddings using Subword RNNs

82% match1 repo1532017

Cite this paper

@article{kiela2018dynamic,
  title  = {Dynamic Meta-Embeddings for Improved Sentence Representations},
  author = {Douwe Kiela and Changhan Wang and Kyunghyun Cho},
  year   = {2018},
  eprint = {1804.07983},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1804.07983}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1804.07983.svg)](https://codeofpaper.com/paper/1804.07983)

Discussion