Factors Influencing the Surprising Instability of Word Embeddings

Laura Wendlandt, Jonathan K. Kummerfeld, Rada Mihalcea
4/25/2018
cs.CL

Official code: laura-burdick/embeddingStability (8 stars, last commit 2020-08-07).

Abstract

Despite the recent popularity of word embedding methods, there is only a small body of work exploring the limitations of these representations. In this paper, we consider one aspect of embedding spaces, namely their stability. We show that even relatively high frequency words (100-200 occurrences) are often unstable. We provide empirical evidence for how various factors contribute to the stability of word embeddings, and we analyze the effects of stability on downstream tasks.

View on arXivView PDF

Code Implementations(1)

laura-burdick/embeddingStabilityOfficial100%
83PythonMar 14, 20186 years ago

Similar Papers

On the Dimensionality of Word Embedding

82% match2 repos3272018

FRAGE: Frequency-Agnostic Word Representation

82% match1 repo1142018

Semantic Structure and Interpretability of Word Embeddings

80% match1 repo102017

On Dimensional Linguistic Properties of the Word Embedding Space

79% match1 repo82019

Examining the Limits of Word2Vec with Toki Pona

78% match2026

An empirical investigation into the properties of standard word embeddings

78% match2026

Cite this paper

@article{wendlandt2018factors,
  title  = {Factors Influencing the Surprising Instability of Word Embeddings},
  author = {Laura Wendlandt and Jonathan K. Kummerfeld and Rada Mihalcea},
  year   = {2018},
  eprint = {1804.09692},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1804.09692}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1804.09692.svg)](https://codeofpaper.com/paper/1804.09692)

Discussion