Sparse Overcomplete Word Vector Representations

Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, Noah Smith
6/5/2015
cs.CL

Official code: mfaruqui/sparse-coding (57 stars, MIT, last commit 2017-10-01).

Abstract

Current distributed representations of words show little resemblance to theories of lexical semantics. The former are dense and uninterpretable, the latter largely based on familiar, discrete classes (e.g., supersenses) and relations (e.g., synonymy and hypernymy). We propose methods that transform word vectors into sparse (and optionally binary) vectors. The resulting representations are more similar to the interpretable features typically used in NLP, though they are discovered automatically from raw corpora. Because the vectors are highly sparse, they are computationally easy to work with. Most importantly, we find that they outperform the original vectors on benchmark tasks.

View on arXivView PDF

Code Implementations(1)

mfaruqui/sparse-codingOfficial100%
MIT

Parallel (asynchronous) sparse coding implementation for obtaining sparse overcomplete word vectors

5719C++, MakefileMar 25, 20159 years agoMIT

Similar Papers

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

83% match1 repo1342024

Interpretable Syntactic Representations Enable Hierarchical Word Vectors

83% match2024

Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec

82% match1 repo3.2k2016

Construction of a Japanese Word Similarity Dataset

82% match1 repo1032017

Efficient Estimation of Word Representations in Vector Space

82% match1 repo242013

WordRank: Learning Word Embeddings via Robust Ranking

82% match1 repo512015

Cite this paper

@article{faruqui2015sparse,
  title  = {Sparse Overcomplete Word Vector Representations},
  author = {Manaal Faruqui and Yulia Tsvetkov and Dani Yogatama and Chris Dyer and Noah Smith},
  year   = {2015},
  eprint = {1506.02004},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1506.02004}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1506.02004.svg)](https://codeofpaper.com/paper/1506.02004)

Discussion