Multimodal Word Distributions

Ben Athiwaratkun, Andrew Gordon Wilson
4/27/2017
stat.MLcs.AIcs.CL

Official code: benathi/word2gm (284 stars, BSD-3-Clause, last commit 2019-07-08).

Abstract

Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information. To learn these distributions, we propose an energy-based max-margin objective. We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.

View on arXivView PDF

Code Implementations(1)

benathi/word2gmOfficial100%
BSD-3-Clause

Word to Gaussian Mixture Model

28471C++, ShellMar 6, 20177 years agoBSD-3-Clause

Similar Papers

Hierarchical Density Order Embeddings

82% match1 repo332018

On the Dimensionality of Word Embedding

81% match2 repos3272018

From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models

79% match2024

Spoken Word2Vec: Learning Skipgram Embeddings from Speech

79% match1 repo42023

Context Mover's Distance & Barycenters: Optimal Transport of Contexts for Building Representations

78% match1 repo212018

Disentangling Latent Embeddings with Sparse Linear Concept Subspaces (SLiCS)

78% match2025

Cite this paper

@article{athiwaratkun2017multimodal,
  title  = {Multimodal Word Distributions},
  author = {Ben Athiwaratkun and Andrew Gordon Wilson},
  year   = {2017},
  eprint = {1704.08424},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1704.08424}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1704.08424.svg)](https://codeofpaper.com/paper/1704.08424)

Discussion