Semantic Structure and Interpretability of Word Embeddings

Lutfi Kerem Senel, Ihsan Utlu, Veysel Yucesoy, Aykut Koc, Tolga Cukur
11/1/2017
cs.CL

Official code: avaapm/SEMCATdataset2018 (10 stars, last commit 2020-09-18).

Abstract

Dense word embeddings, which encode semantic meanings of words to low dimensional vector spaces have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word embeddings are substantially successful in capturing semantic relations among words, so a meaningful semantic structure must be present in the respective vector spaces. However, in many cases, this semantic structure is broadly and heterogeneously distributed across the embedding dimensions, which makes interpretation a big challenge. In this study, we propose a statistical method to uncover the latent semantic structure in the dense word embeddings. To perform our analysis we introduce a new dataset (SEMCAT) that contains more than 6500 words semantically grouped under 110 categories. We further propose a method to quantify the interpretability of the word embeddings; the proposed method is a practical alternative to the classical word intrusion test that requires human intervention.

View on arXivView PDF

Code Implementations(1)

avaapm/SEMCATdataset2018Official100%

This repository is to download the SEMCATdataset 2018 for the publication "Senel L. K., Utlu I., Yucesoy V., Koc A., Cukur T., Semantic Structure and Interpretability of Word Embeddings, IEEE/ACM Transactions on Audio, Speech, and Language Processing".

101Apr 5, 20186 years ago

Similar Papers

Imparting Interpretability to Word Embeddings while Preserving Semantic Structure

86% match2018

Examining the Limits of Word2Vec with Toki Pona

83% match2026

Joint Embedding of Words and Labels for Text Classification

82% match1 repo3232018

On Evaluation of Bangla Word Analogies

82% match2023

Finding Meaning in Embeddings: Concept Separation Curves

81% match2026

Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings

81% match1 repo2025

Cite this paper

@article{senel2017semantic,
  title  = {Semantic Structure and Interpretability of Word Embeddings},
  author = {Lutfi Kerem Senel and Ihsan Utlu and Veysel Yucesoy and Aykut Koc and Tolga Cukur},
  year   = {2017},
  eprint = {1711.00331},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1711.00331}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1711.00331.svg)](https://codeofpaper.com/paper/1711.00331)

Discussion