Discovering Structure in High-Dimensional Data Through Correlation Explanation

Greg Ver Steeg, Aram Galstyan
6/4/2014
cs.LGcs.AIstat.ML

Official code: gregversteeg/CorEx (308 stars, GPL-2.0, last commit 2017-06-02).

Abstract

We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective. Intuitively, the optimization searches for a set of latent factors that best explain the correlations in the data as measured by multivariate mutual information. The method is unsupervised, requires no model assumptions, and scales linearly with the number of variables which makes it an attractive approach for very high dimensional systems. We demonstrate that Correlation Explanation (CorEx) automatically discovers meaningful structure for data from diverse sources including personality tests, DNA, and human language.

View on arXivView PDF

Code Implementations(1)

gregversteeg/CorExOfficial100%
GPL-2.0

CorEx or "Correlation Explanation" discovers a hierarchy of informative latent factors. This reference implementation has been superseded by other versions below.

30854PythonOct 10, 20149 years agoGPL-2.0
information-theorymachine-learningpythonunsupervised-learning

Similar Papers

Maximally Informative Hierarchical Representations of High-Dimensional Data

83% match1 repo3082014

Fast structure learning with modular regularization

81% match1 repo362017

Approximating mutual information of high-dimensional variables using learned representations

81% match2024

Isolating Sources of Disentanglement in Variational Autoencoders

80% match1 repo3542018

The Information Sieve

80% match1 repo362015

Surveying the space of descriptions of a composite system with machine learning

80% match1 repo2024

Cite this paper

@article{steeg2014discovering,
  title  = {Discovering Structure in High-Dimensional Data Through Correlation Explanation},
  author = {Greg Ver Steeg and Aram Galstyan},
  year   = {2014},
  eprint = {1406.1222},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1406.1222}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1406.1222.svg)](https://codeofpaper.com/paper/1406.1222)

Discussion