Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

Nat Dilokthanakul, Pedro A. M. Mediano, Marta Garnelo, Matthew C. H. Lee, Hugh Salimbeni +2 more
11/8/2016
cs.LGcs.NEstat.ML

Official code: Nat-D/GMVAE (216 stars, last commit 2020-03-04).

Abstract

We study a variant of the variational autoencoder model (VAE) with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the known problem of over-regularisation that has been shown to arise in regular VAEs also manifests itself in our model and leads to cluster degeneracy. We show that a heuristic called minimum information constraint that has been shown to mitigate this effect in VAEs can also be applied to improve unsupervised clustering performance with our model. Furthermore we analyse the effect of this heuristic and provide an intuition of the various processes with the help of visualizations. Finally, we demonstrate the performance of our model on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving competitive performance on unsupervised clustering to the state-of-the-art results.

View on arXivView PDF

Code Implementations(1)

Nat-D/GMVAEOfficial100%

Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders

21652Lua, ShellNov 9, 20166 years ago

Similar Papers

Deep Generative Clustering with VAEs and Expectation-Maximization

85% match2025

ClusterDDPM: An EM clustering framework with Denoising Diffusion Probabilistic Models

84% match1 repo2023

Scalable Robust Bayesian Co-Clustering with Compositional ELBOs

83% match2025

Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering

83% match1 repo3152016

A Contrastive Variational Graph Auto-Encoder for Node Clustering

82% match1 repo102023

VAE with a VampPrior

81% match1 repo2312017

Cite this paper

@article{dilokthanakul2016deep,
  title  = {Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders},
  author = {Nat Dilokthanakul and Pedro A. M. Mediano and Marta Garnelo and Matthew C. H. Lee and Hugh Salimbeni and Kai Arulkumaran and Murray Shanahan},
  year   = {2016},
  eprint = {1611.02648},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1611.02648}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1611.02648.svg)](https://codeofpaper.com/paper/1611.02648)

Discussion