Inference Suboptimality in Variational Autoencoders

Chris Cremer, Xuechen Li, David Duvenaud
1/10/2018
cs.LGstat.ML

Official code: lxuechen/inference-suboptimality (16 stars, MIT, last commit 2020-03-17). 1 more implementation below.

Abstract

Amortized inference allows latent-variable models trained via variational learning to scale to large datasets. The quality of approximate inference is determined by two factors: a) the capacity of the variational distribution to match the true posterior and b) the ability of the recognition network to produce good variational parameters for each datapoint. We examine approximate inference in variational autoencoders in terms of these factors. We find that divergence from the true posterior is often due to imperfect recognition networks, rather than the limited complexity of the approximating distribution. We show that this is due partly to the generator learning to accommodate the choice of approximation. Furthermore, we show that the parameters used to increase the expressiveness of the approximation play a role in generalizing inference rather than simply improving the complexity of the approximation.

View on arXivView PDF

Code Implementations(2)

lxuechen/inference-suboptimalityOfficial100%
MIT

Code for "Inference Suboptimality in Variational Autoencoders"

165PythonApr 13, 20186 years agoMIT
chriscremer/Inference-SuboptimalityOfficial100%

Code for 'Inference Suboptimality in Variational Autoencoders'

112Shell, PythonApr 13, 20186 years ago

Similar Papers

Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders

87% match2024

On the Expressiveness of Approximate Inference in Bayesian Neural Networks

86% match1 repo132019

Lagging Inference Networks and Posterior Collapse in Variational Autoencoders

86% match1 repo1852019

Instance-Adaptive Parametrization for Amortized Variational Inference

85% match2026

Uncertainty in latent representations of variational autoencoders optimized for visual tasks

84% match2 repos42024

A Note on Generalization in Variational Autoencoders: How Effective Is Synthetic Data & Overparameterization?

84% match2023

Cite this paper

@article{cremer2018inference,
  title  = {Inference Suboptimality in Variational Autoencoders},
  author = {Chris Cremer and Xuechen Li and David Duvenaud},
  year   = {2018},
  eprint = {1801.03558},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1801.03558}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1801.03558.svg)](https://codeofpaper.com/paper/1801.03558)

Discussion