A Stein variational Newton method

Gianluca Detommaso, Tiangang Cui, Alessio Spantini, Youssef Marzouk, Robert Scheichl
6/8/2018
stat.MLcs.LGmath.NA

Official code: gianlucadetommaso/Stein-variational-samplers (28 stars, last commit 2019-05-25).

Abstract

Stein variational gradient descent (SVGD) was recently proposed as a general purpose nonparametric variational inference algorithm [Liu & Wang, NIPS 2016]: it minimizes the Kullback-Leibler divergence between the target distribution and its approximation by implementing a form of functional gradient descent on a reproducing kernel Hilbert space. In this paper, we accelerate and generalize the SVGD algorithm by including second-order information, thereby approximating a Newton-like iteration in function space. We also show how second-order information can lead to more effective choices of kernel. We observe significant computational gains over the original SVGD algorithm in multiple test cases.

View on arXivView PDF

Code Implementations(1)

gianlucadetommaso/Stein-variational-samplersOfficial100%

Python and MATLAB code for Stein Variational sampling methods

284M, HTMLMar 14, 20187 years ago
bayesian-inferencechangepointhawkes-processlstm-neural-networksnewton-method+6 more

Similar Papers

Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

88% match1 repo1042016

Kernel Semi-Implicit Variational Inference

83% match1 repo92024

Semi-Implicit Variational Inference via Kernelized Path Gradient Descent

82% match2025

Rényi Divergence Variational Inference

82% match1 repo282016

Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold

81% match2024

Optimization Guarantees for Square-Root Natural-Gradient Variational Inference

81% match2025

Cite this paper

@article{detommaso2018stein,
  title  = {A Stein variational Newton method},
  author = {Gianluca Detommaso and Tiangang Cui and Alessio Spantini and Youssef Marzouk and Robert Scheichl},
  year   = {2018},
  eprint = {1806.03085},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1806.03085}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1806.03085.svg)](https://codeofpaper.com/paper/1806.03085)

Discussion