Variational Neural Networks

Illia Oleksiienko, Dat Thanh Tran, Alexandros Iosifidis
7/4/2022
cs.LGstat.ML

No official code found. Best implementation: iliiliiliili/variational-nn-pytorch (5 stars, Apache-2.0, last commit 2023-08-30).

Abstract

Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other single-bin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods.

View on arXivView PDF

Code Implementations(1)

iliiliiliili/variational-nn-pytorchCommunity100%
Apache-2.0

Variational Neural Networks Pytorch implementation

53R, TeXAug 13, 20203 years agoApache-2.0

Similar Papers

Be Bayesian by Attachments to Catch More Uncertainty

89% match2023

Uncertainty in Neural Networks: Approximately Bayesian Ensembling

86% match1 repo892018

Noise Contrastive Priors for Functional Uncertainty

85% match2 repos1272018

On weight and variance uncertainty in neural networks for regression tasks

85% match2025

Toward Robust Uncertainty Estimation with Random Activation Functions

85% match2023

An Isotropic Approach to Efficient Uncertainty Quantification with Gradient Norms

85% match2026

Cite this paper

@article{oleksiienko2022variational,
  title  = {Variational Neural Networks},
  author = {Illia Oleksiienko and Dat Thanh Tran and Alexandros Iosifidis},
  year   = {2022},
  eprint = {2207.01524},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2207.01524v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2207.01524.svg)](https://codeofpaper.com/paper/2207.01524)

Discussion