Virtual Adversarial Ladder Networks For Semi-supervised Learning

Saki Shinoda, Daniel E. Worrall, Gabriel J. Brostow
11/20/2017
cs.LGcs.CVstat.ML

Official code: sakishinoda/tf-ssl (13 stars, MIT, last commit 2017-10-27).

Abstract

Semi-supervised learning (SSL) partially circumvents the high cost of labeling data by augmenting a small labeled dataset with a large and relatively cheap unlabeled dataset drawn from the same distribution. This paper offers a novel interpretation of two deep learning-based SSL approaches, ladder networks and virtual adversarial training (VAT), as applying distributional smoothing to their respective latent spaces. We propose a class of models that fuse these approaches. We achieve near-supervised accuracy with high consistency on the MNIST dataset using just 5 labels per class: our best model, ladder with layer-wise virtual adversarial noise (LVAN-LW), achieves 1.42% +/- 0.12 average error rate on the MNIST test set, in comparison with 1.62% +/- 0.65 reported for the ladder network. On adversarial examples generated with L2-normalized fast gradient method, LVAN-LW trained with 5 examples per class achieves average error rate 2.4% +/- 0.3 compared to 68.6% +/- 6.5 for the ladder network and 9.9% +/- 7.5 for VAT.

View on arXivView PDF

Code Implementations(1)

sakishinoda/tf-sslOfficial100%
MIT

Virtual Adversarial Ladder Networks for Semi-Supervised Learning

134PythonJul 6, 20178 years agoMIT

Similar Papers

Distributional Smoothing with Virtual Adversarial Training

85% match1 repo1092015

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

84% match2 repos2542017

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

83% match1 repo4602018

DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification

82% match1 repo522018

CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

82% match1 repo1332020

Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Training

82% match2024

Cite this paper

@article{shinoda2017virtual,
  title  = {Virtual Adversarial Ladder Networks For Semi-supervised Learning},
  author = {Saki Shinoda and Daniel E. Worrall and Gabriel J. Brostow},
  year   = {2017},
  eprint = {1711.07476},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1711.07476}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1711.07476.svg)](https://codeofpaper.com/paper/1711.07476)

Discussion