Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels

Massimiliano Patacchiola, Jack Turner, Elliot J. Crowley, Michael O'Boyle, Amos Storkey
10/11/2019
cs.LGstat.ML

Official code: BayesWatch/deep-kernel-transfer (208 stars, last commit 2022-01-19).

Abstract

Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.

View on arXivView PDF

Code Implementations(1)

BayesWatch/deep-kernel-transferOfficial100%

Official pytorch implementation of the paper "Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels" (NeurIPS 2020)

20831Shell, PythonOct 10, 20194 years ago
bayesian-methodsclassificationdeep-kernel-learningdeep-learningfew-shot-learning+12 more

Similar Papers

Meta-learning approaches for few-shot learning: A survey of recent advances

89% match2023

Meta-Transfer Learning for Few-Shot Learning

87% match1 repo7852018

Unsupervised Meta-Learning via Few-shot Pseudo-supervised Contrastive Learning

87% match1 repo582023

Multimodal Model-Agnostic Meta-Learning via Task-Aware Modulation

86% match1 repo192019

Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks

86% match1 repo52021

Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimation

86% match2023

Cite this paper

@article{patacchiola2019bayesian,
  title  = {Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels},
  author = {Massimiliano Patacchiola and Jack Turner and Elliot J. Crowley and Michael O'Boyle and Amos Storkey},
  year   = {2019},
  eprint = {1910.05199},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1910.05199}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1910.05199.svg)](https://codeofpaper.com/paper/1910.05199)

Discussion