Accelerating Neural Self-Improvement via Bootstrapping

Kazuki Irie, Jürgen Schmidhuber
5/2/2023
cs.LG

No official code found. Best implementation: IDSIA/modern-srwm (177 stars, PyTorch, last commit 2025-06-11).

Abstract

Few-shot learning with sequence-processing neural networks (NNs) has recently attracted a new wave of attention in the context of large language models. In the standard N-way K-shot learning setting, an NN is explicitly optimised to learn to classify unlabelled inputs by observing a sequence of NK labelled examples. This pressures the NN to learn a learning algorithm that achieves optimal performance, given the limited number of training examples. Here we study an auxiliary loss that encourages further acceleration of few-shot learning, by applying recently proposed bootstrapped meta-learning to NN few-shot learners: we optimise the K-shot learner to match its own performance achievable by observing more than NK examples, using only NK examples. Promising results are obtained on the standard Mini-ImageNet dataset. Our code is public.

View on arXivView PDF

Code Implementations(1)

IDSIA/modern-srwmCommunity100%
PyTorch

Official repository for the paper "A Modern Self-Referential Weight Matrix That Learns to Modify Itself" (ICML 2022 & NeurIPS 2021 Deep RL Workshop) and "Accelerating Neural Self-Improvement via Bootstrapping" (ICLR 2023 Workshop)

17717C++, CudaNov 25, 20211 years ago
fast-weight-programmersfast-weightspytorchreinforcement-learningrnn-based-few-shot-learning+4 more

Similar Papers

Neural Fine-Tuning Search for Few-Shot Learning

82% match1 repo102023

An experimental approach on Few Shot Class Incremental Learning

82% match2025

On the Role of Neural Collapse in Meta Learning Models for Few-shot Learning

82% match2023

Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning

81% match1 repo612026

NTK-Guided Few-Shot Class Incremental Learning

81% match1 repo262024

UNEM: UNrolled Generalized EM for Transductive Few-Shot Learning

81% match1 repo122026

Cite this paper

@article{irie2023accelerating,
  title  = {Accelerating Neural Self-Improvement via Bootstrapping},
  author = {Kazuki Irie and Jürgen Schmidhuber},
  year   = {2023},
  eprint = {2305.01547},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.01547v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.01547.svg)](https://codeofpaper.com/paper/2305.01547)

Discussion