Neural Fine-Tuning Search for Few-Shot Learning

Panagiotis Eustratiadis, Łukasz Dudziak, Da Li, Timothy Hospedales
6/15/2023
cs.CVcs.LG

No official code found. Best implementation: peustr/nfts-public (10 stars, last commit 2023-06-15).

Abstract

In few-shot recognition, a classifier that has been trained on one set of classes is required to rapidly adapt and generalize to a disjoint, novel set of classes. To that end, recent studies have shown the efficacy of fine-tuning with carefully crafted adaptation architectures. However this raises the question of: How can one design the optimal adaptation strategy? In this paper, we study this question through the lens of neural architecture search (NAS). Given a pre-trained neural network, our algorithm discovers the optimal arrangement of adapters, which layers to keep frozen and which to fine-tune. We demonstrate the generality of our NAS method by applying it to both residual networks and vision transformers and report state-of-the-art performance on Meta-Dataset and Meta-Album.

View on arXivView PDF

Code Implementations(1)

peustr/nfts-publicCommunity100%

Supporting code for the paper: Neural Fine-Tuning Search for Few-Shot Learning. Eustratiadis et al. (ICLR 2023)

101PythonJun 15, 20233 years ago

Similar Papers

MoRe Fine-Tuning with 10x Fewer Parameters

84% match1 repo222024

Flatness Improves Backbone Generalisation in Few-shot Classification

83% match1 repo22024

InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning

83% match2023

How Much Is Hidden in the NAS Benchmarks? Few-Shot Adaptation of a NAS Predictor

83% match2023

Enhancing Test Time Adaptation with Few-shot Guidance

82% match2024

Accelerating Neural Self-Improvement via Bootstrapping

82% match1 repo1772023

Cite this paper

@article{eustratiadis2023neural,
  title  = {Neural Fine-Tuning Search for Few-Shot Learning},
  author = {Panagiotis Eustratiadis and Łukasz Dudziak and Da Li and Timothy Hospedales},
  year   = {2023},
  eprint = {2306.09295},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.09295v1},
  journal = {ICLR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.09295.svg)](https://codeofpaper.com/paper/2306.09295)

Discussion