Bayesian Metaplasticity from Synaptic Uncertainty

Djohan Bonnet, Tifenn Hirtzlin, Tarcisius Januel, Thomas Dalgaty, Damien Querlioz +1 more
12/15/2023
cs.LGcs.AI

No confident code implementation found for this paper.

Abstract

Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios. In this study, we introduce MEtaplasticity from Synaptic Uncertainty (MESU), inspired by metaplasticity and Bayesian inference principles. MESU harnesses synaptic uncertainty to retain information over time, with its update rule closely approximating the diagonal Newton's method for synaptic updates. Through continual learning experiments on permuted MNIST tasks, we demonstrate MESU's remarkable capability to maintain learning performance across 100 tasks without the need of explicit task boundaries.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Bayesian continual learning and forgetting in neural networks

90% match1 repo162025

Neuromimetic metaplasticity for adaptive continual learning

88% match2024

Learning to Continually Learn with the Bayesian Principle

88% match1 repo202024

Learning to Continually Learn

87% match1 repo1182020

Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization

86% match2 repos72023

Order parameters and phase transitions of continual learning in deep neural networks

85% match2024

Cite this paper

@article{bonnet2023bayesian,
  title  = {Bayesian Metaplasticity from Synaptic Uncertainty},
  author = {Djohan Bonnet and Tifenn Hirtzlin and Tarcisius Januel and Thomas Dalgaty and Damien Querlioz and Elisa Vianello},
  year   = {2023},
  eprint = {2312.10153},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.10153v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.10153.svg)](https://codeofpaper.com/paper/2312.10153)

Discussion