On the Computation of the Fisher Information in Continual Learning

Gido M. van de Ven
2/17/2025
cs.LGcs.AIcs.CV

No official code found. Best implementation: GMvandeVen/continual-learning (1,866 stars, MIT, last commit 2025-11-05).

Abstract

One of the most popular methods for continual learning with deep neural networks is Elastic Weight Consolidation (EWC), which involves computing the Fisher Information. The exact way in which the Fisher Information is computed is however rarely described, and multiple different implementations for it can be found online. This blog post discusses and empirically compares several often-used implementations, which highlights that many currently reported results for EWC could likely be improved by changing the way the Fisher Information is computed.

View on arXivView PDF

Code Implementations(1)

GMvandeVen/continual-learningCommunity100%
MIT

PyTorch implementation of various methods for continual learning (XdG, EWC, SI, LwF, FROMP, DGR, BI-R, ER, A-GEM, iCaRL, Generative Classifier) in three different scenarios.

1,866347Shell, PythonSep 26, 201811 months agoMIT
artificial-neural-networksclass-incremental-learningcontinual-learningdeep-learningdistillation+11 more

Similar Papers

Generalizable Two-Branch Framework for Image Class-Incremental Learning

82% match2024

Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers

82% match1 repo22026

Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers

82% match1 repo22024

Continual Learning and Catastrophic Forgetting

81% match2024

Decodable and Sample Invariant Continuous Object Encoder

80% match1 repo42026

Order parameters and phase transitions of continual learning in deep neural networks

80% match2024

Cite this paper

@article{ven2025computation,
  title  = {On the Computation of the Fisher Information in Continual Learning},
  author = {Gido M. van de Ven},
  year   = {2025},
  eprint = {2502.11756},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.11756v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.11756.svg)](https://codeofpaper.com/paper/2502.11756)

Discussion