Can We Understand Plasticity Through Neural Collapse?

Guglielmo Bonifazi, Iason Chalas, Gian Hess, Jakub Łucki
4/3/2024
cs.LGcs.AI

No confident code implementation found for this paper.

Abstract

This paper explores the connection between two recently identified phenomena in deep learning: plasticity loss and neural collapse. We analyze their correlation in different scenarios, revealing a significant association during the initial training phase on the first task. Additionally, we introduce a regularization approach to mitigate neural collapse, demonstrating its effectiveness in alleviating plasticity loss in this specific setting.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Understanding plasticity in neural networks

88% match2026

Plasticity Loss in Deep Reinforcement Learning: A Survey

88% match2024

Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay

86% match2023

The Prevalence of Neural Collapse in Neural Multivariate Regression

86% match2024

Directions of Curvature as an Explanation for Loss of Plasticity

85% match2023

Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity

85% match2025

Cite this paper

@article{bonifazi2024understand,
  title  = {Can We Understand Plasticity Through Neural Collapse?},
  author = {Guglielmo Bonifazi and Iason Chalas and Gian Hess and Jakub Łucki},
  year   = {2024},
  eprint = {2404.02719},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.02719v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.02719.svg)](https://codeofpaper.com/paper/2404.02719)

Discussion