The Dormant Neuron Phenomenon in Deep Reinforcement Learning

Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, Utku Evci
2/24/2023
cs.LG

No official code found. Best implementation: google/dopamine (10,877 stars, Apache-2.0, TensorFlow, last commit 2026-03-24). 1 more below.

Abstract

In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.

View on arXivView PDF

Code Implementations(2)

google/dopamineCommunity100%
TensorFlowApache-2.0

Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.

10,8771,392HTML, ShellJul 26, 20187 months agoApache-2.0
aigooglemlrltensorflow
timoklein/redoCommunity71%

ReDo: The Dormant Neuron Phenomenon in Deep Reinforcement Learning (pytorch)

345Shell, PythonApr 12, 20231 years ago

Similar Papers

The Dormant Neuron Phenomenon in Multi-Agent Reinforcement Learning Value Factorization

90% match1 repo52026

Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning

85% match2025

Understanding and Exploiting Plasticity for Non-stationary Network Resource Adaptation

84% match2025

Addressing the Plasticity-Stability Dilemma in Reinforcement Learning

84% match2025

Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning

84% match2025

Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration

83% match1 repo62025

Cite this paper

@article{sokar2023dormant,
  title  = {The Dormant Neuron Phenomenon in Deep Reinforcement Learning},
  author = {Ghada Sokar and Rishabh Agarwal and Pablo Samuel Castro and Utku Evci},
  year   = {2023},
  eprint = {2302.12902},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.12902v2},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.12902.svg)](https://codeofpaper.com/paper/2302.12902)

Discussion