Reinforcement Learning with Fast and Forgetful Memory

Steven D. Morad, Ryan Kortvelesy, Stephan Liwicki, Amanda Prorok
2/12/2026

Official code: proroklab/ffm (24 stars, MIT, last commit 2023-08-15).

Abstract

Nearly all real world tasks are inherently partially observable, necessitating the use of memory in Reinforcement Learning (RL). Most model-free approaches summarize the trajectory into a latent Markov state using memory models borrowed from Supervised Learning (SL), even though RL tends to exhibit different training and efficiency characteristics. Addressing this discrepancy, we introduce Fast and Forgetful Memory, an algorithm-agnostic memory model designed specifically for RL. Our approach constrains the model search space via strong structural priors inspired by computational psychology. It is a drop-in replacement for recurrent neural networks (RNNs) in recurrent RL algorithms, achieving greater reward than RNNs across various recurrent benchmarks and algorithms without changing any hyperparameters. Moreover, Fast and Forgetful Memory exhibits training speeds two orders of magnitude faster than RNNs, attributed to its logarithmic time and linear space complexity. Our implementation is available at https://github.com/proroklab/ffm.

View on arXivView PDF

Code Implementations(1)

proroklab/ffmOfficial100%
MIT

Reinforcement Learning with Fast and Forgetful Memory

242Shell, PythonApr 26, 20233 years agoMIT

Similar Papers

Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning

84% match2026

A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control

83% match2025

Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning

83% match2026

Reservoir Computing for Fast, Simplified Reinforcement Learning on Memory Tasks

83% match2024

Mastering Memory Tasks with World Models

83% match1 repo842024

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

83% match1 repo72023

Cite this paper

@article{morad2026reinforcement,
  title  = {Reinforcement Learning with Fast and Forgetful Memory},
  author = {Steven D. Morad and Ryan Kortvelesy and Stephan Liwicki and Amanda Prorok},
  year   = {2026},
  eprint = {2310.04128},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2310.04128},
  journal = {NEURIPS 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.04128.svg)](https://codeofpaper.com/paper/2310.04128)

Discussion