Prediction and Control in Continual Reinforcement Learning

N. Anand, D. Precup
2/12/2026

No official code found. Best implementation: NishanthVAnand/prediction-and-control-in-continual-reinforcement-learning (13 stars, MIT, last commit 2024-05-10).

Abstract

Temporal difference (TD) learning is often used to update the estimate of the value function which is used by RL agents to extract useful policies. In this paper, we focus on value function estimation in continual reinforcement learning. We propose to decompose the value function into two components which update at different timescales: a permanent value function, which holds general knowledge that persists over time, and a transient value function, which allows quick adaptation to new situations. We establish theoretical results showing that our approach is well suited for continual learning and draw connections to the complementary learning systems (CLS) theory from neuroscience. Empirically, this approach improves performance significantly on both prediction and control problems.

View on arXivView PDF

Code Implementations(1)

NishanthVAnand/prediction-and-control-in-continual-reinforcement-learningCommunity100%
MIT

Code to reproduce results from the paper: Prediction and Control in Continual Reinforcement Learning, NeurIPS 2023.

132Shell, PythonDec 30, 20232 years agoMIT

Similar Papers

On the continuity and smoothness of the value function in reinforcement learning and optimal control

84% match2024

Rethinking the Foundations for Continual Reinforcement Learning

84% match2025

Inferring Transition Dynamics from Value Functions

82% match2025

Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning

82% match1 repo22025

Policy Optimization for Continuous Reinforcement Learning

81% match2026

Leveraging Prior Knowledge in Reinforcement Learning via Double-Sided Bounds on the Value Function

81% match2023

Cite this paper

@article{anand2026prediction,
  title  = {Prediction and Control in Continual Reinforcement Learning},
  author = {N. Anand and D. Precup},
  year   = {2026},
  eprint = {2312.11669},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2312.11669},
  journal = {NEURIPS 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.11669.svg)](https://codeofpaper.com/paper/2312.11669)

Discussion