Full Gradient Deep Reinforcement Learning for Average-Reward Criterion

Tejas Pagare, Vivek Borkar, Konstantin Avrachenkov
4/7/2023
eess.SYcs.LG

No confident code implementation found for this paper.

Abstract

We extend the provably convergent Full Gradient DQN algorithm for discounted reward Markov decision processes from Avrachenkov et al. (2021) to average reward problems. We experimentally compare widely used RVI Q-Learning with recently proposed Differential Q-Learning in the neural function approximation setting with Full Gradient DQN and DQN. We also extend this to learn Whittle indices for Markovian restless multi-armed bandits. We observe a better convergence rate of the proposed Full Gradient variant across different tasks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning

85% match1 repo82024

On Convergence of Average-Reward Q-Learning in Weakly Communicating Markov Decision Processes

84% match2024

Efficient Q-Learning and Actor-Critic Methods for Robust Average Reward Reinforcement Learning

84% match2025

GB-DQN: Gradient Boosted DQN Models for Non-stationary Reinforcement Learning

84% match2025

A Differential Perspective on Distributional Reinforcement Learning

84% match2025

Burning RED: Unlocking Subtask-Driven Reinforcement Learning and Risk-Awareness in Average-Reward Markov Decision Processes

83% match1 repo2024

Cite this paper

@article{pagare2023full,
  title  = {Full Gradient Deep Reinforcement Learning for Average-Reward Criterion},
  author = {Tejas Pagare and Vivek Borkar and Konstantin Avrachenkov},
  year   = {2023},
  eprint = {2304.03729},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.03729v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.03729.svg)](https://codeofpaper.com/paper/2304.03729)

Discussion