Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

R. Srikant
1/28/2024
math.PRcs.LGeess.SY

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains. We then show that these results can be applied to establish a non-asymptotic central limit theorem for Temporal Difference (TD) learning with averaging.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

100% match2024

Central Limit Theorem for Two-Timescale Stochastic Approximation with Markovian Noise: Theory and Applications

85% match2024

Uncertainty quantification for Markov chain induced martingales with application to temporal difference learning

84% match2025

Central Limit Theorems for Asynchronous Averaged Q-Learning

81% match2025

Statistical Inference for Temporal Difference Learning with Linear Function Approximation

80% match2024

Distributed TD(0) with Almost No Communication

80% match2023

Cite this paper

@article{srikant2024rates,
  title  = {Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning},
  author = {R. Srikant},
  year   = {2024},
  eprint = {2401.15719},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.15719v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.15719.svg)](https://codeofpaper.com/paper/2401.15719)

Discussion