Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation

Zechu Li, Tao Chen, Zhang-Wei Hong, Anurag Ajay, Pulkit Agrawal

2/12/2026

No official code found. Best implementation: Improbable-AI/pql (78 stars, MIT, last commit 2023-08-02).

DOI Semantic Scholar

Code Implementations(1)

Improbable-AI/pqlCommunity60%

MIT

Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation

785Shell, PythonJul 25, 20233 years agoMIT

Cite this paper

@article{li2026parallel,
  title  = {Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation},
  author = {Zechu Li and Tao Chen and Zhang-Wei Hong and Anurag Ajay and Pulkit Agrawal},
  year   = {2026},
  doi    = {10.48550/arXiv.2307.12983},
  url    = {https://arxiv.org/pdf/2307.12983},
  journal = {ICML 2023 2023}
}

Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation

Code Implementations(1)

Cite this paper

Discussion