Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning

Cameron Voloshin, Hoang M. Le, Nan Jiang, Yisong Yue
11/15/2019
cs.LGcs.AIcs.RO

Official code: clvoloshin/COBS (61 stars, last commit 2022-08-09).

Abstract

We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications. Given the increasing interest in deploying learning-based methods, there has been a flurry of recent proposals for OPE method, leading to a need for standardized empirical analyses. Our work takes a strong focus on diversity of experimental design to enable stress testing of OPE methods. We provide a comprehensive benchmarking suite to study the interplay of different attributes on method performance. We distill the results into a summarized set of guidelines for OPE in practice. Our software package, the Caltech OPE Benchmarking Suite (COBS), is open-sourced and we invite interested researchers to further contribute to the benchmark.

View on arXivView PDF

Code Implementations(1)

clvoloshin/COBSOfficial100%

OPE Tools based on Empirical Study of Off Policy Policy Estimation paper.

6113Shell, PythonNov 14, 20194 years ago

Similar Papers

NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning

90% match2 repos1512021

SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation

88% match1 repo1432023

Active Offline-to-Online Reinforcement Learning

86% match2026

When is Offline Policy Selection Sample Efficient for Reinforcement Learning?

84% match2023

Benchmarks for Deep Off-Policy Evaluation

84% match1 repo882021

Counterfactual-Augmented Importance Sampling for Semi-Offline Policy Evaluation

84% match2023

Cite this paper

@article{voloshin2019empirical,
  title  = {Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning},
  author = {Cameron Voloshin and Hoang M. Le and Nan Jiang and Yisong Yue},
  year   = {2019},
  eprint = {1911.06854},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1911.06854}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1911.06854.svg)](https://codeofpaper.com/paper/1911.06854)

Discussion