Sequential Counterfactual Risk Minimization

Houssam Zenati, Eustache Diemert, Matthieu Martin, Julien Mairal, Pierre Gaillard
2/23/2023
cs.LG

No official code found. Best implementation: criteo-research/sequential-conterfactual-risk-minimization (6 stars, last commit 2023-01-26).

Abstract

Counterfactual Risk Minimization (CRM) is a framework for dealing with the logged bandit feedback problem, where the goal is to improve a logging policy using offline data. In this paper, we explore the case where it is possible to deploy learned policies multiple times and acquire new data. We extend the CRM principle and its theory to this scenario, which we call "Sequential Counterfactual Risk Minimization (SCRM)." We introduce a novel counterfactual estimator and identify conditions that can improve the performance of CRM in terms of excess risk and regret rates, by using an analysis similar to restart strategies in accelerated optimization methods. We also provide an empirical evaluation of our method in both discrete and continuous action settings, and demonstrate the benefits of multiple deployments of CRM.

View on arXivView PDF

Code Implementations(1)

criteo-research/sequential-conterfactual-risk-minimizationCommunity100%
62Shell, PythonJan 26, 20233 years ago

Similar Papers

A Simpler Alternative to Variational Regularized Counterfactual Risk Minimization

80% match2024

Randomized Confidence Bounds for Stochastic Partial Monitoring

78% match2024

Eventual Discounting Temporal Logic Counterfactual Experience Replay

78% match2023

Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization

78% match1 repo2023

Doubly Robust Counterfactual Classification

78% match2023

Practical and Robust Safety Guarantees for Advanced Counterfactual Learning to Rank

78% match2024

Cite this paper

@article{zenati2023sequential,
  title  = {Sequential Counterfactual Risk Minimization},
  author = {Houssam Zenati and Eustache Diemert and Matthieu Martin and Julien Mairal and Pierre Gaillard},
  year   = {2023},
  eprint = {2302.12120},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.12120v2},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.12120.svg)](https://codeofpaper.com/paper/2302.12120)

Discussion