Explaining Reinforcement Learning with Shapley Values

Daniel Beechey, Thomas M. S. Smith, Özgür Şimşek
6/9/2023
cs.LG

No official code found. Best implementation: djeb20/SVERL_icml_2023 (23 stars, last commit 2023-12-14). 1 more below.

Abstract

For reinforcement learning systems to be widely adopted, their users must understand and trust them. We present a theoretical analysis of explaining reinforcement learning using Shapley values, following a principled approach from game theory for identifying the contribution of individual players to the outcome of a cooperative game. We call this general framework Shapley Values for Explaining Reinforcement Learning (SVERL). Our analysis exposes the limitations of earlier uses of Shapley values in reinforcement learning. We then develop an approach that uses Shapley values to explain agent performance. In a variety of domains, SVERL produces meaningful explanations that match and supplement human intuition.

View on arXivView PDF

Code Implementations(2)

djeb20/SVERL_icml_2023Community100%

ICML paper on Shapley Values for Explaining Reinforcement Learning. XRL feature-influence method.

233PythonMay 3, 20232 years ago
djeb20/sverlCommunity73%

A theoretical framework for explaining reinforcement learning with Shapley values

01PythonApr 1, 20251 years ago

Similar Papers

A Theoretical Framework for Explaining Reinforcement Learning with Shapley Values

95% match1 repo2025

Approximating Shapley Explanations in Reinforcement Learning

91% match2025

Shapley Value Based Multi-Agent Reinforcement Learning: Theory, Method and Its Application to Energy Network

83% match2024

Beyond Shapley Values: Cooperative Games for the Interpretation of Machine Learning Models

83% match2025

Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learning

83% match2025

Introduction to Reinforcement Learning

81% match1 repo22.2k2024

Cite this paper

@article{beechey2023explaining,
  title  = {Explaining Reinforcement Learning with Shapley Values},
  author = {Daniel Beechey and Thomas M. S. Smith and Özgür Şimşek},
  year   = {2023},
  eprint = {2306.05810},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.05810v1},
  journal = {ICML 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.05810.svg)](https://codeofpaper.com/paper/2306.05810)

Discussion