Explanation through Reward Model Reconciliation using POMDP Tree Search

Benjamin D. Kraske, Anshu Saksena, Anna L. Buczak, Zachary N. Sunberg
5/1/2023
cs.AIcs.HCcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

As artificial intelligence (AI) algorithms are increasingly used in mission-critical applications, promoting user-trust of these systems will be essential to their success. Ensuring users understand the models over which algorithms reason promotes user trust. This work seeks to reconcile differences between the reward model that an algorithm uses for online partially observable Markov decision (POMDP) planning and the implicit reward model assumed by a human user. Action discrepancies, differences in decisions made by an algorithm and user, are leveraged to estimate a user's objectives as expressed in weightings of a reward function.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Expectation Alignment: Handling Reward Misspecification in the Presence of Expectation Mismatch

81% match2024

Repairing Reward Functions with Human Feedback to Mitigate Reward Hacking

80% match2025

Sail into the Headwind: Alignment via Robust Rewards and Dynamic Labels against Reward Hacking

79% match2024

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms

79% match2024

Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models

79% match1 repo22025

Reward Machines for Deep RL in Noisy and Uncertain Environments

79% match1 repo62024

Cite this paper

@article{kraske2023explanation,
  title  = {Explanation through Reward Model Reconciliation using POMDP Tree Search},
  author = {Benjamin D. Kraske and Anshu Saksena and Anna L. Buczak and Zachary N. Sunberg},
  year   = {2023},
  eprint = {2305.00931},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.00931v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.00931.svg)](https://codeofpaper.com/paper/2305.00931)

Discussion