Acting for the Right Reasons: Creating Reason-Sensitive Artificial Moral Agents

Kevin Baum, Lisa Dargasz, Felix Jahn, Timo P. Gros, Verena Wolf
9/23/2024
cs.AIcs.CYcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

We propose an extension of the reinforcement learning architecture that enables moral decision-making of reinforcement learning agents based on normative reasons. Central to this approach is a reason-based shield generator yielding a moral shield that binds the agent to actions that conform with recognized normative reasons so that our overall architecture restricts the agent to actions that are (internally) morally justified. In addition, we describe an algorithm that allows to iteratively improve the reason-based shield generator through case-based feedback from a moral judge.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Integrating Reason-Based Moral Decision-Making in the Reinforcement Learning Architecture

94% match2025

Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning

85% match1 repo82023

MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes

84% match2025

Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making

83% match2025

MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents

83% match2026

Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignment

82% match2026

Cite this paper

@article{baum2024acting,
  title  = {Acting for the Right Reasons: Creating Reason-Sensitive Artificial Moral Agents},
  author = {Kevin Baum and Lisa Dargasz and Felix Jahn and Timo P. Gros and Verena Wolf},
  year   = {2024},
  eprint = {2409.15014},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.15014v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.15014.svg)](https://codeofpaper.com/paper/2409.15014)

Discussion