CoinRun: Solving Goal Misgeneralisation

Stuart Armstrong, Alexandre Maranhão, Oliver Daniels-Koch, Patrick Leask, Rebecca Gorman
9/28/2023
cs.AI

No confident code implementation found for this paper.

Abstract

Goal misgeneralisation is a key challenge in AI alignment -- the task of getting powerful Artificial Intelligences to align their goals with human intentions and human morality. In this paper, we show how the ACE (Algorithm for Concept Extrapolation) agent can solve one of the key standard challenges in goal misgeneralisation: the CoinRun challenge. It uses no new reward information in the new environment. This points to how autonomous agents could be trusted to act in human interests, even in novel and critical situations.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization

87% match2026

Agents Need Not Know Their Purpose

82% match2024

Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment

82% match1 repo2024

Multi-Agent Systems Should be Treated as Principal-Agent Problems

81% match2026

Goal Alignment: A Human-Aware Account of Value Alignment Problem

81% match2023

Moral Alignment for LLM Agents

80% match1 repo92026

Cite this paper

@article{armstrong2023coinrun,
  title  = {CoinRun: Solving Goal Misgeneralisation},
  author = {Stuart Armstrong and Alexandre Maranhão and Oliver Daniels-Koch and Patrick Leask and Rebecca Gorman},
  year   = {2023},
  eprint = {2309.16166},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.16166v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.16166.svg)](https://codeofpaper.com/paper/2309.16166)

Discussion