Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning

Filippos Christianos, Lukas Schäfer, Stefano V. Albrecht
6/12/2020
cs.MAcs.LG

Official code: uoe-agents/lb-foraging (61 stars, MIT, last commit 2024-09-15). 2 more implementations below.

Abstract

Exploration in multi-agent reinforcement learning is a challenging problem, especially in environments with sparse rewards. We propose a general method for efficient exploration by sharing experience amongst agents. Our proposed algorithm, called Shared Experience Actor-Critic (SEAC), applies experience sharing in an actor-critic framework. We evaluate SEAC in a collection of sparse-reward multi-agent environments and find that it consistently outperforms two baselines and two state-of-the-art algorithms by learning in fewer steps and converging to higher returns. In some harder environments, experience sharing makes the difference between learning to solve the task and not learning at all.

View on arXivView PDF

Code Implementations(3)

uoe-agents/lb-foragingOfficial100%
MIT

Level-Based Foraging (LBF): A multi-agent reinforcement learning environment

6117PythonJun 11, 20202 years agoMIT
uoe-agents/seacOfficial100%

The official code base of Shared Experience Actor-Critic (NeurIPS2020)

4113PythonOct 19, 20202 years ago
uoe-agents/robotic-warehouseCommunity100%
MIT

Multi-Robot Warehouse (RWARE): A multi-agent reinforcement learning environment

7414PythonJun 11, 20202 years agoMIT

Similar Papers

Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning via Incorporating Generalized Human Expertise

89% match2025

CPIG: Leveraging Consistency Policy with Intention Guidance for Multi-agent Exploration

88% match2024

DIFFER: Decomposing Individual Reward for Fair Experience Replay in Multi-Agent Reinforcement Learning

88% match2023

CoDreamer: Communication-Based Decentralised World Models

87% match2024

Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

87% match1 repo252024

GOV-REK: Governed Reward Engineering Kernels for Designing Robust Multi-Agent Reinforcement Learning Systems

87% match2024

Cite this paper

@article{christianos2020shared,
  title  = {Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning},
  author = {Filippos Christianos and Lukas Schäfer and Stefano V. Albrecht},
  year   = {2020},
  eprint = {2006.07169},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2006.07169}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2006.07169.svg)](https://codeofpaper.com/paper/2006.07169)

Discussion