Deep Coordination Graphs

Wendelin Böhmer, Vitaly Kurin, Shimon Whiteson
9/27/2019
cs.LGcs.AIcs.MA

Official code: wendelinboehmer/dcg (77 stars, Apache-2.0, last commit 2024-06-02).

Abstract

This paper introduces the deep coordination graph (DCG) for collaborative multi-agent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factoring the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks that employ parameter sharing and low-rank approximations to significantly improve sample efficiency. We show that DCG can solve predator-prey tasks that highlight the relative overgeneralization pathology, as well as challenging StarCraft II micromanagement tasks.

View on arXivView PDF

Code Implementations(1)

wendelinboehmer/dcgOfficial100%
Apache-2.0
7723Shell, PythonJun 3, 20202 years agoApache-2.0

Similar Papers

Coordination Graphs for Constrained Multi-Agent Reinforcement Learning

88% match1 repo102026

Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems

87% match1 repo172024

Multi-agent Reinforcement Learning with Deep Networks for Diverse Q-Vectors

87% match2024

PACT: Plan-Augmented Coordination via Trace-Graphs for Dynamic Multi-Agent Combinatorial Optimization

86% match2026

Reinforcement Networks: novel framework for collaborative Multi-Agent Reinforcement Learning tasks

86% match2025

Joint Intrinsic Motivation for Coordinated Exploration in Multi-Agent Deep Reinforcement Learning

86% match1 repo72024

Cite this paper

@article{bhmer2019deep,
  title  = {Deep Coordination Graphs},
  author = {Wendelin Böhmer and Vitaly Kurin and Shimon Whiteson},
  year   = {2019},
  eprint = {1910.00091},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1910.00091}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1910.00091.svg)](https://codeofpaper.com/paper/1910.00091)

Discussion