Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

Ardian Selmonaj, Oleg Szehr, Giacomo Del Rio, Alessandro Antonucci, Adrian Schneider +1 more
9/20/2023
cs.LGcs.AIcs.MA

Official code: IDSIA/hhmarl_2D (165 stars, last commit 2025-04-03).

Abstract

The application of artificial intelligence to simulate air-to-air combat scenarios is attracting increasing attention. To date the high-dimensional state and action spaces, the high complexity of situation information (such as imperfect and filtered information, stochasticity, incomplete knowledge about mission targets) and the nonlinear flight dynamics pose significant challenges for accurate air combat decision-making. These challenges are exacerbated when multiple heterogeneous agents are involved. We propose a hierarchical multi-agent reinforcement learning framework for air-to-air combat with multiple heterogeneous agents. In our framework, the decision-making process is divided into two stages of abstraction, where heterogeneous low-level policies control the action of individual units, and a high-level commander policy issues macro commands given the overall mission targets. Low-level policies are trained for accurate unit combat control. Their training is organized in a learning curriculum with increasingly complex training scenarios and league-based self-play. The commander policy is trained on mission targets given pre-trained low-level policies. The empirical validation advocates the advantages of our design choices.

View on arXivView PDF

Code Implementations(1)

IDSIA/hhmarl_2DOfficial68%

Heterogeneous Hierarchical Multi Agent Reinforcement Learning for Air Combat

16523PythonSep 14, 20231 years ago
air-combatheterogeneous-agentshierarchical-reinforcement-learningmulti-agent-reinforcement-learning

Similar Papers

Enhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learning

96% match2025

Coordinated Strategies in Realistic Air Combat by Hierarchical Multi-Agent Reinforcement Learning

95% match2025

A Hierarchical Hybrid AI Approach: Integrating Deep Reinforcement Learning and Scripted Agents in Combat Simulations

86% match2025

A survey of air combat behavior modeling using machine learning

85% match2024

Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning

85% match2024

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions

85% match2023

Cite this paper

@article{selmonaj2023hierarchical,
  title  = {Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering},
  author = {Ardian Selmonaj and Oleg Szehr and Giacomo Del Rio and Alessandro Antonucci and Adrian Schneider and Michael Rüegsegger},
  year   = {2023},
  eprint = {2309.11247},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.11247v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.11247.svg)](https://codeofpaper.com/paper/2309.11247)

Discussion