Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

Jun Wang, Hosein Hasanbeig, Kaiyuan Tan, Zihe Sun, Yiannis Kantaros
11/28/2023
cs.ROcs.AIcs.LG

No official code found. Best implementation: spec-driven-rl/spec-driven-rl.github.io (0 stars, Apache-2.0, last commit 2025-03-28). 1 more below.

Abstract

This paper addresses the problem of designing control policies for agents with unknown stochastic dynamics and control objectives specified using Linear Temporal Logic (LTL). Recent Deep Reinforcement Learning (DRL) algorithms have aimed to compute policies that maximize the satisfaction probability of LTL formulas, but they often suffer from slow learning performance. To address this, we introduce a novel Deep Q-learning algorithm that significantly improves learning speed. The enhanced sample efficiency stems from a mission-driven exploration strategy that prioritizes exploration towards directions likely to contribute to mission success. Identifying these directions relies on an automaton representation of the LTL task as well as a learned neural network that partially models the agent-environment interaction. We provide comparative experiments demonstrating the efficiency of our algorithm on robot navigation tasks in unseen environments.

View on arXivView PDF

Code Implementations(2)

spec-driven-rl/spec-driven-rl.github.ioCommunity63%
Apache-2.0

Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

00CSS, HTMLMar 28, 20251 years agoApache-2.0
kantaroslab/BiasedRL_LTLCommunity58%
Apache-2.0

Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications

01Shell, PythonNov 8, 20241 years agoApache-2.0

Similar Papers

Signal Temporal Logic Neural Predictive Control

86% match2023

TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks

85% match1 repo22025

LTL-Constrained Policy Optimization with Cycle Experience Replay

85% match2024

Exploiting Hybrid Policy in Reinforcement Learning for Interpretable Temporal Logic Manipulation

84% match1 repo132024

Directed Exploration in Reinforcement Learning from Linear Temporal Logic

84% match2024

Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments

84% match1 repo12025

Cite this paper

@article{wang2023missiondriven,
  title  = {Mission-driven Exploration for Accelerated Deep Reinforcement Learning with Temporal Logic Task Specifications},
  author = {Jun Wang and Hosein Hasanbeig and Kaiyuan Tan and Zihe Sun and Yiannis Kantaros},
  year   = {2023},
  eprint = {2311.17059},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.17059v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.17059.svg)](https://codeofpaper.com/paper/2311.17059)

Discussion