LLM-Empowered State Representation for Reinforcement Learning

Boyuan Wang, Yun Qu, Yuhang Jiang, Jianzhun Shao, Chang Liu +2 more
2/12/2026

No official code found. Best implementation: thu-rllab/LESR (42 stars, last commit 2024-06-14).

Abstract

Conventional state representations in reinforcement learning often omit critical task-related details, presenting a significant challenge for value networks in establishing accurate mappings from states to task rewards. Traditional methods typically depend on extensive sample learning to enrich state representations with task-specific information, which leads to low sample efficiency and high time costs. Recently, surging knowledgeable large language models (LLM) have provided promising substitutes for prior injection with minimal human intervention. Motivated by this, we propose LLM-Empowered State Representation (LESR), a novel approach that utilizes LLM to autonomously generate task-related state representation codes which help to enhance the continuity of network mappings and facilitate efficient training. Experimental results demonstrate LESR exhibits high sample efficiency and outperforms state-of-the-art baselines by an average of 29% in accumulated reward in Mujoco tasks and 30% in success rates in Gym-Robotics tasks.

View on arXivView PDF

Code Implementations(1)

thu-rllab/LESRCommunity100%

LLM-Empowered State Representation for Reinforcement Learning (ICML2024 Accepted paper)

423PythonMay 20, 20242 years ago

Similar Papers

A Survey of State Representation Learning for Deep Reinforcement Learning

86% match2025

Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control

86% match2026

For SALE: State-Action Representation Learning for Deep Reinforcement Learning

85% match1 repo1702023

Auxiliary Reward Generation with Transition Distance Representation Learning

85% match2024

Improving Intrinsic Exploration by Creating Stationary Objectives

85% match2026

Choices are More Important than Efforts: LLM Enables Efficient Multi-Agent Exploration

84% match1 repo7112024

Cite this paper

@article{wang2026llmempowered,
  title  = {LLM-Empowered State Representation for Reinforcement Learning},
  author = {Boyuan Wang and Yun Qu and Yuhang Jiang and Jianzhun Shao and Chang Liu and Wenming Yang and Xiangyang Ji},
  year   = {2026},
  eprint = {2407.13237},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2407.13237},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.13237.svg)](https://codeofpaper.com/paper/2407.13237)

Discussion