Control of Rayleigh-Bénard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime

Thorben Markmann, Michiel Straat, Sebastian Peitz, Barbara Hammer
4/16/2025
physics.flu-dyncs.LG

No official code found. Best implementation: HammerLabML/RBC-Control-SARL (1 star, MIT, last commit 2025-11-11).

Abstract

Data-driven flow control has significant potential for industry, energy systems, and climate science. In this work, we study the effectiveness of Reinforcement Learning (RL) for reducing convective heat transfer in the 2D Rayleigh-Bénard Convection (RBC) system under increasing turbulence. We investigate the generalizability of control across varying initial conditions and turbulence levels and introduce a reward shaping technique to accelerate the training. RL agents trained via single-agent Proximal Policy Optimization (PPO) are compared to linear proportional derivative (PD) controllers from classical control theory. The RL agents reduced convection, measured by the Nusselt Number, by up to 33% in moderately turbulent systems and 10% in highly turbulent settings, clearly outperforming PD control in all settings. The agents showed strong generalization performance across different initial conditions and to a significant extent, generalized to higher degrees of turbulence. The reward shaping improved sample efficiency and consistently stabilized the Nusselt Number to higher turbulence levels.

View on arXivView PDF

Code Implementations(1)

HammerLabML/RBC-Control-SARLCommunity100%
MIT

Control of Rayleigh-Bénard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime

10Shell, PythonMar 20, 202511 months agoMIT

Similar Papers

Improving the Robustness of Control of Chaotic Convective Flows with Domain-Informed Reinforcement Learning

92% match2025

Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need

92% match1 repo222023

Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-Bénard convection

91% match1 repo32024

Model-based deep reinforcement learning for accelerated learning from flow simulations

82% match2024

Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality

82% match1 repo22024

HydroGym: A Reinforcement Learning Platform for Fluid Dynamics

82% match2025

Cite this paper

@article{markmann2025control,
  title  = {Control of Rayleigh-Bénard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime},
  author = {Thorben Markmann and Michiel Straat and Sebastian Peitz and Barbara Hammer},
  year   = {2025},
  eprint = {2504.12000},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.12000v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.12000.svg)](https://codeofpaper.com/paper/2504.12000)

Discussion