Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

2026-01-21

AI Summary

This work addresses the lack of a unified and reproducible benchmark for reinforcement learning (RL) in active flow control, a limitation stemming from reliance on external computational fluid dynamics (CFD) solvers, non-differentiability, and insufficient support for three-dimensional and multi-agent scenarios. To overcome these challenges, we introduce FluidGym—the first fully PyTorch-based, end-to-end differentiable RL benchmark suite for active flow control. FluidGym integrates a GPU-accelerated PICT fluid solver, eliminating dependence on external CFD software. The platform supports 3D and multi-agent configurations, provides standardized evaluation protocols, and is accompanied by open-sourced environments, datasets, trained models, and baseline results using PPO and SAC algorithms, thereby establishing a foundation for reproducible and scalable learning-based flow control research.

Technology Category

• Machine Learning: Reinforcement Learning
• Multiagent Systems: Adversarial Agents
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Reinforcement learning (RL) has shown promising results in active flow control (AFC), yet progress in the field remains difficult to assess as existing studies rely on heterogeneous observation and actuation schemes, numerical setups, and evaluation protocols. Current AFC benchmarks attempt to address these issues but heavily rely on external computational fluid dynamics (CFD) solvers, are not fully differentiable, and provide limited 3D and multi-agent support. To overcome these limitations, we introduce FluidGym, the first standalone, fully differentiable benchmark suite for RL in AFC. Built entirely in PyTorch on top of the GPU-accelerated PICT solver, FluidGym runs in a single Python stack, requires no external CFD software, and provides standardized evaluation protocols. We present baseline results with PPO and SAC and release all environments, datasets, and trained models as public resources. FluidGym enables systematic comparison of control methods, establishes a scalable foundation for future research in learning-based flow control, and is available at https://github.com/safe-autonomous-systems/fluidgym.

Problem

• Reinforcement Learning
• Active Flow Control
• Benchmarking
• Computational Fluid Dynamics
• Multi-agent

Innovation

• differentiable simulation
• reinforcement learning
• active flow control
• benchmark suite
• GPU-accelerated CFD