🤖 AI Summary
This work addresses the performance degradation and unphysical bang-bang control often observed when gradient-based multi-agent reinforcement learning methods, trained on small domains, are transferred to large-scale turbulent channel flows. For the first time, evolutionary strategies (ES) are directly applied to optimize a recurrent neural network-based closed-loop controller in a full-scale turbulent channel. The approach employs an energy-aware reward function that evaluates the entire flow evolution and leverages parallel policy evaluation for efficient training. By circumventing the control degeneracy inherent in gradient-based methods, the proposed strategy yields physically plausible actuation aligned with near-wall turbulence dynamics. In direct numerical simulations, it achieves approximately 26% drag reduction in skin friction, substantially outperforming both GRU-MARL trained on small domains (17%) and classical opposition control (22%).
📝 Abstract
Closed-loop wall control learnt by multi-agent reinforcement learning can lower skin-friction drag in turbulent channels, but these gradient-based policies are trained on small periodic boxes and exhibit reduced performance when carried over to a larger domain. We recently showed that such policies are also prone to saturated bang-bang actuations that collapse into standing streamwise waves whose scale is set by the computational box rather than by the near-wall cycle, and proposed architectural fixes that avoid these degeneracies. Here, we employ Evolution Strategy (ES) to optimise a recurrent closed-loop controller directly on a large turbulent channel at $\mathit{Re}_τ\simeq180$, evaluating policy performance over full flow episodes using an energy-aware criterion and processing candidate policies in parallel. To our knowledge, this is the first application of an evolution strategy to the control of a turbulent flow. The ES controller reduces the skin friction by about $26\%$, exceeding the gradient-based multi-agent controller of Cavallazzi et al. (2026), GRU-MARL, trained on a minimal box ($17\%$), and marginally exceeding classic opposition control (OC, $22\%$). A wall-normal decomposition of the friction, Reynolds-stress profiles and anisotropy invariants show that the ES and opposition-controlled flows follow separate trajectories through the buffer layer, reaching comparable drag reduction by different reorganisations of the near-wall turbulence. In particular, the ES actuation correlates predominantly with the streamwise velocity fluctuations rather than with the wall-normal velocity that classical OC targets.