Sparse Sensor Placement in Multi-Agent Reinforcement Learning Control of Rayleigh-Bénard Convection

2026-06-29

AI Summary

This work addresses the high sensor deployment cost in Rayleigh-Bénard convection control by proposing a structured pruning–based sparse observation learning framework. Starting from a densely trained expert policy, the approach employs supervised distillation combined with ordered non-convex group regularization and iterative reweighted group regularization, augmented by a consistency-aware pruning mechanism across overlapping observation windows to yield an interpretable and hardware-friendly sensor layout. The method reduces the per-agent observation dimensionality from 360 to 12—achieving maximal sparsity—while maintaining control performance close to that of the dense policy under both fixed and varying initial conditions. By integrating multi-agent Transformers with windowed observation modeling, the framework substantially lowers data throughput without compromising efficiency or performance.

Abstract

This paper studies sparse sensor placement for control of Rayleigh-Bénard convection with multi-agent reinforcement learning. We train dense expert policies with windowed observations and distill sparse apprentice policies by supervised learning with grouped regularization on encoder input weights. The framework combines ordered non-convex grouped regularization and iterative reweighted grouped regularization, and uses a grouping construction that enforces consistent pruning across overlapping observation windows. Experiments with fixed and varying initial conditions show that Multi-Agent Transformer policies train more stably than proximal policy optimization baselines, while sparse apprentices retain control behavior comparable to dense experts. Sparsity results are strong for the proposed grouped methods across settings, including maximal sparsity in all fixed-initial-condition setting variants and maximal or near-maximal sparsity in varying-initial-condition setting variants. As an additional proof of concept, training from learned minimal sensor sets reduces per-agent observation size from 360 to 12 and preserves the overall training trend in simulation while reducing data throughput. The results provide both an interpretable basis for identifying control-relevant spatial regions and state components, and a practical pathway toward sensor-efficient control under realistic hardware constraints.

Problem

• sparse sensor placement
• Rayleigh-Bénard convection
• multi-agent reinforcement learning
• sensor-efficient control

Innovation

• sparse sensor placement
• grouped regularization
• multi-agent reinforcement learning
• policy distillation
• Rayleigh-Bénard convection