Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions

2026-04-15

AI Summary

This study addresses the challenge of stabilizing rotating detonation engines (RDEs), which are prone to nonlinear oscillations and chaotic wave propagation during operation. To tackle this, the authors propose a symmetry-aware moving reference frame modeling approach that reformulates the deep reinforcement learning (DRL) control problem in a coordinate system co-moving with the detonation wave. For the first time, a fast–slow timescale separation mechanism is integrated into the DRL framework to manage the multiscale nature of reactive flow control. Using a one-dimensional reduced-order RDE model, rapid switching between phase-locked states is achieved through spatially segmented modulation of injection pressure. Experimental results demonstrate that the proposed method significantly outperforms fixed-reference-frame approaches in robustness and generalizability across diverse driving periods, initial conditions, and target detonation modes.

Technology Category

• Machine Learning: Reinforcement Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Rotating detonation engines (RDEs) are a promising propulsion concept that may offer higher thermodynamic efficiency and specific impulse than conventional systems, but nonlinear phenomena, including transitions to oscillatory or chaotic propagation modes, can hinder practical operation. Deep Reinforcement Learning (DRL) has emerged as a promising method for controlling complex nonlinear dynamics such as those observed in RDEs. However, the multi-timescale nature of the RDE system makes direct application of DRL challenging. We address this challenge by reformulating the DRL problem in a moving reference frame that follows the detonation-wave pattern, making the wave structure appear quasi-steady to the agent. This reformulation enables scale separation between fast detonation propagation and slower operating-mode dynamics. We train DRL controllers to modulate spatially segmented injection pressure in a one-dimensional reduced-order RDE model and induce rapid transitions between different mode-locked states. Across a range of actuation periods, initial states, and target modes, controllers trained in the moving frame learn more reliably than those trained in a stationary frame and remain effective over a broader range of actuation periods. These results suggest that symmetry-aware moving reference frame formulations may be useful for related multiscale flow-control problems and that scale separation should be exploited whenever possible to enable DRL control of multi-timescale systems.

Problem

• Rotating Detonation Engine
• multi-timescale dynamics
• mode transitions
• deep reinforcement learning
• flow control

Innovation

• Timescale Separation
• Deep Reinforcement Learning
• Rotating Detonation Engine
• Moving Reference Frame
• Mode Transition Control