Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning

2025-12-03

AI Summary

To address performance limitations in active suspension systems caused by fixed hardware configurations and environment-invariant control strategies, this paper proposes a synergistic optimization framework integrating digital twin (DT) technology with deep reinforcement learning (DRL). The framework establishes a high-fidelity vehicle–road coupled digital twin, incorporates a multi-generational self-evolution mechanism, and employs a quantile-aware DRL algorithm. Leveraging automatic differentiation, it enables joint online optimization of control policies and hardware parameters, effectively mitigating challenges arising from partial observability and data uncertainty. Experimental results demonstrate that, under moderate and aggressive driving conditions, the system reduces energy consumption by 43% and 52%, respectively, while significantly enhancing ride comfort, handling stability, and safety. These outcomes validate the framework’s capability for personalized, dynamic-adaptive control under varying operational conditions.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework remains an open challenge. This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions using multi-generation design concepts. By integrating automatic differentiation into DRL, we jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties. DRL also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to capture data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of suspension systems under two distinct driving settings (mild and aggressive). Results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 43% and 52% for mild and aggressive, respectively, while maintaining ride comfort and stability. Contributions include: developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, introducing a multi-generation design strategy for self-improving systems, and demonstrating personalized optimization of active suspension systems for distinct driver types.

Problem

• Integrates digital twins and deep reinforcement learning for active suspension co-design
• Optimizes suspension components and control under varying driver behaviors and uncertainties
• Addresses partial observability by learning control from limited sensor data

Innovation

• Digital twin integrates deep reinforcement learning for suspension co-design
• Automatic differentiation jointly optimizes hardware and control policies
• Quantile learning updates models to handle data uncertainty adaptively