Reinforcement learning-based control co-design of digital twin-enabled full-vehicle active suspension systems

Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in Digital Twins (DTs) and Reinforcement Learning (RL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework for co-optimizing physical and control systems remains an open challenge. This work presents an RL-based Control Co-Design (CCD) framework for full-vehicle active suspensions using multi-generation design and DT concepts. Through integrating automatic differentiation into Deep Reinforcement Learning (DRL), we jointly optimize physical components of suspension systems and control policies under varying driver behaviors and environmental uncertainties. The DRL technique also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to quantify data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of autonomous suspension systems under two distinct driving settings (mild and aggressive). The results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 58% and 12% for mild and aggressive while improving ride comfort by approximately 17% and 28%, respectively. Contributions of this work include: (1) developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, (2) introducing a multi-generation design framework for self-improving systems across the whole lifecycle, and (3) demonstrating personalized optimization of active suspension systems for distinct types of drivers.