Natural sit-to-stand control for biomimetic musculoskeletal robots with synergy-based deep reinforcement learning and...

Bio-inspired musculoskeletal humanoids offer inherent compliance and robustness, yet their control remains challenging due to nonlinear dynamics, redundant muscle-tendon actuation, and strong inter-joint coupling. This study proposes a neuromechanics-inspired control framework integrating deep reinforcement learning (DRL) with biologically grounded reward shaping and a physiological dual-pathway muscle synergy architecture to generate human-like (i.e., kinematically consistent) and energy-efficient sit-to-stand (STS) motions in a sagittal-plane musculoskeletal model.

The proposed framework consists of three main components. First, human kinematic and surface electromyography (sEMG) data are collected to provide reference trajectories and extract non-negative muscle synergies. Second, a DRL controller integrated with bio-inspired reward shaping is designed using a physiological dual-pathway architecture. One synergistic pathway outputs activation coefficients for surface muscles, while a direct-control pathway independently regulates deep muscles. These combined signals drive the musculoskeletal model through forward dynamics. Finally, two state-of-the-art DRL algorithms (PPO and SAC) are implemented to evaluate the robustness and generality of the framework.

Experimental results demonstrate that the dual-pathway approach generates natural STS motions that closely resemble human kinematics and temporal patterns. Compared to pure high-dimensional DRL control, the synergy-based framework acts as a powerful neuromuscular regularizer. It significantly improves joint coordination, prevents non-physiological jittering, and achieves higher human-likeness (quantified by higher Pearson correlation coefficients and lower root-mean-square errors, RMSE, against human trajectories). Furthermore, the bio-inspired reward shaping accelerates learning convergence and minimizes cumulative muscle activation, thereby ensuring energy efficiency.

The proposed method successfully synthesizes energy-efficient , reproducible, and highly human-like STS behaviors in a musculoskeletal model. By explicitly embedding physiological constraints into the exploration space, synergy-informed DRL effectively overcomes the curse of dimensionality, offering a promising approach to enhance biomechanical realism and neuromechanical control in assistive robotics and humanoid motion.