PBO-SAC: optimized soft actor-critic for a 7-DoF pneumatic humanoid robotic arm with Bowden cable transmission

Area of Science:

• Robotics
• Artificial Intelligence
• Biomimetic Engineering

Background:

• Humanoid robots require human-like appearance and motion for seamless human-robot interaction.
• Pneumatic artificial muscles (PAMs) offer biological similarity but have limited contraction ratios, constraining robot design and movement.
• Existing robotic arms face challenges in achieving anthropomorphic scaling and complex motion control.

Purpose of the Study:

• To develop a 7-degrees-of-freedom (DoF) pneumatic humanoid robotic arm that replicates human arm appearance and movement.
• To address the control challenges of nonlinear dynamics in PAMs and Bowden cables for robotic arms.
• To propose and validate a novel model-free reinforcement learning framework for on-hardware control policy learning.

Main Methods:

• A hybrid actuation scheme combining direct PAM actuation and PAM with Bowden cable transmission for shoulder and distal joints.
• Development of the Pneumatic Bowden cable Optimized Soft Actor-Critic (PBO-SAC) framework, a model-free reinforcement learning approach.
• Incorporation of posture-perturbed decoupled training and local recurrent fusion networks within PBO-SAC for safe and smooth learning.

Main Results:

• The hybrid actuation scheme enabled anthropomorphic scaling with a lightweight and compliant structure.
• PBO-SAC demonstrated efficient on-hardware control policy learning for the multi-DoF robotic arm.
• Hardware experiments validated trajectory tracking and teleoperated stacking task performance, showcasing effective multi-DoF coordination control.

Conclusions:

• The developed 7-DoF pneumatic robotic arm successfully mimics human arm appearance and movement capabilities.
• The PBO-SAC framework provides an effective solution for controlling complex robotic systems with nonlinear dynamics.
• This research advances human-robot interaction through more capable and adaptable robotic systems.