Cross-Modality Alignment Perception and Multi-Head Self-Attention Mechanism for Vision-Language-Action of Humanoid...

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Humanoid robots face challenges in precise motion trajectory prediction for complex, multi-step tasks, often resulting in robotic arm jitter.
• End-to-end imitation learning and Vision-Language-Action (VLA) models, while promising, can suffer from high computational complexity, particularly within self-attention mechanisms.

Purpose of the Study:

• To address the limitations of current imitation learning methods in humanoid robotics.
• To reduce the computational complexity of self-attention modules in VLA operations.
• To enhance the motion prediction accuracy and reduce jitter in humanoid robot arms during complex tasks.

Main Methods:

• Proposed a novel memory-gated filtering attention model to improve the multi-head self-attention mechanism.
• Developed a cross-modal alignment perception strategy for enhanced training.
• Implemented a few-shot data-collection approach for critical task steps.

Main Results:

• Significantly improved task success rates in humanoid robots.
• Effectively alleviated the robot arm jitter problem.
• Reduced video memory usage by 72% and improved training speed by over 10x (from 1.35s to 0.129s per batch).

Conclusions:

• The proposed memory-gated filtering attention model enhances VLA operations for humanoid robots.
• The approach successfully improves task performance, reduces computational load, and increases training efficiency.
• This method offers a more robust and accurate solution for complex robotic manipulation tasks.