Vision-Based Efficient Robotic Manipulation with a Dual-Streaming Compact Convolutional Transformer.

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Robotic manipulation via Reinforcement Learning (RL) faces challenges in learning from visual observation.
• Convolutional Neural Network (CNN) based visual encoders limit multi-task performance and increase optimization costs.
• Attention mechanisms offer potential for improved visual understanding in RL.

Purpose of the Study:

• To develop an efficient and generalizable robotic manipulation method using RL.
• To enhance multi-task performance by improving the visual encoder's efficacy.
• To reduce the data and training time required for effective visual learning in robots.

Main Methods:

• Utilized a lite Transformer-based visual encoder with an attention mechanism.
• Incorporated unsupervised learning and data augmentation techniques.
• Combined third-person and egocentric views, prioritizing the master view for visual input.

Main Results:

• Achieved 92% success rate on OpenAI Gym Fetch tasks, significantly outperforming baselines (65%-81%).
• Demonstrated comparable Transformer performance with substantially less training data.
• Showcased the effectiveness of the master view in assimilating global and local visual information.

Conclusions:

• The proposed method significantly improves general multi-task robotic manipulation performance.
• The Transformer-based visual encoder offers an efficient and effective alternative to CNNs.
• The strategy of combining multiple camera views enhances visual perception for robotic tasks.