A Novel Robotic Pushing and Grasping Method Based on Vision Transformer and Convolution.

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Robotic grasping in cluttered environments presents significant challenges due to limited space.
• Existing methods struggle to identify optimal grasp poses when objects are closely packed.

Purpose of the Study:

• To develop an effective method for robotic grasping in cluttered scenes.
• To enhance grasp pose detection and execution using a combined pushing and grasping strategy.

Main Methods:

• Proposed a pushing-grasping combined grasping network (GN) utilizing a transformer and convolution (PGTC) approach.
• Introduced a vision transformer (ViT)-based pushing transformer network (PTNet) for predicting object positions after pushing.
• Developed a cross-dense fusion network (CDFNet) for accurate grasp detection using fused RGB and depth images.

Main Results:

• PTNet effectively captures global and temporal features for improved object position prediction post-push.
• CDFNet demonstrates superior accuracy in detecting optimal grasping positions by effectively fusing multi-modal sensor data.
• The combined PGTC method achieved state-of-the-art (SOTA) performance in both simulation and real-world UR3 robot experiments.

Conclusions:

• The proposed pushing and grasping (PG) strategy significantly enhances robotic grasping capabilities in cluttered scenes.
• The developed PGTC network, incorporating PTNet and CDFNet, offers a robust solution for complex grasping tasks.
• This research advances the field of robotic manipulation by enabling more reliable grasping in previously challenging environments.