Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset

Area of Science:

• Robotics
• Computer Vision
• Neuromorphic Engineering

Background:

• Current robotic grasping relies on conventional RGB-D cameras, limiting agility.
• Neuromorphic vision offers potential for enhanced robotic perception but lacks large-scale datasets.
• Annotating event-based data is challenging, hindering neuromorphic vision dataset development.

Purpose of the Study:

• To develop a robust robotic grasping detection system using neuromorphic vision.
• To create a comprehensive, high-frequency annotated dataset for neuromorphic grasping.
• To advance the application of neuromorphic vision in agile robotic systems.

Main Methods:

• Utilized a Dynamic and Active-pixel Vision Sensor (DAVIS) attached to a robot gripper.
• Constructed the 'Event-Grasping' dataset with 91 objects, annotated at 1 kHz using a spatial-temporal mixed particle filter.
• Developed a deep neural network treating grasp angle learning as classification.

Main Results:

• Achieved 93% precision in robotic grasping detection on the Event-Grasping dataset.
• Demonstrated the effectiveness of neuromorphic vision for agile robotic grasping.
• Provided a large-scale, high-frequency annotated dataset for future research.

Conclusions:

• The Event-Grasping dataset and proposed deep learning method significantly improve robotic grasping detection.
• Neuromorphic vision sensors are viable for agile robotic perception and grasping tasks.
• This work facilitates further research and application of neuromorphic vision in robotics.