Complementary visual localization and tactile mapping approach for robotic perception of millimeter-sized objects...

Area of Science:

• Robotics
• Human-Machine Interaction
• Sensor Technology

Background:

• Humanoid robots require advanced perception for manipulating small, irregular objects in extreme environments (e.g., space, hazardous zones).
• Vision-based systems face limitations in detecting object size and are sensitive to occlusion and lighting.
• Tactile perception provides crucial data on shape, surface texture, and forces, complementing visual information.

Purpose of the Study:

• To develop a complementary visual localization and tactile mapping framework for robots.
• To enable effective perception and manipulation of small, irregular objects in visually restricted environments.
• To enhance humanoid robot autonomy and adaptability in unpredictable conditions.

Main Methods:

• Utilized an RGB-Depth camera for visual perception (localization and size detection).
• Employed an inkjet-printed soft pressure sensor array for tactile perception (surface identification and 3D reconstruction).
• Drew inspiration from human sequential vision-tactile sensory processing.

Main Results:

• Demonstrated sensory substitution for object detection and localization using visual data.
• Successfully identified object surfaces and reconstructed 3D profiles via tactile scanning.
• Validated the framework's effectiveness in environments with limited visual information.

Conclusions:

• The proposed framework enhances humanoid robot capabilities for precise manipulation in challenging, unstructured environments.
• This research lays the technological foundation for more autonomous and adaptable robots in extreme conditions.
• Integrating vision and tactile sensing offers a robust solution for complex object interaction tasks.