Visual and tactile 3D point cloud data from real robots for shape modeling and completion

Area of Science:

• Robotics
• Computer Vision
• Computer Graphics
• 3D Geometry

Background:

• Robotic applications require accurate 3D shape perception from sensory data.
• Noisy and incomplete data present significant challenges for object shape modeling.
• Existing methods for shape estimation and completion need robust datasets for validation.

Purpose of the Study:

• To provide a comprehensive dataset of real sensory data for 3D shape modeling.
• To facilitate the analysis and comparison of novel methods for shape estimation, completion, and exploration.
• To support research in robotic perception using combined visual and tactile sensing.

Main Methods:

• Acquired 3D point cloud data using two robot setups with integrated visual and tactile sensors.
• Collected data through predefined exploration paths, ensuring visual and tactile points share a common coordinate frame.
• Focused on sensing unseen object parts via tactile sensors at touched areas to complement visual data.

Main Results:

• The dataset comprises visual and tactile readings from diverse objects of varying complexity.
• Data was utilized in studies on shape completion and modeling using Implicit Surface representation and Gaussian-Process-based regression.
• Partially used in research on enhancing visual perception through tactile feedback and efficient object exploration.

Conclusions:

• The provided dataset is valuable for advancing research in 3D shape representation and reconstruction in robotics.
• It enables the development of more robust algorithms for handling noisy and incomplete sensory information.
• Facilitates the study of efficient exploration strategies combining visual and tactile sensing for comprehensive object modeling.