Geometric Affordance Perception: Leveraging Deep 3D Saliency With the Interaction Tensor

Area of Science:

• Robotics
• Computer Vision
• Artificial Intelligence

Background:

• Agents acting in their surroundings benefit from understanding interaction possibilities (affordances).
• Existing methods may lack efficiency or generalization capabilities for real-world affordance perception.

Purpose of the Study:

• To develop a computationally efficient and generalizable method for affordance perception in 3D environments.
• To combine geometric scene understanding with deep learning for robust affordance identification.

Main Methods:

• Utilized the Interaction Tensor, a geometric representation for object-scene interactions.
• Integrated deep learning saliency for rapid parsing of environmental affordances from 3D pointclouds.
• Developed a one-shot generalization approach for interaction descriptions.

Main Results:

• Successfully identified affordance candidate locations (e.g., sitting, riding, placing objects) in novel synthetic and real RGB-D scenes.
• Demonstrated one-shot generalization, predicting affordances from a single training example.
• Achieved fast processing rates and high parallelizability for multiple affordance representations.

Conclusions:

• The combined approach of deep learning saliency and one-shot geometric representation provides a perceptually direct and economical model for affordance estimation.
• This method enhances agent interaction capabilities in complex 3D environments.