FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidance.

Area of Science:

• Computer Vision
• Robotics
• Human-Computer Interaction

Background:

• Synthesizing 3D hand-object interactions (HOI) is complex due to intricate hand movements, contact dynamics, and geometric ambiguities.
• Current methods often require 3D HOI annotations, object templates, or interaction priors, limiting their use in diverse grasping scenarios.

Purpose of the Study:

• To develop an annotation-free framework for synthesizing functional 3D HOI from natural language prompts.
• To address the limitations of existing methods in open-ended functional grasping scenarios.

Main Methods:

• Introduced FunHOI, a two-stage, annotation-free framework for functional-intent-conditioned 3D HOI synthesis.
• Utilized a Functional Grasp Generator (FGG) to translate natural language prompts into 2D HOI guidance images.
• Employed a Functional Grasp Refiner (FGR) for reconstructing and refining 3D hand-object composition, focusing on scale-aware alignment and contact-driven improvements.

Main Results:

• FunHOI demonstrated competitive reconstruction quality for 3D HOI.
• Achieved improved plausibility in hand-object contacts and better alignment with functional intent.
• Successfully synthesized functional 3D HOI without relying on 3D HOI annotations.

Conclusions:

• FunHOI offers a novel approach to generating functional 3D HOI from text, overcoming limitations of previous methods.
• The framework's ability to work without 3D annotations broadens its applicability to real-world scenarios.
• This research advances the field of human-object interaction synthesis, enabling more versatile and context-aware robotic grasping.