FunHOI: Annotation-Free 3D Hand-Object Interaction Generation via Functional Text Guidance

Hand-object interaction (HOI) is central to human manipulation, but synthesizing functional 3D HOI remains challenging due to dexterous hand motion, complex contact patterns, and strong geometric ambiguity. Existing methods often rely on 3D HOI annotations, predefined object templates, or supervised interaction priors. Such requirements limit their applicability to open-ended functional grasping scenarios, where the same object may require different contact regions and poses depending on the intended use. We present FunHOI, an annotation-free two-stage framework for functional-intent-conditioned 3D HOI synthesis from natural-language prompts. FunHOI represents each prompt using action-object-purpose semantics and first translates the functional intent into a semantically aligned 2D HOI guidance image through the Functional Grasp Generator (FGG). Rather than learning a direct text-to-3D generative prior, FunHOI reconstructs the hand and object from this 2D semantic guidance and composes them in 3D using the Functional Grasp Refiner (FGR), which performs scale-aware alignment, object pose approximation, and contact-driven refinement to improve spatial consistency and physical plausibility. Extensive experiments show that FunHOI achieves competitive reconstruction quality, improved hand-object contact plausibility, and better alignment with functional intent, without requiring 3D HOI annotations.