ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusion.

Area of Science:

• Robotics and Artificial Intelligence
• Computer Vision and Graphics

Background:

• Dynamic object manipulation requires synchronized hand motions and object trajectories for plausible interactions.
• Accurate modeling of contact correspondences between hands and objects is critical for dexterous manipulation.

Purpose of the Study:

• To introduce ManiDext, a unified hierarchical diffusion-based framework for generating hand manipulation and grasp poses.
• To enable seamless and physically plausible human-object interactions in robotic systems.

Main Methods:

• Developed a continuous correspondence embedding representation for detailed, vertex-level object-hand contact modeling.
• Implemented a self-supervised optimization of the hand mesh embedding, reflecting geodesic distance.
• Integrated iterative refinement into the diffusion process by using hand pose residuals as refinement targets during denoising.

Main Results:

• Generated physically plausible and highly realistic motions for diverse manipulation tasks.
• Demonstrated success in single and bimanual hand grasping scenarios.
• Showcased effective manipulation of both rigid and articulated objects.

Conclusions:

• ManiDext offers a unified framework merging generation and refinement for dexterous hand manipulation.
• The proposed continuous correspondence embedding and iterative refinement within diffusion significantly improve motion realism and plausibility.