Spatial-Temporal Relation Guided Motion Transfer via Diffusion Model

Area of Science:

• Robotics and Virtual Reality
• Computer Vision
• Human-Computer Interaction

Background:

• Existing methods for Human-Object Interaction (HOI) motion transfer primarily focus on surface correspondences.
• These traditional approaches neglect crucial internal topological structures of humans and objects, as well as non-surface spatial and temporal relationships.
• This limitation hinders the accurate and adaptable transfer of HOI motions to novel objects.

Purpose of the Study:

• To develop an advanced framework for transferring HOI motion to novel objects.
• To address the limitations of existing methods by incorporating internal topologies and temporal dynamics.
• To generate spatially, semantically, and temporally consistent HOI motions adapted to new object geometries.

Main Methods:

• Introduction of a spatial-temporal relation interaction graph representation (STRIG) to model human and object internal topologies, global human-object topology, and temporal motion relations.
• Development of a STRIGs-guided motion transfer diffusion model for generating HOI motions.
• Implementation of a spatial-temporal relation optimization strategy to address the lack of ground-truth data in motion transfer.

Main Results:

• The proposed method demonstrates superior performance in motion transfer quality, efficiency, and sequence stability compared to existing approaches.
• The framework exhibits robustness, particularly when dealing with target objects possessing significant topological variations.
• Generated motions are spatially, semantically, and temporally consistent with the interaction context.

Conclusions:

• The spatial-temporal relation guided motion transfer framework offers a significant advancement in HOI motion adaptation.
• STRIGs provide a comprehensive representation for capturing complex interaction dynamics.
• The diffusion model and optimization strategy effectively generate high-quality, stable, and robust HOI motions for novel objects.