HiTMM: Generative Temporal Masked Modeling of Human Interactive Motions

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Generating realistic two-person interactive motions is challenging due to overlooked temporal dependencies and participant roles.
• Existing models often use independent timelines, leading to unnatural motion generation.

Purpose of the Study:

• To propose HiTMM, a novel framework for human interaction generation using Temporal Masked Modeling.
• To improve the realism and naturalness of generated two-person motions by explicitly modeling temporal dependencies and roles.

Main Methods:

• Decomposing human interaction into separate single-person motions mapped to a shared latent space via multi-layer discrete tokens.
• Arranging all motion tokens along a single timeline to capture temporal order and initiating roles.
• Employing masked and residual transformers to model motion tokens on the shared timeline.

Main Results:

• Achieved a lower FID score of 5.017 on the InterHuman dataset compared to the state-of-the-art (5.154).
• Achieved a lower FID score of 0.373 on the InterX dataset compared to the state-of-the-art (0.399).
• Demonstrated the capability for temporal editing within human interaction sequences due to the shared temporal representation.

Conclusions:

• HiTMM effectively models temporal dependencies and participant roles in human interaction generation.
• The proposed Temporal Masked Modeling approach significantly enhances the realism of generated two-person motions.
• HiTMM offers a promising direction for future research in human-human interaction generation and temporal editing.