MARRS: Masked Autoregressive Unit-Based Reaction Synthesis.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human Motion Synthesis

Background:

• Autoregressive models with vector quantization (VQ) are common for motion generation.
• VQ has limitations like information loss and low codebook utilization.
• Existing methods often neglect mutual perception between body parts.

Purpose of the Study:

• To develop a novel framework (MARRS) for human action reaction synthesis.
• To generate coordinated and fine-grained human reaction motions.
• To overcome limitations of current VQ-based approaches.

Main Methods:

• Proposed MARRS framework utilizing continuous representations.
• Unit distinguished Motion Variational AutoEncoder (UD-VAE) for independent body and hand unit encoding.
• Action Conditioned Fusion (ACF) for extracting information from reactive tokens.
• Mutual Unit Modulation (MUM) for adaptive interaction between body and hand units.
• Diffusion model with compact MLP noise predictor and diffusion loss.

Main Results:

• Achieved superior performance in quantitative and qualitative evaluations.
• Demonstrated effective generation of coordinated and fine-grained reaction motions.
• Successfully addressed limitations of VQ in motion synthesis.

Conclusions:

• MARRS framework offers an effective approach for human action reaction synthesis.
• The proposed methods (UD-VAE, ACF, MUM) enhance motion generation quality.
• The approach generates more coordinated and detailed human reactions compared to existing methods.