MoFTSS: Motion Generation With Frequency and Text State Space Models

Area of Science:

• Computer Vision
• Artificial Intelligence
• Human Motion Generation

Background:

• Text-driven diffusion models show promise in human motion generation.
• Existing models struggle with fine-grained motion detail and text-motion consistency.
• Challenges include distinguishing motion representations in latent diffusion and multimodal space misalignment.

Purpose of the Study:

• To propose a novel framework, Motion generation with Frequency and Text State Space models (MoFTSS), to improve text-driven human motion generation.
• To address limitations in fine-grained motion modeling and multimodal alignment.
• To generate higher-quality and more consistent human motion from textual descriptions.

Main Methods:

• Introduced MoFTSS, comprising frequency state space model (FreqSSM) and text state space model (TextSSM).
• FreqSSM decomposes motion sequences into low and high-frequency components for detailed pose and transition generation.
• TextSSM integrates text features as a semantic modulation term for dynamic filtering of motion features, ensuring text-motion alignment.

Main Results:

• MoFTSS demonstrated superior performance in text-to-motion generation tasks.
• Achieved a significantly lower Frechet Inception Distance (FID) of 0.181 on the HumanML3D dataset compared to the previous 0.421.
• Showcased improved generation of static poses and fine-grained motions, along with enhanced consistency with textual descriptions.

Conclusions:

• MoFTSS effectively overcomes limitations in current text-driven motion generation models.
• The proposed FreqSSM and TextSSM modules significantly enhance motion quality and semantic consistency.
• MoFTSS represents a substantial advancement in generating realistic and text-aligned human motion.