Text-Controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Human motion understanding is vital for applications like robotics and animation.
• Existing text-motion tasks include generation and editing, but temporal localization remains challenging.
• Transformer models struggle with long sequences due to quadratic computational costs.

Purpose of the Study:

• Introduce text-based human motion grounding (THMG) for precise temporal segment localization.
• Address computational challenges of Transformers in handling long, untrimmed motion sequences.
• Propose a novel, efficient model for text-controlled motion analysis.

Main Methods:

• Developed Text-controlled Motion Mamba (TM-Mamba), a unified model with linear memory cost.
• Integrated temporal global context, language query control, and spatial graph topology.
• Introduced a text-controlled selection mechanism for dynamic global information incorporation.
• Enhanced model awareness through relational embeddings for spatial graph topology.

Main Results:

• TM-Mamba demonstrated effectiveness in text-based human motion grounding.
• The model efficiently processes long untrimmed sequences with linear memory complexity.
• Achieved strong performance on the newly introduced BABEL-Grounding dataset.

Conclusions:

• TM-Mamba offers an efficient and effective solution for text-based human motion grounding.
• The proposed method overcomes limitations of Transformer models for long sequence analysis.
• The BABEL-Grounding dataset facilitates future research in this domain.