M² Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Diffusion models show promise for embodied AI but struggle with mobile manipulation due to complex action spaces and environmental interactions.
• Coordinating navigation and manipulation in robots remains a significant challenge for current generative AI techniques.
• Existing methods often fail to adequately address the high-dimensional action spaces and extended trajectories required for mobile manipulation.

Purpose of the Study:

• Introduce M² Diffuser, a novel diffusion-based generative model for mobile manipulation.
• Enable the direct generation of coordinated, efficient whole-body motion trajectories for robots.
• Integrate physical constraints and task objectives into the generative process for improved safety and robustness.

Main Methods:

• Developed M² Diffuser, a scene-conditioned generative model utilizing robot-centric 3D scans.
• Trained the model on expert-planned mobile manipulation trajectories to learn distributions.
• Incorporated a differentiable optimization module to enforce physical constraints and task objectives during inference.

Main Results:

• M² Diffuser generated efficient and coordinated motion trajectories for mobile manipulation tasks.
• Benchmarking across three task types and over 20 scenes demonstrated superior performance compared to state-of-the-art neural planners.
• Successfully transferred generated trajectories to a real-world robot, validating practical applicability.

Conclusions:

• Generative AI, enhanced by M² Diffuser, can significantly improve the generalization of robotic planning and learning methods.
• Enforcing physical constraints during motion generation is crucial for safe and robust robotic execution.
• M² Diffuser represents a significant step towards enabling advanced mobile manipulation capabilities in embodied AI.