Dynamic Gaussian-Based Digital Twin Reconstruction of Articulated Multi-Joint Objects

Area of Science:

• Computer Graphics
• Robotics
• 3D Reconstruction

Background:

• Existing 3D Gaussian Splatting (3DGS) methods struggle with dynamic, multi-joint objects, causing inconsistencies in simulations.
• Accurate digital twins of articulated objects are crucial for advanced graphics, simulation, and embodied AI applications.

Purpose of the Study:

• To develop a structure-aware framework for dynamic and editable digital twin reconstruction of articulated objects.
• To overcome the limitations of static scene reconstruction in existing 3DGS approaches.

Main Methods:

• A novel framework attaching articulated Gaussian primitives to a kinematic hierarchy.
• Utilizing differentiable rendering for reconstruction from multi-view observations.
• Explicitly modeling link-level geometry and motion coupling for temporal consistency.

Main Results:

• Achieved temporally consistent digital twin reconstructions that generalize to new viewpoints and joint configurations.
• Demonstrated improved visual fidelity, geometric accuracy, and cross-view consistency on articulated robotic systems.
• Successfully facilitated scalable simulation and enhanced sim-to-real policy transfer.

Conclusions:

• The proposed structure-aware framework enables high-fidelity, dynamic digital twin reconstruction for articulated objects.
• Dynamic Gaussian-based modeling is vital for realistic digital twins in simulation and robotics.
• The method enhances the utility of digital twins for applications like sim-to-real policy transfer.