RobustFusion: Robust Volumetric Performance Reconstruction Under Human-Object Interactions From Monocular RGBD...

Area of Science:

• Computer Vision
• Computer Graphics
• Human-Computer Interaction

Background:

• High-quality 4D reconstruction of human performance with object interactions is crucial for immersive virtual and augmented reality (VR/AR).
• Existing methods struggle with complex interactions and occlusions, particularly in monocular settings.

Purpose of the Study:

• To propose RobustFusion, a robust volumetric performance reconstruction system for human-object interaction using a single RGBD sensor.
• To address challenges of complex interactions and severe occlusions in monocular 4D reconstruction.

Main Methods:

• A semantic-aware scene decoupling scheme for explicit occlusion modeling.
• Segmentation refinement and robust object tracking for temporal consistency.
• A data-driven performance capture scheme with spatial relation priors and interaction cues.
• Adaptive fusion with occlusion analysis and human parsing for coherent reconstruction.

Main Results:

• RobustFusion achieves high-quality 4D human performance reconstruction in complex human-object interaction scenarios.
• The system effectively handles severe occlusions and disentanglement uncertainty.
• Maintains temporal consistency and natural motion, even in occluded regions.

Conclusions:

• RobustFusion provides a lightweight yet effective solution for 4D human performance reconstruction from a single RGBD sensor.
• The proposed methods significantly improve reconstruction quality under challenging human-object interaction conditions.
• Enables more realistic and immersive VR/AR experiences through accurate human-object interaction capture.