Dual-Expert Landmark Localization in 3D Facial Point Clouds Under Controlled Synthetic Local Surface Loss for Rigid...

Area of Science:

• Computer Vision
• 3D Reconstruction
• Geometric Deep Learning

Background:

• Landmark-based rigid initialization in 3D facial point clouds is sensitive to local surface loss.
• Existing methods struggle with occluded or incomplete facial data.

Purpose of the Study:

• To develop a robust dual-expert framework for accurate landmark localization in 3D facial point clouds with synthetic surface loss.
• To improve the reliability of 3D facial reconstruction and analysis under challenging data conditions.

Main Methods:

• A dual-expert framework combining a Clean expert for complete surfaces and an Occlusion expert for handling surface loss.
• The Occlusion expert utilizes local coordinate regression, visibility estimation, heteroscedastic modeling, and a global structural prior.
• A reliability gate and validation-selected residual application manage expert output based on data completeness.

Main Results:

• Achieved mean localization errors of 0.470±0.688 mm (0% loss), 1.159±1.093 mm (30% loss), and 2.216±2.169 mm (50% loss) on the FaceScape dataset.
• Significantly outperformed Unified OASR and the Occlusion expert at 30% and 50% surface loss.
• Demonstrated 100% iterative closest point (ICP) success in large-pose stress tests, improving coarse alignment.

Conclusions:

• The proposed dual-expert framework enables robust rigid initialization of 3D facial point clouds under controlled synthetic surface loss.
• The method shows superior performance compared to existing models in handling occluded facial data.
• Further research is needed to validate generalization to real-world sensor-acquired point clouds.