Meta-evaluation for 3D Face Reconstruction Via Synthetic Data.

Area of Science:

• Computer Vision
• 3D Geometry Processing
• Machine Learning

Background:

• Geometric error is the standard metric for 3D face reconstruction.
• Current methods rely on the Chamfer criterion for point correspondence on real scans.
• The appropriateness of Chamfer error as a benchmark metric is questioned.

Purpose of the Study:

• To introduce a meta-evaluation framework for assessing geometric error estimators in 3D face reconstruction.
• To compare the fairness and accuracy of different geometric error estimators.
• To address fundamental questions about benchmark metric quality in 3D face reconstruction.

Main Methods:

• Development of a meta-evaluation framework utilizing synthetic data.
• Experimental comparison of four geometric error estimators, including Chamfer and non-rigid ICP.
• Analysis of error underestimation and ranking consistency across reconstruction methods.

Main Results:

• The standard Chamfer error metric significantly underestimates geometric error in 3D face reconstruction.
• Chamfer error exhibits inconsistent underestimation across different reconstruction methods, altering their performance rankings.
• Non-rigid ICP shows reduced bias but still fails to rank all methods correctly and is computationally expensive.

Conclusions:

• The current benchmarking approach for 3D face reconstruction using Chamfer error is inadequate.
• A novel meta-evaluation framework using synthetic data provides a more reliable method for assessing benchmark metrics.
• The findings necessitate a re-evaluation of standard practices in 3D face reconstruction evaluation.