Real-Time 3D Facial Tracking via Cascaded Compositional Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Biomedical Imaging

Background:

• Real-time 3D facial tracking is crucial for applications like augmented reality and animation.
• Existing methods often require extensive training data or high computational resources.
• Handling multi-modal facial motion and diverse output variables presents a significant challenge.

Purpose of the Study:

• To develop an efficient and robust method for real-time 3D facial tracking from monocular RGB cameras.
• To address the challenges of multi-modal facial motion regression and variable modality.
• To investigate the impact of synthetic data on training non-deep learning facial tracking models.

Main Methods:

• Proposed globally-optimized modular boosted ferns (GoMBF), a cascade of regression models.
• Each GoMBF module is initially trained on partial motion parameters and globally optimized.
• Cascaded GoMBFs (GoMBF-Cascade) were used for 3D facial tracking, with experiments involving real and synthetic training data.

Main Results:

• GoMBF-Cascade achieved competitive tracking performance on in-the-wild videos compared to state-of-the-art methods.
• The method demonstrates increased fitting power and faster learning speed than conventional boosted ferns.
• Models trained purely on synthetic data showed poor generalization to real-world data; mixed data offered limited benefits.

Conclusions:

• GoMBF-Cascade offers a robust, elegant, and practical solution for real-time 3D facial tracking with minimal training data.
• Synthetic data alone is insufficient for training effective non-deep learning facial tracking models.
• Careful consideration of synthetic data integration is necessary for non-deep learning facial image analysis tasks.