Dense 3D Face Alignment from 2D Video for Real-Time Use

Area of Science:

• Computer Vision
• 3D Reconstruction
• Biometrics

Background:

• Accurate 3D facial reconstruction from 2D images is crucial for various applications.
• Existing methods often struggle with real-time performance and pose variations.

Purpose of the Study:

• To develop a real-time, person-independent 3D facial registration system from 2D video.
• To achieve robust 3D face shape reconstruction across a wide range of poses and imaging conditions.

Main Methods:

• A 3D cascade regression approach was developed, ensuring facial landmarks remain invariant across pose variations (approx. 60 degrees).
• The algorithm estimates dense landmark locations and visibility, then reconstructs 3D face shapes using a part-based 3D model.
• The framework is trained on high-resolution 3D face scans, accommodating posed and spontaneous expressions.

Main Results:

• Real-time, dense 3D face shape registration from single 2D images was achieved.
• The method demonstrated robustness across various imaging conditions, including different illumination and surface properties.
• Validation confirmed precision in 3D reconstruction, multi-view extension, temporal integration, and 3D head-pose estimation.

Conclusions:

• The developed 3D cascade regression method enables valid real-time 3D registration and reconstruction from 2D video.
• The approach is applicable to diverse imaging scenarios, including uncalibrated multi-view video.
• The software is publicly available for further research and application.