Fast 3D Face Reconstruction from a Single Image Using Different Deep Learning Approaches for Facial Palsy Patients

Area of Science:

• Medical Imaging
• Computer Vision
• Deep Learning

Background:

• Accurate 3D face models are crucial for clinical decision support.
• Existing methods using medical imaging or depth sensors lack portability and ease of use.
• 3D face reconstruction for facial palsy patients from single images remains an underexplored challenge.

Purpose of the Study:

• To apply state-of-the-art deep learning (DL) methods for 3D face shape reconstruction of facial palsy patients from single images.
• To evaluate the accuracy of DL-based reconstruction against Kinect-driven and MRI-based 3D models.
• To explore novel approaches for fast and accurate 3D facial reconstruction in clinical settings.

Main Methods:

• Utilized three DL methods: 3D Basel Morphable model and two 3D Deep Pre-trained models.
• Applied methods to a dataset including healthy subjects and facial palsy patients in natural and mimic poses.
• Compared reconstructed 3D face shapes against Kinect-driven and MRI-based ground truth data.

Main Results:

• Achieved a best mean error of 1.5±1.1 mm compared to Kinect-driven reconstructions.
• Obtained a best error range of 1.9±1.4 mm when compared to MRI-based reconstructions.
• Identified areas for improving reconstruction accuracy in future studies.

Conclusions:

• Deep learning methods show potential for accurate 3D face reconstruction from single images for facial palsy patients.
• The findings open new avenues for rapid 3D facial modeling in diagnosing and managing facial disorders.
• The best-performing DL method is planned for integration into a computer-aided decision support system for facial disorders.