Dynamic 4D facial capture pipeline with appearance driven progressive retopology based on optical flow

Area of Science:

• Computer Vision
• Computer Graphics
• 3D Imaging

Background:

• Current 3D facial reconstruction methods often focus on single frames and require manual annotation.
• Existing techniques can suffer from mesh drifting and jitter, hindering temporal coherence.
• Preserving artist-specified wireframe structures in dynamic reconstructions remains a challenge.

Purpose of the Study:

• To develop a production-oriented 4D facial reconstruction pipeline for high-fidelity facial mesh sequences.
• To create a system that autonomously establishes feature correspondence and temporal coherence.
• To preserve the wireframe structure specified by artists throughout the reconstruction process.

Main Methods:

• A compact, efficient, and fast optical capture system using synchronized camera arrays for dynamic 3D facial imaging.
• Exploiting appearance consistency constraints for autonomous feature correspondence and temporal coherence.
• Utilizing progressive retopology with artist-guided templates to maintain wireframe structures.

Main Results:

• Generated high-fidelity facial mesh sequences with consistently structured topology.
• Successfully eliminated mesh drifting and jitter, enabling full parallelization for dynamic facial expression capture.
• Demonstrated faithful reproduction of character expressions from photographs with artist-friendly stable facial movements.

Conclusions:

• The proposed pipeline offers a robust solution for production-quality 4D facial reconstruction.
• The method achieves high accuracy in capturing dynamic facial expressions while preserving essential structural details.
• This approach advances the field of 3D facial imaging for animation and digital character creation.