Adaptive 3D Model-Based Facial Expression Synthesis and Pose Frontalization.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Biometrics

Background:

• Facial expressions are crucial for non-verbal communication and emotion analysis.
• Modeling facial expressions from images, especially with diverse face poses, is challenging due to complex muscle movements.

Purpose of the Study:

• To develop a robust method for acquiring and reproducing facial expressions from non-frontal single photographs.
• To enhance facial expression modeling accuracy and enable realistic synthesis for various applications.

Main Methods:

• A 3D-aided approach for facial expression acquisition from single, non-frontal images.
• A contour-fitting method to improve 3D contour landmark accuracy against 2D image landmarks.
• Parametric manipulation of expressions using blendshape or expression transfer based on the Facial Action Coding System (FACS).
• An exemplar-texture wrinkle synthesis method using a constructed wrinkle table from 400 individuals.

Main Results:

• Successful acquisition and parametric manipulation of facial expressions from single images.
• Improved modeling accuracy through the contour-fitting method.
• Realistic facial expression synthesis, including wrinkles, via the exemplar-texture method.
• Demonstrated suitability for expression-invariant face recognition through quantitative and qualitative evaluations.

Conclusions:

• The proposed 3D-aided method effectively models facial expressions from single images, overcoming pose variations.
• The system enables realistic expression synthesis and has practical applications in fields like face recognition, HCI, and deep learning data augmentation.