Recovering Joint and Individual Components in Facial Data

Area of Science:

• Computer Vision
• Statistical Learning
• Image Analysis

Background:

• Facial image analysis requires distinguishing shared (joint) and unique (individual) components for tasks like expression transfer and age progression.
• Unconstrained facial images often contain noise, outliers, and missing data, posing significant challenges for component discovery.

Purpose of the Study:

• To adapt and enhance the Joint and Individual Variance Explained (JIVE) method for robust facial component recovery.
• To develop JIVE alternatives that are resilient to sparse, non-Gaussian noise and missing data while automatically determining component ranks.

Main Methods:

• Investigated the statistical Joint and Individual Variance Explained (JIVE) method for facial component recovery.
• Proposed novel JIVE-based alternatives robust to sparse gross errors, non-Gaussian noise, and missing data.
• Developed methods for automatic rank determination of individual components.

Main Results:

• Demonstrated the effectiveness of the proposed robust JIVE alternatives for facial component analysis.
• Successfully applied the methods to challenging 'in-the-wild' facial datasets.
• Achieved accurate facial expression synthesis and 2D/3D age progression.

Conclusions:

• The proposed robust JIVE methods significantly improve the recovery of joint and individual components from challenging facial image data.
• These advancements enable more reliable facial expression synthesis and age progression, particularly in unconstrained environments.