Facial Expression Recognition with Geometric Scattering on 3D Point Clouds

Area of Science:

• Computer Vision
• Geometric Deep Learning
• 3D Data Analysis

Background:

• Point clouds are crucial for detailed geometry but susceptible to noise in raw sensor data.
• Existing 3D Facial Expression Recognition (3D FER) methods struggle with high-dimensional data and environmental variations.
• Grid-based methods entangle features with pose and illumination variations.

Purpose of the Study:

• To develop an efficient and robust framework for 3D Facial Expression Recognition (3D FER) directly from point cloud data.
• To address challenges in feature abstraction and stabilization for noisy 3D facial data.
• To enable accurate expression recognition without requiring predefined meshes or additional signals.

Main Methods:

• Proposed a localized and smoothed overlapping kernel for extracting discriminative geometric features from point clouds.
• Utilized manifold scattering transform to associate deformation stability with external perturbations.
• Developed a novel framework that directly processes point cloud coordinates for 3D FER.

Main Results:

• Achieved 78.33% accuracy on the Bosphorus dataset for expression recognition.
• Attained 77.55% accuracy on the 3D-BUFE dataset.
• Demonstrated a compact framework capable of direct point cloud consumption for FER.

Conclusions:

• The proposed framework offers a robust and efficient solution for 3D Facial Expression Recognition (3D FER) using raw point cloud data.
• The method effectively extracts inherent geometric features, providing stability against extrinsic variations.
• This approach advances the field by enabling direct point cloud analysis for complex facial expression tasks.