Joint Local and Global Information Learning With Single Apex Frame Detection for Micro-Expression Recognition.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Affective Computing

Background:

• Micro-expressions (MEs) are subtle, rapid facial movements crucial for emotion recognition.
• Existing methods often struggle with ME detection and recognition, despite using spatio-temporal information.
• The apex frame, conveying peak emotional information, is key but its contribution to ME recognition remains unclear.

Purpose of the Study:

• To develop a robust method for detecting the apex frame in micro-expressions.
• To propose a joint feature learning architecture for improved ME recognition using the apex frame.
• To investigate the contribution of the apex frame to micro-expression recognition.

Main Methods:

• A novel apex frame detection method estimating pixel-level change rates in the frequency domain.
• A joint feature learning architecture that integrates local facial region information with global facial information.
• Extensive evaluation on multiple benchmark datasets (CASME, CASME II, SAMM, SMIC).

Main Results:

• The proposed frequency-domain method outperforms existing spatio-temporal methods for apex frame spotting.
• The joint local-global feature learning architecture achieves promising ME recognition performance.
• The apex frame significantly contributes to micro-expression recognition, outperforming methods using the entire sequence.

Conclusions:

• The apex frame is a critical component for accurate micro-expression recognition.
• Frequency domain analysis offers a more effective approach to apex frame detection.
• The proposed joint feature learning method enhances ME recognition by leveraging focused and holistic facial information.