AM3F-FlowNet: Attention-Based Multi-Scale Multi-Branch Flow Network.

Area of Science:

• Computer Vision
• Affective Computing
• Machine Learning

Background:

• Micro-expressions are brief facial changes indicating emotions, but their annotation complexity results in scarce data.
• Extracting meaningful features from limited micro-expression datasets is challenging.

Purpose of the Study:

• To develop an effective method for learning micro-expression features from limited data.
• To improve the accuracy of micro-expression recognition by leveraging multi-modal optical flow and attention mechanisms.

Main Methods:

• Proposed an attention-based multi-scale, multi-modal, multi-branch flow network.
• Extracted horizontal optical flow, vertical optical flow, and optical strain.
• Utilized spatial and channel attention for feature fusion and reweighting.
• Introduced a logarithmically adjusted prior knowledge weighting loss to handle class imbalance.

Main Results:

• The proposed network effectively learns motion information from micro-expression videos.
• Multi-scale fusion and feature reweighting modules enhance feature representation.
• The novel loss function mitigates the impact of imbalanced micro-expression samples.
• Experimental results on benchmark datasets demonstrate performance comparable to state-of-the-art methods.

Conclusions:

• The developed attention-based network offers a robust solution for micro-expression analysis with limited data.
• The integration of multi-modal optical flow and advanced attention mechanisms significantly improves feature learning.
• The proposed loss function effectively addresses the challenge of class imbalance in micro-expression recognition.