PASTFNet: a paralleled attention spatio-temporal fusion network for micro-expression recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Biomedical Engineering

Background:

• Micro-expressions (MEs) are crucial for understanding genuine human emotions.
• Recognizing MEs is challenging due to their short duration and subtle intensity.
• Existing methods struggle to effectively capture spatio-temporal features of MEs.

Purpose of the Study:

• To develop a novel network for improved micro-expression recognition.
• To effectively capture and fuse spatio-temporal features for enhanced emotion prediction.
• To address the limitations of current ME recognition techniques.

Main Methods:

• A paralleled dual-branch attention-based spatio-temporal fusion network (PASTFNet) was proposed.
• The spatial branch extracts short- and long-range spatial relationships.
• An attention-based multi-scale feature fusion network (AMFNet) encodes temporal features using CNN and LSTM principles.

Main Results:

• PASTFNet demonstrated promising micro-expression recognition performance.
• The model effectively fuses spatial and temporal features.
• Experiments on CASMEII and SAMM datasets validated the model's effectiveness.

Conclusions:

• The proposed PASTFNet model significantly advances micro-expression recognition.
• Effective fusion of spatio-temporal features is key to improving accuracy.
• This research contributes to more reliable emotion prediction systems.