MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Micro-expression spotting is crucial for understanding human emotions and behavior.
• Existing methods struggle with spontaneous micro-expressions in long videos due to scale and temporal variations.

Purpose of the Study:

• To develop a novel convolutional neural network (CNN) for effective spotting of multi-scale spontaneous micro-expression intervals in long videos.
• To introduce a new evaluation metric for micro-expression spotting.
• To assess the performance of the proposed network against state-of-the-art methods.

Main Methods:

• Proposed Micro-Expression Spotting Network (MESNet) utilizing a 2+1D Spatiotemporal Convolutional Network for feature extraction.
• Incorporated a Clip Proposal Network for generating potential micro-expression clips.
• Employed a Classification Regression Network for clip classification and temporal boundary regression.
• Conducted experiments on CAS(ME)2 and SAMM datasets using leave-one-subject-out cross-validation.

Main Results:

• MESNet effectively enhances the F1-score, a key metric for spotting performance.
• The proposed network achieved superior performance compared to other state-of-the-art methods.
• MESNet demonstrated particularly strong results on the SAMM dataset.

Conclusions:

• The developed MESNet is a highly effective deep learning model for spontaneous micro-expression spotting in long videos.
• The proposed network offers a significant advancement in micro-expression analysis accuracy.
• MESNet provides a robust solution for identifying subtle, short-duration facial expressions.