Related Experiment Video
Updated: Jun 21, 2025
04:23
A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images
Published on: April 21, 2023
1.8K
Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression
Nianfeng Li1, Yongyuan Huang1, Zhenyan Wang1
1College of Computer Science and Technology, Changchun University, No. 6543, Satellite Road, Changchun 130022, China.
Sensors (Basel, Switzerland)
|July 13, 2024
Summary
This study introduces a lightweight hybrid vision transformer for facial expression recognition (FER). The novel method enhances feature extraction and reduces computational costs, achieving high accuracy on benchmark datasets.
Area of Science:
- Computer Science
- Artificial Intelligence
- Machine Learning
Background:
- Convolutional Neural Networks (CNNs) show promise in facial expression recognition (FER) but struggle with real-world challenges like occlusion and lighting variations.
- CNNs primarily use local spatial features, lacking global context and efficient computational complexity-accuracy balance for robust FER.
- Existing CNN-based models are insufficient for accurate and efficient facial expression recognition in diverse conditions.
Purpose of the Study:
- To develop a lightweight and accurate facial expression recognition method overcoming limitations of current CNN-based approaches.
- To enhance feature extraction by integrating multi-scale facial information and improving focus on salient expression regions.
- To reduce computational complexity while maintaining high recognition accuracy through a novel attention mechanism.
Main Methods:
- Proposed a lightweight facial expression recognition method utilizing a hybrid vision transformer architecture.
- Incorporated an improved attention module to capture multi-scale facial features and enhance feature integration.
- Introduced a patch dropping (PD) module to emulate human visual attention, focusing on discriminative features and reducing computational load.
Main Results:
- Achieved a high accuracy of 86.51% on the RAF-DB dataset and nearly 70% on the FER2013 dataset.
- The proposed model has a significantly small model size of only 3.64 MB, indicating high efficiency.
- Demonstrated superior performance compared to existing facial expression recognition methods in extensive experiments.
Conclusions:
- The hybrid vision transformer approach offers a new perspective for advancing facial expression recognition.
- The method effectively addresses challenges like occlusion and lighting variations by capturing richer, multi-scale features.
- The lightweight design and high accuracy make the proposed method suitable for real-world applications.
