使用可见和IR的面部表情识别通过深度学习与注意力机制的早期融合
Muhammad Tahir Naseem1, Chan-Su Lee1, Tariq Shahzad2
1Department of Electronic Engineering, Yeungnam University, Gyeongsan, Republic of Korea.
PeerJ. Computer science
|March 26, 2025
概括
这项研究引入了一种先进的早期融合方法,用于使用可见和红外数据进行面部表情识别 (FER). 多模式方法显著提高了情绪识别的准确性,实现了最先进的结果.
科学领域:
- 计算机视觉 计算机视觉
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 面部表情识别 (FER) 对人工智能应用,如驾驶员监控和医疗保健至关重要.
- 准确的FER是具有挑战性的,因为表达变化和情绪之间的相似之处.
- 深度学习技术已经提升了FER的能力.
研究的目的:
- 为准确的面部表情识别开发一种强大的多模式方法.
- 克服表达变化和类间相似性引起的FER挑战.
- 为了利用可见和红外数据来增强情绪识别.
主要方法:
- 提出了结合可见和红外面部特征的早期融合策略.
- 使用了ResNet-18架构,并为单模模式模型提供了注意力机制.
- 扩展到一个整合卷积神经网络 (CNN) 和转移学习 (TL) 的多模式模型.
- 雇员VIRI和NVIE公开可访问的数据库用于模型培训和验证.
主要成果:
- 带有注意力的单模模式实现了强的性能.
- 多模式早期核聚变方法显著优于单模式模型.
- 在VIRI数据库中达到84.44%的准确率,在NVIE数据库中达到85.20%.
- 与现有的FER方法相比,已证明具有更高的性能.
结论:
- 拟议的多模式早期融合方法在面部表情识别方面实现了最先进的性能.
- 将可见和红外模式结合起来,可以提高FER的准确性.
- 基于注意力的ResNet-18架构对单模和多模FER都有效.
更多相关视频
03:31Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
Published on: December 15, 2023
442
07:12Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation
Published on: August 26, 2016
9.4K
相关概念视频
01:21Association Areas of the Cortex
4.8K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
4.8K
01:24Facial Feedback Hypothesis
107
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
107
01:20Parallel Processing
141
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
141