Detecting changes in facial temperature induced by a sudden auditory stimulus based on deep learning-assisted face...

Area of Science:

• Psychophysiology
• Neuroscience
• Biomedical Engineering

Background:

• Traditional psychophysiological measures lack ecological validity due to contact requirements.
• Facial thermal signal extraction using Infrared Imaging (IRI) is hindered by head motion artifacts.
• Advancements in analytical methods are crucial for leveraging IRI's potential.

Purpose of the Study:

• To develop a semi-automated method for extracting thermal signals from facial regions using deep learning.
• To assess the feasibility of detecting facial temperature changes in response to auditory stimuli.
• To compare IRI-based thermal responses with traditional measures like galvanic skin response (GSR) and electrocardiography (ECG).

Main Methods:

• Developed a novel semi-automated thermal signal extraction method utilizing deep learning for facial landmark identification.
• Applied the method to analyze facial temperature changes following a sudden auditory stimulus.
• Compared thermal responses with concurrent GSR and ECG measurements.

Main Results:

• Significant decrease in facial temperatures, particularly at the nose tip, detected post-auditory stimulus.
• Rapid thermal response observed within 4-5 seconds, initiating less than 2 seconds after GSR changes.
• Demonstrated sensitivity and robustness of the new methodology in capturing facial physiological changes.

Conclusions:

• The developed deep learning-based method provides a sensitive and robust tool for extracting facial thermal signals with minimal manual intervention.
• Infrared Imaging (IRI) shows promise as an ecologically valid technique in social and affective neuroscience.
• Methodological advancements in thermal signal extraction can enhance the application of IRI in psychophysiological research.