Application of Deep Multi-Scale Representation Learning Based on Eye-Tracking and Facial Expression Data in Cognitive...

Area of Science:

• Neuroscience
• Biomedical Engineering
• Computer Science

Background:

• Digital biomarkers from eye-tracking and facial expressions show promise for non-invasive cognitive decline (CD) screening.
• Current unimodal methods struggle with complex temporal patterns, and deep learning (DL) applications in this multimodal domain are underexplored.

Purpose of the Study:

• To develop and validate a deep learning framework for multimodal eye-tracking and facial expression analysis to screen for cognitive decline.
• To address the limitations of existing unimodal and feature-engineered approaches.

Main Methods:

• Designed an assessment system with five cognitive paradigms, collecting eye-tracking and facial expression data from 20 healthy controls (HC) and 20 individuals with CD.
• Proposed a deep neural network for multi-scale representation learning using subspace exploration and multi-scale convolutions.
• Implemented a decision fusion mechanism for enhanced diagnostic robustness.

Main Results:

• The proposed DL method achieved 90% classification accuracy, outperforming traditional machine learning models.
• Validated known CD-associated features and identified novel potential behavioral patterns.
• Demonstrated the feasibility of DL for objective CD screening using eye-tracking and facial expression signals.

Conclusions:

• A DL framework leveraging eye-tracking and facial expression data is feasible for identifying cognitive decline.
• This approach offers a reference for developing objective and efficient digital screening tools for cognitive decline.
• The study highlights the potential of multimodal deep learning in behavioral neuroscience and digital health.