A Cross-modality Deep Learning Method for Measuring Decision Confidence from Eye Movement Signals.

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Electroencephalography (EEG) signals accurately measure human decision confidence.
• Acquiring EEG signals is challenging due to high costs and complex procedures.
• Eye movement signals offer a more accessible alternative for data acquisition.

Purpose of the Study:

• To develop a practical method for assessing human decision confidence using easily obtainable eye movement signals.
• To transfer knowledge from Electroencephalography (EEG) to eye movement signal analysis for confidence prediction.
• To improve the feasibility of decision confidence monitoring in real-world applications.

Main Methods:

• Proposed a cross-modality deep learning approach utilizing deep canonical correlation analysis (CDCCA).
• Transformed EEG and eye movement signals into a shared hyperspace using canonical correlation analysis constraints.
• Trained the model using both EEG and eye movement data, but tested using only eye movement signals.

Main Results:

• The proposed method achieved superior performance compared to single-modal approaches using only eye movement signals.
• The model maintained competitive accuracy when compared against other multimodal models.
• Demonstrated the effectiveness of transferring knowledge from EEG to eye movement signals for confidence prediction.

Conclusions:

• The cross-modality deep learning method provides an effective and practical solution for measuring human decision confidence.
• This approach overcomes the limitations of EEG acquisition by leveraging accessible eye movement data.
• The findings suggest a promising direction for developing non-invasive and user-friendly decision confidence assessment tools.