Novel methods for elucidating modality importance in multimodal electrophysiology classifiers.

Area of Science:

• Neuroscience
• Computational Biology
• Medical Informatics

Background:

• Multimodal classification is prevalent in electrophysiology.
• Deep learning models using raw time-series data lack explainability, hindering clinical applications.
• There is a need for novel multimodal explainability methods in electrophysiology.

Purpose of the Study:

• To develop and evaluate novel multimodal explainability methods for electrophysiology.
• To compare global and local explainability approaches for automated sleep stage classification.
• To investigate subject-level differences and the influence of clinical/demographic variables on classifier explanations.

Main Methods:

• Trained a convolutional neural network on electroencephalogram (EEG), electrooculogram, and electromyogram data for sleep stage classification.
• Developed a global explainability approach adapted for electrophysiology and compared it with an existing method.
• Introduced the first two local multimodal explainability approaches and analyzed subject-level differences and variable effects.

Main Results:

• High agreement was observed between different explainability methods.
• Electroencephalogram (EEG) was identified as the most important modality for most sleep stages globally.
• Local explanations revealed subject-level differences not apparent in global methods, and sex, medication, and age significantly affected classifier patterns.

Conclusions:

• Novel multimodal explainability methods enhance classifier transparency in electrophysiology.
• These methods offer insights into personalized medicine and the impact of demographic/clinical factors.
• The developed approaches facilitate the clinical implementation of multimodal electrophysiology classifiers.