Explainable Multi-Modal Deep Learning for Recording-Level Classification of Respiratory Audio Signals Under Internal...

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Signal Processing

Background:

• Respiratory diseases pose a global health challenge, with diagnosis often hindered by subjective assessments and noisy data.
• Current methods for respiratory audio analysis face limitations in accuracy and consistency due to environmental noise and inter-clinician variability.

Purpose of the Study:

• To develop and evaluate an explainable multimodal deep learning framework for multiclass classification of respiratory audio signals at the recording level.
• To compare the performance of the proposed hybrid model against conventional machine learning algorithms.
• To assess the model's explainability and robustness across different datasets.

Main Methods:

• A multimodal deep learning framework integrating a CNN-BiLSTM-attention spectro-temporal encoder and a handcrafted acoustic-feature encoder.
• Late-stage fusion of the two encoder branches to combine data-driven and domain-informed features.
• Training and internal evaluation on the Asthma Detection Dataset Version 2, with pre-processing including resampling, filtering, and feature extraction.
• Explainability techniques such as Grad-CAM, Integrated Gradients, and SHAP were employed.

Main Results:

• The proposed hybrid model achieved a mean held-out recording-level test accuracy of 0.9099 and a macro ROC-AUC of 0.9867.
• The deep spectro-temporal branch was the primary driver of performance, with the handcrafted branch offering complementary interpretable information.
• Domain-shift evaluation revealed dataset shift effects, indicating limited external transferability and potential inflation of internal performance estimates.

Conclusions:

• The developed explainable multimodal deep learning framework demonstrates strong internal performance for respiratory audio classification.
• The model's explainability features provide insights into acoustic cues, though external validation is crucial before clinical deployment.
• Further work is needed to address domain-shift effects and enhance the model's generalizability for real-world clinical applications.