Fused Audio Instance and Representation for Respiratory Disease Detection.

Area of Science:

• Medical Informatics
• Biomedical Signal Processing
• Machine Learning for Healthcare

Background:

• Audio-based classification of body sounds is crucial for diagnosing respiratory diseases.
• While coughs are primary biomarkers, breath and speech sounds also show potential, especially for COVID-19 detection.

Purpose of the Study:

• To propose and evaluate the Fused Audio Instance and Representation (FAIR) method for enhanced respiratory disease detection.
• To investigate the efficacy of combining diverse audio representations (waveform and spectrogram) and body sounds (cough, breath, speech).

Main Methods:

• Developed FAIR, a method constructing joint feature vectors from multiple body sound representations.
• Utilized self-attention mechanisms to integrate features extracted from cough, breath, and speech sounds.
• Experimented with combining waveform and spectrogram data for COVID-19 detection.

Main Results:

• The FAIR method achieved an Area Under the Curve (AUC) of 0.8658, with 0.8057 sensitivity and 0.7958 specificity for COVID-19 detection.
• Combining both waveform and spectrogram representations significantly improved AUC scores compared to single-representation models.
• Self-attention integration of cough, breath, and speech sound features yielded the highest performance.

Conclusions:

• FAIR demonstrates superior performance in respiratory disease detection by fusing multi-modal audio features.
• Combining waveform and spectrogram representations enriches feature extraction, outperforming single-representation approaches.
• The FAIR method's flexibility supports broader applications in multi-modal diagnostic systems for various diseases.