A Multimodal Approach for Deep-Learning Classification of Vocal Fold Pathologies in Stroboscopy

Area of Science:

• Otolaryngology
• Artificial Intelligence
• Medical Imaging

Background:

• Vocal fold (VF) pathologies, including unilateral paralysis (UVFP) and lesions, require accurate diagnosis.
• Current diagnostic methods may benefit from advanced computational analysis.

Purpose of the Study:

• To develop and validate a multimodal deep-learning classifier for differentiating between healthy vocal folds (HVF), UVFP, and VF lesions.
• To compare the performance of a multimodal classifier against unimodal image-only and audio-only classifiers.

Main Methods:

• Retrospective analysis of patients with UVFP, VF lesions, and HVF.
• Extraction of image frames and voice samples from stroboscopic videos; collection of clinicodemographic data.
• Development of a multimodal classifier combining visual, acoustic, and demographic features using deep learning.

Main Results:

• The multimodal classifier achieved 76.9% accuracy on a test set, outperforming image-only (61.5%) and audio-only (65.4%) models.
• On an external dataset, the multimodal classifier accuracy was 45%, still superior to unimodal models (42% and 31%).

Conclusions:

• A multimodal deep-learning approach combining stroboscopic images, voice, and clinical data shows potential for diagnosing vocal fold pathologies.
• This proof-of-concept study highlights the benefit of integrating diverse data modalities for improved diagnostic accuracy.