HSNet: An adaptive fusion network based on laryngoscope-speech multimodal data for laryngeal disease classification

To design and implement a deep learning-based multimodal data fusion classification model that integrates laryngoscope images and voice signals to improve the diagnostic accuracy of laryngeal diseases, enabling rapid and precise identification for clinical support.

The model demonstrated high classification accuracy and robustness, achieving an overall accuracy of 87.92 % on the independent test set. Precision, recall, specificity, and F1-score were 0.879, 0.887, 0.966, and 0.883, respectively. The model outperformed single-modal approaches and existing multimodal frameworks.

The proposed HSNet effectively integrates hierarchical features from laryngoscope images and voice modalities, enabling accurate classification of six laryngeal diseases. This method holds significant potential for clinical applications.