Pathological Voice Detection and Classification Based on Multimodal Transmission Network.

Area of Science:

• Speech pathology
• Biomedical signal processing
• Machine learning

Background:

• Accurate diagnosis of pathological voice types is crucial for effective treatment.
• Multimodal analysis of voice signals can provide a more comprehensive understanding of vocal pathologies.

Purpose of the Study:

• To propose a pathological voice detection and classification algorithm using a multimodal transmission network.
• To leverage both sound and electroglottography (EGG) signals for improved diagnostic accuracy.

Main Methods:

• Utilized short-time Fourier transform (STFT) and Mel filters to generate Mel spectrograms from sound and EGG signals.
• Developed a multimodal transmission network incorporating a Multimodal Transfer Module (MMTM).
• Employed a fusion layer for integrating multimodal information and a fully connected layer for classification.

Main Results:

• Achieved high performance on the Saarbrücken voice database (SVD) with 1179 subjects.
• Reported average accuracy of 98.02%, recall of 98.23%, specificity of 97.82%, and F1 score of 97.95% for pathological voice classification.
• Demonstrated significant improvement in classification accuracy compared to existing algorithms.

Conclusions:

• The proposed multimodal model effectively integrates diverse signal information for robust voice feature extraction.
• Enhanced accuracy in pathological voice classification was achieved through comprehensive and stable feature representation.
• Future work will focus on optimizing the model for reduced computational time and complexity.