Computing nasalance with MFCCs and Convolutional Neural Networks.

Area of Science:

• Speech Acoustics and Signal Processing
• Computational Linguistics and Phonetics
• Biomedical Engineering and Clinical Biomarkers

Background:

• Nasalance measurement is crucial for diagnosing hypernasality.
• Traditional eNasalance calculation has limitations.
• Developing advanced computational methods can improve biomarker accuracy.

Purpose of the Study:

• To introduce and evaluate a novel approach for nasalance computation using Convolutional Neural Networks (CNNs) and Mel-Frequency Cepstrum Coefficients (mfccNasalance).
• To assess the accuracy of mfccNasalance across different dialects and speech dynamicity.
• To compare the performance of mfccNasalance with traditional eNasalance.

Main Methods:

• Utilized dual-channel Nasometer speech data from healthy speakers across different dialects (Costa Rica, Spain, Chile).
• Trained CNN models using sequences of 39 MFCC vectors from 250 ms moving windows.
• Evaluated accuracy using Spearman correlation against expert perceptual nasality scores on varied test data (short words, sentences, diadochokinetic syllables).

Main Results:

• mfccNasalance demonstrated higher accuracy than eNasalance in same-dialect conditions, irrespective of CNN configuration.
• A 1x1 kernel improved accuracy for dynamic utterances, while kernel shape significantly impacted non-dynamic utterances.
• Performance decreased in different-dialect conditions, especially for models trained on Costa Rican data.

Conclusions:

• mfccNasalance presents a flexible and effective alternative to eNasalance for measuring nasalance.
• CNN model selection should consider the dynamicity of speech data for optimal mfccNasalance performance.
• Further research is needed to refine CNN model optimization for diverse speech conditions.