Detection of Pathological Voice Samples Using a Text-to-Speech-Inspired Approach With Multiple Model Architectures:...

Voice disorders are common, yet access to timely specialist assessment is often limited. Automated analysis of short voice recordings may support early screening, but achieving robust performance on heterogeneous clinical datasets remains challenging.

This study aims to investigate the impact of synthetic speech augmentation on voice disorder recognition and classification using the Saarbrücken Voice Database (SVD) and to develop an adaptive ensemble with learnable weighting and optional sex embedding.

Sentence recordings from the SVD were used for binary voice disorder recognition (VDR, n=1988) and 4-class voice disorder classification (VDC; n=1199). Two text-to-speech (TTS) models, XTTS and Kartoffelbox, generated 1 synthetic sample for each training instance. Synthetic speech quality was assessed in a blinded listening evaluation with 40 samples per TTS model. Baseline models included support vector machine (SVM), ResNet-18, long short-term memory, and transformer-based architectures, including Hidden-Unit BERT (HuBERT), Wav2Vec 2.0, and WavLM. Ensemble learning and learnable fusion (LF), with and without a sex embedding (LFS), were evaluated using bootstrapped performance estimates and standard classification metrics. Area under the receiver operating characteristic curve (AUC) was reported for VDR.

In the blinded evaluation, Kartoffelbox-generated synthetic samples were less often identified as synthetic than XTTS-generated synthetic samples (10/19, 52.6% vs 17/19, 89.5%). In pooled original and synthetic ratings, Kartoffelbox showed higher pathological-status identification than XTTS for pathological samples (22/29, 75.9% vs 19/29, 65.5%) and nonpathological samples (8/10, 80.0% vs 6/10, 60.0%), as well as higher diagnostic-class assignment (20/39, 51.3% vs 14/39, 35.9%). Among the baseline models, HuBERT showed the most balanced VDR performance (accuracy=0.85, F₁-score=0.84). Kartoffelbox-based augmentation further improved performance, with HuBERT achieving 0.88 sensitivity, 0.87 accuracy, and 0.85 F₁-score. LF showed minor improvements in threshold-dependent metrics (accuracy=0.87, F₁-score=0.86), while LFS further balanced sensitivity and specificity (accuracy=0.88, AUC=0.92). The equal-weight ensemble achieved the highest AUC among ensemble approaches (AUC=0.93). For VDC, HuBERT was the strongest baseline (accuracy=0.72, median F₁-score=0.63), and LF achieved the highest overall fusion performance (accuracy=0.73, F₁-score=0.66), while laryngitis remained one of the more challenging classes.

High-quality synthetic speech augmentation can improve automated voice disorder analysis across heterogeneous clinical data, with effectiveness varying by the TTS model. Multimodel fusion of complementary architectures provided robust performance for screening and multiclass classification, with the strongest approach varying by metric and task. These findings support further evaluation of AI-assisted preassessment in clinical and telemedicine settings.