Related Experiment Videos
A novel class-attention transformer-driven feature fusion technique-based speech disorder classification
Abdul Rahaman Wahab Sait1, Haitham Ahmed Jamil Mohammed2, Taqwa Ali Mohammad Bani Awad3,4
1Department of Documents and Archive, Center of Documents and Administrative Communication, King Faisal University, Al-Ahsa, Saudi Arabia.
Introduction:
Speech disorders (SD) present significant diagnostic challenges due to the complex and indistinct acoustic characteristics embedded within speech signals.
Problem Statement:
The standalone convolutional neural networks (CNNs) and vision transformers (ViT) struggle to capture SD temporal and spectral dynamics.
Objectives:
To address these limitations, this study proposes an end-to-end binary pathological SD detection framework that processes raw audio waveforms to differentiate disordered speech from healthy speech while improving feature representation, interpretability, and generalization.
Methods:
A hybrid feature extraction integrating one-dimensional CNNs and ViT's self-attention mechanism is developed to extract crucial SD features. An adaptive fusion and a Class-attention transformer (CaiT)-based feature refinement is introduced to transform the extracted features into a classification-optimized global representation. Through the integration of gradient-weighted class activation mapping (Grad-CAM) and attention-based visualization with the model architecture, the temporal-localization of disorder-relevant acoustic patterns is enabled. Two benchmark datasets are utilized for model's performance evaluation, benchmarking against state-of-the-art CNNs and ViT architectures. The model is trained and internally validated on the SVD dataset using subject-level splitting, while the PD and VOICED datasets are used exclusively for external validation to assess cross-dataset generalization across neurological and heterogeneous pathological voice conditions.
Results:
The proposed model achieves 97.50% accuracy on both SVD and PD datasets and 95.80% accuracy on VOICED, while maintaining a lightweight design with 5.2 million parameters. Statistical analysis further confirms the results' reliability and significance.
Conclusion:
The integration of adaptive fusion and transformer-based refinement significantly enhances SD detection performance while ensuring interpretability. The suggested framework offers a sound and proof-of-concept for diagnosing SD, allowing clinicians and speech-language pathologists to make reliable predictions and pay attention to diagnostically significant time intervals.
