FedSER-XAI: PSO-optimized multi-stream cross-attention transformer with graph features for explainable federated...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Speech Processing

Background:

• Federated learning for speech emotion recognition faces challenges in balancing performance, privacy, and interpretability.
• Existing methods often struggle to capture complex speech relationships while maintaining data confidentiality.

Purpose of the Study:

• To introduce FedSER-XAI, a novel explainable federated learning framework for speech emotion recognition.
• To enhance performance, privacy, and interpretability in federated speech emotion recognition systems.

Main Methods:

• Integration of Particle Swarm Optimization (PSO)-based feature selection for dimensionality reduction.
• Utilizing multi-stream cross-attention mechanisms and graph-based feature extraction (visibility and correlation graphs).
• Employing Vision Transformer for mel-spectrograms and temporal-spatial graph convolutional networks.

Main Results:

• Achieved 78.1% dimensionality reduction with PSO, improving discriminative power.
• Demonstrated high accuracy in centralized (99.9% EMODB, 97.2% SAVEE) and federated settings (99.7% EMODB, 97.2% SAVEE).
• Framework showed rapid convergence (10 rounds) with minimal performance degradation and strong privacy preservation.

Conclusions:

• FedSER-XAI is the first explainable federated speech emotion recognition system.
• The framework effectively balances performance, privacy, and interpretability for trustworthy AI applications.
• Graph-based features were validated as significant contributors to emotion discrimination through explainability methods (SHAP, LIME).