Meta-learning with Latent Space Clustering in Generative Adversarial Network for Speaker Diarization.

Area of Science:

• Speech processing
• Machine learning
• Speaker recognition

Background:

• Speaker diarization systems using x-vector embeddings struggle with noisy environments and lack domain robustness.
• Generative adversarial network (GAN) based ClusterGAN showed promise for speaker diarization on meeting data by projecting x-vectors into a latent space.

Purpose of the Study:

• To extend ClusterGAN for improved speaker diarization robustness and rapid generalization across challenging domains.
• To investigate the effectiveness of proposed ClusterGAN and meta-ClusterGAN (MCGAN) embeddings over standard x-vectors.

Main Methods:

• Fine-tuning a pre-trained ClusterGAN encoder using prototypical loss within a meta-learning framework to create meta-ClusterGAN (MCGAN) embeddings.
• Evaluating MCGAN and ClusterGAN embeddings on diverse datasets including CALLHOME, AMI, DIHARD-II, ADOS, and BOSCC.
• Utilizing normalized maximum eigengap spectral clustering (NME-SC) and embedding fusion with x-vectors for diarization.

Main Results:

• The proposed MCGAN and ClusterGAN embeddings with NME-SC consistently outperformed the Kaldi state-of-the-art x-vector diarization system.
• Embedding fusion with x-vectors achieved relative diarization error rate (DER) improvements ranging from 6.67% to 53.93% across datasets.
• MCGAN embeddings demonstrated superior performance in speaker count estimation and short speech segment diarization compared to x-vectors and ClusterGAN on telephonic data.

Conclusions:

• The proposed meta-learning approach (MCGAN) significantly enhances speaker diarization robustness and domain generalization.
• The developed embeddings offer substantial improvements over traditional x-vectors, particularly in challenging and multi-domain scenarios.
• MCGAN shows potential for real-world applications requiring accurate speaker identification in diverse and noisy conditions.