Combination of deep speaker embeddings for diarisation.

Area of Science:

• Speech processing
• Machine learning
• Artificial intelligence

Background:

• Speaker diarisation has advanced with neural network (NN) derived d-vectors for speaker embeddings.
• Existing d-vectors face limitations in performance and robustness for clustering speech segments.

Purpose of the Study:

• To propose a novel c-vector method for extracting more robust and higher-performing speaker embeddings.
• To develop a unified neural-based single-pass speaker diarisation pipeline.

Main Methods:

• Combining complementary d-vectors using 2D self-attentive, gated additive, and bilinear pooling structures.
• Implementing a neural network pipeline for voice activity detection, speaker change point detection, and embedding extraction.
• Conducting experiments on the AMI and NIST RT05 datasets.

Main Results:

• Relative speaker error rate (SER) reductions of 13% and 29% on AMI dev and eval sets using c-vectors over d-vectors.
• A 15% relative SER reduction on the RT05 dataset, demonstrating robustness.
• Further improvements with the best c-vector system achieving 7-17% relative SER reduction when incorporating VoxCeleb data.

Conclusions:

• The proposed c-vector method offers significant improvements in speaker diarisation accuracy and robustness compared to d-vectors.
• The neural-based single-pass pipeline effectively integrates multiple diarisation tasks.
• The findings highlight the potential of advanced embedding techniques for complex acoustic environments.