An enhanced deep learning approach for speaker diarization using TitaNet, MarbelNet and time delay network

Area of Science:

• Speech Processing and Signal Analysis
• Machine Learning and Artificial Intelligence
• Computational Linguistics

Background:

• Speaker diarization is crucial for speech transcription, AI, and audio analysis, but traditional methods struggle with noise and overlapping speech.
• Existing clustering-based approaches exhibit limitations in accuracy and robustness, particularly in challenging acoustic conditions.

Purpose of the Study:

• To introduce a novel deep learning framework, the Neuro-TM Diarizer, for improved speaker diarization.
• To enhance performance in complex acoustic environments by integrating noise reduction, adaptive beamforming, and neural diarization.

Main Methods:

• Developed a multimodal deep learning framework combining Marble-Net for voice activity detection and Tita-Net for speaker embeddings.
• Employed time-delay neural networks for neural diarization and speaker identification.
• Evaluated the Neuro-TM Diarizer on VoxConverse and VoxCeleb datasets, comparing it against clustering-based methods.

Main Results:

• The Neuro-TM Diarizer achieved low Diarization Error Rates (DER) of 6.89% on VoxConverse and 6.93% on VoxCeleb.
• Demonstrated significant improvements over clustering-based approaches, reducing DER by 12.60% on VoxConverse and 14.01% on VoxCeleb.
• The framework effectively handles noise and overlapping speech, outperforming traditional methods in accuracy and robustness.

Conclusions:

• The proposed Neuro-TM Diarizer offers a substantial advancement in speaker diarization technology.
• This deep learning framework provides a robust solution for real-world applications including speech transcription, speaker authentication, and audio archiving.
• The integration of noise reduction and advanced neural network architectures leads to superior performance in complex acoustic scenarios.