Recurrence plot embeddings as short segment nonlinear features for multimodal speaker identification using air, bone...

Area of Science:

• Speech processing and acoustics
• Nonlinear dynamics in biological systems
• Machine learning for biometrics

Background:

• Speech production involves a nonlinear Vocal Tract (VT) system.
• Speaker characteristics are often modeled using linear spectral features.
• Nonlinear dynamics of the VT system are underexplored for speaker recognition.

Purpose of the Study:

• To propose Recurrent Plot (RP) embeddings as stand-alone, nonlinear speaker-discriminating features.
• To evaluate the efficacy of RP embeddings for speaker identification across different speech transmission modes (air, bone, throat).
• To assess the performance of unimodal and multimodal systems using RP embeddings.

Main Methods:

• Utilized two datasets: TIMIT speech corpus and a consonant-vowel unimodal syllable dataset.
• Employed Recurrent Plot (RP) embeddings as nonlinear features.
• Conducted closed-set speaker identification experiments using unimodal (Air, Bone, Throat) and multimodal (bimodal, trimodal) systems.

Main Results:

• Unimodal systems trained on RP embeddings achieved high accuracies: Air (95.81%), Bone (98.18%), and Throat (99.74%).
• The best trimodal system (Air-Bone-Throat) reached 99.84% accuracy, comparable to systems using spectrograms and MFCCs.
• A bimodal Bone-Throat system achieved 98.84% accuracy, demonstrating effectiveness without air conduction speech.

Conclusions:

• RP embeddings are significant nonlinear features capable of independent speaker recognition.
• The nonlinear dynamics of the Vocal Tract (VT) system, captured by RP embeddings, are highly speaker-specific.
• This nonlinear feature representation holds potential for advancing both speaker and speech recognition technologies.