Spoof detection with dynamic learnable sparse attention and tri-modal fusion in resource-constrained audio systems

Area of Science:

• Speech Processing
• Machine Learning
• Signal Processing

Background:

• Automatic speaker verification (ASV) systems are vulnerable to audio spoofing attacks.
• Existing spoof detection methods, like Multi-Head Attention (MHA), are computationally expensive and memory-intensive, limiting their use in resource-constrained environments.

Purpose of the Study:

• To develop an efficient and robust spoof detection framework for resource-constrained ASV systems.
• To reduce the computational complexity and memory footprint of spoof detection methods.

Main Methods:

• Proposed a novel Dynamic Learnable Sparse Attention (DLSA) framework integrating Mel-Frequency Cepstral Coefficients (MFCC), Constant-Q Transform (CQT), and raw waveform.
• Employed a ResNet backbone for raw waveform feature extraction and a hybrid loss function (cross-entropy and center loss) for optimized feature representation.
• Implemented a learnable attention mechanism for dynamic cross-modal fusion of spectral and temporal features.

Main Results:

• Achieved an 80% reduction in computational costs compared to MHA-based methods.
• Attained an Equal Error Rate (EER) of 0.68% and a minimum tandem Detection Cost Function (t-DCF) of 0.0173 on the ASVspoof 2019 LA dataset.
• Demonstrated a 33.6% improvement in EER reduction over existing methods.

Conclusions:

• The DLSA framework offers an efficient and effective solution for spoof detection in ASV systems with limited resources.
• The proposed method enhances robustness against sophisticated audio spoofing techniques.
• This work paves the way for deploying advanced security features on edge devices.