Forensic deepfake audio detection using segmental speech features

Area of Science:

• Acoustic Phonetics
• Digital Forensics
• Artificial Intelligence

Background:

• Deepfake audio poses a significant challenge to authenticity verification.
• Current deepfake detection methods often rely on global audio features.
• Replicating fine-grained articulatory speech characteristics is difficult for deepfake generation models.

Purpose of the Study:

• To investigate the efficacy of segmental speech sound features for audio deepfake detection.
• To compare the performance of segmental versus global features in identifying deepfakes.
• To propose and evaluate a novel speaker-specific framework for deepfake detection.

Main Methods:

• Analysis of acoustic features of segmental speech sounds.
• Utilizing features common in forensic voice comparison (FVC).
• Development and testing of a speaker-specific deepfake detection framework.

Main Results:

• Certain segmental features, particularly those used in FVC, are effective in detecting audio deepfakes.
• Global audio features showed limited value in distinguishing deepfakes.
• The proposed speaker-specific framework demonstrated potential advantages over speaker-independent systems.

Conclusions:

• Segmental acoustic features offer a promising avenue for audio deepfake detection, distinct from traditional FVC approaches.
• A speaker-specific detection framework is advantageous for forensic applications requiring high interpretability and sensitivity.
• Future research should focus on refining speaker-specific models for robust deepfake identification.