Fine-Tuning of Wav2Vec 2.0 for Multimodal Classification of Abnormal Heart Sounds

Cardiovascular diseases (CVDs) are the leading cause of death, resulting in approximately 17.9 million deaths every year. Treatment of CVDs is most effective when detected early, creating a need for accurate and inexpensive pre-screening methods. Recently, deep learning has been explored to classify abnormal heart sounds indicative of CVDs using phonocardiogram (PCG) and electrocardiogram (ECG) signals. However, due to limited synchronised PCG and ECG data, state-of-the-art (SOTA) architectures, such as transformers, have not seen much use within this domain. This work explores the use of traditional signal processing, along with denoising diffusion models WaveGrad and DiffWave, to create an augmented dataset that is used to fine-tune a Wav2Vec 2.0-based classifier. The proposed methods achieve greater than state-of-the-art accuracy, unweighted average recall, sensitivity, specificity and Matthew's correlation coefficient of 93.14% and 92.21%, 93.35%, 90.10% and 0.838, respectively, in the Computing in Cardiology (CinC) 2016 training-a data which consists of synchronised PCG and ECG signals. These methods also achieved 92.98%, 92.48%, 93.63%, 92.48%, and 0.8238 when trained on all CinC databases utilising only PCG signals, which is also greater performance than SOTA methods. These results demonstrate the performance of fine-tuning self-supervised transformer-based models when trained on larger amounts of data, which an augmented dataset can supplement.