GAMMNet: Gating Multi-head Attention in a Multi-modal Deep Network for Sound Based Respiratory Disease Detection

Respiratory diseases present significant challenges to global health due to their high morbidity and mortality rates. Traditional diagnostic methods, such as chest radiographs and blood tests, often lead to unnecessary costs and resource strain, as well as potential risks of cross-contamination during these procedures. In recent years, contactless sensing and intelligent technologies, particularly multi-modal sound-based deep learning methods, have emerged as promising solutions for the early detection of respiratory diseases. While these methods have shown encouraging results, the integration of multi-modal features has not been sufficiently explored, which limits the enhancement of diagnostic accuracy. To address this issue, we introduce GAMMNet, a novel multi-modal neural network designed to enhance the detection of respiratory diseases by leveraging multi-modal sound data collected from contactless recording devices. GAMMNet utilizes a unique gating mechanism that adaptively regulates the influence of each modality on the classification results. Additionally, our model incorporates multi-head attention and linear transformation modules to further enhance classification performance. Our GAMMNet achieves state-of-the-art classification results, compared to existing deep learning based methods, on real-world multi-modal respiratory sound datasets. These findings demonstrate the robustness and effectiveness of GAMMNet in the contactless monitoring and early detection of respiratory diseases.