Multimodal Triplet Attention Network for Brain Disease Diagnosis

Area of Science:

• Neuroimaging
• Medical Data Analysis
• Artificial Intelligence in Medicine

Background:

• Multi-modal imaging data fusion enhances medical data analysis by integrating complementary information.
• Functional MRI (fMRI) and Diffusion Tensor Imaging (DTI) are crucial for diagnosing brain diseases like epilepsy.
• Existing methods often overlook high-order relationships and discriminative features in multi-modal data fusion.

Purpose of the Study:

• To propose a novel framework for epilepsy diagnosis by fusing fMRI and DTI data.
• To capture complementary information and discriminative features using high-order feature extraction and attention mechanisms.
• To improve the accuracy of multi-modal data fusion for brain disease diagnosis.

Main Methods:

• Developed a novel framework fusing functional MRI (fMRI) and diffusion tensor imaging (DTI) data.
• Employed a triple network for high-order representation learning and discriminative feature extraction.
• Utilized self-attention and cross-attention mechanisms to weigh brain region importance and extract complementary information.
• Applied a triple loss function to optimize sample distances in a common representation space.

Main Results:

• The proposed method significantly outperforms several state-of-the-art diagnosis approaches on an epilepsy dataset.
• Demonstrated the effectiveness of high-order feature extraction with attention mechanisms in multi-modal fusion.
• Validated the framework's ability to capture complementary information and discriminative features from fMRI and DTI.

Conclusions:

• The novel framework effectively fuses fMRI and DTI for enhanced epilepsy diagnosis.
• High-order feature extraction with attention mechanisms is crucial for improving multi-modal data fusion.
• The proposed method shows significant superiority over existing approaches in brain disease diagnosis.