One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Informatics

Background:

• Multimodal models outperform single-modality approaches in various tasks, including disease diagnosis.
• Current multimodal research primarily focuses on vision-language applications with limited modalities (typically <= 4).
• Healthcare data often involves numerous modalities (e.g., X-rays, PET, MRI, genomics, clinical notes), necessitating efficient integration methods.

Purpose of the Study:

• To address the computational bottleneck of existing attention mechanisms in integrating numerous modalities.
• To propose a novel, scalable attention mechanism for efficient multimodal data integration in healthcare.
• To evaluate the performance and efficiency of the proposed method against state-of-the-art techniques.

Main Methods:

• Developed a new attention mechanism named One-Versus-Others (OvO) attention.
• OvO attention exhibits linear scalability with the number of modalities, unlike quadratic scaling of cross-attention and self-attention.
• Evaluated OvO attention on three diverse clinical datasets with multiple modalities.

Main Results:

• OvO attention significantly reduces computational complexity, scaling linearly with the number of modalities.
• The method demonstrated substantial reductions in floating point operations (FLOPs) by at least 91.98% across clinical datasets.
• OvO attention maintained or improved performance compared to existing multimodal integration techniques.

Conclusions:

• One-Versus-Others (OvO) attention offers a computationally efficient solution for integrating a large number of modalities.
• The proposed method overcomes the limitations of traditional attention mechanisms, enabling broader adoption of multimodal models in healthcare.
• OvO attention facilitates efficient and accurate multi-modal predictions, advancing AI applications in clinical settings.