Adaptive Vision-Language Transformer for Multimodal CNS Tumor Diagnosis

Area of Science:

• Neuroimaging
• Artificial Intelligence
• Medical Diagnostics

Background:

• Accurate Central Nervous System (CNS) tumor identification is challenged by varied MRI protocols, tumor heterogeneity, and difficulties integrating imaging with clinical data.
• Existing diagnostic models struggle with domain shifts and lack robust multimodal fusion capabilities.

Purpose of the Study:

• To develop and validate the Adaptive Vision-Language Transformer (AVLT), a novel multimodal diagnostic infrastructure.
• To enhance the accuracy, robustness, and interpretability of CNS tumor diagnosis by integrating multi-sequence MRI with clinical notes.
• To address challenges posed by divergent MRI acquisition protocols and unequal tumor morphology.

Main Methods:

• The AVLT model integrates multi-sequence MRI (T1, T1c, T2, FLAIR) and clinical note text using normalized cross-attention for joint processing.
• An Adaptive Normalization Module (ANM) was employed to mitigate distribution shifts across datasets by adapting feature statistics.
• Auxiliary semantic and alignment losses were incorporated to stabilize multimodal fusion.

Main Results:

• AVLT demonstrated superior classification accuracy compared to existing CNN-, transformer-, radiogenomic-, and multimodal fusion-based models across multiple datasets.
• Achieved high accuracy rates: 84.6% on BraTS (OS), 92.4% on TCGA-GBM/LGG, 89.5% on REMBRANDT, and 90.8% on GLASS.
• Attained Area Under the Curve (AUC) values exceeding 90% for all evaluated domains, indicating strong performance.

Conclusions:

• AVLT offers a reliable and generalizable approach for the accurate diagnosis of CNS tumors.
• The model's interpretability is enhanced, providing clinically actionable insights.
• This multimodal diagnostic infrastructure represents a significant advancement in neuro-oncology imaging analysis.