Few-Shot and Zero-Shot Learning for MRI Brain Tumor Classification Using CLIP and Vision Transformers

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computational Biology

Background:

• Accurate brain tumor classification from MRI is crucial but hindered by limited annotated data.
• Data-efficient learning paradigms like few-shot learning (FSL) and zero-shot learning (ZSL) offer potential solutions.

Purpose of the Study:

• To compare FSL and ZSL for brain tumor diagnosis using deep learning and vision-language models.
• To establish a benchmark for data-efficient MRI classification under severe label constraints.

Main Methods:

• Evaluated Prototypical Network (ProtoNet) with CNN, ResNet-18, and vision transformer backbones.
• Tested under 1000 randomly sampled five-shot, four-way episodes.
• Compared against a fine-tuned ResNet-50 baseline and CLIP (ZSL) model.

Main Results:

• ResNet-18 ProtoNet achieved 85% ± 8% accuracy (F1 = 0.85).
• This surpassed the ResNet-50 baseline (42% ± 12%) and CLIP (ZSL) (30% ± 10%).
• A visual-only ZSL baseline achieved 54% ± 11% accuracy.

Conclusions:

• Metric-based FSL offers a 43% absolute improvement over standard fine-tuning for MRI classification.
• FSL provides a robust benchmark for data-efficient brain tumor diagnosis with limited labels.