Benchmarking foundation models and parameter-efficient fine-tuning for prognosis prediction in medical imaging

Area of Science:

• Artificial Intelligence in Medical Imaging
• Machine Learning for Prognosis Prediction
• Transfer Learning Strategies

Background:

• Foundation Models (FMs) show potential in medical imaging but face challenges like data scarcity and class imbalance for prognosis prediction.
• Clinical adoption of AI for COVID-19 outcome prediction is limited by these challenges.
• A structured benchmark is needed to compare AI strategies under realistic constraints.

Purpose of the Study:

• To benchmark transfer learning strategies for Foundation Models (FMs) against Convolutional Neural Networks (CNNs) for COVID-19 prognosis.
• To systematically compare fine-tuning methods (full, linear probing, parameter-efficient) under data scarcity and class imbalance.
• To provide empirical guidance for deploying AI in clinical workflows for medical imaging analysis.

Main Methods:

• Utilized four public COVID-19 chest X-ray datasets with varying sample sizes and class imbalances for mortality, severity, and ICU admission prediction.
• Adapted CNNs (ImageNet pretrained) and FMs (general/biomedical pretrained) using full fine-tuning, linear probing, and parameter-efficient fine-tuning (PEFT).
• Evaluated models in full-data and few-shot regimes using Matthews Correlation Coefficient (MCC) and Precision-Recall AUC (PR-AUC), employing cross-validation and class-weighted losses.

Main Results:

• CNNs with full fine-tuning demonstrated robustness on small, imbalanced datasets.
• FMs with PEFT (LoRA, BitFit) achieved competitive results on larger datasets, though performance degraded with severe class imbalance.
• In few-shot scenarios, FMs exhibited limited generalization; linear probing offered the most stable performance.

Conclusions:

• No single fine-tuning strategy is universally optimal for COVID-19 prognosis prediction using medical imaging.
• CNNs remain a reliable choice for low-resource clinical scenarios.
• FMs, particularly with parameter-efficient methods, are effective when sufficient data is available, guiding AI deployment strategies.