From Embeddings to Accuracy: Comparing Foundation Models for Radiographic Classification

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Machine Learning

Background:

• Foundation models offer robust, transferable embeddings for machine learning tasks.
• These embeddings are applicable to diverse domains, including medical imaging diagnostics.
• Lightweight adapter models can leverage these embeddings for efficient model training.

Purpose of the Study:

• To evaluate foundation model-derived embeddings for training lightweight adapter models in multi-class radiography classification.
• To compare the performance of adapter models against end-to-end training of a convolutional neural network.
• To assess the computational efficiency and fairness of the adapter models.

Main Methods:

• Extracted embeddings from seven foundation models (DenseNet121, BiomedCLIP, Med-Flamingo, MedImageInsight, MedSigLIP, Rad-DINO, CXR-Foundation) using 8842 radiographs.
• Trained adapter models (KNN, LR, SVM, RF, MLP) using these embeddings for classification.
• Compared performance using mean area under the curve (mAUC) and conducted Wilcoxon signed-rank tests and fairness analyses.

Main Results:

• MedImageInsight embeddings with SVM or MLP adapters achieved the highest mAUC (93.1%), surpassing a fully finetuned DenseNet121 (87.2%).
• Most adapter models demonstrated computational efficiency, training in minutes and inferring in seconds on CPU.
• Fairness analysis showed minimal performance disparities across gender and age groups.

Conclusions:

• Foundation model embeddings, particularly from MedImageInsight, enable accurate, efficient, and equitable radiographic diagnostic classification via lightweight adapters.
• This approach offers a practical alternative to computationally intensive end-to-end training for clinical applications.
• The study confirms the utility of foundation model embeddings for developing high-performing and fair medical imaging AI tools.