Training a high-performance retinal foundation model with half-the-data and 400 times less compute

Area of Science:

• Medical Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Medical AI development is constrained by large training datasets.
• Foundation models like RETFound-MEH offer adaptability but require substantial data and compute.
• Existing data-efficient models still demand significant computational resources.

Purpose of the Study:

• To develop a highly efficient and environmentally friendly medical AI foundation model.
• To reduce the data and computational requirements for training advanced AI models in healthcare.
• To improve the accessibility and sustainability of medical AI.

Main Methods:

• Proposed RETFound-Green, a novel foundation model trained on a significantly smaller dataset (75,000 images).
• Utilized a new Token Reconstruction objective for training.
• Evaluated performance on diverse downstream tasks using geographically varied datasets.

Main Results:

• RETFound-Green achieved comparable performance to existing models using 400x less compute.
• Training costs were drastically reduced to under $100, compared to $10,000-$14,000 for previous models.
• Demonstrated faster download, embedding computation, and reduced storage requirements.
• Achieved over double the statistically significant wins on downstream tasks compared to the next best model.

Conclusions:

• RETFound-Green offers a sustainable and cost-effective solution for medical AI foundation models.
• The novel training approach significantly lowers barriers to entry for developing and deploying medical AI.
• This advancement has the potential to accelerate AI adoption in healthcare globally.