Towards practical federated learning and evaluation for medical prediction models

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Privacy-Preserving Technologies

Background:

• Federated learning (FL) enables collaborative AI model training while protecting sensitive patient data.
• Healthcare faces privacy and regulatory hurdles for centralized data sharing, making FL a promising alternative.
• FL has demonstrated potential in disease detection, matching centralized system performance, but real-world application is still developing.

Purpose of the Study:

• To assess the practical effectiveness of federated learning for predicting the need for prostate cancer biopsies.
• To introduce and evaluate a novel federated learning assessment strategy, Leave-Silo-Out (LSO).
• To compare federated learning models against locally trained models, focusing on local patient diagnosis improvement.

Main Methods:

• Utilized 14 public prostate cancer datasets from 10 countries for evaluation.
• Proposed and benchmarked the Leave-Silo-Out (LSO) strategy to measure federated learning performance against non-contribution (free-riding).
• Investigated the performance of multi-hospital federated learning models versus single-institution local models.

Main Results:

• Federated learning benefits are contingent on the volume of local annotated data.
• Hospitals with minimal data showed negligible gains from FL compared to free-riding.
• Moderate datasets may see improvements with FL, while extensive datasets often yield no advantage or even performance degradation versus local training.

Conclusions:

• Federated learning offers potential for data-scarce environments in medical AI.
• The practical utility of FL in healthcare is highly context-specific, influenced by data volume and task demands.
• Further research is needed to optimize FL implementation for diverse clinical settings.