Bridging the Generalisation Gap: Synthetic Data Generation for Multi-Site Clinical Model Validation.

Area of Science:

• Clinical Machine Learning
• Data Science
• Healthcare Informatics

Background:

• Clinical machine learning (ML) model generalisability is challenged by healthcare setting variability.
• Current evaluation methods using real-world data are limited by availability, bias, and lack of experimental control.
• Generative models often lack transparency and control over data distributional shifts.

Purpose of the Study:

• To propose a novel structured synthetic data framework for controlled benchmarking of clinical ML models.
• To enable systematic evaluation of model robustness, fairness, and generalisability.
• To provide a tool for investigating model responses to specific distributional shifts and biases.

Main Methods:

• Developed a structured synthetic data framework with explicit control over data generation.
• Incorporated site-specific prevalence variations, hierarchical subgroup effects, and feature interactions.
• Conducted controlled experiments to benchmark model performance under varying conditions.

Main Results:

• Demonstrated the framework's ability to isolate the impact of site variations on ML models.
• Showcased support for fairness-aware audits and identification of generalisation failures.
• Highlighted the interaction between model complexity and site-specific effects.

Conclusions:

• The proposed framework offers a reproducible, interpretable, and configurable tool for clinical ML.
• It facilitates targeted investigation into factors affecting model performance and reliability.
• Aims to advance the dependable deployment of machine learning in clinical practice.