Multi-source data integration through pooling and transfer learning improves generalizability and specialization of...

Most Artificial Intelligence prognostic models in intensive care are trained and validated on a single source, limiting their reliability in external settings. This study evaluated generalizability and specialization for ICU mortality and remaining length of stay (RLoS) prediction across international databases, and explored multi-source strategies to mitigate the performance cost of external deployment.

Using the Temporal Pointwise Convolution (TPC) architecture, we conducted external validation across four harmonized BlendedICU databases (eICU-CRD, MIMIC-IV, AmsterdamUMCdb, HiRID), extracting nearly 20,000 unique patients from each. Four training configurations were compared: single-source training on AmsterdamUMC (N = 3,574) or MIMIC-IV (N = 10,915), data pooling, and transfer learning (both N = 14,489). Performance was assessed using AUROC and AUPRC for mortality, MAPE for RLoS, and a composite metric (Mcomposite).

Internal validation consistently overestimated external performance, with MAPE increasing by up to 51.8% and AUROC dropping by up to 13.8% on external databases. Site-specific features such as drug exposure acted as "shortcuts" degrading portability. Data pooling emerged as the superior strategy, achieving the best generalization (Mcomposite improvement up to 8.0% over single-source models) while matching or outperforming transfer learning for specialization - particularly for RLoS (MAPE 84.49 vs. 89.24, non-overlapping confidence intervals).

Single-source training compromises clinical applicability. Data pooling across harmonized international cohorts is the most effective strategy for both generalization and specialization, supporting a "Data-Centric AI" approach. We advocate for the democratization of local ICU datasets and open-source customizable models to empower independent clinical validation.