Knowledge distillation for TBI prognosis: Addressing feature mismatch in heterogeneous clinical datasets

Traumatic Brain Injury (TBI) remains a major public health concern, requiring accurate prognostic models to support clinical decision-making. While high-performance models have been developed using large, feature-rich datasets, their applicability is often limited by the variability and restricted availability of clinical data across hospitals. This study explores knowledge distillation as a strategy to adapt complex models for resource-limited settings with fewer available variables.

A teacher model was trained on the MIMIC-III dataset using 326 features, while a student model, constrained to 20 features common to both MIMIC-III and eICU, learned from the teacher's probabilistic outputs (soft labels). The student model was designed to approximate the teacher's predictions while enhancing generalization. To address differences in outcome distributions between datasets (MIMIC-III: 20.9% mortality; eICU: 8.4% mortality), isotonic regression calibration was applied to refine predicted probabilities. Model performance was evaluated and compared against baseline approaches.

The knowledge distillation model outperformed baseline feature selection approaches, achieving higher accuracy (AUC: 0.864; AUPRC: 0.499), significantly surpassing the feature selection model (AUC: 0.856; AUPRC: 0.458; p = 8.069e-8). Calibration further improved model performance (AUC: 0.872; AUPRC: 0.537; p = 4.413e-15), aligning predicted probabilities with observed outcomes and mitigating dataset distribution disparities.

Knowledge distillation effectively transfers predictive power from a complex, feature-rich model to a feature-constrained environment, improving TBI prognostication despite substantial feature mismatch. These findings highlight the potential for implementing advanced predictive models in diverse clinical settings, with prospective validation needed to confirm their real-world impact.