Knowledge distillation for TBI prognosis: Addressing feature mismatch in heterogeneous clinical datasets

Area of Science:

• Medical Informatics
• Machine Learning in Healthcare
• Clinical Prognostics

Background:

• Traumatic Brain Injury (TBI) poses a significant public health challenge, necessitating accurate prognostic models for clinical decision support.
• Existing high-performance TBI models often face limitations in real-world application due to data variability and restricted availability across healthcare institutions.
• This study investigates knowledge distillation to adapt sophisticated models for resource-constrained clinical environments with reduced variable availability.

Purpose of the Study:

• To explore the efficacy of knowledge distillation in transferring predictive capabilities from a complex TBI prognostic model to a simpler model with fewer features.
• To assess the performance of a feature-constrained model adapted via knowledge distillation compared to traditional feature selection methods.
• To evaluate the impact of isotonic regression calibration on model performance and its ability to address outcome distribution disparities between datasets.

Main Methods:

• A teacher model was trained on the MIMIC-III dataset utilizing 326 features.
• A student model, restricted to 20 common features from MIMIC-III and eICU, learned from the teacher model's soft labels.
• Isotonic regression calibration was applied to address differing mortality rates (MIMIC-III: 20.9%, eICU: 8.4%) and refine probability predictions.

Main Results:

• The knowledge distillation model demonstrated superior performance over baseline feature selection, achieving higher accuracy (AUC: 0.864 vs. 0.856) and Area Under the Precision-Recall Curve (AUPRC: 0.499 vs. 0.458).
• Calibration further enhanced the model's predictive power (AUC: 0.872, AUPRC: 0.537), aligning predicted probabilities with observed outcomes.
• The results indicate significant improvement (p = 8.069e-8 for initial comparison, p = 4.413e-15 for calibrated model).

Conclusions:

• Knowledge distillation successfully transfers predictive accuracy to feature-constrained settings, enhancing TBI prognostication despite significant feature mismatches.
• This approach shows promise for deploying advanced predictive models in diverse clinical settings with limited data.
• Further prospective validation is recommended to confirm the real-world clinical impact and utility of this method.