Pathophysiological Features in Electronic Medical Records Sustain Model Performance under Temporal Dataset Shift

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Health data science

Background:

• Electronic medical records (EMRs) are crucial for developing supervised machine learning (ML) models in clinical settings.
• Limited research exists on how specific EMR features affect ML model performance during temporal dataset shift.

Purpose of the Study:

• To analyze the impact of EMR feature groups and categories on initial and sustained ML model performance.
• To understand how different feature types influence model robustness against temporal dataset shift.

Main Methods:

• Features from EMRs were aggregated into source-based feature groups (e.g., medications, diagnoses, labs).
• Features were categorized based on patient pathophysiology or healthcare processes.
• Adapted Shapley values were used to quantify the marginal contribution of feature groups and categories.

Main Results:

• Feature contributions to initial model performance varied across different clinical prediction tasks.
• Features reflecting patient pathophysiology were found to be crucial in reducing performance decline over time.
• Specific feature groups demonstrated varying impacts on both initial performance and robustness to temporal shifts.

Conclusions:

• Interpretable insights into feature contributions can guide the development of more robust clinical ML models.
• Prioritizing pathophysiological features may enhance the long-term reliability of ML models facing temporal dataset shift.
• This study provides a framework for assessing feature impact in dynamic healthcare data environments.