Understanding palliative care trajectories through clustering, calibrated prediction models, and explainable AI

Area of Science:

• Palliative Care Research
• Machine Learning in Healthcare
• Health Services Research

Background:

• Palliative care access is limited and often initiated late in life.
• Current machine learning models lack integration of validated symptom and function measures, hindering interpretability of care trajectories.

Purpose of the Study:

• Identify distinct patient subgroups based on symptom burden and functional status.
• Predict palliative care episode duration using these identified subgroups.

Main Methods:

• Retrospective cohort study of 261,290 Australian palliative care patients (2014-2023).
• Employed unsupervised clustering (K-means, GMM, etc.) for symptom-function clusters.
• Utilized supervised models (Elastic Net, Random Forest, XGBoost) for duration prediction with calibration and explainability analyses.

Main Results:

• Clustering revealed a continuous severity gradient with rapid escalation in mid-range functional states.
• Predictive performance was modest (R² ≈ 0.2), improved by calibration, especially for longer episodes.
• Clustering enhanced interpretability by summarizing patient heterogeneity; prediction accuracy varied by cluster and disease stage.

Conclusions:

• Palliative care needs exhibit a continuous severity gradient, with prediction accuracy linked to functional status.
• Prediction is more accurate near end-of-life, creating a paradox where intervention potential is highest amid uncertainty.
• Machine learning's value lies in combining calibrated predictions with interpretable stratification for stage-specific care planning.