Understanding palliative care trajectories through clustering, calibrated prediction models, and explainable AI

Area of Science:

• Palliative Care Research
• Machine Learning in Healthcare
• Health Services Research

Background:

• Palliative care access is limited and often initiated late.
• Current machine learning models lack integration of validated symptom and functional status measures.
• This limits interpretability of patient care trajectories.

Purpose of the Study:

• Identify distinct patient subgroups based on symptom burden and functional status.
• Predict palliative care episode duration using these identified subgroups.

Main Methods:

• Retrospective cohort study of 261,290 Australian palliative care patients (2014-2023).
• Unsupervised clustering (K-means, GMM, etc.) to derive symptom-function clusters.
• Supervised prediction models (Elastic Net, Random Forest, XGBoost) for care duration prediction.

Main Results:

• Clustering revealed a continuous severity gradient, with rapid escalation in mid-range functional states.
• Predictive performance was modest (R² ≈ 0.2), improving with calibration, especially for longer episodes.
• Clustering enhanced interpretability but not predictive accuracy; functional status and episode type were key predictors.

Conclusions:

• Palliative care needs follow a continuous severity gradient, with prediction accuracy linked to functional status.
• Prediction is more accurate near end-of-life, creating a paradox of high uncertainty when intervention potential is greatest.
• Machine learning value lies in combining calibrated predictions with interpretable stratification for stage-specific care planning.