Plug-and-play use of tree-based methods: consequences for clinical prediction modeling

Area of Science:

• Machine Learning in Healthcare
• Clinical Prediction Modeling
• Data Science Applications

Background:

• Tree-based models (e.g., random forest, XGBoost) are widely adopted for clinical prediction.
• Their behavior regarding predictor-outcome associations and extrapolation is often overlooked.
• Understanding these behaviors is critical for reliable clinical decision-making.

Purpose of the Study:

• To illustrate the behavior of tree-based models in clinical prediction.
• To discuss the implications of their "plug-and-play" use.
• To highlight limitations in learning smooth, monotonic, and additive associations, and in extrapolation.

Main Methods:

• A simulation study with standard normal predictors and logistic outcomes.
• Analysis of a real-world clinical example: post-radiotherapy toxicity prediction.
• Assessment using a public dataset of head and neck cancer patients.
• Visualization of learned predictor-outcome associations across varying sample sizes.

Main Results:

• Tree-based models exhibit stepwise, non-smooth, and non-monotonic predictor-outcome associations.
• They struggle to learn additive effects due to orthogonal splitting.
• Extrapolation results in constant predictions beyond observed data ranges.
• Learned associations can be biologically implausible, impacting trustworthiness and generalizability.

Conclusions:

• "Plug-and-play" use of tree-based models can yield undesirable predictor-outcome associations.
• Careful consideration of model behavior during development and evaluation is essential.
• Further research into constrained and soft-split decision trees is warranted.