Training and validating a treatment recommender with partial verification evidence

Area of Science:

• Medical Informatics
• Clinical Decision Support
• Biostatistics

Background:

• Clinical decision support systems (DSS) are typically trained on observational data from a specific clinic.
• This limits their application to treatments already validated in randomized clinical trials (RCTs) but not yet in clinical practice.
• A method is needed to train and validate DSS using existing RCT data before clinical implementation.

Purpose of the Study:

• To develop and validate a method for training and validating DSS core using data from randomized clinical trials (RCTs).
• To address challenges of missing treatment rationale and verification evidence inherent in RCT data.
• To enable the use of RCT data for pre-clinical DSS training and validation.

Main Methods:

• Re-modeling the target variable to control for general treatment effects rather than random individual assignments.
• Utilizing a machine learning core robust to missing features and employing ensemble methods for small patient numbers.
• Introducing counterfactual treatment verification to compare DSS recommendations against RCT assignments.

Main Results:

• The developed approach successfully leverages RCT data for DSS learning and verification.
• The DSS demonstrated the ability to suggest treatments that improve patient outcomes.
• Results are constrained by the limited number of patients per treatment group in the RCT data.

Conclusions:

• A foundation is established for creating decision support tools for treatments validated in RCTs but not yet clinically deployed.
• Practitioners can utilize this method to train and validate DSS using available RCT data.
• Future work should focus on enhancing predictor robustness, potentially exploring synthetic data generation.