A Framework for Using Real-World Data and Health Outcomes Modeling to Evaluate Machine Learning-Based Risk Prediction...

Area of Science:

• Health outcomes research
• Clinical decision support
• Predictive modeling

Background:

• Lung transplant (LTx) referral in cystic fibrosis is complex.
• Current referral policies may not optimize patient outcomes.
• Real-world data (RWD) can inform model evaluation.

Purpose of the Study:

• To evaluate the clinical utility of novel machine learning (ML) risk prediction models for LTx referral.
• To compare ML-based referrals with existing methods using RWD.
• To develop a simulation model for projecting LTx referral and survival outcomes.

Main Methods:

• Utilized longitudinal RWD from 4247 adult cystic fibrosis patients.
• Compared LTx referral policies: ML mortality predictions vs. FEV1 alone vs. usual care (UC).
• Developed a patient-level simulation model incorporating transplant availability and allocation policies.

Main Results:

• ML referral rates (20%) exceeded FEV1 (19%) and UC (12%).
• ML could have referred 40% of patients who died before UC referral.
• Differences in referral timing did not significantly impact 5-year transplant numbers or survival.

Conclusions:

• Health outcomes modeling with RWD can identify ML models with high clinical utility.
• This framework aids in prioritizing investment in effective predictive tools.
• It helps to avoid investing in models unlikely to provide real-world benefits.