Evaluating predictive modeling algorithms to assess patient eligibility for clinical trials from routine data

Area of Science:

• Biomedical Informatics
• Clinical Trial Management
• Machine Learning in Healthcare

Background:

• Translating free-text eligibility criteria into electronic health record (EHR) compatible rules is a major challenge for clinical trial recruitment systems.
• Case-based reasoning offers an alternative by using past cases, reducing reliance on explicit rules and EHR terminology.
• Evaluating predictive modeling for patient eligibility assessment is crucial for improving recruitment efficiency.

Purpose of the Study:

• To assess the feasibility of predictive modeling for patient eligibility in clinical trials.
• To evaluate the performance of a prototype system using various configurations.
• To compare predictive modeling against traditional rule-based recruitment systems.

Main Methods:

• Developed a prototype using existing patient data (eligible/ineligible) to induce prediction models.
• Measured performance retrospectively across three clinical trials using receiver operating characteristic curves (ROC-AUC).
• Tested different prediction algorithms, learning set sizes, and patient attribute aggregation levels.

Main Results:

• Random forests demonstrated strong performance, achieving maximum ROC-AUC values of 0.81 (Trial A), 0.96 (Trial B), and 0.99 (Trial C).
• Optimal performance for random forests was achieved with approximately 200 manually screened patients.
• Aggregation levels of diagnosis and procedure codes did not significantly impact algorithm performance.

Conclusions:

• Predictive modeling is a feasible approach for supporting clinical trial patient recruitment.
• Key advantages include independence from specific eligibility criteria and EHR data representations.
• The potential for automation makes predictive modeling a promising alternative to current rule-based systems.