Unsupervised Coverage Sampling to Enhance Clinical Chart Review Coverage for Computable Phenotype Development:...

Area of Science:

• Health Informatics
• Machine Learning
• Clinical Research

Background:

• Developing computable phenotypes (CPs) from electronic health records (EHR) relies on clinician chart review for gold-standard labels.
• Random sampling of patient charts may not capture population diversity, leading to biased and poorly performing CPs, especially for smaller subpopulations.

Purpose of the Study:

• To propose an unsupervised coverage sampling approach for EHR data.
• To enhance patient cohort diversity and improve information coverage in chart review samples for better CP development.

Main Methods:

• Implemented an unsupervised coverage sampling method involving patient population clustering and stratified sampling.
• Introduced a nearest neighbor distance metric to evaluate sample coverage.
• Compared coverage sampling against random sampling via simulation studies and a real-world COVID-19 hospitalization CP development.

Main Results:

• Coverage sampling demonstrated broader patient population coverage than random sampling in simulations.
• When subpopulations exist, coverage sampling improved the area under the receiver operating characteristic curve (AUC) by approximately 0.03-0.05.
• In a real-world COVID-19 application, coverage sampling yielded a more representative sample and a 0.02 AUC improvement over random sampling.

Conclusions:

• The proposed coverage sampling method is easy to implement and generates more representative chart review samples.
• This leads to CPs with improved performance for both subpopulations and the overall cohort.
• Alternative sampling strategies beyond random selection should be considered for CP development.