Applying active learning to high-throughput phenotyping algorithms for electronic health records data.

Area of Science:

• Clinical informatics
• Biomedical data science
• Machine learning in healthcare

Background:

• Supervised machine learning (ML) algorithms can accelerate the use of electronic health records (EHRs) for clinical and translational research.
• High-throughput phenotyping using ML often requires extensive annotated samples, which are resource-intensive to obtain.
• Active learning (AL) offers a potential solution to reduce the annotation burden in ML-based phenotyping.

Purpose of the Study:

• To investigate the efficacy of active learning (AL) in ML-based phenotyping algorithms.
• To compare the performance of AL with passive learning (PL) using uncertainty sampling and random sampling, respectively.
• To evaluate the impact of feature sets (unrefined vs. refined) on phenotyping performance.

Main Methods:

• Integrated an uncertainty sampling AL approach with support vector machines (SVMs).
• Evaluated performance across three disease cohorts: rheumatoid arthritis (RA), colorectal cancer (CRC), and venous thromboembolism (VTE).
• Compared AL against passive learning (PL) using both unrefined and expert-defined refined feature sets.

Main Results:

• Active learning (AL) outperformed passive learning (PL) in all three phenotyping tasks.
• AL reduced the number of required annotated samples by up to 68% for achieving a 0.95 area under the curve (AUC) in RA and CRC.
• AL achieved a 68% reduction in annotated samples for VTE with an optimal AUC of 0.70 using refined features; refined features generally improved performance.

Conclusions:

• Active learning (AL) is a valuable method for enhancing ML-based phenotyping.
• Combining AL with domain knowledge-driven feature engineering can lead to efficient and generalizable phenotyping methods.
• This study demonstrates a practical approach to overcome annotation limitations in EHR-based research.