Validation of a Major and Clinically Relevant Nonmajor Bleeding Phenotyping Algorithm on Electronic Health Records

Bleeding is an important health outcome of interest in epidemiological studies. We aimed to develop and validate rule-based algorithms to identify (1) major bleeding and (2) all clinically relevant bleeding (CRB) (composite of major and all clinically relevant nonmajor bleeding) within real-world electronic healthcare data.

We took a random sample (n = 1630) of inpatient admissions to Singapore public healthcare institutions in 2019 and 2020, stratifying by hospital and year. We included patients of all age groups, sex, and ethnicities. Presence of major bleeding and CRB were ascertained by two annotators through chart review. A total of 630 and 1000 records were used for algorithm development and validation, respectively. We formulated two algorithms: sensitivity- and positive predictive value (PPV)-optimized algorithms. A combination of hemoglobin test patterns and diagnosis codes were used in the final algorithms.

During validation, diagnosis codes alone yielded low sensitivities for major bleeding (0.16) and CRB (0.24), although specificities and PPV were high (>0.97). For major bleeding, the sensitivity-optimized algorithm had much higher sensitivity and negative predictive values (NPVs) (sensitivity = 0.94, NPV = 1.00), however false positive rates were also relatively high (specificity = 0.90, PPV = 0.34). PPV-optimized algorithm had improved specificity and PPV (specificity = 0.96, PPV = 0.52), with little reduction in sensitivity and NPV (sensitivity = 0.88, NPV = 0.99). For CRB events, our algorithms had lower sensitivities (0.50-0.56).

The use of diagnosis codes alone misses many genuine major bleeding events. We have developed major bleeding algorithms with high sensitivities, which can ascertain events within populations of interest.