Developing and Evaluating a Natural Language Processing Algorithm to Detect Insulin Pump and Continuous Glucose...

We aimed to develop and validate natural language processing (NLP) algorithms to identify insulin pump and continuous glucose monitor (CGM) users using unstructured clinical note data from the electronic health record (EHR).

We reviewed a random sample of outpatient clinical notes from endocrinologists to catalog how insulin pump and CGM use was documented. We translated these patterns into regular expressions and used them to build rule-based NLP algorithms, which we iteratively refined. We evaluated the final algorithms in a University of California Los Angeles (UCLA) holdout dataset that included the most recent note from 667 unique patients. We then externally validated the algorithms in a second health system with a different EHR and patient population. Manual chart review served as the gold standard. We assessed performance with measures including sensitivity and specificity. To contextualize algorithm performance, we evaluated the accuracy of billing codes for insulin pump and CGM use within the same UCLA holdout dataset.

In the UCLA holdout dataset, our insulin pump algorithm achieved a sensitivity of 0.90 and specificity of 0.89. The CGM algorithm achieved a sensitivity of 0.85 and specificity of 0.84. The combined algorithm identifying both insulin pump and CGM use showed a sensitivity of 0.76 and specificity of 0.92. In comparison, billing codes underperformed: International Classification of Diseases/Current Procedural Terminology (CPT) codes identified insulin pump use with a sensitivity of 0.09 and specificity of 1.00, whereas CPT codes identified CGM use with a sensitivity of 0.68 and specificity of 0.86. For combined device use, billing codes had a sensitivity of 0.06 and specificity of 1.00. External validation demonstrated similarly strong algorithm performance in the second health system.

We showed that NLP can accurately identify insulin pump and CGM users from unstructured EHR notes, substantially outperforming billing code-based methods. This scalable approach can support system- and population-level evaluations of diabetes technologies.