Natural Language Processing in Clinical Quality Measures of Diagnostic Performance: Learnings from Three Case Reports

Clinical quality measures (CQMs) have historically relied on structured data elements in electronic health records; however, unstructured fields more reliably capture information needed to assess quality of care. Unstructured fields such as clinical notes and imaging results are particularly important for assessing diagnostic performance. Natural language processing (NLP) can efficiently capture nuanced information from unstructured fields to assess healthcare quality. This article describes three CQM projects that incorporated NLP in ways that are scalable for healthcare systemwide implementation.

The projects leveraged NLP to measure diagnostic performance related to venous thromboembolism, pneumonia, and pulmonary embolism, each of which is characterized by nonspecific symptoms and patient safety risk.

NLP was used to extract patient symptoms, procedures, and findings from clinical notes in outpatient settings, emergency department visits, radiology reports, and discharge summaries. One NLP algorithm developed in the US Department of Veterans Affairs system performed comparably in an academic medical system. Other measures were successfully implemented in a data registry. Two of these CQMs were endorsed by the Centers for Medicare & Medicaid Services consensus-based entity. The third CQM set will be available as registry measures.

These projects demonstrate that CQMs leveraging NLP can achieve the high levels of reliability and validity necessary for national-level implementation. This will support efforts to assess diagnostic performance, which is often challenging to assess using structured data.