Provider-specific quality measurement for ERCP using natural language processing

Natural language processing (NLP) is an information retrieval technique that has been shown to accurately identify quality measures for colonoscopy. There are no systematic methods by which to track adherence to quality measures for ERCP, the highest risk endoscopic procedure widely used in practice. Our aim was to demonstrate the feasibility of using NLP to measure adherence to ERCP quality indicators across individual providers.

ERCPs performed by 6 providers at a single institution from 2006 to 2014 were identified. Quality measures were defined using society guidelines and from expert opinion, and then extracted using a combination of NLP and data mining (eg, ICD9-CM codes). Validation for each quality measure was performed by manual record review. Quality measures were grouped into preprocedure (5), intraprocedure (6), and postprocedure (2). NLP was evaluated using measures of precision and accuracy.

A total of 23,674 ERCPs were analyzed (average patient age, 52.9 ± 17.8 years, 14,113 were women [59.6%]). Among 13 quality measures, precision of NLP ranged from 84% to 100% with intraprocedure measures having lower precision (84% for precut sphincterotomy). Accuracy of NLP ranged from 90% to 100% with intraprocedure measures having lower accuracy (90% for pancreatic stent placement).

NLP in conjunction with data mining facilitates individualized tracking of ERCP providers for quality metrics without the need for manual medical record review. Incorporation of these tools across multiple centers may permit tracking of ERCP quality measures through national registries.