Multi-center colonoscopy quality measurement utilizing natural language processing.

Area of Science:

• Medical Informatics
• Gastroenterology
• Artificial Intelligence in Healthcare

Background:

• Effective colorectal cancer (CRC) screening and surveillance rely on accurate tracking of colonoscopy quality and patient follow-up intervals.
• Current methods for tracking these metrics can be inconsistent and costly across different healthcare institutions.
• Natural Language Processing (NLP) offers a potential solution for automating and standardizing this process.

Purpose of the Study:

• To develop and validate an NLP-based system for assessing colonoscopy quality and determining surveillance intervals.
• To evaluate the system's accuracy and consistency across multiple institutions with varying reporting methods.

Main Methods:

• An NLP program was trained and tested on 750 paired colonoscopy and pathology reports from 13 centers (42,569 total reports).
• The system was designed to extract 19 key quality and surveillance measures.
• Performance was benchmarked against blinded, expert manual review, with additional testing on non-annotated reports to assess consistency.

Main Results:

• The NLP system demonstrated high accuracy in identifying colorectal cancer (99.6%) and various polyp types, including advanced adenomas (95%) and sessile serrated polyps (99.8%).
• Accuracy for lesion location and adenoma count was also high (87.0-99.8% and 92%, respectively).
• While paired annotators had a slightly lower error rate than NLP overall, NLP performance was consistent across measures and sites.

Conclusions:

• NLP can accurately extract critical data from colonoscopy reports to determine adenoma detection rates.
• The system effectively identifies components needed for guideline-adherent colonoscopy surveillance intervals.
• This NLP approach shows promise for improving the standardization and efficiency of CRC screening and surveillance across diverse healthcare settings.