Enhancing and Not Replacing Clinical Expertise: Improving Named-Entity Recognition in Colonoscopy Reports Through...

Area of Science:

• Natural Language Processing
• Medical Informatics
• Artificial Intelligence in Healthcare

Background:

• Colonoscopy findings are typically unstructured free text, hindering secondary data use for quality monitoring, personalized medicine, and research.
• Accurate named-entity recognition (NER) is crucial for extracting valuable clinical information from these free-text reports.

Purpose of the Study:

• To compare the performance of NER models trained on real, synthetic, and mixed colonoscopy report data.
• To evaluate if privacy-preserving synthetic reports can enhance clinical information extraction accuracy.

Main Methods:

• Three Spark NLP biLSTM CRF models were trained: ModelR (real data), ModelS (synthetic data), and ModelM (mixed real/synthetic data).
• Models were evaluated on unseen real and synthetic colonoscopy reports for seven entity types.
• Performance metrics included micro-averaged precision, recall, and F1-score, with McNemar tests for statistical significance.

Main Results:

• The mixed-data model (ModelM) achieved the highest performance (F1 = 0.94), significantly outperforming single-source models (ModelR F1 = 0.70, ModelS F1 = 0.64).
• ModelR performed well on real data (F1 = 0.90) but poorly on synthetic data (F1 = 0.47); ModelS showed the opposite trend (F1 = 0.99 synthetic, F1 = 0.33 real).
• Statistical analysis confirmed the significant superiority of the mixed-data approach.

Conclusions:

• Synthetic colonoscopy reports are a valuable supplement but not a replacement for real annotated data.
• Training NER models on a balanced mix of real and synthetic data yields robust, generalizable models for structuring colonoscopy reports.
• This approach supports large-scale, privacy-preserving colorectal cancer surveillance and personalized follow-up strategies.