Automatic image and text-based description for colorectal polyps using BASIC classification.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in gastroenterology

Background:

• Colorectal polyps (CRP) are precursors to colorectal cancer (CRC).
• Accurate in-vivo identification of CRPs relies on endoscopist expertise and classification models like BASIC.
• Current computer-aided detection (CADe) and diagnosis (CADx) systems lack robust clinical information translation.

Purpose of the Study:

• To develop a CADx system combining image classification and language models for automatic generation of descriptive text for CRP images.
• To create text output comparable to human endoscopist language for enhanced clinical communication.
• To evaluate the system's performance in describing polyps across White Light (WL), Blue Light Imaging (BLI), and Linked Color Imaging (LCI).

Main Methods:

• An AI model integrating an image feature encoder and a BERT module was developed.
• The system was trained to generate text captions for colorectal polyp images.
• Performance was evaluated using an external test set and linguistic metrics (BLEU-1, ROUGE-L, METEOR).

Main Results:

• The AI system successfully generated complete sentences describing colorectal polyp images.
• Achieved linguistic metric scores: BLEU-1 = 0.67, ROUGE-L = 0.83, METEOR = 0.50.
• Demonstrated capability to describe images acquired with WL, BLI, and LCI.

Conclusions:

• The developed CADx system provides automatic image captioning for colorectal polyps.
• This facilitates advancements in automatic reporting of endoscopic findings.
• The system has the potential to reduce the need for time-consuming histology assessments.