Large language model for interpreting the Paris classification of colorectal polyps

Area of Science:

• Gastroenterology
• Artificial Intelligence
• Medical Imaging

Background:

• Accurate colorectal polyp morphology reporting is crucial for patient management.
• Current reporting methods using the Paris classification often lack accuracy.
• Multimodal large language models (M-LLMs) show potential for improving morphological assessment.

Purpose of the Study:

• To evaluate the accuracy of an M-LLM (GPT-4o) in classifying colorectal polyp morphology.
• To compare M-LLM performance against expert and non-expert endoscopists.
• To assess the M-LLM's ability to differentiate polypoid from non-polypoid and sessile from pedunculated lesions.

Main Methods:

• Utilized the SUN dataset of 100 colonoscopy videos with Paris classification labels.
• An M-LLM (GPT-4o) classified five frames per lesion.
• Expert and non-expert endoscopists independently classified the same lesions.

Main Results:

• M-LLM accuracy in differentiating non-polypoid from polypoid lesions was 73%, comparable to experts (75%) and non-experts (77%).
• M-LLM accuracy in differentiating sessile from pedunculated lesions was 55%, significantly lower than experts (76%) and non-experts (77%).
• The M-LLM demonstrated poor specificity (12%) in distinguishing sessile from pedunculated lesions compared to experts (82%) and non-experts (88%).

Conclusions:

• M-LLMs demonstrate comparable performance to endoscopists in distinguishing non-polypoid from polypoid colorectal lesions.
• M-LLMs currently fail to reliably identify pedunculated morphology, indicating a need for further refinement.
• The findings suggest M-LLMs may assist in polyp morphology assessment, but require improvement for specific classifications.