Multimodal Large Language Models for Cystoscopic Image Interpretation and Bladder Lesion Classification: Comparative...

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Urology

Background:

• Cystoscopy is the standard for bladder lesion diagnosis but is operator-dependent and can miss subtle abnormalities.
• Current AI systems lack interpretive capabilities and explanatory reporting for cystoscopic findings.
• Multimodal large language models (MM-LLMs) offer integrated visual recognition, reasoning, and language generation for enhanced interpretation.

Purpose of the Study:

• To evaluate state-of-the-art MM-LLMs for cystoscopic image interpretation and lesion classification.
• To assess diagnostic accuracy, reasoning quality, and clinical relevance using challenging, clinician-defined datasets.
• To compare the performance of different MM-LLMs on specific diagnostic tasks.

Main Methods:

• Four MM-LLMs (OpenAI-o3, ChatGPT-4o, Gemini 2.5 Pro, MedGemma-27B) were tested on free-text interpretation (n=401) and 7-class lesion classification (n=113).
• Outputs were scored by blinded raters using a 5-point Likert scale.
• Classification metrics including accuracy, sensitivity, specificity, Youden J, and MCC were calculated for lesion detection, biopsy indication, and malignancy.

Main Results:

• OpenAI-o3 showed strong performance in anatomy and findings interpretation but struggled with higher-order reasoning and diagnosis.
• OpenAI-o3 achieved the most balanced performance for clinically relevant endpoints, with 88.3% lesion detection accuracy.
• In 7-class classification, OpenAI-o3 outperformed other models, particularly for malignant lesions, though overall accuracy for malignancy was 62.8%.

Conclusions:

• MM-LLMs show significant potential as assistive tools for cystoscopy interpretation, biopsy triage, and training.
• Current MM-LLM performance in complex differential diagnoses is modest and requires further optimization.
• Clinical integration necessitates further development to ensure safe and reliable application in diagnosing bladder lesions.