Domain specific multimodal large language model for automated endoscopy reporting with multicenter prospective...

Accurate endoscopy reports are crucial for the diagnosis and management of patients with upper gastrointestinal (UGI) diseases, yet errors and omissions are common. The preparation of routine reports for common diseases is labor-intensive and time-consuming. To address this, here we developed Report-Angel, an integrated AI system based on a multi-modal large language model (MLLM) and conventional deep learning models and trained it on 20,617 image-text pairs to automatically generate detailed draft reports for UGI endoscopy. Report-Angel achieved a clinically acceptable report rate of 79.3% (95% CI: 74.4-83.5%) in the prospective internal and 83.3% (95% CI: 78.7-87.3%) in the prospective external cohort. At the case level, Report-Angel achieved a report completeness of 88.51% (95% CI: 84.64-92.38%) and a report accuracy of 78.93% (95% CI: 73.98-83.88%), with average processing time of 1.5 s per lesion in the internal prospective video dataset. Lesion-level reporting accuracies were 91.92% (95% CI: 90.58-93.25%), 89.07% (95% CI: 87.57-90.57%), and 83.94% (95% CI: 81.58-86.31%) on retrospective image and prospective single- and multi-center video datasets, respectively. Report-Angel generates expert-level draft endoscopy reports and demonstrates robust generalizability. By providing reliable foundation draft reports, this system has the potential to effectively standardize reporting and reduce endoscopists' workloads.