Endo-Histo foundational fusion model: a novel artificial intelligence for assessing histologic remission and response...

Artificial intelligence (AI)-enabled endoscopy and histology offer accurate, objective, and rapid assessment of disease activity in ulcerative colitis (UC). Emerging multi-source AI models may enhance standardized disease evaluation and outcome prediction. This investigation aimed to develop a novel AI model fusing endoscopic and histologic features to improve the assessment of disease remission and response to therapy in UC clinical trials.

A novel multimodal AI model was developed that fuses endoscopic videos and histologic whole-slide images from a Phase 2 clinical trial of Mirikizumab in UC (NCT02589665). Informative endoscopic frames were predicted using convolutional neural networks and processed with BioMedCLIP, while histologic features were extracted using the CONCH foundational model. Multimodal features were then integrated via multi-head self-attention to generate a patient-level assessment. Model performance for assessing histologic remission (HR) and treatment response at weeks 12 and 52, based on histologic endpoints, was evaluated by cross-validation.

The fusion model outperformed single-modality assessments for HR, achieving a sensitivity of 89.72% (95% CI, 82.35-94.76), specificity of 89.67% (95% CI, 84.34-93.67), and accuracy of 89.69% (95% CI, 85.61-92.94). It showed a sensitivity of 97.96% (95% CI, 89.15-99.95), specificity of 86.84% (95% CI, 71.91-95.59), and accuracy of 93.10% (95% CI, 85.59-97.43) for assessing HR at week 52. Substantial agreement was observed between the AI-fusion model and central readout.

This novel tool significantly advances precision medicine in clinical trials by potentially standardizing central readouts and enabling automated disease assessment.