A deep-learning based system using multi-modal data for diagnosing gastric neoplasms in real-time (with video)

Area of Science:

• Gastroenterology
• Medical Imaging
• Artificial Intelligence

Background:

• White light (WL) and weak-magnifying (WM) endoscopy are crucial for diagnosing gastric neoplasms.
• Accurate and real-time diagnosis of gastric neoplasms remains a clinical challenge.

Discussion:

• Feature-level fusion of WL and WM data yielded the optimal multi-modal model (Model 4, ENDOANGEL-MM).
• ENDOANGEL-MM demonstrated superior accuracy compared to single-modal models and outperformed endoscopists in specific diagnostic scenarios.
• The system's assistance significantly improved the diagnostic accuracy of non-expert endoscopists, bringing their performance close to that of experts.

Key Insights:

• ENDOANGEL-MM achieved higher accuracy than single-modal approaches in still images, real-time videos, and prospective patient evaluations.
• The multi-modal system significantly outperformed endoscopists using WM data and multi-modal data.
• Assisted by ENDOANGEL-MM, non-expert endoscopists showed significant accuracy improvement, comparable to expert performance.

Outlook:

• ENDOANGEL-MM shows promise for integration into real-world clinical settings for gastric neoplasm diagnosis.
• Further validation and clinical trials are warranted to establish its widespread utility.
• The development highlights the potential of AI in enhancing endoscopic diagnostics and improving patient outcomes.