A video based benchmark data set (ENDOTEST) to evaluate computer-aided polyp detection systems.

Area of Science:

• Gastroenterology
• Medical Imaging
• Artificial Intelligence in Medicine

Background:

• Computer-aided polyp detection (CADe) systems are emerging as standard tools for colonoscopy.
• Preclinical assessment of CADe systems is crucial before large-scale clinical trials.
• A novel video-based benchmark technique, ENDOTEST, is introduced for evaluating CADe systems.

Purpose of the Study:

• To introduce and validate the ENDOTEST benchmark for preclinical assessment of CADe systems.
• To compare the performance of a commercially available CADe system (GI-Genius) with a newly developed system (ENDOMIND).

Main Methods:

• The ENDOTEST benchmark utilizes two datasets: a validation set (48 video-snippets) and a performance set (10 full-length colonoscopies).
• Performance was assessed based on polyp detection accuracy and First Polyp Detection Time (FDT).
• Two CADe systems, GI-Genius and ENDOMIND (a convolutional neural network), were evaluated.

Main Results:

• Both CADe systems detected all polyps in at least one image within the ENDOTEST.
• ENDOMIND showed a per-frame sensitivity of 54% and specificity of 92.7%.
• ENDOMIND's median FDT (217 ms) was significantly faster than GI-Genius (1050 ms, p=0.003).

Conclusions:

• The ENDOTEST benchmark is a valuable tool for preclinical evaluation of CADe devices.
• A shorter FDT may correlate with higher sensitivity and lower specificity in polyp detection.