Creating a standardized tool for the evaluation and comparison of artificial intelligence-based computer-aided...

Area of Science:

• Gastroenterology
• Medical Imaging
• Artificial Intelligence in Medicine

Background:

• Computer-aided detection (CADe) software is increasingly used for colorectal cancer screening.
• Existing CADe algorithms lack standardized evaluation methods, creating uncertainty in performance comparison.
• Regulatory approval has been granted for CADe in multiple global regions.

Purpose of the Study:

• To establish priority scoring metrics for the evaluation and comparison of CADe algorithms in colonoscopy.
• To develop a consensus-based framework for assessing CADe performance in clinical practice.

Main Methods:

• A modified Delphi approach involving 25 global experts in CADe for colonoscopy.
• An 8-month online survey process including criteria generation, open commenting, and multi-round priority scoring.
• Utilized a 5-point Likert scale for ranking 121 initially generated scoring criteria, refined to a final set of 6.

Main Results:

• Sensitivity and separate/independent validation of CADe algorithms were identified as the top priority metrics (tied).
• Adenoma detection rate, false-positive rate, latency, and adenoma miss rate were also ranked as high-priority evaluation criteria.
• The study identified 6 key metrics through international expert consensus.

Conclusions:

• This research presents the first international consensus on priority scoring metrics for CADe in colonoscopy.
• The identified metrics should guide the development and refinement of CADe software for improved colorectal cancer screening.
• Future research should focus on validating these metrics using benchmark datasets to create a standardized scoring instrument.