Assessing clinical efficacy of polyp detection models using open-access datasets.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Colorectal Cancer Research

Background:

• Accurate polyp detection during colonoscopy is crucial for colorectal cancer (CRC) prevention.
• Deep learning-based computer-aided detection (CADe) systems show promise in improving endoscopist performance.
• Effective CADe systems require high polyp detection rates and low false positive (FP) rates.

Purpose of the Study:

• To develop and benchmark computer-aided detection (CADe) systems for colonoscopy using a unified platform of open-access datasets.
• To introduce novel performance metrics that align with clinical needs for evaluating CADe systems.
• To assess CADe system performance under realistic clinical conditions using full-procedure videos.

Main Methods:

• Integrated four open-access datasets (340,000+ images, 380 polyps) and the REAL-Colon dataset (60 videos) for model development and benchmarking.
• Employed YOLOv7 for training and testing CADe models across diverse data splits.
• Utilized traditional object detection metrics alongside new metrics, including per-polyp recall at early detection and average per-patient FPs, to generate Free-Response Receiver Operating Characteristic (FROC) curves.

Main Results:

• Demonstrated the robustness of the open-access platform for CADe system development and benchmarking.
• Showcased how new metrics enable optimization of CADe operational parameters based on clinical criteria (e.g., per-patient FPs, early polyp detection).
• Highlighted that omitting full-procedure videos leads to unrealistic assessments and that detecting small polyps is challenging.

Conclusions:

• Newly available open-access data facilitates research in clinically relevant settings.
• Introduced metrics and FROC curves effectively illustrate CADe clinical efficacy.
• The developed platform and metrics aid in tuning CADe hyperparameters for improved performance.