Current validation practice undermines surgical AI development

Area of Science:

• Surgical Data Science
• Artificial Intelligence in Medicine
• Medical Imaging Analysis

Background:

• Clinical adoption of AI in surgery is limited, partly due to inadequate validation methods.
• Current validation practices for AI in surgical video analysis often overlook temporal and hierarchical data structures, leading to unreliable results.

Purpose of the Study:

• To identify and catalog validation pitfalls in AI-based surgical video analysis.
• To propose a framework of best practices for rigorous validation of surgical AI algorithms.

Main Methods:

• A multi-stage Delphi process involving 92 international experts to identify validation pitfalls.
• Systematic review of surgical AI literature to assess the prevalence of identified pitfalls.
• Empirical experiments on real surgical video datasets to demonstrate the impact of validation pitfalls.

Main Results:

• A comprehensive catalog of validation pitfalls across data, metrics, and reporting was established.
• A systematic review confirmed widespread use of flawed validation practices in surgical AI research.
• Experiments showed that ignoring temporal and hierarchical structures can lead to understated uncertainty and altered algorithm performance.

Conclusions:

• Addressing identified validation pitfalls is crucial for improving the reliability of AI in surgical video analysis.
• The proposed framework and best practices can guide more rigorous validation, benchmarking, and clinical translation of surgical AI.
• This work aims to enhance the trustworthiness and clinical utility of AI tools in surgery.