Methods for Clinical Evaluation of Artificial Intelligence Algorithms for Medical Diagnosis

Area of Science:

• Medical Informatics
• Clinical Trial Design
• Artificial Intelligence in Healthcare

Background:

• Clinical evaluation of artificial intelligence (AI) algorithms is crucial before healthcare adoption.
• AI evaluation must confirm performance via external testing and demonstrate benefits over conventional care.
• Prospective studies are preferred for robust clinical evaluation of AI.

Purpose of the Study:

• To outline fundamental methodological considerations for designing and appraising clinical evaluations of AI algorithms in medical diagnosis.
• To provide guidance on effective external testing, performance metrics, study designs, and reporting guidelines for AI clinical studies.

Main Methods:

• Discussion of external testing strategies for AI algorithms.
• Review of AI performance metrics and graphical evaluation methods.
• Explanation of paired and parallel study designs, including randomized clinical trials.
• Emphasis on EQUATOR Network guidelines for reporting AI clinical studies.

Main Results:

• Highlights the importance of external validation for AI diagnostic tools.
• Details various metrics and study designs for evaluating AI performance and impact.
• Stresses the need for adherence to established reporting guidelines.

Conclusions:

• Sound methodological knowledge is essential for designing, executing, reporting, and appraising AI clinical evaluations.
• Proper evaluation ensures the safe and effective integration of AI into medical practice.