How to apply evidence-based practice to the use of artificial intelligence in radiology (EBRAI) using the data...

Area of Science:

• Radiology
• Artificial Intelligence
• Medical Informatics

Background:

• The increasing volume of radiology artificial intelligence (AI) research presents unique review challenges.
• Existing evidence-based practice (EBP) frameworks require adaptation for AI-specific literature.
• Radiologists need structured methods to critically appraise AI research.

Purpose of the Study:

• To introduce a novel systematic method, Data, Algorithm, Training, Output (DATO), to complement existing frameworks like PICO.
• To provide a tool for navigating and evaluating the growing body of radiology AI literature.
• To assist professionals in applying evidence-based radiology (EBR) principles to AI research.

Main Methods:

• The Data, Algorithm, Training, Output (DATO) method is proposed to systematically assess AI studies.
• DATO considers key components: Data, Algorithm, Training, and Output.
• The method is illustrated using bone age assessment from skeletal radiographs.

Main Results:

• A systematic search identified 17 bone age estimation papers, with 5 offering external validation.
• The DATO method was applied to appraise these studies.
• The best-performing AI models achieved error rates comparable to human performance, particularly ensemble models.

Conclusions:

• The DATO method offers a simple, systematic approach to applying evidence-based radiology (EBR) to AI.
• This framework is essential for radiologists and professionals engaging with AI research.
• Integrating DATO facilitates a more robust evaluation of AI in clinical radiology.