Approaches to Sampling for Quality Control of Artificial Intelligence in Biomedical Research

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Healthcare
• Statistical Quality Control

Background:

• Periodic quality control of artificial intelligence (AI) results is crucial in biomedical practice.
• Various sampling approaches exist, each with different statistical underpinnings and applicability.
• The integration of AI in healthcare necessitates robust methods for validating AI-generated data.

Purpose of the Study:

• To evaluate the efficacy of different sampling approaches for periodic quality control of AI results in biomedical settings.
• To determine the optimal sampling strategy for AI-driven medical image analysis within a healthcare system.

Main Methods:

• Analysis of sampling methods including point statistical estimation, hypothesis testing, statistical tables, and GOST R ISO 2859-1-2007.
• Consideration of sample sizes for populations ranging from 1000 to 100,000 medical studies.
• Experimentation within the Moscow healthcare system using AI for medical image analysis.

Main Results:

• Point statistical estimation is suitable when only Type I errors are critical.
• Statistical hypothesis testing allows for the consideration of both Type I and Type II errors.
• An optimal sample size of 80 studies was identified for AI quality control in medical image analysis, balancing representativeness, risk, and cost.

Conclusions:

• The optimal sample size of 80 studies ensures representativeness and efficient quality control for AI in medical image analysis.
• The choice of sampling method depends on the specific statistical parameters and error tolerance required.
• Effective sampling strategies are essential for the reliable implementation of AI in healthcare.