Related Experiment Video
Updated: Oct 5, 2025

Author Spotlight: Advancing CBCT and Digital Dental Image Integration with AI-Assisted Digitization
Published on: February 23, 2024
Quality assessment standards in artificial intelligence diagnostic accuracy systematic reviews: a meta-research study
Shruti Jayakumar1, Viknesh Sounderajah1,2, Pasha Normahani1,2
1Department of Surgery and Cancer, Imperial College London, London, UK.
Quality assessment of artificial intelligence (AI) diagnostic accuracy studies is inconsistent. Many reviews fail to use quality assessment tools, hindering safe clinical implementation of AI healthcare solutions.
Area of Science:
- Medical Informatics
- Health Services Research
- Artificial Intelligence in Healthcare
Background:
- Artificial intelligence (AI) diagnostic systems are increasingly integrated into healthcare.
- Secondary research studies on AI diagnostic accuracy are vital for clinical and policy decisions.
- Accurate appraisal of methodological quality and risk of bias in these studies is essential.
Purpose of the Study:
- To evaluate the adherence to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool in systematic reviews of AI diagnostic accuracy studies.
- To identify the extent and nature of bias reporting in these reviews.
Main Methods:
- A meta-research study was conducted, analyzing systematic reviews of AI diagnostic accuracy published between 2000 and December 2020.
- Fifty systematic reviews were included, with a focus on those utilizing the QUADAS-2 tool for quality assessment.
Main Results:
- Of 50 reviews, 36 performed quality assessment, and 27 used the QUADAS-2 tool.
- Significant bias was reported across QUADAS-2 domains: patient selection (57.5%), index test (26%), reference standard (28.6%), and flow and timing (37.1%).
- Incomplete uptake and inconsistent reporting of quality assessment were observed.
Conclusions:
- There is a need for improved adherence to quality assessment tools in reviews of AI diagnostic accuracy studies.
- Inconsistent reporting standards act as barriers to the clinical implementation of AI tools.
- Developing an AI-specific extension for quality assessment tools could aid the safe translation of AI into clinical practice.
Related Concept Videos
Quality Assurance
Measures of Intelligence
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this;...
Documentation of Nursing Diagnosis
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
Sensitivity, Specificity, and Predicted Value
Sensitivity is the...
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...

