Related Experiment Video
Updated: Jun 12, 2025
15:00
A Tablet-Based Curriculum-Based Measurement Protocol for Kindergarten Writing
Published on: February 7, 2025
527
Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies Using AI (QUADAS-AI): Protocol for a
Ahmad Guni1, Viknesh Sounderajah1, Penny Whiting2
1Institute of Global Health Innovation, Imperial College London, London, United Kingdom.
JMIR Research Protocols
|September 18, 2024
Summary
A new tool, QUADAS-AI, is being developed to assess the quality of artificial intelligence (AI) diagnostic accuracy studies. This AI-specific quality assessment tool will help evaluate bias and applicability in AI diagnostic systems.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
- Diagnostic Accuracy Research
Background:
- Existing quality assessment tools like QUADAS-2 do not adequately address AI-specific diagnostic systems.
- The rapid advancement of AI in diagnostics necessitates specialized tools for quality evaluation.
- Assessing bias and applicability is crucial for the clinical translation of AI diagnostic tools.
Purpose of the Study:
- To develop QUADAS-AI, a novel tool tailored for appraising the methodological quality of AI diagnostic accuracy studies.
- To outline the systematic process for creating the QUADAS-AI tool.
Main Methods:
- A multi-stage development process involving project organization, item generation, and piloting.
- Item generation includes mapping reviews, meta-research, expert surveys, and patient/public engagement.
- Consensus will be achieved through a modified Delphi methodology with international experts.
Main Results:
- Project organization, mapping review, and meta-research are complete as of July 2024.
- Item generation and Delphi consensus are planned for completion by the end of 2024.
- The finalized QUADAS-AI tool is anticipated by early 2025.
Conclusions:
- QUADAS-AI will provide a standardized framework for evaluating AI diagnostic accuracy studies.
- The tool aims to identify potential bias and applicability issues in AI diagnostic research.
- QUADAS-AI is expected to support clinical, governmental, and regulatory evaluation of AI diagnostic systems worldwide.
Keywords:
AIAI-drivenAI-specific quality assessment of diagnostic accuracy studiesQUADAS-AIaccuracyartificial intelligenceassessment toolbiasclinical practicediagnosticdiagnostic servicediagnosticsevaluationevidence synthesisquality assessmenttranslationMore Related Videos
Related Concept Videos
01:19Quality Assurance
117
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
117
01:15Receiver Operating Characteristic Plot
95
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
95