Related Experiment Video
Updated: Feb 10, 2026

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
Published on: December 15, 2023
AI-RADS: A Framework for Assessment of Artificial Intelligence Output in Radiology-Development and Multireader
Background:
Despite the growing number of artificial intelligence (AI)-based applications used in radiology, no structured framework exists to assess their case-level reliability or to document overridden outputs in reports.
Purpose:
To develop and evaluate the Artificial Intelligence Reporting and Data System (AI-RADS), a structured framework for an objective, case-level assessment of AI output reliability, clinical utility, and recommended actions in radiology.
Materials And Methods:
The AI-RADS framework was tested in a retrospective, multireader study. Here, 5 board-certified radiologists independently evaluated 350 cases processed by 7 representative AI applications for image-based and generative tasks. Each case was assigned one of 5 AI-RADS categories, applicable modifiers, and an independent correctness rating as a reference. Interreader agreement was quantified using Krippendorff's α with 95% CIs.
Results:
Substantial interreader agreement was observed for the core AI-RADS categories in both image-based (Krippendorff's α=0.87; 95% CI: 0.83-0.91) and generative AI tasks (Krippendorff's α=0.93; 95% CI: 0.91-0.95). Reader-assigned correctness aligned well with AI-RADS categories 1 to 2, which indicate outputs suitable for integration into clinical workflows. Outputs rated as "incorrect" were predominantly assigned to categories 4 to 5, warranting override or removal from display.
Conclusion:
AI-RADS provides a structured framework for the case-level evaluation of AI output reliability, clinical utility, and consequences for report communication. This multireader study demonstrated substantial interreader agreement and applicability across various AI applications.
Related Concept Videos
Cardiac Output II: Effect of Stroke Volume on Cardiac Output
Preload
Preload refers to the initial elongation of the cardiac myocytes before contraction and is related to the volume of blood filling the heart at the end of diastole, or end-diastolic volume. The...
Intelligence
Cardiac Output I:Effect of Heart Rate on Cardiac Output
Cardiac output (CO) refers to the total amount of blood ejected by one of the ventricles in liters per minute (L/min). In a resting adult, CO ranges from 5 to 6 L/min, adjusting according to the body's metabolic requirements.
Effect of Heart Rate on Cardiac Output
Cardiac output adapts to metabolic demands during stress, physical activity, or illness. The autonomic nervous system regulates heart rate via the sinoatrial node. The parasympathetic nervous system decreases heart...
Radiological Investigation I: X-ray and CT
Exercise and Cardiac Output
Sustained exercise increases the muscles' oxygen demand, which can be...
Imbalances in Cardiac Output
CHF can occur due to the failure of either side of the heart. Left-side failure leads to pulmonary congestion—the right side continues to send...

