Performance assessment for radiologists interpreting screening mammography | JoVE Visualize

Area of Science:

Medical Imaging and Diagnostics
Biostatistics and Health Informatics
Radiology and Oncologic Imaging

Background:

Radiologist interpretation of screening mammograms significantly impacts patient recall decisions for further testing.
Existing performance assessments show wide variations in radiologist false-positive and false-negative rates, with concerns about current assessment adequacy.
U.S. government mandates performance assessments for individual radiologists, highlighting the need for robust evaluation techniques.

Purpose of the Study:

To develop and demonstrate a hierarchical modeling technique for inferring individual radiologist interpretive performance in screening mammography.
To account for patient mix and radiologist attributes (experience, volume) in performance assessments.
To provide novel performance metrics, including sensitivity, specificity, and performance curves, with interval estimates for uncertainty.

Main Methods:

Utilized Bayesian hierarchical models to analyze mammogram-level data from registries and radiologist surveys.
Modeled false-positive and false-negative rates separately using logistic regression, incorporating patient risk factors and radiologist random effects.
Regressed radiologist random effects on attributes like years in practice to assess individual performance against a standard and predict performance on hypothetical cases.

Main Results:

Developed methods to assess radiologist performance by comparing individual rates to a hypothetical standard, considering patient mix and radiologist attributes.
Introduced metrics to predict radiologist performance on different patient risk profiles ('typical', 'high-risk', 'low-risk').
Demonstrated the ability to estimate multiple performance rates (sensitivity, specificity) and performance curves, and infer performance for new radiologists without refitting models.

Conclusions:

Hierarchical modeling offers a robust framework for accurate and nuanced assessment of radiologist performance in screening mammography.
The proposed methods provide valuable metrics for comparing radiologists and understanding performance variations.
The approach allows for flexible performance inference, including for new radiologists and varying case mixes, enhancing quality assurance in breast cancer screening.