Impact of Different Study Populations on Reader Behavior and Performance Metrics: Initial Results

Area of Science:

• Medical Imaging Analysis
• Radiology Research
• Regulatory Science

Background:

• The FDA's Validation of Imaging Premarket Evaluation and Regulation (VIPER) study addressed design methodologies for medical imaging evaluations.
• Previous studies have not fully explored the impact of different patient populations on reader performance in mammography.

Purpose of the Study:

• To compare the impact of different study populations and prevalence levels on reader performance metrics (sensitivity, specificity, AUC) in mammography.
• To evaluate the performance of full-field digital mammography (FFDM) against screen-film mammography (SFM) in women with dense breasts.

Main Methods:

• The VIPER study involved five reader sub-studies using data from the Digital Mammographic Imaging Screening Trial (DMIST).
• Two non-cancer patient sampling methods (screening and challenge populations) and varying prevalence levels were investigated.
• Twenty radiologists participated in a split-plot study design, assessing recall decisions and ROC scores.

Main Results:

• Full-field digital mammography (FFDM) showed a moderate increase in sensitivity and decrease in specificity as prevalence rose to 50%, with largely stable AUC.
• Statistical efficiency of sensitivity and specificity relative to AUC was at best 0.66 and decreased with increasing prevalence.

Conclusions:

• Reader performance in mammography is influenced by prevalence and population sampling methods.
• Ongoing analyses will further compare FFDM and SFM, and screening versus challenge populations.