Certain performance values arising from mammographic test set readings correlate well with clinical audit

Area of Science:

• Radiology and Medical Imaging
• Biostatistics
• Public Health

Background:

• Breast screening programs increasingly use test sets to supplement clinical audits.
• The precise relationship between mammographic test set performance and clinical audit outcomes has remained unclear.
• This study aimed to clarify the association between mammographic test set performance and clinical audit data.

Purpose of the Study:

• To investigate the correlation between mammographic test set performance metrics and clinical audit data for radiologists.
• To determine if test set performance can serve as a reliable indicator of clinical performance in breast cancer screening.

Main Methods:

• Twenty radiologists' clinical audit data over two years was analyzed.
• A test set of 60 mammographic examinations (40 normal, 20 cancer cases) was used.
• Performance metrics including Jackknifing free-response operating characteristic (JAFROC) figure-of-merit (FOM), location sensitivity, and specificity were calculated and compared to audit data using Spearman's rho.

Main Results:

• JAFROC FOM significantly correlated with recall rate, detection of small invasive cancers, missed cancers, and overall sensitivity.
• Location sensitivity showed significant correlations with the detection rates of small invasive cancers, ductal carcinoma in situ (DCIS), and overall cancer detection.
• Several strong correlations were observed between test set performance indicators and key clinical audit metrics.

Conclusions:

• Performance indicators derived from mammographic test sets show significant correlations with specific aspects of clinical performance in breast cancer screening.
• While test sets offer valuable insights, caution is advised when generalizing test set specificity to real-world clinical scenarios.
• The study supports the utility of test sets in evaluating radiologist performance but highlights the need for careful interpretation.