Human-AI Interaction in the ScreenTrustCAD Trial: Recall Proportion and Positive Predictive Value Related to...

Area of Science:

• Radiology
• Artificial Intelligence
• Medical Imaging

Background:

• The ScreenTrustCAD trial investigated cancer detection rates using artificial intelligence (AI) computer-aided detection (CAD) combined with radiologists.
• Concerns arose regarding radiologists' agreement with AI CAD, either excessively or insufficiently.

Purpose of the Study:

• To compare recall proportions and positive predictive values (PPV) based on whether AI CAD or radiologists flagged mammograms for consensus discussion.
• To analyze the impact of different reader combinations (AI CAD and/or radiologists) on screening outcomes.

Main Methods:

• Prospective study (April 2021-June 2022) involving 54,991 women.
• Each mammogram was interpreted by two radiologists and AI CAD.
• Recall proportion and PPV were calculated for examinations flagged by single or multiple readers.

Main Results:

• Mammograms initially flagged by radiologists had higher recall proportions (14.2%-57.2%) but lower positive predictive values (PPV) (2.5%-3.4%) compared to those flagged by AI CAD (4.6%-38.6% recall, 22%-25% PPV).
• When all three readers flagged an examination, recall proportion was 82.6% with a PPV of 34.2%.

Conclusions:

• Radiologists' initial flagging led to a greater proportion of women being recalled for further assessment compared to AI CAD flagging.
• Mammograms flagged by AI CAD demonstrated a higher proportion of cancer detection (higher PPV) than those flagged by radiologists alone.