Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology

Area of Science:

• Radiology
• Medical Imaging Analysis
• Diagnostic Accuracy

Background:

• The satisfaction of search (SOS) effect describes a phenomenon in diagnostic imaging.
• Previous studies suggested the SOS effect in chest radiology, but its statistical validation required further investigation.

Purpose of the Study:

• To validate the satisfaction of search (SOS) effect in chest radiology.
• To assess the utility of proper receiver operating characteristic (ROC) curves and joint detection and localization ROC curves for demonstrating the SOS effect.

Main Methods:

• Analysis of existing ROC data from a chest radiology study on SOS.
• Application of three proper ROC models and one joint detection and localization ROC model.
• Replication of the study with a new reader sample and identical analytical methods.

Main Results:

• The contaminated binormal model provided a superior fit to the data compared to other ROC models.
• Both the original and replication experiments demonstrated the SOS effect, evidenced by a reduced area under the ROC curve when nodules were added.
• Joint ROC analysis, avoiding inappropriate chance line crossing, yielded consistent results.

Conclusions:

• The satisfaction of search (SOS) effect in chest radiology is demonstrable using appropriate ROC methodologies.
• Employing ROC methods that prevent inappropriate chance line crossing enhances measurement accuracy and statistical test power.
• These findings support the use of refined ROC techniques for robust analysis in diagnostic radiology.