[Effect of signal selection in receiver operating characteristics (ROC) analysis]

Selection of the signal is one of the major factors in a receiver operating characteristic (ROC) study for evaluating the diagnostic accuracy of medical imaging systems. We investigated the effect of signal selections in ROC studies which were applied to evaluate a difference between the two systems. Each of 50 positive and 50 negative images obtained with and without two different types of phantom signals, but with the same background noise and two different digital imaging systems, were used as case samples. We assumed that two different types of phantom signal could be related to image resolution and graininess, respectively. We employed two systems which were assumed to have advantages in resolution and graininess, respectively. Twelve observers participated in this ROC study, which aimed to compare two parameter settings in computed radiography systems. A statistical significance test considering case and reader variations was conducted for each of the ROC data sets with two different signals. As results, p values obtained in the statistical significance test were varied by changing the type of signal even if the same observer group participated and the same two different systems were compared (p value = 0.0003 and 0.0944). In conclusion, it was suggested that the selection of phantom signal for an ROC study could lead to different conclusions if the type of phantom signal was not matched to the purpose of the ROC study.