Related Experiment Videos
A comparison of two data analyses from two observer performance studies using Jackknife ROC and JAFROC.
Bin Zheng1, Dev P Chakraborty, Howard E Rockette
1Department of Radiology, University of Pittsburgh, 300 Halket Street, Suite 4200, Pittsburgh, Pennsylvania 15213-3180, USA. zhengb@msx.upmc.edu
Medical Physics
|May 18, 2005
Summary
This study compared Jackknife ROC and JAFROC methods for analyzing free-response receiver operating characteristics (FROC) data. JAFROC analysis often yielded lower p-values, indicating significant differences more frequently than Jackknife ROC analysis.
Area of Science:
- Medical Imaging Analysis
- Statistical Methods in Radiology
Background:
- Observer performance studies are crucial for evaluating medical imaging tools.
- Free-response receiver operating characteristics (FROC) analysis is a common method for such studies.
- Comparing analytical approaches like Jackknife ROC and JAFROC is essential for accurate interpretation.
Purpose of the Study:
- To compare the performance of Jackknife ROC and JAFROC methodologies in analyzing FROC data.
- To assess differences in statistical significance and p-values generated by each method.
Main Methods:
- Two observer performance studies involving radiologists interpreting mammograms with and without computer-aided detection (CAD) were analyzed.
- Data were analyzed using Jackknife ROC (LABMRMC software) and JAFROC methods.
- Statistical significance (p-values) for overall modes and paired data sets were computed and compared.
Main Results:
- Overall results were significant in the first study (p < 0.05) but not the second (p > 0.05) for both methods.
- In the first study, JAFROC found more significant paired differences (4 pairs) than LABMRMC (3 pairs).
- JAFROC consistently produced lower p-values than LABMRMC across paired comparisons in both studies.
Conclusions:
- Both Jackknife ROC and JAFROC can detect significant differences in observer performance.
- JAFROC appears more sensitive in detecting differences, as indicated by lower p-values in paired comparisons.
- The choice of analytical method may influence the interpretation of observer performance in FROC studies.