Related Experiment Videos
Receiver operating characteristic (ROC) analysis for diagnostic examinations with uninterpretable cases
Ying Lu1, Daniel N Heller, Shoujun Zhao
1Department of Radiology, Box 1290, University of California, San Francisco, 94143-1290, USA. ying.lu@radiology.ucsf.edu
Statistics in Medicine
|July 12, 2002
Summary
This study introduces a modified Receiver Operating Characteristic (ROC) curve method to accurately assess diagnostic test performance when uninterpretable results are present. The new approach improves discriminatory power analysis for medical tests in real-world scenarios.
Area of Science:
- Biostatistics
- Medical Diagnostics
- Health Services Research
Background:
- Traditional Receiver Operating Characteristic (ROC) analysis assumes tests yield interpretable results, categorizing populations as 'normal' or 'abnormal'.
- Many diagnostic tests inherently produce uninterpretable results, complicating accurate performance evaluation.
- Existing ROC methods may not adequately account for non-informative uninterpretable results, potentially misrepresenting a test's discriminatory power.
Purpose of the Study:
- To develop a novel statistical method for evaluating diagnostic test discriminatory power in the presence of uninterpretable results.
- To introduce a mixed model modified ROC curve that accounts for non-informative uninterpretable test outcomes.
- To provide formulae for estimating the area under the modified ROC curve and compare it with conventional ROC analysis.
Main Methods:
- Development of a mixed model modified Receiver Operating Characteristic (ROC) curve.
- Derivation of formulae for estimating the area under the modified ROC curve.
- Mathematical comparisons and simulated experiments contrasting conventional and mixed model ROC analyses.
Main Results:
- The mixed model modified ROC curve provides a method to describe test discrimination when uninterpretable results occur.
- Formulae for estimating the area under the modified ROC curve are presented.
- Simulations and mathematical comparisons demonstrate the utility of the modified ROC approach.
Conclusions:
- The proposed mixed model modified ROC analysis offers a more accurate assessment of diagnostic test performance in the presence of uninterpretable results.
- This method enhances the ability to differentiate between populations when test outcomes are not definitively normal or abnormal.
- The approach is applicable to real-world diagnostic challenges, as illustrated by a spinal fracture diagnosis example.