On Estimating Diagnostic Accuracy From Studies With Multiple Raters and Partial Gold Standard Evaluation

Area of Science:

• Biostatistics
• Medical Diagnostics
• Health Services Research

Background:

• Estimating diagnostic test accuracy (sensitivity, specificity) is crucial but often hindered by expensive or invasive gold standard evaluations.
• Latent modeling approaches exist for estimating diagnostic error without a gold standard, but are susceptible to bias from misspecified dependence structures between tests.
• Choosing between complex latent models is practically difficult, despite the persistent challenge of obtaining gold standard verification.

Purpose of the Study:

• To develop a robust statistical approach for estimating diagnostic accuracy when gold standard verification is only available for a subset of subjects.
• To provide a compromise between fully utilizing latent models and the practical constraints of gold standard testing.
• To improve the reliability and ease of model selection in diagnostic accuracy studies.

Main Methods:

• Extension of two classes of latent models to incorporate partial gold standard information.
• Estimation methods that utilize data from both verified and non-verified subjects.
• Analysis of robustness and model selection under different verification scenarios (random vs. dependent on test results).

Main Results:

• The proposed method allows for more reliable estimation of diagnostic error by incorporating partial gold standard data.
• Model selection between competing approaches is facilitated compared to methods relying solely on latent modeling.
• The approach is robust to different verification schemes, including those dependent on actual test results.

Conclusions:

• Partial gold standard data can be effectively integrated into statistical models to improve diagnostic accuracy estimation.
• This methodology offers a practical solution for studies facing limitations in gold standard verification.
• The approach was successfully applied to estimate the diagnostic error of digital radiography for gastric cancer.