How independent are multiple 'independent' diagnostic classifications?

Area of Science:

• Biostatistics
• Epidemiology
• Medical Diagnostics

Background:

• Multiple diagnostic tests are common in clinical practice and research for reliability assessment, sensitivity/specificity enhancement, and bias correction.
• A key assumption is the independence of test results conditional on the true value.
• This independence assumption is often violated for dichotomous tests due to underlying trait continua and proximity to diagnostic thresholds.

Purpose of the Study:

• To assess the magnitude of correlated diagnostic errors.
• To evaluate the influence of underlying trait distribution, measurement error, and diagnostic threshold on error correlation.
• To determine the impact of correlated errors on reliability indices.

Main Methods:

• Mathematical assessment of diagnostic error correlation.
• Analysis as a function of underlying trait distribution, measurement error, and diagnostic threshold.
• Numerical examples to illustrate findings.

Main Results:

• Diagnostic test errors can be strongly correlated even when perception errors are independent.
• The correlation of errors depends on trait distribution, measurement error, and the diagnostic threshold.
• Positive correlation of diagnostic errors can substantially inflate reliability indices like the kappa coefficient.

Conclusions:

• The assumption of independence of diagnostic test results is often not tenable.
• Correlated diagnostic errors can lead to an overestimation of test reliability.
• Findings highlight the need to consider error correlation in diagnostic test evaluation.