Diagnosing tests: using and misusing diagnostic and screening tests

Area of Science:

• Psychometrics
• Medical Diagnostics
• Biostatistics

Background:

• Diagnostic tests serve dual roles: confirming suspected conditions and screening asymptomatic populations.
• A test's utility can drastically change based on its application, with diagnostic accuracy potentially differing from screening effectiveness.
• Prevalence significantly impacts false positive and false negative rates, complicating test interpretation in low-prevalence screening scenarios.

Purpose of the Study:

• To elucidate how the prevalence (base rate) of a disorder influences diagnostic test accuracy.
• To review various diagnostic efficiency statistics derived from 2x2 tables.
• To extend the Standards for Reporting of Diagnostic Accuracy guidelines for psychological tests.

Main Methods:

• Analysis of diagnostic test performance metrics in relation to disease prevalence.
• Review of statistical measures including overall correct classification rate, kappa, phi, odds ratio, predictive values, and likelihood ratios.
• Examination and adaptation of the Standards for Reporting of Diagnostic Accuracy (SRDA) guidelines.

Main Results:

• Test accuracy is highly sensitive to the prevalence of the condition being tested.
• Different statistical measures provide varied insights into diagnostic efficiency.
• Existing reporting standards for diagnostic accuracy require adaptation for psychological assessments.

Conclusions:

• The prevalence of a condition is a critical factor in interpreting diagnostic test results.
• A comprehensive understanding of diagnostic efficiency statistics is essential for appropriate test application.
• Standardized reporting of diagnostic accuracy in psychology is necessary for reliable and valid assessments.