Validation of Diagnostic Groups Based on Health Care Utilization Data Should Adjust for Sampling Strategy

Area of Science:

• Health Informatics
• Biostatistics
• Population Health Surveillance

Background:

• Validating health outcomes from healthcare utilization data is crucial for learning health systems.
• Existing validation methods often lack awareness of stratified sampling needs for complex diagnostic groups.
• Low-prevalence diagnostic codes present unique challenges in accurate measurement.

Purpose of the Study:

• To propose a novel statistical method for validating diagnostic group measurements.
• To address challenges posed by varying prevalences and low prevalence of diagnostic codes.
• To enhance the accuracy of outcome measurement in healthcare data.

Main Methods:

• Oversampling low-prevalence diagnostic codes to estimate the positive predictive value (PPV) as a weighted average.
• Oversampling claims within low-prevalence diagnostic groups for bias-adjusted sensitivity and specificity estimation.
• Utilizing a stratified sampling design and corresponding statistical methods.

Main Results:

• The proposed method improves the accuracy of positive predictive value (PPV) estimation for diagnostic groups.
• Bias-adjusted estimators for sensitivity and specificity are generated, accounting for oversampling.
• Illustrative example demonstrates application in acute respiratory illness surveillance.

Conclusions:

• Failure to account for diagnostic code prevalence underestimates PPV due to false positives in low-prevalence codes.
• Improper adjustment for oversampling leads to overestimated sensitivity and underestimated specificity.
• The proposed method offers a robust approach to validating complex diagnostic groups in healthcare data.