Data-Driven Cutoff Selection for the Patient Health Questionnaire-9 Depression Screening Tool

Area of Science:

• Psychometrics
• Medical Informatics
• Biostatistics

Background:

• Test accuracy studies often use small datasets for simultaneous optimal cutoff score selection and accuracy estimation.
• This approach can lead to deviations from population-level optimal scores and biased accuracy estimates.

Purpose of the Study:

• To evaluate how data-driven methods for simultaneously selecting an optimal Patient Health Questionnaire-9 (PHQ-9) cutoff score and estimating accuracy affect results.
• Specifically, to assess differences in optimal cutoff scores and accuracy estimates compared to population-level values.

Main Methods:

• Utilized cross-sectional data from an individual participant data meta-analysis (IPDMA) database for PHQ-9 screening accuracy.
• Resampled 1000 studies (N=100, 200, 500, 1000) from the IPDMA population to simulate smaller datasets.
• Selected optimal cutoff scores using the Youden index and compared accuracy estimates with the full population data.

Main Results:

• Optimal cutoff scores in simulated studies varied significantly (e.g., 2+ to 21+ in N=100 samples) compared to the population optimum (8+).
• The true optimal cutoff score (8+) was identified in only 17% of N=100 samples and 33% of N=1000 samples.
• Sensitivity was overestimated in smaller samples (e.g., 6.4 percentage points in N=100), while specificity remained largely unaffected.

Conclusions:

• Simultaneously selecting optimal cutoff scores and estimating accuracy using data-driven methods on small datasets leads to substantial differences from population values.
• Diagnostic accuracy evidence derived from inadequately powered studies or meta-analyses should be interpreted with caution.
• Recommendations for cutoff scores require robust evidence from adequately powered research or well-conducted meta-analyses.