Epidemiology: reflections on testing the validity of psychiatric interviews

Area of Science:

• Psychiatry
• Clinical Psychology
• Psychometrics

Background:

• Reliable diagnosis of psychiatric disorders is challenging due to the absence of objective laboratory tests.
• Structured diagnostic interviews are widely used but require rigorous validation.
• The Diagnostic Interview Schedule (DIS) is one such interview tool whose validity has been previously assessed.

Purpose of the Study:

• To evaluate the methodological limitations of the double-blind test-retest design in assessing the validity of structured psychiatric diagnostic interviews.
• To identify shortcomings in using sensitivity, specificity, and concordance (K) statistics for interview validation.
• To explore alternative approaches for more accurate assessment of diagnostic interview validity.

Main Methods:

• The study critically examines the double-blind test-retest design comparing a structured interview (DIS) against a clinician's assessment.
• It analyzes the impact of factors such as respondent changes, clinical fluctuations, and the clinician's interview variability on validation statistics.
• The influence of disorder prevalence and severity on statistical measures like sensitivity, specificity, and K is discussed.

Main Results:

• The double-blind test-retest design is found to be inadequate for validating psychiatric interviews.
• Key statistics (sensitivity, specificity, K) are susceptible to confounding factors, including changes in the respondent and variability of the clinician's assessment.
• These statistics may not accurately reflect the interview's ability to estimate prevalence or identify disorder correlates.

Conclusions:

• The current methods for validating structured psychiatric interviews, particularly the double-blind test-retest design, are methodologically flawed.
• Alternative validation strategies are necessary to ensure the accuracy and reliability of psychiatric diagnostic tools.
• Improved validation methods are crucial for accurate psychiatric diagnosis and research.