False-positive rates associated with the use of multiple performance and symptom validity tests

Area of Science:

• Neuropsychology
• Clinical Psychology
• Forensic Psychology

Background:

• Performance validity tests (PVTs) and symptom validity tests (SVTs) are crucial for detecting invalid performance in clinical assessments.
• Monte Carlo simulations are often used to estimate error rates for PVTs and SVTs.
• Previous research has raised concerns about the accuracy of Monte Carlo simulations in predicting real-world PVT failure rates.

Purpose of the Study:

• To compare error rates from Monte Carlo simulations with actual PVT and SVT failure rates in non-malingering clinical samples.
• To evaluate the accuracy of Monte Carlo simulations in estimating the overestimation of PVT/SVT failures.
• To provide evidence supporting the clinical utility of using multiple PVT/SVT failures to identify invalid presentations.

Main Methods:

• Comparison of Monte Carlo simulation data with failure rates from two non-malingering clinical samples.
• Analysis of PVT and SVT failure rates using established criteria (e.g., ≥2 failures out of 5 or 7 tests).
• Examination of the distributional properties of PVT scores in clinical populations versus simulation assumptions.

Main Results:

• Monte Carlo simulations overestimated PVT/SVT error rates when using ≥2 failures out of 5 or 7 tests.
• Overestimation occurred across different test combinations and simulation parameters (e.g., 10% false-positive rate).
• PVT scores in clinical samples exhibit skewed distributions, deviating from the standard normal distribution assumed in simulations.

Conclusions:

• The findings support the use of ≥2 PVT/SVT failures as a reliable indicator of a probable invalid clinical presentation.
• Monte Carlo simulations may not accurately reflect real-world PVT performance due to atypical score distributions in clinical populations.
• Clinical practice should rely on empirical data from clinical samples rather than solely on simulation-derived error rates for interpreting PVT/SVT results.