From "below chance" to "a single error is one too many": Evaluating various thresholds for invalid performance on two...

Area of Science:

• Neuropsychology
• Psychometrics

Background:

• Performance validity tests (PVTs) are crucial for detecting invalid response styles in cognitive assessments.
• Various definitions of invalid performance exist, necessitating empirical evaluation of their accuracy.

Purpose of the Study:

• To empirically evaluate the classification accuracy of different definitions of invalid performance using two common PVTs.
• To compare binomial theory-based thresholds with empirical data for identifying non-credible responding.

Main Methods:

• The study analyzed data from 470 participants across two mixed clinical samples from the US and Canada.
• Two PVTs, the FCR_CVLT-II and Test of Memory Malingering (TOMM-2), were used.
• Proportions of chance-level responding and error rates were computed and compared against criterion PVTs.

Main Results:

• There was minimal overlap between binomial and empirical distributions for invalid performance.
• Over 95% of participants who passed all PVTs achieved perfect scores.
• Chance-level responding was predominantly observed in individuals failing two or more PVTs (91% failed three).
• No participants scored below chance on FCR_CVLT-II or TOMM-2; all dementia patients scored above chance.
• A single error on FCR_CVLT-II or TOMM-2 demonstrated high specificity (0.95) for psychometrically defined invalid performance.

Conclusions:

• Scores at or below chance level on PVTs provide strong evidence of non-credible responding.
• Scores above chance level on PVTs have no negative predictive value for invalid performance.
• Defining non-credible responding solely by below-chance scores is overly restrictive and may lead to false passes.