Reply to commentary by Bilder, Sugar, and Helleman (2014 this issue) on minimizing false positive error with multiple...

Area of Science:

• Neuropsychology
• Psychometrics
• Clinical Psychology

Background:

• Recent publications on performance validity test (PVT) failure in clinical samples have faced criticism.
• The criticisms by Bilder and colleagues present an idiosyncratic interpretation of research and inconsistent application of statistical principles.

Purpose of the Study:

• To address and discuss problematic aspects of recent criticisms regarding performance validity tests.
• To evaluate the proposed higher psychometric standards for PVTs compared to conventional neuropsychological tests.

Main Methods:

• Critical analysis of existing research and criticisms.
• Examination of the application of null hypothesis significance testing principles.
• Discussion of the psychometric standards applied to performance validity tests.

Main Results:

• The criticisms by Bilder and colleagues demonstrate an idiosyncratic interpretation of research.
• Principles of null hypothesis significance testing appear inconsistently applied in the criticisms.
• The proposed psychometric standards for PVTs are argued to be disproportionately high compared to conventional neuropsychological tests.

Conclusions:

• The criticisms of performance validity tests by Bilder and colleagues are based on flawed interpretations and inconsistent methodologies.
• PVTs should not be held to a higher psychometric standard than other neuropsychological tests.
• Further consideration of research aims and findings is necessary to accurately evaluate PVTs.