Within-Person Score Distribution Measures as Performance Validity Indicators in Service Members and Veterans with and...

Area of Science:

• Neuropsychology
• Cognitive Assessment
• Performance Validity Testing

Background:

• Performance validity tests (PVTs) are crucial for identifying invalid cognitive test performance.
• Within-person score distribution measures (WPSD) offer a potential method for embedded PVTs.
• Previous research has explored various WPSD metrics, but their comparative accuracy needs further investigation.

Purpose of the Study:

• To evaluate the classification accuracy of different WPSD measures as performance validity indicators.
• To compare the diagnostic utility of the overall test battery mean (OTBM), coefficient of variation (CV), number of abnormal scores (ABN), and other WPSD metrics.
• To assess the potential of WPSD measures as embedded PVTs in cognitive test batteries.

Main Methods:

• Secondary analysis of data from the Chronic Effects of Neurotrauma Consortium Study 1 (N=1431).
• Participants were classified into valid and questionable performance validity groups using established PVTs.
• Seven WPSD measures (OTBM, CV, SD-B, kurtosis, skew, range, ABN) were calculated from T scores of 24 cognitive tests; receiver operating characteristic (ROC) curve analysis was used to compare classification accuracy (AUC).

Main Results:

• Significant group differences were found for OTBM, CV, and ABN.
• Excellent classification accuracy was observed for OTBM (AUC=0.83) and ABN (AUC=0.84), with acceptable accuracy for CV (AUC=0.74).
• Minimum specificity (≥90%) was achieved with specific cutoffs for OTBM, CV, and ABN, though minor adjustments were needed for diverse sample subsets.

Conclusions:

• OTBM and ABN show strong potential as embedded PVTs due to their excellent classification accuracy.
• CV demonstrated lower classification accuracy, primarily reflecting changes in mean performance rather than score dispersion.
• Further research and cross-validation of WPSD-based PVTs are recommended, particularly in civilian populations.