Impaired or invalid? Limitations of assessing performance validity using the Boston Naming Test

The Boston Naming Test (BNT) has been proposed as an embedded performance validity test (PVT), though replication is needed to provide further empirical support of its simultaneous use as a cognitive ability measure and embedded PVT. This cross-sectional study examined BNT performance in a mixed neuropsychiatric sample of 137 patients with/without cognitive impairment. Four independent criterion PVTs classified 109 (80%) as valid and 28 (20%) as invalid. BNT raw and demographically-corrected T-scores were significantly higher among the valid group with small effect sizes (η = 0.04-0.05). Raw/T-scores differentiated valid/invalid groups, but with low classification accuracy (areas under the curve [AUCs] = 0.68/0.63), and unacceptably weak sensitivities (i.e. 7%/18%). When separated by impairment status, raw score accuracy appreciably increased (AUC = 0.87; 61% sensitivity/89% specificity) among unimpaired patients, whereas T-score accuracy, while significant, remained low (AUC = 0.68; 21% sensitivity/89% specificity). Conversely, among impaired patients, neither the raw (AUC = 0.59) nor T-score (AUC = 0.60) accurately identified invalid performance. In sum, BNT scores were not able to differentiate valid from invalid performance when cognitive impairment was present, and therefore showed limited overall utility as embedded PVTs. These findings further caution against inferring performance validity from measures in which a single score is used to assess both cognitive ability and validity.