Half of Purposeful Baseline Sandbaggers Undetected by ImPACT's Embedded Invalidity Indicators

The main objectives of this study were to determine how accurately the embedded invalidity indicators (EIIs) identify purposeful underperformers on the baseline Immediate Post-concussion Assessment and Cognitive Test (ImPACT); and to assess the effectiveness of each individual EII.

A randomized controlled trial was conducted in which all participants completed a baseline ImPACT assessment. Participants were randomized into a control or purposeful underperformance (sandbagging) group. The primary outcomes measured were the number of participants identified as invalid (via any EII), as well as the ability of each individual EII to detect purposeful sandbagging. Additionally, participants mean raw composite scores and percentiles were evaluated.

Seventy-seven participants completed the study (control n = 37, sandbag n = 40.) None of the participants in the control group, and 50% of the purposeful sandbaggers were identified as invalid via the current EIIs. Of the five EIIs, three were unable to identify more than 15% of purposeful sandbaggers. The best performing EIIs were Word Memory and Three Letters, identifying 40% and 35% of purposeful sandbaggers, respectively. Sixty- five percent of the purposeful sandbaggers had at least one composite score ≤1st percentile. Using a composite score ≤1st percentile as potential marker of invalidity would have accurately identified more purposeful sandbaggers than all existing EIIs combined.

Half of purposeful sandbaggers were not identified by ImPACT's current EIIs. Multiple EIIs were only able to identify <15% of purposeful underperformers, suggesting that reevaluation and/or recalibration of EII cutoffs may be appropriate.