Attention Promises Fail to Improve Accuracy on Online Attention Checks in College Sample: A Test Across IRI and IMC...

This study tested whether a brief pre-survey "attention promise" improves attention-check performance and whether accuracy differs between Instructed Response Items (IRIs) and an Instructional Manipulation Check (IMC). Undergraduate participants (N = 249) were randomly assigned to an attention-promise or control condition and completed two IRIs and one IMC embedded within personality, self-control, and attention measures. Accuracy was re-expressed as proportion correct (0-1). A mixed ANOVA (Check Type: IRI vs. IMC; Attention Promise: present vs. absent) showed a large main effect of check type (IRI > IMC), with no main effect of the attention promise and no interaction. A trial-level mixed-effects logistic regression with a random intercept for participants converged on the same pattern (substantially lower odds of passing the IMC; no effects of the promise). As an external validity check, internal consistency (Cronbach's α) for the multi-item scales was generally higher among participants who passed all attention checks than among those who failed at least one, suggesting that attention-check performance tracks broader response quality. Overall, accuracy was substantially lower on the IMC than on the IRIs, and a simple pre-survey commitment did not meaningfully improve attentiveness in this undergraduate sample.