Test engagement in first-year university students seeking course credits

This preregistered study examined test engagement among Dutch university students participating in psychological research in exchange for course credit. We hypothesized that at least 10% of students would demonstrate invalid performance, that engagement would decline over the course of the assessment, and that the commonly used self-rated test engagement question administered at the end of the assessment would show only limited utility.

A total of 222 first-year psychology students completed a 45-minute assessment battery and were instructed to maximize their performance on five performance validity tests (PVTs). Test order was counterbalanced across participants to examine the effects of test position within the assessment. In addition, self-rated test engagement was associated with objective task performance.

The results confirmed our first hypothesis, demonstrating that 11.3% of students showed strong indications of underperformance. Evidence of invalid performance was observed primarily on timed attention tasks, whereas memory-based measures yielded unremarkable findings. Effects of test order within the 45-minute battery were observed only for attention-based validity test performance. Self-rated test engagement scores were clustered near the ceiling and showed only weak associations with objective validity test performance.

Researchers should exercise caution when interpreting data from university students participating in research for course credit. Including PVTs may improve the quality of assessment data. However, self-ratings of test engagement do not appear to be a valid method for identifying or excluding invalid student data. Future research should further investigate the potential effects of boredom and declining test engagement during longer cognitive assessment protocols.