Beyond the overt response: Reaction time modeling in self-report surveys across administration modes and item formats

Reaction times (RTs) offer valuable insights into both latent traits and underlying cognitive processes. Yet, they remain underutilized in social psychological measurement due to their statistical distributions and sensitivity to varied survey contexts. In a preregistered 2 × 2 experiment (N = 268 Czech students), we examined the robustness of reaction time modeling (RTM), combining a shifted-lognormal RT component with a graded response model for Likert data, across two common methodological variations: administration mode (online vs. laboratory) and item visualization (single- vs. multiple-item format). Equivalence testing showed that latent trait estimates (θₚ) were practically invariant across all conditions, indicating that RTM is resilient to survey design and platform differences. Complementary eye-tracking and heart rate data revealed that multiple-item formats reduced perceptual and physiological demands, while single-item presentations elicited more intensive visual and pre-decisional processing. These findings demonstrate that RTM provides robust, psychologically meaningful estimates of both latent traits and underlying cognitive processes, supporting flexible survey implementation across contexts. The study also introduces open-source software and modeling tools, lowering barriers for researchers to incorporate reaction time modeling in both laboratory and online environments.