Analytic flexibility in laboratory aggression paradigms: Relations with personality traits vary (slightly) by...

Competitive reaction time tasks (CRTTs) have been used widely in social science research, but recent criticism has been directed at the flexible quantification strategies used with this methodology. A recent review suggests that over 150 different quantification strategies have been used in this literature, and there is evidence to suggest that different operationalizations can affect the results and interpretations of experiments using CRTTs. In the current investigation, we reanalyze data from four existing samples from two different sites (total N = 600) to examine how the relations between a range of personality traits and aggression vary based on how aggression is operationalized. Our results suggest that there is a modest degree of heterogeneity in effect size and direction for these relations, and that effect size and direction were most consistent for traits more generally related to lab aggression (e.g., psychopathy, low Five-Factor Model agreeableness). In addition, profile matching analyses suggest that different operationalizations yield empirical correlates that are quite similar to one another, even when quantifying absolute rather than relative similarity. These results were consistent across site, methodology, and type of sample, suggesting that these issues are likely generalizable across most labs using CRTTs. We conclude with suggestions for future directions, particularly emphasizing the need for adequately-powered samples, and for researchers to preregister a plan for how CRTT data will be analyzed.