How to use a multicriteria comparison procedure to improve modeling competitions: A comment on Erev et al. (2017)

Modeling competitions are a promising method for advancing psychological science. In this commentary to Erev et al. (Psychological Review, 2017, 124, p. 369), we highlight how this promise could be enhanced through modifying competition structures to produce insights more directly in line with the goals of promoting psychological knowledge. We argue that a single criterion on which models is compared limits the diversity of models entered into competitions, restricting the number and type of insights that can be gained consequently. We propose an alternative competition structure with multiple evaluative criteria and outline a quantitative selection method for choosing a winner. Our proposed competition structure has the advantages of (a) increasing the diversity of models entered, (b) incentivizing desirable qualities of models, (c) disambiguating competition winners, and (d) enhancing the impact and possible insights gained from competitions, all these while allowing flexibility for competition organizers to emphasize specific qualities of models. (PsycInfo Database Record (c) 2021 APA, all rights reserved).