Where is the criterion noise in recognition? (Almost) everyplace you look: comment on Kellen, Klauer, and Singmann...

Recent articles, including Benjamin, Diaz, and Wee (2009), have argued that recognition memory may be better understood if consideration is given to sources of noise in the decisions, as well as to those in the representations, underlying recognition judgments. They based that conclusion on a wide consideration of persisting mysteries in recognition research as well as a new experimental paradigm involving ensemble recognition. Kellen, Klauer, and Singmann (2012) reanalyzed Benjamin et al.'s data and introduced their own new experimental paradigm to this debate. They concluded that criteria do not vary much from trial to trial in recognition testing and, thus, that decision noise in recognition is small or nonexistent. However, their alternative interpretation of Benjamin et al.'s data relies on a questionable conclusion to reject all models in which the locations of criteria are restricted to be the same across ensembles and a meta-assumption that a model should be rejected as false if it yields unconventional parameters. In addition, their experimental logic relies on the assumption that ranking tasks are always bias-free. Here, I question these assumptions and suggest avenues for reconciliation between these contrasting claims.