Avoiding pitfalls: Bayes factors can be a reliable tool for post hoc data selection in implicit learning

Research on implicit processes has revealed problems with awareness categorizations based on nonsignificant results. Moreover, post hoc categorizations result in regression to the mean (RTM), by which aware participants are wrongly categorized as unaware. Using Bayes factors to obtain sensitive evidence for participants' lack of knowledge may deal with nonsignificance being nonevidential, but also may prevent regression-to-the-mean effects. Here, we examine the reliability of a novel Bayesian awareness categorization procedure. Participants completed a reward learning task followed by a flanker task measuring attention towards conditioned stimuli. They were categorized as B_Aware and B_Unaware of stimulus-outcome contingencies, and those with insensitive Bayes factors were deemed B_Insensitive. We found that performance for B_Unaware participants was below chance level using unbiased tests. This was further confirmed using a resampling procedure with multiple iterations, contrary to the prediction of RTM effects. Conversely, when categorizing participants using t tests, t_Unaware participants showed RTM effects. We also propose a group boundary optimization procedure to determine the threshold at which regression to the mean is observed. Using Bayes factors instead of t tests as a post hoc categorization tool allows evaluating evidence of unawareness, which in turn helps avoid RTM. The reliability of the Bayesian awareness categorization procedure strengthens previous evidence for implicit reward conditioning. The toolbox used for the categorization procedure is detailed and made available. Post hoc group selection can provide evidence for implicit processes; the relevance of RTM needs to be considered for each study and cannot simply be assumed to be a problem.