Increasing power for detecting awareness: a new approach to test group level objective performance

Demonstrating reliable unconscious effects is notoriously challenging, and the scope of unconscious processing remains debated. Establishing unconscious effects requires two critical pieces of evidence: first, that the stimulus of interest was processed, and second, that participants were not consciously aware of that stimulus. Here, we focus on the second requirement which has faced criticism due to the low statistical power of awareness tests, undermining the reliability of evidence for unawareness. Consequently, reported unconscious effects might in fact stem from conscious processing. To better detect awareness, we suggest using two tests: the frequentist Group Binomial or Chi test and the Bayesian Group Binomial Bayesian test. Both tests were tailored for maximizing power in detecting consciousness, when using objective measures of awareness. We used simulations to compare the sensitivity and specificity of these tests with other commonly used ones (t-test, Mixed Model Logistic Regression, and Bayesian t-test). Our results show a clear power advantage for the proposed tests, across different scenarios. Furthermore, we reanalyzed 79 previously reported effects, from 15 papers on unconscious processing, and found cases where the proposed test revealed affects that were not detected by the other tests. We suggest that our systematic approach towards assessing and comparing the power of awareness tests will advance the field of unconscious processing and reduce the risk of contaminating reports of unconscious effects by undetected conscious processes.