Reliability of attention bias and attention bias variability to climate change images in the dot-probe task

Climate change is one of the most pressing issues of the 21st century, which is perhaps why information about climate change has been found to capture observers' attention. One of the most common ways of assessing individual differences in attentional processing of climate change information is through the use of reaction time difference scores. However, reaction time-based difference scores have come under scrutiny for their low reliability. Given that a primary goal of the field is to link individual differences in attention processing to participant variables (e.g., environmental attitudes), we assessed the reliability of reaction time-based measures of attention processing of climate change information utilizing an existing dataset with three variations of the dot-probe task. Across all three samples, difference score-based measures of attentional bias were generally uncorrelated across task blocks (r = -0.25 to 0.31). We also assessed the reliability of newer attention bias variability measures that are thought to capture dynamic shifts in attention toward and away from salient information. Although these measures were initially found to be correlated across task blocks (r = 0.17-0.67), they also tended to be highly correlated with general reaction time variability (r = 0.49-0.83). When controlling for general reaction time variability, the correlations across task blocks for attention bias variability were much weaker and generally nonsignificant (r = -0.25 to 0.33). Furthermore, these measures were unrelated to pro-environmental disposition indicating poor predictive validity. In short, reaction time-based measures of attentional processing (including difference score and variability-based approaches) have unacceptably low levels of reliability and are therefore unsuitable for capturing individual differences in attentional bias to climate change information.