Ecological validity of common behavioral decision making tasks: evidence across two samples

Clinicians and scholars routinely use behavioral decision tasks to assess real-world decision making capabilities. However, many common behavioral decision making tasks lack data on the extent to which they predict real-world risky behaviors. Across two pre-registered studies, and two timepoints, we assessed decision making abilities using common behavioral tasks and predicted participants' real-world risky decision making from task performance.

In Study 1, 918 Mechanical Turk (MTurk) workers completed three decision making tasks in addition to assessments of real-world risk behavior: preventive health behaviors, COVID-19 vaccination status, and virtual social distancing task performance. In Study 2, 221 college student participants completed the Study 1 tasks plus additional assessments of decision making and real world risk and protective behaviors.

Across both studies, the selected behavioral decision tasks rarely predicted real world behavior and, when they did, the relationship was weak at best.

These data suggest that these behavioral decision making tasks may not be good predictors of real world risky behavior at present, with some evidence that the specificity of the behavior being assessed matters (i.e. the closer the task was to the specific behavior being predicted), calling for additional ecological validity research, with a greater variety of tasks in the future.