Experiment aversion does not appear to generalize

Area of Science:

• Behavioral economics
• Social psychology
• Policy analysis

Background:

• Behavioral insights teams often advocate for randomized experiments to inform policy.
• Prior research suggested individuals may perceive randomized experiments as less appropriate than the policies themselves.
• Concerns were raised that this perception could lead to reduced use of field experiments and potential bias.

Purpose of the Study:

• To test the generalizability and robustness of findings on relative experiment aversion.
• To investigate whether people consistently rate randomized experiments as less appropriate than policies.
• To provide evidence for policymakers regarding the use of evidence-based practices.

Main Methods:

• Conducted one direct and six conceptual preregistered replications of prior studies.
• Utilized a total sample size of 5,200 participants (N = 5,200).
• Varied scenario wording, recruitment platforms, and countries; included measures of participant views.

Main Results:

• The original findings regarding relative experiment aversion did not appear to generalize across replications.
• Insufficient evidence was found to conclude that people exhibit a common pattern of relative experiment aversion.
• Results support recent studies questioning the prevalence of experiment aversion.

Conclusions:

• Policymakers may not need to be concerned about relative experiment aversion when implementing evidence-based practices.
• The findings suggest a lower risk of policymakers avoiding or being opaque about randomized experiments.
• Confidence in employing evidence-based practices is supported by the lack of generalized experiment aversion.