A randomized experiment comparing random and cutoff-based assignment.

Area of Science:

• Econometrics
• Causal Inference
• Experimental Design

Background:

• Randomized experiments are the gold standard for causal inference.
• Regression discontinuity designs offer an alternative when randomization is not feasible.
• Previous comparisons have shown mixed results due to methodological differences.

Purpose of the Study:

• To directly compare randomized experiments and regression discontinuity designs under controlled conditions.
• To investigate the impact of parameter estimation on the agreement between the two methods.
• To assess the validity of regression discontinuity designs as an alternative to randomized experiments.

Main Methods:

• Randomly assigned 588 participants to either a randomized experiment or a regression discontinuity design.
• Ensured participants were treated identically across both designs.
• Employed parametric, semiparametric, and nonparametric methods to model nonlinearities.
• Compared estimates for both identical and different parameters.

Main Results:

• Estimates from regression discontinuity designs closely approximated those from randomized experiments.
• Significant exceptions were observed, highlighting the importance of parameter definition.
• The choice of modeling approach (parametric, semiparametric, nonparametric) influenced the results.
• The study identified challenges in defining "agreement" between the two estimation methods.

Conclusions:

• Regression discontinuity designs provide a reliable approximation of randomized experiment results when implemented carefully.
• Methodological rigor, particularly in parameter estimation and nonlinearity modeling, is essential for valid comparisons.
• Further research is needed to establish clear criteria for assessing the concordance between these causal inference techniques.