Effect sizes in ANCOVA and difference-in-differences designs.

Area of Science:

• Statistics
• Experimental Design
• Biostatistics

Background:

• Adjusting for baseline characteristics is standard in randomized and quasi-experiments to improve intervention effect estimates.
• Pre-tests reduce standard error and bias in non-randomized designs.
• Standardized effect sizes are crucial for comparing interventions but their estimation with covariate adjustment is understudied.

Purpose of the Study:

• To provide a framework for defining effect sizes in pre-tested experimental designs.
• To propose estimators for these effect sizes.
• To evaluate the proposed estimators and their sampling distributions.

Main Methods:

• Developed a framework for effect size estimation in designs with pre-tests (e.g., difference-in-differences, analysis of covariance).
• Proposed novel estimators for these effect sizes.
• Utilized simulation studies to evaluate estimator performance and sampling distributions.

Main Results:

• The proposed framework and estimators are effective for calculating standardized effect sizes in pre-tested designs.
• Simulation results demonstrate the validity and accuracy of the proposed methods.
• The methods were successfully applied to a real-world data example.

Conclusions:

• This research addresses a gap in the literature by providing robust methods for estimating standardized effect sizes in pre-tested designs.
• The proposed framework and estimators enhance the comparability and interpretability of intervention effects.
• The findings have implications for researchers conducting experiments with baseline measurements.