Time on Your Side: Aggregating Data in Difference-In-Differences Studies.

Area of Science:

• Econometrics
• Causal Inference
• Statistical Modeling

Background:

• Difference-in-differences (DID) estimators are widely used for causal inference.
• The choice of time scale for data aggregation can influence DID estimator performance.
• Understanding this influence is crucial for accurate policy and treatment effect evaluation.

Purpose of the Study:

• To compare the performance of DID estimators when applied to data aggregated at different time scales (monthly, quarterly, yearly).
• To assess how time aggregation affects the estimation of static and dynamic average treatment effects on the treated (ATT).

Main Methods:

• Simulations using parametric and resampling models with varying panel balance and treatment timing.
• Estimation of static and dynamic ATT using linear regression and Callaway and Sant'Anna (2021) estimators.
• Comparison of monthly, quarterly, and yearly aggregated data based on bias, standard error, RMSE, power, and Type I error.
• A case study using police retraining data to illustrate real-world impacts.

Main Results:

• Time aggregation's impact varied by performance metric, estimation method, and data structure.
• Dynamic treatment effects, unbalanced panel data, and resampling simulations showed greater sensitivity to time aggregation.
• Coarser aggregation was favored in resampling simulations with staggered timing.
• Re-analysis of police training data demonstrated sensitivity to time aggregation.

Conclusions:

• Time aggregation often has a negligible effect on DID estimators.
• Estimating dynamic effects with staggered timing and unbalanced data presents a precision-power tradeoff favoring finer aggregations for power but reducing precision.
• Single reference time point estimators are more susceptible to noise at finer time scales.