Related Experiment Video
Updated: Jun 11, 2025

Effects of Surgical Masks on Cardiopulmonary Function in Healthy Subjects
Published on: February 12, 2021
School Mask Mandates and COVID-19: The Challenge of Using Difference-in-Differences Analysis of Observational Data to
Ambarish Chandra1, Tracy Beth Høeg2, Shamez Ladhani3
1Department of Management, University of Toronto Scarborough, and Rotman School of Management, University of Toronto, Toronto, Ontario, Canada (A.C.).
Background:
There are considerable challenges when using difference-in-differences (DiD) analysis of ecological data to estimate the effectiveness of public health interventions in rapidly changing situations.
Objective:
To discuss the shortcomings of DiD methodology for the estimation of the effects of public health interventions using ecological data.
Design:
As an example, the authors consider an analysis that used DiD methodology and reported a causal reduction in COVID-19 cases due to the maintenance of school mask mandates. They did alternate analyses using various control groups to assess the robustness of the prior analysis.
Setting:
School districts in the greater Boston area and Massachusetts during the 2021-to-2022 academic year.
Participants:
Students and school staff.
Measurements:
Changes in COVID-19 case rates in districts that did and did not lift mask mandates.
Results:
Important potential confounders rendered DiD methodology inappropriate for causal inference, including prior immunity, temporal variation in rates of infection, and changes in testing practices. The racial composition and income of intervention and control groups also differed substantially. Compared with maintaining the mask requirement, dropping the requirement was associated with anywhere from an increase of 5.64 cases (95% CI, 3.00 to 8.29 cases) per 1000 persons to a decrease of 2.74 cases (CI, 0.63 to 4.85 cases) per 1000 persons, depending on choice of control group and whether students or staff were examined.
Limitation:
Ecological data were used; detailed data on all potential confounders were unavailable.
Conclusion:
Alternate analyses yielded estimates consistent with a wide range of both negative and positive associations in COVID-19 case rates after removal of mask mandates. The findings highlight the challenges of using DiD analysis of ecological data to estimate the effectiveness of interventions in divergent intervention and control groups during rapidly changing circumstances.
Primary Funding Source:
None.
More Related Videos
15:00Setup of Consumer Wearable Devices for Exposure and Health Monitoring in Population Studies
Published on: February 3, 2023
06:55Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Related Concept Videos
Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Confounding in Epidemiological Studies
Comparing the Survival Analysis of Two or More Groups
Steps in Outbreak Investigation
Statistical Methods for Analyzing Epidemiological Data
Statistical Significance