Causal definitions versus casual estimation: Reply to Valente et al. (2022)

Area of Science:

• Psychology
• Statistics
• Research Methodology

Background:

• Mediation models are used to estimate causal effects in research.
• Single-case experimental designs (SCEDs) are employed in various fields.
• Valente et al. (2022) proposed a mediation model for SCEDs.

Purpose of the Study:

• To critically evaluate the plausibility and applicability of the mediation model proposed by Valente et al. (2022) for SCEDs.
• To examine the causal effects estimation under the assumption of sequential ignorability (no unmeasured confounders).
• To demonstrate the potential for artifacts and inaccuracies in mediation analyses.

Main Methods:

• Analytic argument on the plausibility of sequential ignorability for SCEDs.
• Reanalysis of the empirical example from Valente et al. (2022).
• Simulation study introducing an unmeasured confounder.
• Analysis of historical data on birth control, stork population, and birth rates.

Main Results:

• The assumption of sequential ignorability is problematic for SCEDs.
• Simulation results show Type I error rates up to 100% and 0% coverage for causal effects with unmeasured confounding.
• Historical data analysis reveals potential artifacts in mediation models that are difficult to detect.

Conclusions:

• The proposed mediation model's assumptions are not tenable for SCEDs.
• Unmeasured confounding can severely distort causal effect estimates in mediation analyses.
• Mediation models require careful scrutiny for potential artifacts and limitations.