Identifying causes of treatment effect heterogeneity across replication studies: Definitions, identification, and...

Replication studies are crucial for evaluating treatment efficacy and exploring the conditions that moderate treatment effects. Understanding this treatment effect heterogeneity across studies is essential for building robust theories. However, identifying the causes of this heterogeneity is challenging, as multiple study characteristics often vary simultaneously, even in replication studies in which many aspects are deliberately held constant across studies. Traditional metaregression typically lacks a clear causal framework, obscuring the causal interpretation of moderating effects. Although a more recent causal method provides a way to identify the replication effect after adjusting for unintended differences, it relies on the strong assumption that all effect moderators that cause the treatment effect heterogeneity across studies must be measured and uses a parametric estimation model. We propose an alternative approach by introducing a causal framework that decomposes sources of treatment effect heterogeneity. Our approach uses a directed acyclic graph to identify these sources and to develop strategies to single out their contributions. By focusing on different (causal and noncausal) quantities, the proposed strategies require weaker assumptions than the earlier causal approach. We also propose robust estimation strategies using a variable selection method designed to model these higher order interactions with minimal bias. We illustrate our method using a real-world example of replication studies from social psychology. Through a simulation and case study, we provide useful recommendations for applied researchers on designing future replication studies and identifying moderators that cause effect heterogeneity. (PsycInfo Database Record (c) 2026 APA, all rights reserved).