Evaluating statistical methods for single-case designs under serial dependence: Precision, validity, and sensitivity

Single-case experimental designs involve short, serially dependent time series that complicate statistical inference. This Monte Carlo simulation evaluated six methods (nonoverlap of all pairs, Tau-U, Hedges's g, piecewise regression, randomization tests, and Bayesian generalized linear models) for Type I error, power, and interval coverage, each assessed for the methods that admit it, across series lengths (N = 15, 30, 60), autocorrelation (ρ = 0, .3, .6), baseline trend, and effect structure within an AB design. The two regression-based methods showed Type I error rates reaching .27-.37 at ρ = .6 and coverage as low as .62. The Type I error rate for nonoverlap of all pairs was inflated with autocorrelation and reached 1.00 under a baseline trend. Coverage for Hedges's g fell from .95 to .69 under a flat baseline and collapsed to .00 under a trending baseline. Because these methods did not control Type I error, their nominally higher power is not commensurable with that of a valid test: At a common calibrated Type I error rate, the randomization test was at least as powerful as either regression method in every equal-allocation level-change condition at N ≥ 30. The apparent validity-sensitivity trade-off is therefore largely an artifact of comparing tests that operate at different actual significance levels, compounded by applying a randomization test to a design containing no randomization. Randomizing the intervention start point restored the test to the largest significance level its discrete reference distribution admits. Type I error control is treated as a prerequisite, and design-level randomization is recommended as the principal remedy. (PsycInfo Database Record (c) 2026 APA, all rights reserved).