Choosing the Best Statistical Test for Within-Groups Analysis for Home Health Practitioners: A Simulation Study

Within-group (pre-post) designs are commonly employed in home healthcare research because patient and professional study populations present with heterogeneous characteristics, recruitment is often challenging, and well-matched control groups are often infeasible to assemble. However, these designs typically involve small sample sizes, outcomes measured in field settings, and data conditions marked by heteroskedasticity, non-normality, and outliers both unplanned and associated with the frequent use of ordinal scales to collect data. To address these issues, a Monte Carlo simulation study was conducted to evaluate the characteristics of four one-sided paired tests used in within-group analyses as to recommend the best baseline test for home healthcare researchers: the paired samples t test, the Wilcoxon signed-rank (WSR) test, and two Bayesian paired-samples t tests with default priors (Cauchy, Normal). The simulated data were designed specifically to reflect conditions commonly encountered in home health research, including small sample sizes (N = 10-50), heavy-tailed distributions, contamination by 5% extreme posttest outliers, and heteroskedasticity introduced by doubling posttest variance relative to pretest. Type I error rates were calibrated to 5% across all methods to ensure fair comparison. Results indicated that the WSR provided the most consistent performance across both normal and non-normal data environments, maintaining 5% Type I error and achieving higher average power under the heavy-tailed, contaminated, and heteroskedastic conditions most representative of home healthcare outcomes. Bayesian procedures outperformed the paired samples t-test in many conditions, but neither consistently exceeded the WSR. The paired samples t-test remained competitive only under clean normal, homoskedastic data with very small effects. Overall, findings support the WSR as the safest default test for small-sample within-group home health studies.