Sample size determination for decision-centered pragmatic trials

To develop a decision-analytic (DA) method for sample size determination in pragmatic randomized controlled trials (RCTs) aimed at selecting the better treatment. The method is anchored to the stakeholder-defined minimally important difference (MID). The MID defines acceptable regret (ARg), the expected loss from a wrong decision that stakeholders are willing to tolerate, and γ, the probability of choosing the wrong direction.

We modeled a two-arm RCT using a decision tree in which net clinical benefit incorporates both treatment benefits and harms, weighted according to stakeholder values and preferences, expressed as the relative value (RV) placed on disease outcomes vs harms. We define the decision rule to choose the treatment with the better sample-analog estimate of net benefit. Sample size is set so that expected loss from wrong decisions (γ·Δ) does not exceed ARg. We compared DA with conventional frequentist designs (type I, α and type II, β error control) in Monte Carlo simulations and illustrated the approach by recalculating sample size for four recent high-impact pragmatic trials. A Stata program and web-based calculator are provided.

Across 10,000 simulations, the DA approach required about half as many participants, on average, as in explanatory, frequentist-oriented RCTs (mean N 89 vs 168). In 3 of 4 recent pragmatic trials, the reduction of sample size was between 47% and 72%; this reduction depended on the assumed RV and ARg. In the fourth trial (ADAPTABLE), the framework revealed tensions among the assumed Δ, stakeholder preferences, and tolerable decision risk - showing that, at very small effect sizes, trialists must either relax ARg, revise the question, or conclude that the expected decision value is insufficient.

We describe a novel method for calculating sample size for pragmatic RCTs that is transparent, ethically coherent, and - when assumptions are reasonable - can substantially reduce required sample size, potentially lowering barriers to accrual and accelerating the generation of actionable pragmatic evidence.

Whether a treatment works is usually first tested under controlled ("laboratory") conditions in "explanatory" trials. But to know what treatment to choose in everyday practice, we need "pragmatic" trials that compare real treatment options with representative patients in real settings.A key challenge is choosing the right number of participants. Too few increases the risk of choosing the wrong treatment; too many wastes resources and exposes people to unnecessary risk. Decision-focused methods for setting sample size in pragmatic trials have been limited. Our research addresses this gap. We developed a patient-centered approach that starts from what patients and clinicians consider an important difference between treatments, and uses this to set an acceptable chance of making the wrong choice ("acceptable regret"). The approach accounts for both benefits and harms of each treatment. Using computer simulations, we identified the smallest sample size that keeps the risk of a wrong decision within the acceptable limit. Across many simulated settings and in three of four well-known pragmatic trials, our approach required fewer participants than traditional methods while still answering the question the trials were designed to answer. In the fourth trial, the framework signaled that the target effect was too small to resolve with any feasible sample size - a result that usefully exposes a question otherwise hidden by conventional methods. The approach can make pragmatic trials faster and less expensive, yield answers sooner, speed up the adoption of better treatments, and use participants' contributions more ethically.