Minimal clinically important differences and the illusion of precision

Minimal clinically important differences (MCIDs) are used in palliative care trials to guide sample size calculations and to judge whether changes in patient-reported outcomes represent meaningful benefit. Several methods are used to estimate MCIDs. For the Total Symptom Distress Score (TSDS), two common approaches are the within-patient mean change and receiver-operating characteristic (ROC)-based method.MCIDs derived using these approaches often differ across otherwise similar studies. We examined whether this variability reflects limited sample size or whether commonly used ROC-based methods contribute to persistent instability.

Data for 186 participants were drawn from two published randomised controlled trials. MCIDs were estimated using (1) the within-patient mean change (anchored to minimal improvement) and (2) ROC analysis to maximise Youden's J. Stratified bootstrap resampling was performed at nominal sample sizes of 50, 100 and 150.

With increasing sample size, variability in ROC-derived MCIDs decreased only modestly and remained substantial even in large samples. Multiple thresholds showed near-equivalent discriminatory performance. At n=150, multiple near-optimal thresholds were observed in up to 85.4% of bootstrap samples. The differences in the threshold values were clinically relevant. In contrast, mean-change MCIDs converged towards more consistent values (approximately -6.5 TSDS points), with markedly lower clinically-relevant variability.

When MCIDs are used as fixed thresholds for trial design or interpretation, mean-change approaches are more reproducible across plausible samples, whereas ROC-based MCIDs show greater variability that can meaningfully affect trial conclusions. Caution is therefore warranted when using single ROC-derived MCIDs in palliative care trials.