Understanding statistical versus clinical significance: a critical appraisal of the CREST-2 study

A widespread practice is to label trials as positive or negative based on whether the P value for the primary outcome is less than 0.05. A more reasonable approach is to examine the totality of evidence, particularly when results approach the threshold of statistical significance.

We review the CREST-2 study which included two parallel trials on ≥70% asymptomatic carotid stenoses: one trial (n = 1245) compared stenting with an intensive medical management group (IMM₁); a second trial (n = 1240) compared endarterectomy with another IMM₂ group.

A significantly lower rate of the 4-year primary composite endpoint of stroke or death (2.8%) was reported with stenting compared to IMM₁ (6.0%; ARR 3.2%; P = 0.02), while results of endarterectomy (3.7%) vs IMM₂ (5.3%) were not significantly better (ARR 1.6%; P = 0.24). Closer examination shows that overall, only 104 infarctions occurred in 2485 patients (4.2%) over 4 years, and most (70 or 67%) were minor strokes. A 'tipping point analysis' revealed that if 3 more primary outcome events had occurred in the stenting group, significance would have been lost; if 6 more events had occurred in the IMM2 group, endarterectomy would have attained significance. Stenting resulted in as many major strokes (n = 4) as IMM₁ (n = 5) over 4 years. Major strokes after endarterectomy (n = 3) and IMM₂ (n = 5) were similar.

Stenting or endarterectomy provide similar results but should be reserved for patients who fail intensive medical management. A global interpretation of all the evidence is required when interpreting results that are close to statistical significance.