Picking the good apples: statistics versus good judgment in choosing stent operators for a multicenter clinical trial

The Carotid Revascularization Endarterectomy Versus Stenting Trial was completed with a low stroke and death rate. A lead-in series of patients receiving carotid artery stenting was used to select the physician-operators for the study, where performance was evaluated by complication rates and by peer review of cases. Herein, we assess the potential contribution of statistical evaluation of complication rates.

The ability to discriminate between stent operators who can successfully meet the published guideline of <3% combined rate of stroke and death is calculated under the binomial distribution, based on a small consecutive case series (n=24 patients).

A criterion of ≤2 stroke or death events among the 24 patients (<8% event rate) was required of operators. Setting such a high criterion, however, ensures an inability to exclude operators who cannot meet the criteria. In fact, if a good operator is defined as having a 2% event rate and a poor operator as a 6% event rate, even a series of 240 patients would (on average) still exclude 5.4% of the good operators and include 4.6% of the poor operators.

The low periprocedural event rates in the trial suggest success in separating skillful operators from less skillful. However, it seems unlikely that statistical assessment of event rates in the lead-in contributed to successful selection, but rather successful selection was more likely because of peer review of subjective and other factors including patient volume and technical approaches.

http://www.clinicaltrials.gov. Unique identifier: NCT00004732.