Exploring and adjusting for potential learning effects in ROLARR: a randomised controlled trial comparing...

Commonly in surgical randomised controlled trials (RCT) the experimental treatment is a relatively new technique which the surgeons may still be learning, while the control is a well-established standard. This can lead to biased comparisons between treatments. In this paper we discuss the implementation of approaches for addressing this issue in the ROLARR trial, and points of consideration for future surgical trials.

ROLARR was an international, randomised, parallel-group trial comparing robotic vs. laparoscopic surgery for the curative treatment of rectal cancer. The primary endpoint was conversion to open surgery (binary). A surgeon inclusion criterion mandating a minimum level of experience in each technique was incorporated. Additionally, surgeon self-reported data were collected periodically throughout the trial to capture the level of experience of every participating surgeon. Multi-level logistic regression adjusting for operating surgeon as a random effect is used to estimate the odds ratio for conversion to open surgery between the treatment groups. We present and contrast the results from the primary analysis, which did not account for learning effects, and a sensitivity analysis which did.

The primary analysis yields an estimated odds ratio (robotic/laparoscopic) of 0.614 (95% CI 0.311, 1.211; p = 0.16), providing insufficient evidence to conclude superiority of robotic surgery compared to laparoscopic in terms of the risk of conversion to open. The sensitivity analysis reveals that while participating surgeons in ROLARR were expert at laparoscopic surgery, some, if not all, were still learning robotic surgery. The treatment-effect odds ratio decreases by a factor of 0.341 (95% CI 0.121, 0.960; p = 0.042) per unit increase in log-number of previous robotic operations performed by the operating surgeon. The odds ratio for a patient whose operating surgeon has the mean experience level in ROLARR - 152.46 previous laparoscopic, 67.93 previous robotic operations - is 0.40 (95% CI 0.168, 0.953; p = 0.039).

In this paper we have demonstrated the implementation of approaches for accounting for learning in a practical example of a surgery RCT analysis. The results demonstrate the value of implementing such approaches, since we have shown that without them the ROLARR analysis would indeed have been confounded by the learning effects.

International Standard Randomised Controlled Trial Number (ISRCTN) registry, ID: ISRCTN80500123. Registered on 27 May 2010.