An example on the value of non-randomisation in clinical trials in complementary medicine

Randomised clinical trials may in principle show a small external validity. Non-randomised clinical trials therefore are sometimes regarded as an appropriate alternative when complementary and conventional treatments are compared.

To assess the value of advanced statistical methods in the process of estimating differences between a complementary and a conventional treatment of acute sinusitis in a non-randomised clinical trial.

Multicentre, non-randomised, controlled clinical trial comparing 2 complementary and 3 conventional ENT centres. Patients were free to choose the physician (and hence the therapy). Treatment differences were estimated by controlling for confounders in analyses of covariance or by propensity score techniques.

Most potential confounders (sex, age, life-style parameters) did not have significant effects on the choice of therapy. Disease severity and previous ENT surgery were the main confounding factors. At study onset they almost cause a defined separation of both treatment groups. As a result estimated treatment differences vary substantially depending on the chosen statistical model.

When comparing complementary and conventional treatments, non-randomised clinical trials may be misleading. Results may be strongly biased even when advanced statistical methods are used. Trials of complex statistical designs are needed to give valid results.