Assessing alternative medicine: methodological and research policy concerns

Setting funding priorities among research projects for complementary and alternative therapies is especially vulnerable to arbitrary, partisan criteria and opportunistic readings of controversial evidence. This study develops a procedural approach to characterizing, in a transparent and even-handed way, the available evidence on these treatments and demonstrates a simple analytical scheme for prioritizing competing, but typically incommensurable, research projects for public support.

A simple five-step scheme for categorizing therapies is developed and justified, based on a taxonomy of the study designs used to generate research evidence. Once identified, candidate therapies are assigned to ordered categories along these design criteria and effectively grouped into equivalence classes by type of evidence. Finally, a partial ordering on the therapies is formed within each class by means of secondary screening criteria.

Twenty-five candidate therapies are assigned to equivalence classes. The intent, in effect, is to restrict comparisons to those therapies that fall within a particular class of similar study designs. Within-class orderings avoid the problem of having better-known or better-supported therapies crowd out lesser known ones when it comes to allocating dollars for more research. A set of criteria and procedure for prioritizing spending for further research is demonstrated.

Relying on an open, formal procedure for comparing unconventional therapies offers protection against prejudgment in setting funding priorities, especially when weak clinical evidence relates more to a low investment in research than lack of efficacy.