Design, Conduct, and Analysis of Randomized Clinical Trials of Chinese Herbal Medicine

Randomized clinical trials (RCTs) provide the optimal design for evaluating the effects of Chinese herbal medicine (CHM) on patient outcomes. However, how trialists have designed, conducted, and analyzed CHM RCTs remains largely unknown.

To investigate the design, conduct, and analysis of CHM RCTs; to examine whether there are differences between RCTs published in English and Chinese and between higher-impact and lower-impact English journals; and to determine whether CHM RCTs have improved over time.

In this cross-sectional study, PubMed, EMBASE, Cochrane Central Register of Controlled Trials, and 4 Chinese databases were searched from inception to April 2024. Parallel CHM RCTs published in journals covered in the Journal Citation Reports or Chinese core journals were included.

The primary outcomes were the general and methodological characteristics of included RCTs published in English vs Chinese, publication year up to vs after 2015, and higher-impact vs lower-impact English journals. To compare characteristics of included RCTs published in different journals, χ2 or Fisher exact test was use for dichotomous variables, and t test was used for continuous variables when the distribution proved normal or Mann-Whitney U test when it did not.

The 400 CHM RCTs (200 from Chinese language journals and 200 from English language journals) enrolled 100 to 4870 patients. Most RCTs (370 RCTs [92.5%]) did not specify the study hypothesis; approximately one-third (102 RCTs [30.6%]) were registered. The protocols were available for 15 RCTs (3.8%), and statistical analysis plans were available for 4 RCTs (1.0%). Approximately two-fifths (159 RCTs [39.8%]) reported inadequate methods of allocation sequence generation, and three-fifths (242 RCTs [60.2%]) described inadequate methods of allocation concealment. More than one-third (138 RCTs [34.5%]) explicitly specified a primary outcome, and 115 RCTs (28.8%) reported sample size estimation. Ony 10 RCTs (2.5%) had an independent data monitoring committee. More than two-thirds (254 RCTs [73.5%]) stated reasons for prescribing CHM, most commonly the limited or no effect of Western medicine (215 RCTs [53.8%]) and adverse effects of Western medicine (80 RCTs [20.0%]). Most RCTs did not mention whether there was prior clinical (279 RCTs [69.8%]), pharmacological (201 RCTs [50.2%]), or toxicological (388 RCTs [97.0%]) evidence to support the trial hypotheses. A minority (146 RCTs [36.5%]) specified the prescription of CHM according to traditional Chinese medicine syndrome diagnosis. Most RCTs with missing data conducted only a complete case analysis (70 RCTs [77.8%] for dichotomous outcomes and 79 RCTs [84.0%] for continuous outcomes). A small proportion of RCTs (62 RCTs [15.5%]) used an intention-to-treat analysis, and trialists rarely performed sensitivity analysis (29 RCTs [7.2%]) and subgroup analysis (30 RCTs [7.5%]). The design, conduct, and analysis of CHM RCTs improved over time, and were superior in English-language journals, especially higher-impact English-language journals.

These findings suggest that the conduct and analysis of CHM RCTs are generally suboptimal, highlighting areas that urgently need improvement, including statement of study hypothesis and provision of a protocol; registration of the trial; implementation of allocation concealment; specification of primary outcome and sample size estimation; mention of prior clinical, pharmacological, and toxicological support for the trial hypotheses; and satisfactory conduct of sensitivity analysis or subgroup analysis. Although improvements occurred over time, further enhancing the fundamental research capabilities and developing methodological guidelines remains necessary.