Problems lowering the study quality in traditional medicine, introspection from an example of meta-analysis of...

Most randomized controlled trials (RCTs) of traditional medicine (such as traditional Chinese medicine (TCM), psychotherapy or behavioral therapy, and dietary interventions, etc.) have reported that they could not provide convincing evidence to support the efficacy because of the low quality of their studies. Here, we aimed to determine the underlying problems of the study quality using standards of evidence-based medicine (EBM) to evaluate the efficacy of traditional medicine.

We conducted an example of meta-analysis to evaluate the efficacy of acupuncture, a classical treatment of TCM, for treatment of intracerebral hemorrhage (ICH). The quality of the included studies was evaluated by using a Jadad score.

A total of 24 Chinese RCTs that enrolled 1815 patients with ICH were included. Although the results suggested that acupuncture had good efficacy for relief of neurological deficits and improvement of the activities of daily living despite the high heterogeneity of the included studies, the low quality of the included literature reduced the worthiness of the evidence. Two systematic problems (lack of blinding and allocation concealment and high heterogeneity) and one non-systematic problem (lack of reports on adverse events and follow-up) of the TCM studies were found in this illustrational meta-analysis. We believed that other interventions of traditional medicine also suffer from these problems.

Non-systematic problems can be improved by perfecting the experimental design, educating the researcher, and improving the reporting system. However, systematic problems are derived from the characteristics of traditional medicine that are difficult to be corrected. We propose that adoption of objective indexes might be a better solution to improve the systematic problems of traditional medicine. We summarized the problems and the underlying solutions, which may contribute to improve the study quality of systematic review in traditional medicine, strictly complying with the principles of EBM.