Abstract
Background:
Cochrane Systematic Reviews (CSRs) are frequently referenced by acupuncture efficacy studies currently. In this study, the CSRs on acupuncture are reviewed, and the disease fields they covered and the conclusions they reached are analyzed. In order to explore the potential contribution to CSRs by Chinese resources, the authors analyzed whether the participation of Chinese reviewers, the utilization of Chinese databases, and the inclusion of Chinese clinical trials would affect the positive conclusion ratios of the CSRs.
Methods:
Acupuncture-related CSRs in the Cochrane Library were searched and classified based on the International Classification of Diseases-10 (ICD-10). The CSRs were further designated as positive or negative according to the conclusion statements. CSRs with the participation of Chinese reviewers, the utilization of Chinese databases, or the inclusion of Chinese clinical trials were extracted, and the positive ratios of conclusions were compared separately with corresponding CSRs without those three Chinese resources.
Results:
Thirty-two (32) CSRs were identified, 9 (28.1%) of which reached positive conclusions. The CSRs with positive conclusions were mainly about multifarious pains, nausea and vomiting, and functional disorders. Seventeen (17; 53.1%) included the participation of Chinese reviewers, 18 (56.3%) involved the utilization of Chinese databases, and 20 (62.5%) included Chinese clinical trials. No differences on the positive conclusion ratios were observed between CSRs with reviewers from Chinese institutions and those that did not (odds ratio [OR]: 0.32, 95% confidence interval [CI]: 0.06, 1.62), the utilization of Chinese databases and those that did not (OR: 0.51, 95% CI: 0.11, 2.44), or the inclusion of Chinese clinical trials and those that did not (OR: 1.29, 95% CI: 0.26, 6.49).
Conclusions:
Most CSRs on acupuncture are inconclusive. No significant differences regarding the positive conclusion ratios were found between the CSRs with or without the utilization of Chinese resources.
Introduction
The Cochrane Collaboration, an international nonprofit organization established in 1992, has prepared and maintained systematic reviews on various types of intervention. In 1996, the Cochrane Collaboration established a field for reviewing trials of complementary medicine. 2 After that, the guidelines and methods of conducting meta-analyses and systematic reviews of acupuncture became more standardized. 3 Cochrane Systematic Reviews (CSRs) on acupuncture, which are regarded as the highest level of evidence, are referenced with increasing frequency.
As the biggest user of complementary and alternative medicine (CAM), China has been publishing a significant number of articles in this field. How CSRs should best make use of Chinese resources caused some controversy. Some opinions held that Chinese reviewers might have a favorable attitude toward acupuncture efficacy, and clinical trials conducted or published in China are prone to have poor methodological quality 4 or positive outcomes 5 and thus introduce bias. However, one argument against this is that a significant number of controlled trials in CAM are published in languages other than English and in journals not indexed with MEDLINE,® so the exclusion of these trials from systematic reviews is likely to introduce certain levels of publication bias. 4,6 Therefore, it is worth considering what impact Chinese resources have on current CSRs and how Chinese resources could better contribute to future CSRs.
The purposes of this study were (1) to review the clinical evidence of acupuncture on different diseases based on CSRs; and (2) to analyze whether the CSRs with the participation of reviewers from Chinese institutions, the utilization of Chinese databases, or the inclusion of Chinese articles had higher positive conclusion ratios than did those without utilization of the corresponding Chinese resources.
Materials and Methods
Search strategy
The strategies that were used to search The Cochrane Library (Issue May 5, 2011) are listed below.
Selection criteria
All CSRs on acupuncture (including electroacupuncture, wrist acupuncture, auricular acupuncture, and acupressure) were included in this review. The CSRs on multiple interventions (such as nonhormonal interventions, complementary and alternative therapies, etc.) or other interventions related to acupuncture (such as moxibustion, massage, herbal medicine, etc.) were excluded.
Data collection
Two (2) authors (SJ, KT) performed the data extraction and classification independently. If their conclusions conflicted, the third author (NH) would help in making the decisions.
In order to have a thorough understanding of the information in each CSR, a table was generated using Manheimer's table 7 as a model. Data were extracted on the number of included trials, the total number of participants, the number of forest plots, and the number of forest plots with meta-analysis.
The CSRs were assigned into groups A, B, and C, based on the conclusion statements. CSRs with conclusions including the phrase “consistent evidence,” “moderate evidence,” etc. were considered as being in group A. CSRs with conclusions including the phrase “may be beneficial,” “more effective than,” “demonstrate benefit,” “show better results,” “have better results,” “low strength evidence,” “have greater improvements,” etc. were considered as being in Group B. CSRs with conclusions including the phrase “insufficient evidence,” “not enough evidence,” “no consistent evidence,” “without clear evidence,” “lack of data,” “not demonstrate benefit,” “does not support,” etc. were considered as being in Group C. CSRs in Groups A and B were further designated as positive and those in Group C were designated as negative.
The CSRs were categorized based on the International Classification of Diseases-10 (ICD-10) 8 in order to further analyze diseases of interest.
CSRs with the participation of reviewers from Chinese institutions, the utilization of Chinese databases, or the inclusion of clinical trials published in journals issued in China were extracted (including mainland China, Hong Kong, Macau, and Taiwan, no language restriction). The positive conclusion ratios of the CSRs with reviewers from Chinese institutions, the utilization of Chinese databases, or the inclusion of Chinese clinical trials were compared in pairs with those not having each of the three corresponding Chinese resources. The associations between the positive conclusion ratios and the utilization of the three Chinese resources were respectively analyzed.
Statistical analysis
The categorical variables were analyzed using χ2 and Fisher's exact tests, as well as odds ratio and 95% confidence interval for comparison of proportions and statistical significance. Variables identified as significant (p<0.05, two-sided) were then included in backward logistic regression models to examine the association between the utilization of Chinese resources and positive conclusions. All statistical analyses were performed using IBM SPSS Statistics 19.
Results
By using the searching strategies, 207 CSRs were identified. Of them, 32 were acupuncture-related and were included in this study (Fig. 1).

Two hundred and seven Cochrane Systematic Reviews (CSRs) and 94 protocols were identified. Of them, 32 CSRs and 31 protocols were acupuncture related. The 32 CSRs on acupuncture were included in the analysis of this study.
Analyses of conclusions and disease fields
In the 32 CSRs on acupuncture, 9 (28.1%) were considered as having positive conclusions. Four (4; 12.5%) were in Group A, including CSRs on migraine prophylaxis, 9 tension-type headache, 10 neck disorder, 11 and postoperative nausea and vomiting 12 (Table 1). The CSRs on migraine prophylaxis 9 and tension-type headache 10 were considered to have positive conclusions only in the latest version after adding supplemental material of several new randomized controlled trials (RCTs). The earlier versions concluded as having insufficient evidence. Five (5; 15.6%) were in Group B, including CSRs on assisted conception, 13 low back pain, 14 primary dysmenorrhea, 15 shoulder pain, 16 and chemotherapy-induced nausea or vomiting 17 (Table 1). Twenty three (23; 71.9%) of the 32 CSRs on acupuncture were in Group C (Appendix 1).
If the Chinese reviewer was the first or corresponding reviewer, “Yes” is shown in parentheses.
Number of Chinese trials (% of Chinese trial in this CSR).
In the nine CSRs, three included the participation of reviewers from Chinese institutions, and one included first or corresponding authors from Chinese institutions.
In the nine CSRs, four searched Chinese databases.
CSRs, Cochrane Systematic Reviews.
The top three disease fields and their CSR quantities were (1) diseases of the nervous system (Chapter VI), 8 reviews; (2) and (3) mental and behavioral disorders (Chapter V) and diseases of the musculoskeletal system and connective tissue (Chapter XIII), 7 reviews (Table 2).
All the CSRs were assigned into three groups, A, B, and C, according to the strength of the evidence based on the CSRs' conclusion statements. CSRs that indicated consistent or moderate evidence for the efficacy of acupuncture were defined in Group A, CSRs that indicated better results than control or low-strength evidence for acupuncture were defined in Group B, and CSRs with no evidence for acupuncture were defined in Group C.
Analyses of CSRs with Chinese resources
Of the 32 CSRs on acupuncture, 17 (53.1%) included the participation of reviewers from Chinese institutions, and 12 (37.5%) included first or corresponding authors from Chinese institutions. Three (17.6%, 3/17) CSRs with the participation of reviewers from Chinese institutions reached positive conclusions, and 1 (8.3%, 1/12) CSR with first/corresponding authors from Chinese institutions reached a positive conclusion. No significant difference in the positive conclusion ratio was found between the CRSs with and without the participation of reviewers from Chinese institutions or first/corresponding authors from Chinese institutions (Table 3).
No significant differences regarding the positive conclusion ratios were found between the CSRs with and without the utilization of Chinese resources.
CSRs, Cochrane Systematic Reviews; OR, odds ratio; CI, confidence interval.
Eighteen (18; 56.3%) CSRs utilized Chinese databases and 4 (4/18; 22.2%,) of these had positive conclusions. No significant difference in the positive conclusion ratio was found between the CSRs with or without the utilization of Chinese databases (Table 3).
Twenty (20; 62.5%) CSRs included clinical trials published in China, and six of these (6/20; 30%,) had positive conclusions. No significant difference in the positive conclusion ratio was found between the CSRs that included or did not include Chinese clinical trials (Table 3).
For all four pairs of variables that did not have a significant difference (Table 3), backward logistic regression was not performed.
Discussion
CSRs and the conclusions
Manheimer and colleagues had completed a review in 2009, which stated that 26.9% (7/26) of CSRs on acupuncture were considered beneficial. 7 The present result, with a positive conclusion ratio of 28.1% (9/32), is consistent with theirs. In the book Trick or Treatment by Simon Singh and Edzard Ernst published in 2008, was the summary that the positive conclusions of CSRs on acupuncture were restricted to certain types of pain and nausea. 18 In this study, in addition to pain and nausea, one CSR also reported beneficial effects for assisted conception. 13
In this study, 71.9% (23/32) of CSRs were inconclusive, most of which were either due to insufficient qualified clinical trials or methodological limitations in the original clinical trials. In the CSRs on acupuncture for dysphagia in acute stroke, 19 only one trial with 66 participants was included, and no conclusive result was reached. In another two CSRs, “Acupuncture for attention deficit hyperactivity disorder in children and adolescents” 20 and “Acupuncture for vascular dementia,” 21 no RCT or quasi-RCT met the qualifying standard to be included. As to what is known from the two CSRs (acupuncture for migraine prophylaxis 9 and tension-type headaches), 10 new RCTs included might provide new evidence and change the conclusions of current CSRs. Therefore, more high-quality clinical trials are needed to help improve the conclusions of current CSRs.
CSRs and Chinese resources
In one review on acupuncture-related CSRs published in 2005, Chinese databases were searched in 2 of 10 CSRs. 22 In this study, the CSRs with reviewers from Chinese institutions, the utilization of Chinese databases, and the inclusion of Chinese clinical trials were 17 (53.1%), 18 (56.3%), and 20 (62.5%), respectively. Compared to the study in 2005, more Chinese resources have been involved in CSRs during recent years.
In this study, the total positive ratio of the identified CSRs was 28.1% (9/32). The positive ratios of CSRs with reviewers from Chinese institutions, first/corresponding author from Chinese institutions, the utilization of Chinese databases, and the inclusion of Chinese clinical trials were 17.6% (3/17), 8.3% (1/12), 22.2% (4/18), and 30% (6/20), respectively. Although no statistical significance on the ratio of positive conclusions was observed between the CSRs that utilized Chinese resources and those that did not, fewer CSRs with reviewers from Chinese institutions, especially with first/corresponding author from Chinese institutions, reached positive conclusions. Due to the small number of the CSRs in this study, the difference might be eliminated or become more distinct in the future by supplements of new CSRs.
Using the same search strategy and selection criteria as the CSRs, the Cochrane Library (issue May 5, 2011) was searched for registered Cochrane protocols on acupuncture. Among the 31 identified protocols on acupuncture, 25 (80.6%) were with the participation of reviewers from Chinese institutions, and 29 (93.5%) were with the utilization of Chinese databases (Appendix 2). In contrast to the CSRs, more Chinese resources were involved in the Cochrane protocols on acupuncture. Compared to the CSRs, registered protocols were developed in some new fields based on ICD-10 and a wider range of conditions is covered, particularly in the protocols utilizing Chinese resources. 23 –28 Thus, how to make the best use of Chinese resources and avoid bias is critical in future CSRs.
In 2011, there were over 100 journals in print in China related to Traditional Chinese Medicine. 29 A search of CNKI (National Knowledge Infrastructure), the most often used Chinese database, for articles published in 2010 containing the keywords “acupuncture” or “electroacupuncture” found 7384 items. Therefore, it is difficult for reviewers to select high-quality Chinese articles efficiently and take full advantage of them. That might also explain part of the reason why Chinese articles of high quality were seldom cited by international peers of CAM. 29
First, the quality of Chinese articles should be improved, and those with poor quality should not be published. As reported by Manheimer, it was common that information about study quality dimensions was not described in published reports on acupuncture. 30 Thus, when reporting a RCT on acupuncture, reference to the CONSORT Statement (Consolidated Standards of Reporting Trials), 31 especially the revised STRICTA (Standards for Reporting Interventions in Controlled Trials of Acupuncture), 32 would make the report more detailed and understandable. Second, a comprehensive literature search and standardized evaluations of individual studies were suggested to be the optimal approach. 22 Third, when including Chinese clinical trials in CSRs, the Cochrane Handbook for Systematic Reviews of Interventions 33 should be strictly followed to select studies, collect data, and assess risk of bias in the included studies. Through these means, the poor-quality trials and reports could be recognized and excluded from CSRs. In addition, the effort made by Chinese reviewers to identify the methodological weaknesses of the Chinese clinical trials and reports would eventually lead to an enhancement of the quality of future RCTs conducted and reported in China, which would help the Chinese articles contribute to the CSRs on a better level in the future.
Conclusions
About 28.1% of CSRs, mainly on multifarious pain, nausea and vomiting, and functional disorders, provide evidence for the efficacy of acupuncture. More high-quality clinical trials may improve future conclusions.
Chinese resources, including reviewers from Chinese institutions and the utilization of Chinese databases or articles, seem to contribute equally to the conclusions of CSRs compared to resources from countries other than China.
Footnotes
Acknowledgments
This research was supported by FY2010 JSPS Postdoctoral Fellowship for Foreign Researchers (ID No.: P10714) provided by the Japan Society for the Promotion of Science (JSPS). We thank Jianping Liu (Beijing University of Chinese Medicine), Ya Tu (Beijing University of Chinese Medicine), and Yutong Fei (Beijing University of Chinese Medicine) for their advice on the current situation of clinical trials in China as well as relevant materials on China. We also thank Lixing Lao (University of Maryland), Eric Manheimer (University of Maryland), and L. Susan Wieland (University of Maryland) for their advice on the methodology of CSRs. We thank Michio Tojima (The University of Tokyo) for his help with statistics.
Disclosure Statement
No competing financial interests exist.
1. Casimiro L, Barnsley L, Brosseau L, et al. Acupuncture and electroacupuncture for the treatment of rheumatoid arthritis. Cochrane Database Syst Rev 2005;4:CD003788.
2. White AR, Rampes H, Liu JP, et al. Acupuncture and related interventions for smoking cessation. Cochrane Database Syst Rev 2011;1:CD000009.
3. Wong V, Cheuk DKL, Lee S, Chu V. Acupuncture for acute management and rehabilitation of traumatic brain injury. Cochrane Database Syst Rev 2011;5:CD007700.
4. Zhang S, Liu M, Asplund K, Li L. Acupuncture for acute stroke. Cochrane Database Syst Rev 2005;2:CD003317.
5. Li S, Yu B, Zhou D, et al. Acupuncture for attention deficit hyperactivity disorder (ADHD) in children and adolescents. Cochrane Database Syst Rev 2011;4:CD007839.
6. Chen N, Zhou M, He L, et al. Acupuncture for Bell's palsy. Cochrane Database Syst Rev 2010;8:CD002914.
7. Paley CA, Johnson MI, Tashani OA, Bagnall AM. Acupuncture for cancer pain in adults. Cochrane Database Syst Rev 2011;1:CD007753.
8. McCarney RW, Brinkhaus B, Lasserson TJ, Linde K. Acupuncture for chronic asthma. Cochrane Database Syst Rev 2003;3:CD000008.
9. Smith CA, Hay PPJ, MacPherson H. Acupuncture for depression. Cochrane Database Syst Rev 2010;1:CD004046.
10. Xie Y, Wang L, He J, Wu T. Acupuncture for dysphagia in acute stroke. Cochrane Database Syst Rev 2008;3:CD006076.
11. Cheuk DKL, Wong V. Acupuncture for epilepsy. Cochrane Database Syst Rev 2008;4:CD005062.
12. Law SK, Li T. Acupuncture for glaucoma. Cochrane Database Syst Rev 2007;4:CD006030.
13. Smith CA, Crowther CA. Acupuncture for induction of labour. Cochrane Database Syst Rev 2004;1:CD002962.
14. Cheuk DKL, Yeung J, Chung KF, Wong V. Acupuncture for insomnia. Cochrane Database Syst Rev 2007;3:CD005472.
15. Green S, Buchbinder R, Barnsley L, et al. Acupuncture for lateral elbow pain. Cochrane Database Syst Rev 2002;1:CD003527.
16. Manheimer E, Cheng K, Linde K, et al. Acupuncture for peripheral joint osteoarthritis. Cochrane Database Syst Rev 2010;1:CD001977.
17. Cui Y, Wang Y, Liu Z. Acupuncture for restless legs syndrome. Cochrane Database Syst Rev 2008;4:CD006457.
18. Rathbone J, Xia J. Acupuncture for schizophrenia. Cochrane Database Syst Rev 2005;4:CD005475.
19. Wu HM, Tang JL, Lin XP, et al. Acupuncture for stroke rehabilitation. Cochrane Database Syst Rev 2006;3:CD004131.
20. Lim B, Manheimer E, Lao L, et al. Acupuncture for treatment of irritable bowel syndrome. Cochrane Database Syst Rev 2006;4:CD005111.
21. Zhang Y, Peng W, Clarke J, Liu Z. Acupuncture for uterine fibroids. Cochrane Database Syst Rev 2010;1:CD007221.
22. Weina P, Zhao H, Zhishun L, Shi W. Acupuncture for vascular dementia. Cochrane Database Syst Rev 2007;2:CD004987.
23. Gates S, Smith LA, Foxcroft D. Auricular acupuncture for cocaine dependence. Cochrane Database Syst Rev 2006;1:CD005192.
1. Bellorini J, Chamberlain IJ, Burton MJ. Acupressure for motion sickness. Cochrane Database Syst Rev 2008;2:CD007165.
2. Kim TH, Lee MS, Kim KH, et al. Acupuncture for angina pectoris (Protocol). Cochrane Database Syst Rev 2011;3:CD009056.
3. Cheuk DKL, Wong V, Chen WX. Acupuncture for autistic spectrum disorder (Protocol). Cochrane Database Syst Rev 2009;2:CD007849.
4. Zhang M, He J, Li J, Ai CL. Acupuncture for children with cerebral palsy (Protocol). Cochrane Database Syst Rev 2008;2:CD007127.
5. Zhao H, Liu JP, Liu Z, Peng W. Acupuncture for chronic constipation (Protocol). Cochrane Database Syst Rev 2003;2:CD004117.
6. Zhang W, Liu Z, Wu T, Peng W. Acupuncture for chronic fatigue syndrome (Protocol). Cochrane Database Syst Rev 2006;2:CD006010.
7. Liu JP, Wang J. Acupuncture for chronic hepatitis B virus infection (Protocol). Cochrane Database Syst Rev 2005;1:CD005163.
8. Lan L, Zeng F, Liu GJ, et al. Acupuncture for functional dyspepsia (Protocol). Cochrane Database Syst Rev 2010;4:CD008487.
9. Cui Y, Liu Z, Zhang W. Acupuncture for Gilles de la Tourette's syndrome (Protocol). Cochrane Database Syst Rev 2009;2:CD007832.
10. Deng Y, Ma X, Peng W, Sanvito L. Acupuncture for Guillain‐Barré syndrome (Protocol). Cochrane Database Syst Rev 2009;2:CD007797.
11. Yang J, Feng Y, Ying L, et al. Acupuncture for hypertension (Protocol). Cochrane Database Syst Rev 2010;11:CD008821.
12. Wong V, Cheuk DKL, Chu V. Acupuncture for hypoxic ischemic encephalopathy in neonates (Protocol). Cochrane Database Syst Rev 2009;3:CD007968.
13. Dodin S, Asselin G, Blanchet C, et al. Acupuncture for menopausal hot flushes. Cochrane Database Syst Rev 2008;4:CD007410.
14. Cui Y, Zhishun L, Marchese M, et al. Acupuncture for multiple sclerosis (Protocol). Cochrane Database Syst Rev 2010;1:CD008210.
15. He J, Zheng M, Zhang M, Jiang H. Acupuncture for mumps in children (Protocol). Cochrane Database Syst Rev 2010;3:CD008400.
16. Wei M, Liu JP, Li N, Liu M. Acupuncture for myopia in children and adolescents (Protocol). Cochrane Database Syst Rev 2009;2:CD007842.
17. Wang E, Saxena M. Acupuncture for neuropathic pain in adults (Protocol). Cochrane Database Syst Rev 2011;2:CD008988.
18. Li J, Lu Y, Shi YM, et al. Acupuncture for overweight or obese people (Protocol). Cochrane Database Syst Rev 2010;7:CD008612.
19. Zhu X, Hamilton KD, McNicol ED. Acupuncture for pain in endometriosis (Protocol). Cochrane Database Syst Rev 2009;3:CD007864.
20. Wang L, Xie Y, Zhang S, Wu T. Acupuncture for Parkinson's Disease (Protocol). Cochrane Database Syst Rev 2006;4:CD006236.
21. Lim DCE, Xue CC, Wong FWS, et al. Acupuncture for polycystic ovarian syndrome (Protocol). Cochrane Database Syst Rev 2009;2:CD007689.
22. Wang P, Zhao J, Wu T. Acupuncture for postherpetic neuralgia (Protocol). Cochrane Database Syst Rev 2009;2:CD007793.
23. Liang CM, Peng W, Ma XJ. Acupuncture for post‐stroke upper limb pain (Protocol). Cochrane Database Syst Rev 2011;4:CD009087.
24. Yu J, Liu B, Liu Z, et al. Acupuncture for premenstrual syndrome (Protocol). Cochrane Database Syst Rev 2005;2:CD005290.
25. Zhao T, Zhang R, Zhao H. Acupuncture for symptomatic treatment of diabetic peripheral neuropathy (Protocol). Cochrane Database Syst Rev 2006;4:CD006280.
26. Manheimer E, Eaton K, Lao L, et al. Acupuncture for the treatment of post‐operative pain (Protocol). Cochrane Database Syst Rev 2006;2:CD006042.
27. Li Y, Zeng RF, Zheng D. Acupuncture for tinnitus (Protocol). Cochrane Database Syst Rev 2009;4:CD008149.
28. Kim TH, Lee MS, Kim KH, et al. Acupuncture for treating acute ankle sprains in adults (Protocol). Cochrane Database Syst Rev 2011;4:CD009065.
29. Deare JC, Zheng Z, Xue CC, et al. Acupuncture for treating fibromyalgia (Protocol). Cochrane Database Syst Rev 2008;2:CD007070.
30. Liu JP, Fei Y, Alraek T. Acupuncture for treatment of erectile dysfunction (Protocol). Cochrane Database Syst Rev 2008;3:CD007241.
31. Lui S, Li C, Xia J, Terplan M. Auricular acupuncture for opiate dependence in substance misuse treatment programmes (Protocol). Cochrane Database Syst Rev 2009;4:CD008043.
