Original Article
Recommendations for reporting randomized controlled trials of herbal interventions: explanation and elaboration

https://doi.org/10.1016/j.jclinepi.2005.12.020Get rights and content

Abstract

Controlled trials that use randomized allocation are the best tool to control for bias and confounding in trials testing clinical interventions. Investigators must be sure to include information that is required by the reader to judge the validity and implications of the findings in the reports of these trials. In part, complete reporting of trials will allow clinicians to modify their clinical practice to reflect current evidence toward the improvement of clinical outcomes. The consolidated standards of reporting trials (CONSORT) statement was developed to assist investigators, authors, reviewers, and editors on the necessary information to be included in reports of controlled clinical trials. The CONSORT statement is applicable to any intervention, including herbal medicinal products. Controlled trials of herbal interventions do not adequately report the information suggested in CONSORT. Recently, reporting recommendations were developed in which several CONSORT items were elaborated to become relevant and complete for randomized controlled trials of herbal medicines. We expect that these recommendations will lead to more complete and accurate reporting of herbal trials. We wrote this explanatory document to outline the rationale for each recommendation and to assist authors in using them by providing the CONSORT items and the associated elaboration, together with examples of good reporting and empirical evidence, where available, for each. These recommendations for the reporting of herbal medicinal products presented here are open to revision as more evidence accumulates and critical comments are collected.

Introduction

Randomized controlled trials (RCTs) provide the best evidence for efficacy of healthcare interventions [1]. Carefully planned and well-executed RCTs give us the best estimates of treatment effect and can thus guide clinical decision making [2], [3], although trials that lack methodological rigor cause over- or underestimation of treatment effect sizes due to bias or confounding factors [4], [5], [6], [7], [8], [9]. Hence, efforts have been undertaken toward improving the design and reporting of RCTs [1], [6], [10], [11].

Current research suggests that reporting quality of complementary and alternative medicine (CAM) trials is poor [12], [13]. Linde et al. [12], [14] found that most CAM trials do not describe the generation of the random sequence, an adequate method of allocation concealment, and the number and reasons for drop outs and withdrawals [12]. Moher et al. [9], [13] reported that a sample of pediatric CAM RCTs reported less than 40% of the consolidated standards of reporting trials' (CONSORT) checklist items with a 24% increase in the number of checklist items included in reports over time. That is, less than half of all information necessary in the reporting of these trials appeared in their reports. Specifically, only 50% of trials reported how random numbers were generated and 25% if allocation concealment was done [13]. The results suggest that a large proportion of CAM trials have poor reporting quality resulting in difficulties with assessment of internal and external validity [12], [13].

Linde et al. (2001) showed that reporting quality may vary across different types of complementary therapies with herbal medicine1 trials being somewhat superior to homeopathy and acupuncture trials [12], although, several systematic reviews state that trials of botanical medicine still fail to report information necessary to judge internal validity, external validity, and reproducibility [15], [16] (Gagnier, JJ., 2003, unpublished data). A study examining the quality of reports of a sample of 206 English language herbal medicine RCTs found that less than 45% of the information suggested within the CONSORT statement was reported [17]. For example, approximately 28% of trials described if the person administering the intervention was blinded to group assignment or not, only 22% described the methods for implementing the allocation sequence, and 21% the method for generating the allocation sequence. Also, reporting quality differed between individual botanical medicines and improved across decades from the 1980s to the 2000s [17]. Furthermore, it has been suggested that trials often do not include detailed information on the herbal product itself [15] (Gagnier, JJ., 2003, unpublished data).

It is known that herbal medicines may vary by part of plant used, time of harvest, active constituent levels, type of extract (aqueous, alcoholic, glycerin), and delivery form. Therefore, results of clinical trials on heterogeneous products may vary considerably even if they are using the same botanical species. Variation in herbal products between trials precludes pooling in systematic reviews of herbal medicines since invalid inferences may result from the combined data [14], [15] (Gagnier, JJ., 2003, unpublished data). It is clear that readers, editors, and reviewers require increased transparency in the reporting of RCTs of botanical medicines. Reporting guidelines for controlled clinical trials have been developed.

The CONSORT statement was first published in 1996 and revised in 2001 [7], [8]. This statement consists of a checklist and flow diagram to guide writers and reviewers on the information that should be available from published reports of two-group parallel RCTs [7], [8]. The CONSORT statement has been endorsed by many leading medical journals, editorial associations, professional societies, and funding agencies [9]. Since its inception, several extensions of the CONSORT statement have been developed [10], [18]. Recently, CONSORT was extended to cluster randomized trials [18] and for trials examining harms [10]. Also, an international group of acupuncture researchers developed a set of recommendations for improving reporting of the interventions in parallel group trials of acupuncture—the Standards for Reporting Interventions in Controlled Trials of Acupuncture or STRICTA [11]. Although not a formal extension of CONSORT, MacPherson et al. [11] described STRICTA as an elaboration of item 4 in CONSORT and suggest STRICTA be used together with CONSORT in reporting acupuncture trials.

In June 2004, an international group of trialists, methodologists, pharmacologists, and pharmacognosists met for a consensus meeting in Toronto, Canada that led to the development of recommendations for the reporting of herbal medicine trials [19] (see Table 1, Table 2). The resulting guidelines amounted to a set of elaborations of current CONSORT items that will aid editors and reviewers in assessing the internal/external validity and reproducibility of herbal medicine trials, allowing an accurate assessment of safety and efficacy.

During the development of the elaborations it became clear that an explanation of the concepts within and underlying the elaborations would aid researchers in planning, conducting, and writing reports of RCTs of herbal medicines. In the current paper, we discuss the rationale and scientific background for each elaboration and provide examples of good reporting for each. Where possible, we discuss empirical evidence for each. It should be noted that each elaboration is an addition to existing CONSORT recommendations. Thus all CONSORT items are first listed in Table 1 and elaborations in Table 2.

Section snippets

CONSORT elaborations for trials of herbal interventions

Where necessary, we have elaborated items to describe information suggested in reports of RCTs of herbal medicinal product interventions. When reporting an RCT of an herbal medicine, authors should consider reporting the information outlined in the CONSORT statement in addition to the information suggested in these elaborations. Below, we list original CONSORT items (in normal type) followed by elaborations (in italicized text) for RCTs of herbal interventions. Excerpts from trials include

Participants

Item 3:

Eligibility criteria for participants and the settings and locations where the data were collected.

If a traditional indication is being tested, a description of how the traditional theories and concepts were maintained. For example, participant inclusion criteria should reflect the theories and concepts underlying the traditional indication.

Example:

There were altogether 118 cases, which were randomly divided into two groups: The QTG (Qingluo Tongbi Granules: A Chinese herbal medicine)-

Baseline data

Item 15:

Baseline demographic and clinical characteristics of each group. Including, concomitant medication use or herbal medicinal product use.

Example:

Eight patients (mean age 44.9 (SEM 4.2) years) received feverfew and nine (mean age 51.2 (2.3) years) received placebo capsules. The patients in the active group had taken 2.44 (0.2) small leaves of feverfew daily for 3.38 (0.58) years before entry to the study, and those in the placebo group had taken 2.33 (0.48) small leaves daily for 4.18

Interpretation

Item 20:

Interpretation of results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes. Interpretation of the results in light of the product and dosage regimen used.

Example:

Although EGb 761 is generally used at a dose of 120 mg/day in treating chronic disease states, we chose to administer the extract at more than twice its usual dose, but for only 5 days before the operation, to cope with the

Comments

Randomized allocation is the best tool to control for bias and confounding in controlled trials testing clinical interventions. Investigators must be sure to include in reports of these trials information that is required by the reader to judge the validity and implications of the findings. In part, complete reporting of trials will allow clinicians to accurately appraise studies so as to modify their clinical practice to reflect current evidence. The CONSORT statement was developed to assist

Focus group participants

The individuals listed below participated in the premeeting phone calls or attended the consensus meeting and provided input toward the elaborations to existing CONSORT checklist items.

Doug Altman (Cancer Research UK Medical Statistics Group, Centre for Statistics in Medicine, Oxford, UK); Joanne Barnes (Centre for Pharmacognosy and Phytotherapy, The School of Pharmacy, University of London, London, UK); Claire Bombardier—Meeting Chair (Department of Health Policy Management and Evaluation,

Acknowledgments

We would like to thank Greer Palloo for aiding in the preparation for the June meeting and Jaime DeMelo and Cyndi Gilbert for assisting Joel Gagnier and Claire Bombardier during the meeting procedures.

This study was funded in part by an operating grant from the Canadian Institutes of Health Research, Clinical Trials Divisions; Grant number: ATF-66679. Dr. Gagnier is supported by a postgraduate fellowship from the Canadian Institutes of Health Research and the Natural Health Products Directorate.

References (77)

  • S. Bent et al.

    A randomized controlled trial of a Chinese herbal remedy to increase energy, memory, sexual function and quality of life in elderly adults in Beijing, China

    Am J Med

    (2003)
  • C.D. Gardner et al.

    The effect of a garlic preparation on plasma lipid levels in moderately hypercholesterolemic adults

    Atherosclerosis

    (2001)
  • D.G. Altman et al.

    The revised consort statement for reporting randomized trials: explanation and elaboration

    Ann Intern Med

    (2001)
  • C.D. Mulrow et al.

    Systematic reviews: critical links in the great chain of evidence

  • D.L. Sackett et al.

    Evidence-based medicine: how to practice and teach EBM

    (1998)
  • K.F. Schulz et al.

    Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials

    JAMA

    (1995)
  • D. Moher et al.

    Assessing the quality of randomized controlled trials: current issues and future directions

    Int J Technol Assess Health Care

    (1996)
  • C.B. Begg et al.

    Improving the quality of reporting of randomized controlled trials: the CONSORT statement

    JAMA

    (1996)
  • D. Moher et al.

    The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials

    JAMA

    (2001)
  • The CONSORT Group
  • J.P.A. Ioannidis et al.

    Improving the reporting of harms in randomized trials: expansion of the CONSORT statement

    Ann Int Med

    (2004)
  • H. MacPherson et al.

    Standards for Reporting Interventions in Controlled Trials of Acupuncture: The STRICTA Recommendations

    Acupunct Med

    (2001)
  • K. Linde et al.

    The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture

    Int J Epidemiol

    (2001)
  • D. Moher et al.

    Assessing the quality of reports of randomized trials in pediatric and complementary and alternative medicine

    BMC Pediatrics

    (2002)
  • K. Linde et al.

    Systematic reviews of complementary therapies—an annotated bibliography. Part 2: Herbal medicine

    BMC Complement Altern Med

    (2001)
  • Gagnier, JJ, VanTulder, M, Bombardier, C, Burman, B. Botanical medicine for low back pain. Cochrane Review (Protocol...
  • C. Little et al.

    Herbal therapy for treating rheumatoid arthritis

    Cochrane Library

    (2000)
  • J.J. Gagnier et al.

    Quality of reports of randomized controlled intervention trials of herbal medicines

    Am J Med

    (2006)
  • M.K. Campbell et al.

    CONSORT statement: extension to cluster randomized trials

    BMJ

    (2004)
  • J.J. Gagnier et al.

    Reporting randomized controlled trials of herbal interventions: an elaborated CONSORT statement

    Ann Int Med

    (2006)
  • J.A. Mix et al.

    A double-blind, placebo-controlled, randomized trial of Ginkgo biloba extract EGb 7611 in a sample of cognitively intact older adults: neuropsychological findings

    Hum Psychopharmacol Clin Exp

    (2002)
  • J.L. Isaacsohn et al.

    Garlic powder and plasma lipids and lipoproteins: a multicenter, randomized, placebo-controlled trial

    Arch Int Med

    (1998)
  • L.S. Murphy et al.

    Searching biomedical databases on complementary medicine: the use of controlled vocabulary among authors, indexers and investigators

    BMC Complement Altern Med

    (2003)
  • B. McGregor

    Medical indexing outside the National Library of Medicine

    J Me Libr Assoc

    (2002)
  • F. Kronenberg et al.

    A comprehensive information resource on traditional, complementary, and alternative medicine: toward an international collaboration

    J Altern Complement Med

    (2001)
  • S. Pietri et al.

    Ginkgo biloba extract (EGb 761) pretreatment limits free radical-induces oxidative stress in patients undergoing coronary bypass surgery

    Cardiovasc Drugs Ther

    (1997)
  • J. Savulescu et al.

    Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability

    BMJ

    (1996)
  • F.M. Ho et al.

    Effect of acupuncture at Nei-Kuan on left ventricular function in patients with coronary artery disease

    Am J Chin Med

    (1999)
  • Cited by (168)

    • Future directions for the discovery of natural product-derived immunomodulating drugs: an IUPHAR positional review

      2022, Pharmacological Research
      Citation Excerpt :

      Weaknesses include small sample size, short duration and non-appropriate information on the mixture under investigation (e.g. the part of the plant used, the Latin name of the plant, solvent/type of extraction and standardization of the mixture) ([207,226]. Reporting recommendations have been developed in which several CONSORT items were elaborated to become relevant and complete for randomized controlled trials of herbal medicines [229]. In conclusion, the current method of drug discovery from plants is not easily adaptable to mixtures, being it typically devoted to reducing complexity and identifying single active plant constituents for drug development.

    • Prioritized research recommendations and potential solutions: addressing gaps surrounding dietary supplement ingredients for boosting brain health and optimizing cognitive performance

      2021, Nutrition Research
      Citation Excerpt :

      Guidance and education on the importance of baseline measurements to understand results reported could close this gap. Guidelines exist for reporting requirements such as the Consolidated Standards of Reporting Trials (CONSORT) [30], which include various extensions to guide researchers in reporting of herbal interventions [31,32]. According to CONSORT guidelines, researchers should display a table showing baseline demographic and clinical characteristics for each group when reporting on a clinical trial.

    View all citing articles on Scopus
    1

    Members of this group are listed in Section 8.

    View full text