Original ArticleRecommendations for reporting randomized controlled trials of herbal interventions: explanation and elaboration
Introduction
Randomized controlled trials (RCTs) provide the best evidence for efficacy of healthcare interventions [1]. Carefully planned and well-executed RCTs give us the best estimates of treatment effect and can thus guide clinical decision making [2], [3], although trials that lack methodological rigor cause over- or underestimation of treatment effect sizes due to bias or confounding factors [4], [5], [6], [7], [8], [9]. Hence, efforts have been undertaken toward improving the design and reporting of RCTs [1], [6], [10], [11].
Current research suggests that reporting quality of complementary and alternative medicine (CAM) trials is poor [12], [13]. Linde et al. [12], [14] found that most CAM trials do not describe the generation of the random sequence, an adequate method of allocation concealment, and the number and reasons for drop outs and withdrawals [12]. Moher et al. [9], [13] reported that a sample of pediatric CAM RCTs reported less than 40% of the consolidated standards of reporting trials' (CONSORT) checklist items with a 24% increase in the number of checklist items included in reports over time. That is, less than half of all information necessary in the reporting of these trials appeared in their reports. Specifically, only 50% of trials reported how random numbers were generated and 25% if allocation concealment was done [13]. The results suggest that a large proportion of CAM trials have poor reporting quality resulting in difficulties with assessment of internal and external validity [12], [13].
Linde et al. (2001) showed that reporting quality may vary across different types of complementary therapies with herbal medicine1 trials being somewhat superior to homeopathy and acupuncture trials [12], although, several systematic reviews state that trials of botanical medicine still fail to report information necessary to judge internal validity, external validity, and reproducibility [15], [16] (Gagnier, JJ., 2003, unpublished data). A study examining the quality of reports of a sample of 206 English language herbal medicine RCTs found that less than 45% of the information suggested within the CONSORT statement was reported [17]. For example, approximately 28% of trials described if the person administering the intervention was blinded to group assignment or not, only 22% described the methods for implementing the allocation sequence, and 21% the method for generating the allocation sequence. Also, reporting quality differed between individual botanical medicines and improved across decades from the 1980s to the 2000s [17]. Furthermore, it has been suggested that trials often do not include detailed information on the herbal product itself [15] (Gagnier, JJ., 2003, unpublished data).
It is known that herbal medicines may vary by part of plant used, time of harvest, active constituent levels, type of extract (aqueous, alcoholic, glycerin), and delivery form. Therefore, results of clinical trials on heterogeneous products may vary considerably even if they are using the same botanical species. Variation in herbal products between trials precludes pooling in systematic reviews of herbal medicines since invalid inferences may result from the combined data [14], [15] (Gagnier, JJ., 2003, unpublished data). It is clear that readers, editors, and reviewers require increased transparency in the reporting of RCTs of botanical medicines. Reporting guidelines for controlled clinical trials have been developed.
The CONSORT statement was first published in 1996 and revised in 2001 [7], [8]. This statement consists of a checklist and flow diagram to guide writers and reviewers on the information that should be available from published reports of two-group parallel RCTs [7], [8]. The CONSORT statement has been endorsed by many leading medical journals, editorial associations, professional societies, and funding agencies [9]. Since its inception, several extensions of the CONSORT statement have been developed [10], [18]. Recently, CONSORT was extended to cluster randomized trials [18] and for trials examining harms [10]. Also, an international group of acupuncture researchers developed a set of recommendations for improving reporting of the interventions in parallel group trials of acupuncture—the Standards for Reporting Interventions in Controlled Trials of Acupuncture or STRICTA [11]. Although not a formal extension of CONSORT, MacPherson et al. [11] described STRICTA as an elaboration of item 4 in CONSORT and suggest STRICTA be used together with CONSORT in reporting acupuncture trials.
In June 2004, an international group of trialists, methodologists, pharmacologists, and pharmacognosists met for a consensus meeting in Toronto, Canada that led to the development of recommendations for the reporting of herbal medicine trials [19] (see Table 1, Table 2). The resulting guidelines amounted to a set of elaborations of current CONSORT items that will aid editors and reviewers in assessing the internal/external validity and reproducibility of herbal medicine trials, allowing an accurate assessment of safety and efficacy.
During the development of the elaborations it became clear that an explanation of the concepts within and underlying the elaborations would aid researchers in planning, conducting, and writing reports of RCTs of herbal medicines. In the current paper, we discuss the rationale and scientific background for each elaboration and provide examples of good reporting for each. Where possible, we discuss empirical evidence for each. It should be noted that each elaboration is an addition to existing CONSORT recommendations. Thus all CONSORT items are first listed in Table 1 and elaborations in Table 2.
Section snippets
CONSORT elaborations for trials of herbal interventions
Where necessary, we have elaborated items to describe information suggested in reports of RCTs of herbal medicinal product interventions. When reporting an RCT of an herbal medicine, authors should consider reporting the information outlined in the CONSORT statement in addition to the information suggested in these elaborations. Below, we list original CONSORT items (in normal type) followed by elaborations (in italicized text) for RCTs of herbal interventions. Excerpts from trials include
Participants
Item 3:
Eligibility criteria for participants and the settings and locations where the data were collected.
If a traditional indication is being tested, a description of how the traditional theories and concepts were maintained. For example, participant inclusion criteria should reflect the theories and concepts underlying the traditional indication.
Example:
There were altogether 118 cases, which were randomly divided into two groups: The QTG (Qingluo Tongbi Granules: A Chinese herbal medicine)-
Baseline data
Item 15:
Baseline demographic and clinical characteristics of each group. Including, concomitant medication use or herbal medicinal product use.
Example:
Eight patients (mean age 44.9 (SEM 4.2) years) received feverfew and nine (mean age 51.2 (2.3) years) received placebo capsules. The patients in the active group had taken 2.44 (0.2) small leaves of feverfew daily for 3.38 (0.58) years before entry to the study, and those in the placebo group had taken 2.33 (0.48) small leaves daily for 4.18
Interpretation
Item 20:
Interpretation of results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes. Interpretation of the results in light of the product and dosage regimen used.
Example:
Although EGb 761 is generally used at a dose of 120 mg/day in treating chronic disease states, we chose to administer the extract at more than twice its usual dose, but for only 5 days before the operation, to cope with the
Comments
Randomized allocation is the best tool to control for bias and confounding in controlled trials testing clinical interventions. Investigators must be sure to include in reports of these trials information that is required by the reader to judge the validity and implications of the findings. In part, complete reporting of trials will allow clinicians to accurately appraise studies so as to modify their clinical practice to reflect current evidence. The CONSORT statement was developed to assist
Focus group participants
The individuals listed below participated in the premeeting phone calls or attended the consensus meeting and provided input toward the elaborations to existing CONSORT checklist items.
Doug Altman (Cancer Research UK Medical Statistics Group, Centre for Statistics in Medicine, Oxford, UK); Joanne Barnes (Centre for Pharmacognosy and Phytotherapy, The School of Pharmacy, University of London, London, UK); Claire Bombardier—Meeting Chair (Department of Health Policy Management and Evaluation,
Acknowledgments
We would like to thank Greer Palloo for aiding in the preparation for the June meeting and Jaime DeMelo and Cyndi Gilbert for assisting Joel Gagnier and Claire Bombardier during the meeting procedures.
This study was funded in part by an operating grant from the Canadian Institutes of Health Research, Clinical Trials Divisions; Grant number: ATF-66679. Dr. Gagnier is supported by a postgraduate fellowship from the Canadian Institutes of Health Research and the Natural Health Products Directorate.
References (77)
- et al.
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Control Clin Trials
(1996) - et al.
Evaluation of commercial ginseng products
J Pharm Sci
(1978) - et al.
Amounts of feverfew in commercial preparations of the herb
Lancet
(1986) - et al.
Analysis of phenolic compounds and radical scavenging activity of Echinacea spp
J Pharm Biomed Anal
(2004) - et al.
American ginseng (Panax quinquefolius L.) attenuates postprandial glycemia in a time-dependent but not dose dependent manner in healthy individuals
Am J Clin Nutr
(2001) - et al.
Determination of pyrrolizidine alkaloids in commercial comfrey products (Symphytum sp.)
J Pharm Sci
(1994) Suggested guidelines for articles about botanical dietary supplements
Am J Clin Nutr
(2002)Some aspects of toxic contaminants in herbal medicines
Chemosphere
(2003)- et al.
Randomized double-blind placebo-controlled trial of feverfew in migraine prevention
Lancet
(1988) - et al.
Why use placebos in clinical trials? A narrative review of the methodological literature
J Clin Epidemiol
(2000)
A randomized controlled trial of a Chinese herbal remedy to increase energy, memory, sexual function and quality of life in elderly adults in Beijing, China
Am J Med
The effect of a garlic preparation on plasma lipid levels in moderately hypercholesterolemic adults
Atherosclerosis
The revised consort statement for reporting randomized trials: explanation and elaboration
Ann Intern Med
Systematic reviews: critical links in the great chain of evidence
Evidence-based medicine: how to practice and teach EBM
Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials
JAMA
Assessing the quality of randomized controlled trials: current issues and future directions
Int J Technol Assess Health Care
Improving the quality of reporting of randomized controlled trials: the CONSORT statement
JAMA
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials
JAMA
Improving the reporting of harms in randomized trials: expansion of the CONSORT statement
Ann Int Med
Standards for Reporting Interventions in Controlled Trials of Acupuncture: The STRICTA Recommendations
Acupunct Med
The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture
Int J Epidemiol
Assessing the quality of reports of randomized trials in pediatric and complementary and alternative medicine
BMC Pediatrics
Systematic reviews of complementary therapies—an annotated bibliography. Part 2: Herbal medicine
BMC Complement Altern Med
Herbal therapy for treating rheumatoid arthritis
Cochrane Library
Quality of reports of randomized controlled intervention trials of herbal medicines
Am J Med
CONSORT statement: extension to cluster randomized trials
BMJ
Reporting randomized controlled trials of herbal interventions: an elaborated CONSORT statement
Ann Int Med
A double-blind, placebo-controlled, randomized trial of Ginkgo biloba extract EGb 7611 in a sample of cognitively intact older adults: neuropsychological findings
Hum Psychopharmacol Clin Exp
Garlic powder and plasma lipids and lipoproteins: a multicenter, randomized, placebo-controlled trial
Arch Int Med
Searching biomedical databases on complementary medicine: the use of controlled vocabulary among authors, indexers and investigators
BMC Complement Altern Med
Medical indexing outside the National Library of Medicine
J Me Libr Assoc
A comprehensive information resource on traditional, complementary, and alternative medicine: toward an international collaboration
J Altern Complement Med
Ginkgo biloba extract (EGb 761) pretreatment limits free radical-induces oxidative stress in patients undergoing coronary bypass surgery
Cardiovasc Drugs Ther
Are research ethics committees behaving unethically? Some suggestions for improving performance and accountability
BMJ
Effect of acupuncture at Nei-Kuan on left ventricular function in patients with coronary artery disease
Am J Chin Med
Cited by (168)
An expert Delphi study to derive minimum reporting set of indices for herbal medicine samples in non-clinical laboratory studies
2023, Journal of Herbal MedicineQuality of reporting of randomized controlled trials of top 10 herbs prescribed in Thailand
2023, Journal of Herbal MedicineFuture directions for the discovery of natural product-derived immunomodulating drugs: an IUPHAR positional review
2022, Pharmacological ResearchCitation Excerpt :Weaknesses include small sample size, short duration and non-appropriate information on the mixture under investigation (e.g. the part of the plant used, the Latin name of the plant, solvent/type of extraction and standardization of the mixture) ([207,226]. Reporting recommendations have been developed in which several CONSORT items were elaborated to become relevant and complete for randomized controlled trials of herbal medicines [229]. In conclusion, the current method of drug discovery from plants is not easily adaptable to mixtures, being it typically devoted to reducing complexity and identifying single active plant constituents for drug development.
Prioritized research recommendations and potential solutions: addressing gaps surrounding dietary supplement ingredients for boosting brain health and optimizing cognitive performance
2021, Nutrition ResearchCitation Excerpt :Guidance and education on the importance of baseline measurements to understand results reported could close this gap. Guidelines exist for reporting requirements such as the Consolidated Standards of Reporting Trials (CONSORT) [30], which include various extensions to guide researchers in reporting of herbal interventions [31,32]. According to CONSORT guidelines, researchers should display a table showing baseline demographic and clinical characteristics for each group when reporting on a clinical trial.