Undisclosed Changes in Outcomes in Randomized Controlled Trials: An Observational Study ======================================================================================= * Robert Ewart * Harald Lausen * Norman Millian ## Abstract **PURPOSE** We wanted to investigate the frequency of undisclosed changes in the outcomes of randomized controlled trials (RCTs) between trial registration and publication. **METHODS** Using a retrospective, nonrandom, cross-sectional study design, we investigated RCTs published in consecutive issues of 5 major medical journals during a 6-month period and their associated trials registry entries. Articles were excluded if they did not have an available trial registry entry, did not have analyzable outcomes, or were secondary publications. The primary outcome was the proportion of publications in which the primary outcome of the trial was, without disclosure, changed between that recorded in the trial registry and that reported in the final publication. The secondary outcome was the proportion of publications in which the secondary outcome was changed without disclosure. **RESULTS** We reviewed 158 reports of RCTs and included 110 in the analysis. In 34 (31%), a primary outcome had been changed, and in 77 (70%), a secondary outcome had been changed. **CONCLUSIONS** There are substantial and important undisclosed changes made to the outcomes of published RCTs between trial registration and publication. This finding has important implications for the interpretation of trial results. Disclosure and discussion of changes would improve transparency in the performance and reporting of trials. * Endpoints * outcomes * trials registration *You want more scepticism at the very foundation of your work. Scepticism, the tonic of minds, the tonic of life, the agent of truth, the way of art and salvation.* Joseph Conrad, in a letter to John Galsworthy, 1901 ## INTRODUCTION The primary outcome (or endpoint) is an essential element of a randomized controlled trial (RCT), and specification of outcomes is central to the design and reporting of these trials. Changing the outcomes of a clinical trial without explanation calls its validity into question. To allow authors to change their outcomes without reason or disclosure threatens the underpinnings of the scientific enterprise and threatens the ability of practicing physicians to apply published research to patient care. Acceptable reasons to change outcomes include results of other trials, intertrial comparisons, theoretical advances in the relevant field, and technical advances (for example, in measurement or treatment). These changes present few problems if they are reported and interpreted appropriately. Unfortunately, outcomes may be changed in ways that violate the fundamental principles of trial design and interpretation1 and, therefore, raise doubts about the validity of the conclusions. These changes, if made, should be carefully reasoned, limited in number, and clearly labeled as hypothesis generating rather than hypothesis supporting. During the past several years efforts have been made to encourage “full transparency with respect to performance and reporting of clinical trials,”2 because the “credibility of research depends on a critical assessment by others of the strengths and weaknesses in study design, conduct, and analysis.”3 As one part of this larger endeavor, most major medical journals and many funding agencies now require that trial details be registered. Registration includes the “definitions of the primary and secondary outcome measures.”2 The purpose of our current research is to estimate the extent to which outcomes are changed, without disclosure in the final report, between trial registration and publication. How prevalent is this problem of changing outcomes? Chan and colleagues4,5 and Hahn and colleagues6 independently have looked at changes in outcomes that occur between trial protocols and publication (or nonpublication). This approach is likely to give an accurate estimate of outcome changes, because researchers are required to submit amendments to trial protocols to their institutional review boards. Protocols, however, are not usually available (although Hawkey7 has argued that they should be), making it nearly impossible for a typical practicing physician to critique trials using this information. Trial registries are an almost ideal solution. They are easy to find on the Internet, and the information is well-organized and concise. This system provides easier access and better transparency for the typical journal reader. With this background in mind, we set out to examine how often outcomes are changed between trial registration and final publication. ## METHODS Our primary outcome was the proportion of trials in which the primary outcome of the trial under consideration had been changed without disclosure from registration to publication. Our secondary outcome was the proportion of trials in which the secondary outcome of the trial under consideration had been changed without disclosure from registration to final publication. To determine the frequency of changes in outcomes, we assembled consecutive issues of 5 major medical journals. These 5 journals were chosen because they have high impact factors and are widely considered of high quality. We identified all reports of randomized controlled trials for a 6-month period. We searched each report for a reference to a trials registry. We then excluded trials for the following reasons: there was no publicly accessible trial registration recorded in the report; the registration database did not record a primary outcome or the outcome recorded was too vague to make any judgments; and the trial publication was not the main report of the trial results. If there were multiple reports from the same trial, we used what appeared to be the main one. If multiple reports considered different outcomes of the same study, we either combined them and considered them together as one study or considered them as separate studies depending on the circumstances. We kept a database of all excluded trials. For each included report we extracted the primary and secondary outcomes as listed in the registration database and in the final publication (or, if necessary, in the associated design and methods publication). We did not search the archives of the trials registries to see whether the outcomes had been changed since registration. Rather, we took them as is on the day we searched. For quality assurance, each of the 3 authors reviewed all the articles independently. The few disagreements were resolved easily by consensus after reviewing the articles a second time. The authors were not masked to the identity of the research reports. We considered outcomes to have been changed only if there were substantive differences, rather than minor differences in wording that did not appear to change the intent of the trial. We counted outcomes as unchanged if the authors acknowledged the change and made any statement indicating that the changes were made before any analyses were done. Secondary outcomes were counted as unchanged if the authors said they would be published separately (although we made no attempt to confirm event). As this sample of articles was not generated randomly, the assumptions underlying the calculation of confidence intervals do not hold; therefore, the confidence intervals presented here (calculated by the modified Wald method and at the 95% level8) should be interpreted cautiously. ## RESULTS We found a total of 158 RCTs in the 5 journals (Table 1⇓). Forty-eight were excluded, and 110 were included in the analysis. Trials were registered in 5 databases. Table 2⇓ gives further details of the included and excluded trials, and Table 3⇓ gives further details on the registry databases. View this table: [Table 1.](http://www.annfammed.org/content/7/6/542/T1) Table 1. **Study Flow (N = 158)** View this table: [Table 2.](http://www.annfammed.org/content/7/6/542/T2) Table 2. **Journals Publishing Trials Reports Reviewed in the Study** View this table: [Table 3.](http://www.annfammed.org/content/7/6/542/T3) Table 3. **Databases of Reported Trials** Thirty-four reports (31%; 95% confidence interval [CI], 23%–40%) had a change in one or more primary outcomes. Seventy-seven reports (70%; 95% CI, 61%–78%) had a change in one or more secondary outcomes. For the primary outcomes, most changes were by deletion. For the secondary outcomes, the changes were more evenly split between addition of outcomes and deletion. Table 4⇓ expands on these results. Although not part of our research question, we noted that there were almost no differences in outcomes when comparing trials funded by pharmaceutical companies with those that had noncommercial sponsorship. In Table 5⇓ are 3 examples of changes made between registration and publication. These were representative of the different types of changes we encountered. View this table: [Table 4.](http://www.annfammed.org/content/7/6/542/T4) Table 4. **Summary of Outcome Changes of Included Studies (n = 110)** View this table: [Table 5.](http://www.annfammed.org/content/7/6/542/T5) Table 5. **Examples of Changes Made Between Trial Registration and Publication of Findings** ## DISCUSSION Several biases could have effected our results. It is possible that the outcomes in the trial registry were incorrect. For example, difficulty in database formats, errors in data entry, misperception of registry importance, or deliberate falsification could all invalidate the entries. Trials registries are not useful, however, if they do not provide an accurate representation as to what was intended. It is also possible that the authors of the articles had made changes in their registry entries after data analysis but before publication, and our search would not have uncovered this change. During our data-checking process, we found 3 instances in which authors had changed their registry entries after trial publication. In all 3 cases, the original registry entry had not listed any outcomes,15–17 but the subsequent changes listed outcomes identical to the final publications.18–20 If these sorts of changes were common, we would have underestimated the frequency of outcome changes. Further, our selection of articles was not random. We selected articles from what are often considered better journals. This focus could have resulted in a lower percentage of changed outcomes (as these articles are presumably reviewed more closely before publication) or a higher percentage (as the financial stakes may be higher with a correspondingly higher incentive to produce good results). Finally, our exclusion criteria were subjective, and other researchers reviewing our database might make slightly different decisions. Although almost all of the decisions on changes in outcomes were straightforward, other reviewers of these same data might arrive at slightly different results. We doubt, however, that the conclusions would change. Outcomes can be changed either by addition, deletion, or by some combination of the 2. The issue with addition of outcomes is that it asks the question after the answer is known. Deletion, while more subtle, may also turn a statistically insignificant result into a significant one. When a study has multiple outcomes, one may be significant by chance alone. In the past, authors have been tempted to mine data for statistically significant outcomes after a study was completed. It is well-understood that such activity is unacceptable.21,22 Instead, authors may be tempted to list multiple outcomes in advance and then eliminate those that were not significant—in essence, data mining in reverse. Any use of multiple outcomes requires *P* value adjustments to account for chance findings of statistical significance. Reducing the number of outcomes can, therefore, make an outcome appear significant where it would not have been before the reduction. Although there is considerable disagreement in the statistical literature about the best approach to doing these adjustments, there is no argument about the necessity.23 We suggest several potential remedies for the issues discussed here. (1) The International Committee of Medical Journal Editors24 requirement that authors register the World Health Organization Minimal Registration Data Set25 should be enforced by all databases (or all journals). For example, the American registry does not require the registration of any outcomes,26 an omission that is easily remedied. (2) Trial registration databases should have the archive of each trial linked directly to the trial registration, which would make it easier to follow changes. (3) Our investigation was about transparency, not misrepresentation. We believe most changes were made in good faith and without an understanding of the issues involved. To aid authors in avoiding these problems, the next revision of the CONSORT statement27 should require reporting of more details about trial registration and changes that have been made therein.28 (4) Journal editors should expect that trials be registered at the outset and certainly before any interim statistical analyses are performed. (5) Reviewers should routinely check the trial protocol, the registry, and any quoted design and methods publications to ensure that the authors have not altered their outcomes without appropriate comment and justification. Although our results must be regarded as only approximate, they nevertheless indicate a substantial and important lack of transparency in the reporting of clinical trials. ## Acknowledgments The authors wish to acknowledge the contributions of Ms Joan Stolz, who reviewed drafts of this article and made helpful editorial comments. The authors also wish to acknowledge the constructive comments of anonymous reviewers of numerous earlier drafts of this article. ## Footnotes * *Conflicts of interest: none reported* * Received for publication March 26, 2008. * Revision received March 5, 2009. * Accepted for publication March 6, 2009. * © 2009 Annals of Family Medicine, Inc. ## REFERENCES 1. Evans S. When and how can endpoints be changed after initiation of a randomized clinical trial? PLoS Clin Trials. 2007;2(4):e18. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1371/journal.pctr.0020018&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17443237&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) 2. De Angelis C, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. [http://www.icmje.org/clin\_trial.pdf](http://www.icmje.org/clin_trial.pdf) Accessed Feb 3, 2007. 3. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–577. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.7326/0003-4819-147-8-200710160-00010&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17938396&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000250437100007&link_type=ISI) 4. Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–2465. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.291.20.2457&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=15161896&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000221599700028&link_type=ISI) 5. Chan A-W, Krleza-Jeri K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–740. [Abstract/FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNzEvNy83MzUiO3M6NDoiYXRvbSI7czoyMjoiL2FubmFsc2ZtLzcvNi81NDIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 6. Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8(3):353–359. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1046/j.1365-2753.2002.00314.x&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=12164983&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000177369000009&link_type=ISI) 7. Hawkey CJ. Journals should see original protocols for clinical trials. [Letter]. BMJ. 2001;323(7324):1309. [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMjMvNzMyNC8xMzA5IjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS83LzYvNTQyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 8. Measuring usability: quantitative usability, statistics & six sigma by Jeff Sauro. Web page. [http://www.measuringusability.com/wald.htm](http://www.measuringusability.com/wald.htm). Accessed May 26, 2008. 9. Pfizer. Effect and safety of detrol LA in men with overactive bladder symptoms with or without bladder outlet obstruction. In: Clinical-Trials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct/show/NCT00147654](http://www.clinicaltrials.gov/ct/show/NCT00147654). Accessed May 25, 2008. 10. Kaplan SA, Roehrborn CG, Rovner ES, Carlsson M, Bavendam T, Guan Z. Tolterodine and tamsulosin for treatment of men with lower urinary tract symptoms and overactive bladder: a randomized controlled trial. JAMA. 2006;296(19):2319–2328. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1001/jama.296.19.2319&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17105794&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) 11. European Organization for Research and Treatment of Cancer. Radiation therapy, surgery, and chemotherapy in treating patients with rectal cancer that can be surgically removed. In: ClinicalTrials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct/show/NCT00002523](http://www.clinicaltrials.gov/ct/show/NCT00002523). Accessed May 25, 2008. 12. Bosset JF, Collette L, Calais G, et al.; EORTC Radiotherapy Group Trial 22921. Chemotherapy with preoperative radiotherapy in rectal cancer. N Engl J Med. 2006;355(11):1114–1123. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa060829&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16971718&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000240462800006&link_type=ISI) 13. University of Minnesota. National Institute of Dental and Craniofacial Research (NIDCR). Obstetrics and periodontal therapy (OPT) study. In: ClinicalTrials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct2/show/NCT00066131](http://www.clinicaltrials.gov/ct2/show/NCT00066131) Accessed May 28, 2008. 14. Michalowicz BS, Hodges JS, DiAngelis AJ, et al; OPT Study. Treatment of periodontal disease and the risk of preterm birth. N Engl J Med. 2006;355(18):1885–1894. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa062249&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17079762&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) 15. Pronyk PM, Hargreaves JR, Kim JC, et al. Effect of a structural intervention for the prevention of intimate-partner violence and HIV in rural South Africa: a cluster randomised trial. Lancet. 2006;368(9551):1973–1983. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1016/S0140-6736(06)69744-4&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17141704&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000242593300029&link_type=ISI) 16. Bussel JB, Kuter DJ, George JN, et al. AMG 531, a thrombopoiesis-stimulating protein, for chronic ITP. N Engl J Med. 2006;355(16): 1672–1681. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa054626&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17050891&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000241357300008&link_type=ISI) 17. National Institute of Allergy and Infectious Diseases (NIAID). CQSP in Malawi: chloroquine and sulfadoxine-pyrimethamine efficacy for the treatment of malaria in Malawi. In: ClinicalTrials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct2/show/NCT00125489](http://www.clinicaltrials.gov/ct2/show/NCT00125489). Accessed May 26, 2008. 18. Sanofi-Aventis. Safety and efficacy of enoxaparin in percutaneous coronary intervention (PCI) patients, an international randomized evaluation (STEEPLE). In: ClinicalTrials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct2/show/NCT00077844](http://www.clinicaltrials.gov/ct2/show/NCT00077844). Accessed May 26, 2008. 19. National Institute of Allergy and Infectious Diseases (NIAID). Islet cell transplants for diabetes. In: ClinicalTrials.gov [Internet]. Bethesda, MD: National Library of Medicine. [http://www.clinicaltrials.gov/ct2/show/NCT00014911](http://www.clinicaltrials.gov/ct2/show/NCT00014911). Accessed May 26, 2008. 20. Laufer MK, Thesing PC, Eddington ND, et al. Return of chloroquine antimalarial efficacy in Malawi. N Engl J Med. 2006;355 (19):1959–1966. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa062032&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17093247&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000241833000004&link_type=ISI) 21. Montalescot G, White HD, Gallo R, et al; STEEPLE Investigators. Enoxaparin versus unfractionated heparin in elective percutaneous coronary intervention. N Engl J Med. 2006;355(10):1006–1017. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa052711&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=16957147&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000240273300006&link_type=ISI) 22. Shapiro AM, Ricordi C, Hering BJ, et al. International trial of the Edmonton protocol for islet transplantation. N Engl J Med. 2006;355(13):1318–1330. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJMoa061267&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=17005949&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) 23. Tannock IF. False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst. 1996;88(3–4):206–207. [FREE Full Text](http://www.annfammed.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJqbmNpIjtzOjU6InJlc2lkIjtzOjEwOiI4OC8zLTQvMjA2IjtzOjQ6ImF0b20iO3M6MjI6Ii9hbm5hbHNmbS83LzYvNTQyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 24. Mills JL. Data torturing. N Engl J Med. 1993;329(16):1196–1199. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1056/NEJM199310143291613&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=8166792&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=A1993MA66700013&link_type=ISI) 25. Ludbrook J. Multiple comparison procedures updated. Clin Exp Pharmacol Physiol. 1998;25(12):1032–1037. [CrossRef](http://www.annfammed.org/lookup/external-ref?access_num=10.1111/j.1440-1681.1998.tb02179.x&link_type=DOI) [PubMed](http://www.annfammed.org/lookup/external-ref?access_num=9888002&link_type=MED&atom=%2Fannalsfm%2F7%2F6%2F542.atom) [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000077928400011&link_type=ISI) 26. International Committee of Medical Journal Editors. Statement of purpose. In: *Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication.* [http://www.icmje.org/index.html#state](http://www.icmje.org/index.html#state). Accessed Jun 26, 2008. 27. International Committee of Medical Journal Editors. Table 1: minimal registration data set. In: *Is This Clinical Trial Fully Registered?: A Statement from the International Committee of Medical Journal Editors.* [http://www.icmje.org/clin\_trialup.htm#table1](http://www.icmje.org/clin_trialup.htm#table1). Accessed May 26, 2008. 28. ClinicalTrials.gov Registration requirements. [http://prsinfo.clinicaltrials.gov/](http://prsinfo.clinicaltrials.gov/). Accessed May 26, 2008. 29. CONSORT Statement web site. [http://www.consort-statement.org/](http://www.consort-statement.org/). 30. Goldbeck-Wood S. Commentary: changes between protocol and manuscript should be declared at submission. BMJ. 2001;322: 1460–1461. [Web of Science](http://www.annfammed.org/lookup/external-ref?access_num=000169389200027&link_type=ISI)