Objectives: Since 2005, the International Committee of Medical Journal Editors (ICMJE) requires randomized controlled trials registration on a public platform as a condition for publication. Study outcomes must be disclosed before enrolment of the first patient. Any protocol modification should be duly declared and justified thereafter. We aimed to evaluate if outcomes of phase III trials regarding the novel widely-prescribed antithrombotic agents were duly registered. To do so, we searched for major registration issues by assessing the data found on the public platform ClinicalTrials.gov and in the corresponding publications.
Methods: We systematically searched on Pubmed for more-than-a-thousand-patient phase III randomized controlled trials on the following novel antiplatelet and anticoagulant drugs: prasugrel, ticagrelor, cangrelor, dabigatran, rivaroxaban, apixaban and edoxaban. The included trials had to be published in high impact factor journals clearly requesting compliance with the ICMJE guidelines. Enrolment had to have started after 2005.
Results: We identified 38 trials published between 2008 and 2017 registered on the public platform ClinicalTrials.gov. Any protocol modification after Publication Date were observed in 31 (82%) trials. Twenty-eight (74%) had Primary Efficacy Outcomes Registration Issues. From these 28 trials, 11 (29%) had no Primary Efficacy Outcome registered before randomization of the first patient. Eight (21%) studies had either no Primary Efficacy Outcome registered or had Primary Outcome discrepancies by Primary Completion Date. Modification of the Time Frame during which the Primary Efficacy Outcomes were assessed occurred in 21 (55%) studies. Secondary Outcomes were not adequately registered for 32 (84%) of the trials.
Conclusion: Despite clearly requesting compliance with ICMJE registration guidelines, high impact factor journals still publish phase III trials that are inadequately or incompletely registered.
Keywords: Antithrombotic Drugs, ClinicalTrials, PubMed.
The Randomized Controlled Trial (RCT) is the preferred modality to test for a treatment’s efficacy in modern evidence-based medicine. Its results might be altered by changing the outcomes definition, modifying the time window of their measure or withholding the publication of some results. These outcome-reporting biases are now well recognized. They involved the “selection of a subset of the original variables recorded, on the basis of the results, for inclusion in the publication of trials”. Eloquent examples in the past have shown the repercussions in clinical practice [2-4]. The US Congress authorized in 2000 the creation of the public registry ClinicalTrials.gov by the National Institute Health to offer more transparency regarding trial methodology. In 2004, Chan et al published in the CAMJ the results of a study where they compared protocols of randomized trials approved for funding by the Canadian Institutes of Health Research (formerly the Medical Research Council of Canada) from 1990 to 1998 with subsequent reports of the trials identified in journal publications
Primary outcomes differed between protocols and publications for 40% of the trials. Statistically significant efficacy outcomes had higher odds than nonsignificant efficacy outcomes of being fully reported (odds ratio 2.7; 95%; confidence interval 1.5-5.0) . Given these results, The International Committee of Medical Journal Editors (ICMJE) began requiring registration as a condition of publication for trials starting recruitment after July 1, 2005 . In 2007, the US Congress expande ClinicalTrials.gov’s mandate by passing the Food and Drug Administration Amendments Act (FDAAA) . A similar legislation was established by the European Medicine Agency (EMA) . Finally, in 2008, the World Medical Association (WMA) revised the declaration of Helsinki to include prospective trials registration as an ethical principle in medical research. Recent studies have consistently outlined poor compliance with the trial-registration policies in various areas of medicine [9-14]. The object of this study is to assess the compliance to the ICMJE guidelines in a set of phase III studies of the popular and expensive novel antithrombotic agents.
Methods Trials Identification On May 17, 2017, we used PubMed to search for all reports of RCTs regarding the following widely-used novel antiplatelet and anticoagulant drugs: prasugrel, ticagrelor, cangrelor dabigatran, rivaroxaban, apixaban and edoxaban. Strategies were the same for each medication. We used the drug name, the filter for RCTs and the presence of an abstract (see Supplementary Appendix 1). Each abstract was screened to select relevant records. Included trials needed to be more-than-a-thousand-patients phase III RCTs. Enrolment had to have started after 2005. Selected trials had to be published in high impact factor (> 10.000 at trial’s publication date) journals stating that they followed the ICMJE recommendations (journals had to be listed in the ICJME’s website) . It was decided that a ClinicalTrials.gov identification number had to be found for each record. Including other registration-platforms would add heterogeneity to the sample and might render the analysis more difficult. Substudies and commentaries related to other trials, phase I, phase II and phase IV RCTs, meta-analyses, observational studies and ongoing trials were excluded. As it would have been surprising for such major trials not to be recorded on PubMed, we therefore searched a single database. We assumed that all high impact relevant trials should be recorded there.
Data Extraction Data extraction was conducted by one investigator (FC) and revised by the others (CL, JLL). The Publication Date, the Journal’s Name, the Principal Sponsor and the investigated condition were identified in each publication. The following data was then extracted from ClinicalTrials.gov: Study Start Date; Trial’s Registration Date on ClinicalTrials.gov; Primary Completion Date (date of database lock); Primary Efficacy Outcomes Registration Date; Primary Efficacy Outcomes Last Modification Date; Time Frame Registration Date and Last Protocol Modification Date. The Last Protocol Modification Date indicates the moment of last recorded protocol update on ClinicalTrials.gov, regardless of the modification made. The Time Frame Registration Date refers to date when the most updated information regarding the Time Frame of the Primary Efficacy Outcomes could be found on ClinicalTrials.gov by Primary Completion Date. Other relevant definitions can be found in the Supplementary Appendix 2. Since the exact dates were not always available, only the month and year were recorded. Finally, the Primary Efficacy Outcomes and corresponding Time Frame as well as the Number of Secondary Outcomes registered on ClinicalTrials.gov and those in the publications were systematically retrieved for comparison.
Choice of Registration Issues We decided to report registration discrepancies that were both objective and clinically relevant. We divided them into the following groups which can be found in Table 1.
They are not mutually exclusive. Primary Efficacy Outcomes Definition Discrepancies Two separate investigators (FC, CL) independently identified discrepancies between the originally registered Primary Efficacy Outcomes on ClinicalTrials.gov and those found in the corresponding publications. A discrepancy was considered an issue when it could favour different results by switching patients from having an event to not having an event, or vice-versa. Disagreements were resolved after a conference with a third party (JLL).
Seven hundred seventy-four records in PubMed were obtained with the previously-described research strategy. After eliminating duplicates and applying the inclusion and exclusion criteria, 36 publications remained that matched 38 ClinicalTrials.gov identification numbers. Two articles reported the results of two RCTs (RE-MEDY and RE-SONATE; ENSTEIN-DVT and EINSTEIN-EXTENSION). The trial selection flowchart may be consulted in Figure 1. A comprehensive description of the selected trials is shown in Table S1 of the supplementary appendix. All studies were multi-centric phase III RCTs sponsored by the industry and published between 2008 and 2017 in high impact factor journals. The investigated conditions were Acute Coronary Syndrome, Percutaneous Coronary Intervention, Atrial Fibrillation, Venous Thromboembolism, Peripheral Artery Disease and Cerebrovascular Disease.
Extracted Data Analysis The main findings of this study are presented in Table 2. In 31 (82%) records, protocol modifications were made on ClinicalTrials.gov after the Publication Date. Primary Efficacy Outcomes Registration Issues were present in 28 (74%) of the studied trials. This includes Timing and Time Frame registration issues as well as outcome registration issues. Eleven (29%) records had a Primary Efficacy Outcomes Registration Timing Issue. Indeed, seven (18%) trials were not registered on ClinicalTrials.gov by the Study start Date, while four (13%) studies had no Primary Efficacy Outcomes registered. Discrepancies between the Primary Efficacy Outcomes found in ClinicalTrials.gov and those in the corresponding publications occurred in 8 (21%) studies. None of these discrepancies were corrected on ClinicalTrials.gov by the Primary Completion Date. In half of these studies, Primary Efficacy Outcomes were modified, while on the other half, no outcome was provided. Discrepancies identification is provided in Table 3. Moreover, 21 (55%) trials had a Primary Efficacy Outcomes Time Frame Registration Issue. The registered time frame was either imprecise, not recorded or not consistent with the publication (see Table S2 of the Supplementary Appendix). The Number of Secondary Outcomes registered in ClinicalTrials.gov was discordant with the number found in the corresponding publication for 32 (84%) of the trials. Records had on average three (range 0-11; median 2) Secondary Outcomes registered on ClinicalTrials.gov by the Primary Completion Date, while it was ten (median: 9; minimum: 1; maximum: 27) in the publication (and/ or in the “Results” section of ClinicalTrials.gov). This situation can be seen in Table S3 of the Supplementary Appendix.
This study assesses compliance to ICMJE registration guidelines in a sample of major practice-changing trials of antithrombotic drugs. Most importantly, 74% of the of the studied trials presented Primary Efficacy Outcomes Registration Issues. We also found that a majority of records were updated on ClinicalTrias.gov after the trial’s publication. We believe most of these changes to be clarifications of minor issues that have no bearing on study results. Finally, the high discrepancy between the Number of Secondary Outcome registered in ClinicalTrials.gov and in the corresponding publications might be explained by the fact that some of the published Secondary Outcomes are actually exploratory or ancillary analyses. In some occasions, modifications in the protocol of ongoing trials are done and are legitimate as long as they are properly explained and justified.
Major protocol modifications have occurred in the past while trials were ongoing. We have no reason to believe that the discrepancies detected in this study reflect an intent to deceive readers. We do not know whether the eventual results would have been different from those published if the outcomes initially registered in ClinicalTrials.gov had been used in the publication’s statistical analyses. We are aware oof the complexity of trials registration especially in the case of multicentric international studies. Our study has some limitations. As we used a convenience sample, our results cannot be generalized to the whole world of clinical research. We did not analyse protocols published separately by investigators since the purpose of this study was mainly to assess the compliance with the ICMJE registration guidelines. Moreover, we did not conduct any analysis to estimate the risk of outcome-reporting bias and its eventual effects on the results. Finally, some outcome modifications could represent clarifications having no effect on the results. In this regard, no conclusion regarding the effect of the registration issues on the trials’ validity can be done.
Prospective outcome registration is instrumental to prevent outcome-registration biases in randomized controlled trials. Proper clinical trials registration offers transparency for physicians and could impact on clinical decisions. This study underlines that the evaluation of the outcome-reporting bias while assessing a trial can be difficult for physicians. Despite requesting compliance with ICMJE registration guidelines, our results suggest that high impact factor journals still publish phase III trials that are inadequately or incompletely registered.
Knowing this, a further study is needed to assess whether specific outcome registration issues are associated with statistically significant results and to assess the risk of an outcome reporting bias for a given outcome registration issue. Moreover, comparison with prior trials could then be done to analyse the evolution of the problematic in time.
Authorship Statement: Frédéric Chalut, Camille Laflamme and Jacques LeLorier participated to the conception and design of the study. Frédéric Chalut and Camille Laflamme conducted the acquisition and analysis of the data. Frédéric Chalut drafted the manuscript, which was revised by Camille Laflamme and Jacques LeLorier. All of the authors approved the final version to be manuscript and agreed to act as guarantors of the work.
Funding: This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Conflict of Interest: The authors have no conflicts of interest to declare.
1. Higgins JPT, Altman DG, Sterne JAC (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011.
2. Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008; 358:252-260.
3. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007; 356: 2457-2471.
4. Curfman GD, Morrissey S, Drazen JM. Expression of concern: Bombardier et al., "Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis," N Engl J Med 2000;343:1520-8. N Engl J Med. 2005; 353:2813-2814.
5. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004; 171:735-740.
6. De Angelis CD, Drazen JM, Frizelle FA, et al. International Committee of Medical Journal Editors. Is this clinical trial fully registered?--A statement from the International Committee of Medical Journal Editors. N Engl J Med. 2005; 352:2436-2438.
7. Food and Drug Administration Amendments Act of 2007. Public Law No. 110-85: 801; 2007.
8. Communication from the Commission regarding the guideline on the data fields contained in the clinical trials database provided for in Article 11 of Directive 2001/20/EC to be included in the database on medicinal products provided for in Article 57 of Regulation (EC) No 726/2004. In: European Commission, ed. Official Journal of the European Union, 2008. (2008/C 168/02)
9. Mathieu S, Boutron I, Moher D, et al. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009; 302:977-984.
10. Zarin DA, Tse T, Williams RJ, et al. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011; 364:852-860.
11. Gill CJ. How often do US-based human subjects research studies register on time, and how often do they post their results? A statistical analysis of the ClinicalTrials.gov database. BMJ Open. 2012; 2:e001186.
12. Anderson ML, Chiswell K, Peterson ED, et al. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med. 2015; 372:1031-1039.
13. Jones CW, Keil LG, Holland WC, et al. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015; 13:282.
14. Pranic S, Marušic A. Changes to registration elements and results in a cohort of ClinicalTrials.gov trials were not reflected in published articles. J Clin Epidemiol. 2016; 70:26- 37.
15. ICMJE. Journals staging that they follow the ICMJE recommandations. 2018.