Support for the All trials campaign has been growing, but progress seems to be slow. There are still many trials that are not published, and the impacts on science based medicine are substantial.

Recently, two articles by former participants in trials have been published, a viewpoint that is very important for these campaigns, I believe.

The first is called “The Outcome of My Clinical Trial Is a Mystery” it is by someone who was part of a trial to evaluate an intracardiac device to close an ASD, the author says :

It’s more than a little unsettling to think that I’ll live for another 70 years or so with this device in my heart, but may never know what the research said about how it works.

The second is called “A Broken Covenant with Patients” from a woman who was part of a preliminary evaluation of a drug for ovarian cancer, and who gives other examples of “broken covenants”.

As for sharing of data, the International Committee of Medical Journal Editors are proposing new rules, the meat of their proposal is here:

As a condition of consideration for publication of a clinical trial report in our member journals, the ICMJE proposes to require authors to share with others the deidentified individual-patient data (IPD) underlying the results presented in the article (including tables, figures, and appendices or supplementary material) no later than 6 months after publication. The data underlying the results are defined as the IPD required to reproduce the article’s findings, including necessary metadata.

I agree with this approach, and I plan to comply, there are details to be worked out, but one that I think is important is how will this be policed and enforced? Many rules of ICMJE are not well followed, including, for example, the requirement for registration of trials, and that publications should follow CONSORT guidelines. There is no point adding new rules if they are just going to be ignored, as happens already with data sharing, in those situation where it is already required.

As one example of this failure you could look at the COMPare website. Ben Goldacre has, with a few colleagues, started to review the randomized controlled trials published in 5 leading medical journals. His team compare the outcomes published in the articles describing the research results to those available in the trial registration documents, or in a published protocol, if published before the trial started. Of course the outcomes described should be those that were initially planned, and if for some reason that was not the case, then any difference should be explained in the trial report. Changing outcomes for a trial means that it cannot necessarily be trusted to answer the question that was initially asked. That is particularly true when the primary outcome is changed when the investigators look at the results. Clearly, if the initially planned primary outcome is not significantly different between groups, but the authors find something else which looks cool, and then claim that was the primary outcome, the risks of making type 1 errors are enormously increased.

Journals are not doing a very good job of comparing reported outcomes to the pre-specified outcomes, nor of insisting that CONSORT standards are followed, and any changes clearly described and justified. The COMPare group as of today have checked 67 trials, they have found only 9 of them reported all the outcomes that were planned. There were 301 outcomes that were planned to be reported by the trials that were not reported, and there were 357 outcomes that appeared for the first time in the published articles.

The editors of the Annals of Internal Medicine have answered in a very haughty and inconsistent way to the letters from COMPare that pointed out the problems in several publications. At one point pointing out their “long experience”, as if that somehow makes them less fallible.

I can only hope that the COMPare team are successful in making journals check publications more carefully for consistency with the CONSORT standards. There are already some signs that they are having an impact.