Although this isn’t neonatal it is a great study with an unfortunate conclusion… There has long been controversy about the place of ECMO for adults with ARDS. To resolve this a French multicentre trial randomized adults who were very sick to ECMO or continuing standard therapy. (Combes A, et al. Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome. The New England journal of medicine. 2018;378(21):1965-75). Just under 1/3 of the patients were enrolled in non-ECMO centers, so the ECMO group was a transport and ECMO group for those patients.

After 2/3 of the sample size was enrolled the study was terminated “for futility”. They had hypothesized a 60% mortality in the controls and a 40% mortality in the ECMO group, for ethical reasons they allowed rescue ECMO in the controls if the physician felt they were still deteriorating and might benefit from ECMO.

At the point the trial was stopped, because they crossed the pre-specified ‘futility’ stopping boundary, there was a 35% mortality in the ECMO group, and a 46% mortality in the controls. 28% of the controls (35 patients) crossed over to ECMO, and of those 57% died.

I don’t know about you, but an 11% absolute reduction in mortality sounds huge to me, and if I was ever unfortunate enough to have severe ARDS, I think the ECMO group would suit me just fine.

Unfortunately, there will now be people saying there was no difference between the groups.  In fact one of the accompanying editorials in the FPNEJM says exactly that “the routine use of ECMO in patients with severe ARDS is not superior to the use of ECMO as a rescue maneuver in patients whose condition has deteriorated further”. That is not true, 35% mortality is superior to 46% mortality! The sentence should state “the routine use of ECMO in patients with severe ARDS is superior to standard care, with the use of ECMO as a rescue maneuver in patients whose condition has deteriorated further, but the difference does not meet conventional norms of statistical significance”.

Indeed if the 15 patients rescued by ECMO in the control group had been allowed to die without ECMO, and if they had indeed all died, then the difference would have been highly statistically significant (p=0.0004). You can see in the online supplement that those who crossed over were indeed extremely sick, all on 100% O2 with a PaO2 averaging 50 despite a PEEP of 10 and a PIP of 33 (Which would probably give an oxygenation index of about 100 if they had been babies!), so the idea that they might have almost all died without ECMO is reasonable, the mortality of this, rescue, group (57%) was substantially higher than the early ECMO group.

This was a very difficult trial to do, it took 6 years to get the 240-0dd patients enrolled, and the stopping boundary for futility was decided prior to starting the trial. I think the decision to stop can be justified, but I do wonder about the stopping boundary that was chosen. The trial showed a potentially substantial benefit of ECMO, and stopping a trial in those circumstances, even when the chance of showing p<0.05 at the end might be considered low, should be questioned. Realize that the relative risk of death was 0.76; a 24% relative reduction in mortality will now be considered “not significant”! I am in favour of sequential designs in general, they may preserve power while potentially decreasing sample size. The problem is in how the stopping boundaries are calculated. If a stopping boundary is calculated which leaves a good chance of a major clinically important benefit, like an 11% absolute mortality reduction, then we have to question the methods. The other editorial about the trial in the FPNEJM says some similar things Harrington D, Drazen JM. Learning from a Trial Stopped by a Data and Safety Monitoring Board. The New England journal of medicine. 2018;378(21):2031-2.

Perhaps the  best answer to all of this is to analyze trials using Bayesian methods, incorporating prior probabilities into the interpretation of new data to produce a posterior probability. At the end of which we can then say not “ECMO decreases mortality” but “the probability that ECMO decreases mortality is X, whereas the probability that ECMO increases mortality is Y”. A perfect example of exactly that technique is the late-hypothermia trial from the NICHD. (Laptook AR, et al. Effect of Therapeutic Hypothermia Initiated After 6 Hours of Age on Death or Disability Among Newborns With Hypoxic-Ischemic Encephalopathy: A Randomized Clinical Trial. JAMA. 2017;318(16):1550-60). The final conclusion of the trial could have been “p=NS, there is no effect of therapeutic hypothermia in this population”, but fortunately the authors had realized when planning the trial that adequate power for classical frequentist p-value testing would be unlikely and planned to perform a Bayesian analysis. That analysis showed that there was about a 75% chance that hypothermia was beneficial.

The authors of that cooling study have also published an article about how you can use Bayesian methods to inform the stopping of an RCT (Pedroza C, et al. Advantages of Bayesian monitoring methods in deciding whether and when to stop a clinical trial: an example of a neonatal cooling trial. Trials. 2016;17(1):335)

It would be interesting to think what would have happened with the ECMO trial if they had used Bayesian methods. Especially given the prior (lower quality) evidence that there may be a reduction in mortality of adults with ARDS who are treated with ECMO. The previous study “CESAR” has been criticized because only 75% of patients randomized to referral for ECMO actually were cannulated, and perhaps respiratory management (in both groups) was not optimal, but it did show a similar-sized advantage of ECMO referral over conventional management. The prior probability of an ECMO advantage for your Bayesian analysis should probably therefore already have favoured ECMO, and  Bayesian analysis might well have shown a high probability that ECMO is preferable to conventional management for adults with ARDS.

I have never done intensive care in adults, but I was a pediatric intensivist for many years as well as a neonatologist, and I cared for a couple of kids with ARDS that we put on ECMO, with good results. I think there is certainly a place for ECMO in such kids, and this trial, to me, confirms that they probably benefit, and probably have a better chance of surviving with ECMO, if they are sick enough to be at substantial risk of death.