The updated Cochrane review of probiotics for prevention of NEC, sepsis and mortality has been published.

Meta-analysis showed that probiotics may reduce the risk of NEC: RR 0.54, 95% CI 0.45 to 0.65 (54 trials, 10,604 infants; I2 = 17%); RD -0.03, 95% CI -0.04 to -0.02; number needed to treat for an additional beneficial outcome (NNTB) 33, 95% CI 25 to 50. Evidence was assessed as low certainty because of the limitations in trials design, and the presence of funnel plot asymmetry consistent with publication bias. Sensitivity meta-analysis of trials at low risk of bias showed a reduced risk of NEC: RR 0.70, 95% CI 0.55 to 0.89 (16 trials, 4597 infants; I2 = 25%); RD -0.02, 95% CI -0.03 to -0.01; NNTB 50, 95% CI 33 to 100. Meta-analyses showed that probiotics probably reduce mortality (RR 0.76, 95% CI 0.65 to 0.89; (51 trials, 10,170 infants; I2 = 0%); RD -0.02, 95% CI -0.02 to -0.01; NNTB 50, 95% CI 50 to 100), and late-onset invasive infection (RR 0.89, 95% CI 0.82 to 0.97; (47 trials, 9762 infants; I2 = 19%); RD -0.02, 95% CI -0.03 to -0.01; NNTB 50, 95% CI 33 to 100). Evidence was assessed as moderate certainty for both these outcomes because of the limitations in trials design. Sensitivity meta-analyses of 16 trials (4597 infants) at low risk of bias did not show an effect on mortality or infection.

I find this extremely interesting, but also somewhat concerning. The recent network meta-analysis that I posted about (https://neonatalresearch.org/2020/08/24/probiotics-save-the-lives-or-preterm-infants-find-a-reliable-source) found the following:

Compared with placebo, a combination of 1 or more Lactobacillus species (spp) and 1 or more Bifidobacterium spp was the only intervention with moderate- or high-quality evidence of reduced all-cause mortality (odds ratio [OR], 0.56; 95% confidence interval [CI], 0.39-0.80). Among interventions with moderate- or high-quality evidence for efficacy compared with placebo, combinations of 1 or more Lactobacillus spp and 1 or more Bifidobacterium spp, Bifidobacterium animalis subspecies lactis, Lactobacillus reuteri, or Lactobacillus rhamnosus significantly reduced severe NEC (OR, 0.35 [95% CI, 0.20-0.59]; OR, 0.31 [95% CI, 0.13-0.74]; OR, 0.55 [95% CI, 0.34-0.91]; and OR, 0.44 [95% CI, 0.21-0.90], respectively).

The differences between the reviews, and between the interpretations of the evidence are fascinating. The Cochrane review did not include 12 trials which are included in the Network Meta-Analysis (NMA), with a total of 3580 subjects; most of those trials are listed in the excluded trials table as having been excluded because “most participants were not very preterm or VLBW”. Three of the trials in the NMA are not listed as excluded in the Cochrane review, one of them has 174 participants, and a mean GA of 29.5 weeks, and is probably eligible for inclusion in the Cochrane review, but has only been published as an abstract in conference proceedings, so may not have been found by their literature search. The other 2 trials, one large (n=524) and one small (n=62) appeared to be mostly larger preterm infants, so probably would have been excluded anyway.

The Cochrane review included 9 trials not in the NMA, it is not clear why they weren’t included, but those trials enrolled a total of 765 infants. Most are limited to VLBW infants, and many are not difficult to find (in JPGEN and PLOS1, for example).

The interpretation of the quality of the data are divergent, the NMA referring to moderate to high-quality data, while the Cochrane review refers to evidence of low certainty. In part, this is based on an analysis of the funnel plot, which looks a bit asymmetric and the statistical test for missing data was just below p=0.05. It is, of course, impossible to be sure if there is missing data or not, if you knew about it it wouldn’t be missing! The statistical test used has been evaluated by using simulations, which is I guess the only way to test such tests, but makes me a little uncertain how reliable it is.

The divergence of opinion also points out that there is some degree of subjectivity in deciding on the quality of the evidence.

Where I start to have concerns about the Cochrane review is that, when restricting the analysis to high-quality trials, there remains a major reduction in NEC, those trials number 16 with 4,597 infants enrolled. Also when analyzing mortality in only trials with a low risk of bias, they state that there was no difference, in fact, the mortality with probiotics was 5.9% and with placebo was 7%, which are 2 different numbers unless I am mistaken. You could say they are not statistically significantly different, or that there is a small difference which may be due to chance, the weighted RR from the meta-analysis is 0.86 (95% CI 0.69, 1.07).

I guess the main issue is : how confident do you have to be to introduce an intervention which has next to no risk, is very cheap, does not prevent you from introducing other interventions to reduce NEC risk, and which decreases NEC in a meta-analysis of high-quality trials? Even though the reductions in mortality and in invasive infection are not below a p-value of 0.05, the differences are in the right direction in the high quality trials.

When you add to the RCTs the real-world experience of introducing probiotics in multiple studies, from large databases in Germany, the USA, and Canada, and individual hospital experiences like ours, and Toronto and Norwich, I think it is hard to avoid the fact that probiotics are almost certainly effective in reducing NEC, and that large enough high-quality studies would likely show a decrease in mortality, which is already evident when the lower quality studies are included in the analysis. It seems likely to me that Bifidobacterium longum Subsp Infantis in a mixture with a Lactobacillus or another Bifido-may be the best, but that is less certain.

Do we really want to spend the next 2 million dollar grant for an RCT comparing probiotics to placebo? Surely cluster randomized trials comparing different preparations could be much more cost-efficient and could quickly give us much larger sample sizes, and would permit an answer to the question of which preparations are most effective.