…oh and by the way, we are the manufacturers making the product in question, so we had objective scientific reasons for doing this, no conflict of interest whatever.

The commercial sponsors of a negative trial decided that they didn’t like the results, so they have re-analyzed it. Two of the 3 authors of the new publication, including the corresponding author, are employees of the company that makes the product tested in the original trial. The original trial found no effect on the duration of intravenous feeding (TPN) in preterm babies randomized to receive a human milk based breast milk fortifier, compared to standard fortifier. So they analyzed the same data in a different way, and lo and behold, found that they were significantly different after all!!

So let’s have a look at the original trial.

The original trial was a multi-center RCT of 3 different regimes of enteral nutrition in infants 500 to 1250 g birth weight, whose mothers wanted to give breast milk. One group received a standard cows-milk-based breast milk fortifier, the other two groups received a new fortifier derived from donor human milk, one of which started the fortifier earlier than the other.  There were 207 babies in total, planned to be 69 per group. The primary outcome variable was duration of TPN (intravenous nutrition). The study was negative, there was no difference in duration of TPN between the groups.

On first reading the trial appears to have been well designed, and well performed (but way too small). I do have problems with the way it was presented. Even the title was misleading (An Exclusively Human Milk-Based Diet Is Associated with a Lower Rate of Necrotizing Enterocolitis than a Diet of Human Milk and Bovine Milk-Based Products).

It would have been more accurate to use a title such as ‘An Exclusively Human Milk-Based Diet has no effect on TPN duration, but Secondary Outcomes show Possible Marginally Significant Reduction in Necrotizing Enterocolitis compared to a Diet of Human Milk and Bovine Milk-Based Products’.

That would be  a more accurate, but less sexy title.

The original study showed a possibly significant (more of that later) difference in the frequency of one of a large number of secondary outcomes: namely Necrotising Enterocolitis. So the title of the publication, and the abstract, and the discussion,  focused on this outcome. You can’t help but wonder if that focus had something to do with the fact that the study was funded by, and several of the authors were employees of, the company that makes the human milk based fortifier.

I once was co-author of a study where the first draft of the publication emphasized the interesting secondary outcome; it did not mention in the title the primary outcome, and concentrated in the abstract on the secondary outcome before describing the negative primary. I understand the desire to do this but it should be resisted by authors and by reviewers. We changed our title, abstract, and presentation to concentrate on the primary outcome, while still describing and discussing the secondary outcome. in the text.

Studies are designed to have power to detect a difference in the primary outcome. They are very rarely powered to be able to study 2 outcomes. So a positive secondary outcome must always be considered suspect. Especially if you examine multiple secondary outcomes. Important secondary outcomes do need to be measured and reported, but they must be considered exploratory and preliminary, needing confirmation.

The original publication of the Sullivan et al study found that the frequency of NEC in one HM (human milk based fortifier) group was 3/67, in the 2nd HM group it was 5/71, and with breast-milk receiving infants who got a cow’s, bovine, milk based fortifier (BOM) the incidence was 11/69. Firstly, that is a very high rate of stage 2 NEC in the breast-milk fed BOM controls (a 17% rate of NEC in preterm babies of this gestation receiving breast milk is highly unusual), so it may just be a quirk. Secondly, I have put these numbers in my software, and if you put them in as 3 groups, they actually are not significant. If you compare the 2 combined HM groups to the BOM group then the p value is 0.03, which is what was reported in the paper. That may be an appropriate way to analyze the data, as long as you planned it that way from the start. The study is way underpowered to compare 3 groups for any of their outcomes.

The trial registration documents state that the sample size was supposed to be 260; and although I don’t find the documents very clear, there were supposed to be 5 groups, the 3 mentioned here and 2 other groups which were of babies who do not receive mothers milk, randomized to either donor human milk preparation plus HM fortifier or premie formula.

NEC was not listed as a secondary outcome variable in the registration page of the website. This may be evidence of one of the limitations of trial registration, and of trial reporting. Maybe the website just doesn’t include all of the pre-specified secondary outcomes. I think CONSORT should require that registration documents are supplied with the final manuscript and any discrepancies be clearly described in the manuscript.

To be honest, I think this is a real shame, I think a human milk based fortifier is a good idea, and if we could really show that it was preferable I would be all in favour. But this study does not prove benefit, just a potential signal that needs to be appropriately investigated, with adequately powered studies, which would have to be much larger than this to have enough power to show a difference in NEC.

Also studies of NEC should be blinded. Diagnosis of NEC is quite subjective, and differentiating stage 1 from stage 2 requires identification of pneumatosis. There are at least 2 publications (here and here) which show that the inter-rater variability for diagnosing pneumatosis gives a Kappa score of between 0.2 and 0.3, which, to interpret for you, is lousy! There is a serious risk of bias in unmasked studies when the important outcome is so subjective. I can’t see any good reason why this study could not have been blinded.

If it were done correctly, and still proved that human milk based fortification reduced NEC then I would be first in line to get my hospital to pay for it. Until then I think we can still accept  the previous evidence from systematic review of 7 trials including 640 preterm infants, that cows-milk protein based fortifiers have no effect on the incidence of NEC.

I agree, your wording is preferable to what I put in the post, there is no strong evidence of adverse effects with the use of cow’s milk based fortifiers, but not enough evidence of safety. We certainly need better studies of fortification which include enough babies who are at risk of adverse outcomes to have the power to say whether they are safe or not.