I mentioned in my previous post, an issue with meta-analyses; there have been several I have read recently which are very problematic. They seem to be produced by groups that have little concern for the quality of their product.

This recent meta-analysis, for example, of transfusion thresholds in the preterm, stated that they found 12 RCTs (Fu et al Ann Hematol 2023). But it includes a trial (Widness JA, et al. Reduction in Red Blood Cell Transfusions Among Preterm Infants: Results of a Randomized Trial With an In-Line Blood Gas and Chemistry Monitor. Pediatrics. 2005;115(5):1299–306.) which was not a trial of differing transfusion thresholds, it was a trial of an in-line device to reduce blood sampling. It also includes data twice from the PINT trial, from the original publication and then includes them again from the long-term follow up publication. Some of the data appear to be made up, PINT never published the “hemoglobin after transfusion” data which are in the first Forest plot, and for which they give different results for the primary PINT publication and the long term outcome publication! According to their fictitious numbers, the Hgb after transfusion was almost identical in the 2 groups (124.9 vs 125.7), whereas, in reality, PINT achieved a clear difference in Hgb concentrations between randomized groups. There were 5 very small Chinese trials included in that SR/MA, only 1 of which I can access (Chen H-L, et al. Effect of Blood Transfusions on the Outcome of Very Low Body Weight Preterm Infants under Two Different Transfusion Criteria. Pediatr Neonatol. 2009;50(3):110–6). The others have no entry in PubMed, and I can’t find them when searching the Chinese national database.

This article is free access, which means that someone paid about $4600 US to put this misleading nonsense on line. Springer journals commonly publish poor quality articles under their pay-to-publish model, and really, if any peer reviewer worth his salt had read this, it is immediately evident that there are huge issues. Just to take one other minor example, they state that the ETTNO trial did not describe the means of randomization, I guess the SR authors just didn’t read the methods which actually describes them in more than the usual detail : “The random sequence was computer generated with variable block size (2-10) using the software RandList version 2.1 (DatInf)”.

They also have weighted some outcomes in a way that the small, unobtainable Chinese trials (n of between 70 and 180) have much more weight in the analysis of duration of oxygen (for example) than the large ETTNO trial (n>1000). This is presumably because of the minuscule SD of the data from those trials, for example Wang 2013 apparently showed a duration of oxygen in the liberal transfusion group of 14 days (SD2) compared to 18 days (SD3). This study of 86 babies has a 40% weight in the analysis as a result, compared to ETTNO, given an 11% weight, probably because the SD of the duration of oxygen therapy is realistic, 50 days (SD33). My guess is that the supposed SD of duration of O2 therapy in Wang, and the other trials with extremely narrow distributions, is actually an SEM, but as the articles are inaccessible there is on way to check that.

To explain further, continuous outcomes in meta-analyses are usually weighted by the inverse of the variance. This is done so that articles with more precision in their estimates (usually the larger trials) have more impact on the calculated overall mean effect. When the variance (however it is reported) is very small, then the article might have an outsized impact on the MA, which is why it is so important to be sure that the data are reliable, and that the reported variability in the data is really a SD, and not a SEM.

If the analyses were redone, giving appropriate weight to the larger trials, then there would be no impact of transfusion threshold on respiratory outcomes.

This matters. Individual carers could give transfusions to preterm infants with the expectation that they will shorten the duration of oxygen therapy, or positive pressure respiratory support, based on this erroneous SR/meta-analysis.

Recently, when I do a lit search, I often find more Systematic Reviews and Meta-Analyses that there are original trials. I think there are academics who think its so much easier to just recycle the results of someone else’s research than to perform a trial themselves.

Reputable journals should be very careful about publishing SR/MA. They should ensure that the SR was registered, and follows PRISMA guidelines and ensure that they are not just re-performing reviews that have already been well done. They should require that the authors provide pdf copies of the original trial publications with the submission, so that peer reviewers can verify the accuracy of what is being presented. Peer reviewers should ensure that the articles included really exist, that they are trials of the intervention being evaluated, and that the results are accurately analysed.

A related issue is the question of whether the original data are reliable or not. I have read, and reviewed, articles which seem to have been written by AI, and which are probably entirely fictitious. Others have probably skewed their results to be more positive, or have reported different outcomes to those planned when they found something interesting post hoc. A new tool has been developed to try and counter these issues, called INSPECT-SR, which is available as a preprint. (Wilkinson J, et al. INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. medRxiv. 2025:2025.09.03.25334905). The tool gives multiple checks to perform when writing an SR, as an attempt to eliminate data which are not reliable. It is a very sad that the integrity of published trials has to be questioned, but it is a reality of our current state of affairs.

Determining the integrity of a Systematic review is even more difficult, as one often does not have access to the original trials, to see if they have been accurately interpreted. The 2 SRs that I have recently criticized, one about Caffeine in the newborn, and this one about transfusions, are both addressing issues for which I was an author of one of the major included trials. My involvement made it immediately obvious to me that there were serious errors in interpretation, and that the SR/MA was very flawed. Systematic reviews of other issues, that I have had less direct involvement with, may have been just as flawed, but it could have escaped my notice.

There is much pressure in some academic circles to “publish or perish”, and to get something, anything, in to print. In some countries medical students are expected to publish an article prior to being awarded their MD degree. In others, junior academics cannot advance unless the combined weight of their output,when printed, exceeds a certain number of kg (or at least it seems that way). Journals now have a major interest in publishing anything that is submitted along with a cheque. Springer seem to be particularly egregious among the older established publishers, but some newer groups, like the Frontiers journals and MDPI have an extremely uneven profile, some of their titles being clearly predatory pay-to-publish journals, and others having higher standards.

It is incumbent on us, in the present day, to be sceptical of everything that we read, primary research and SRs. Pre-registration of trials and SRs, and data sharing are essential to ensure the integrity of the research on which we base our clinical decisions for critically sick babies.