A shiny new test

I have invented a shiny new test, which detects brain injury in preterm infants just before they go home. It can be used at discharge to predict whether a former extremely preterm baby will have developmental delay or neurologic impairment. The test has a threshold of 1 unit for increased risk of poor outcome, and 2 units is a really abnormal test.

When the result is below 2 (that is normal or mildly abnormal) about 5% of babies will have cerebral palsy, and 25 to 35 % will have delay in various domains. When the result is above 2 (moderately or severely abnormal) about 25% will have cerebral palsy, and about 40 to 45% will have developmental delay.

In other words, when the test is severely abnormal, most of the babies will not have delay or movement disorder (even though there is a statistically significant difference in outcomes between groups); I can express the results differently to try and convince you to use my shiny test: there is a highly significant linear correlation between test results and developmental screening test results at 2 years.

What I want to know is; are you buying? It only costs about 600$ (Canadian) per test.

Hopefully in the era of modern neonatology, before doing this test you would discuss it with the parents of each baby, they might well ask “Do you recommend the test, doctor?”

Well?

Extremely preterm infants have increased long term problems, and need to be followed up in focussed programs. Parents are generally aware of the increased risks of their babies by the time they are about to go home. For a pre-discharge test to help families it should reliably inform them of the likely outcome of their infant. In other words it should be both sensitive and specific. Most extremely preterm babies do well, but a test with low sensitivity will have a low positive predictive value for important long term outcomes.

For some outcomes (relatively rare outcomes such as critical congenital heart disease) a screening test with low positive predictive value is acceptable, there are more false positives than real positives (even though many of the positive results of pulse oximetry screening actually have problems that need addressing) and a 0.5% false positive rate, for a condition with an incidence of 1:10,000 does not create huge problems. Especially when the patients who are truly positive need urgent intervention to prevent serious complications.

But when the risk of a particular outcome is moderately common, a test with the same characteristics is much less acceptable, large numbers of families will be identified as being at risk, and most of them will end up OK. Plus, in the case of former preterm infants at discharge, you actually don’t change your intervention based on the test results.

You could actually get almost as good discrimination by sitting in the waiting room of the test station, pointing at all the babies and saying “You will probably be OK”.

Are you takers for my shiny new test?

What I am trying to get at, is just because a test slightly improves prediction of delay or motor issues, does not necessarily mean that all of our patients should have that test. To be useful you need a test that discriminates between babies who will need follow-up and intervention, and those that do not. I think my shiny new test might not be so useful after all.

What is my shiny new test?

It will be obvious to many of my readers that my shiny new test is a term-equivalent cerebral magnetic resonance image.

What I have described is where we are with predischarge MRI (and probably term equivalent head ultrasound also). Most babies with abnormalities on the MRI have good outcomes. The positive predictive value of anything seen on the MRI is less than 50%. MRIs are really good at finding imaging abnormalities, they are relatively useless at finding developmental problems! Which is as it should be… surely, MRIs can be spectacularly succesful at showing brain structure, mapping tracts and developing connections and showing how some brains are different to others, including the brains of former preterm babies.

A new publication from Melbourne of 186 babies under 30 weeks gestation (or under 1250 g) who had pre-discharge MRI illustrates much of this. (Anderson PJ, et al. Associations of Newborn Brain Magnetic Resonance Imaging with Long-Term Neurodevelopmental Impairments in Very Preterm Children. The Journal of pediatrics. 2017).

This group of infants had the usual amazing high quality follow up from Peter Anderson and their team, and it shows a statistically significant association between abnormal MRI and various aspects of their cognitive abilities at 7 years of age.

But because there is a significant association, that does not mean that it is useful to parents, or that everyone should have the MRI.

This new publication doesn’t present the data in a way that you can calculate the PPV or specificity. What it does show is that if you have a semi-objective MRI scoring system, in a group of former very preterm babies who are now 7 years of age, the mean IQ of those with a normal pre-discharge MRI is 100 (SD 15) those with moderate to severe abnormalities on the MRI it is 89 (SD 15). (For those with mild abnormalities the IQ was 97 (SD 15)).

To show it graphically here are the mean plus or minus 2SD, to show the range of results of 95% of the babies in each group (about).

I think these data are amazing, they show, perhaps a bit surprisingly, that pre-discharge MRI findings at 40 weeks PMA have a statistically significant association with standardized testing 8 years later, after 8 more years of growth and development and stimulation and education and love and warmth in a family, there are still measurable impacts of how the brain developed in those first few weeks of life.

But,  I fail to see how this information is of any value for individual parents, although the authors try to use their results to promote routine pre-discharge MRI. What are you supposed to say to parents when you have a moderate to severe abnormality on the term-equivalent pre-discharge MRI?

Here is an evidence based suggestion for what to say to a parent when you get back the MRI with severely abnormal scores.:

“Your babies predicted IQ at 8 years of age is 95% likely to be somewhere between 59 and 119, if the MRI had been normal the predicted IQ at 8 years of age would have been 95% likely to be between 70 and 130.”

Now if the pre-discharge MRI predicted a response to early intervention programs, for example, then maybe they would be of value, but I don’t think there is any evidence of that; what does seem to correlate with an advantage from early intervention is poorer social circumstances. Perhaps if the sensitivity and the negative predictive value were high enough, you could decide not to follow-up some babies, but, although the NPV is high for cerebral palsy in a few studies, it is not very good for developmental delay. Which means, I think, that all very preterm babies should be in focussed follow-up programs regardless of MRI findings.

The authors of the article state the following in the discussion:

This study confirms that newborn MRI identifies brain abnormalities in CWM, DGM, and CBL that have long-term impact on neurodevelopmental outcomes, independent of perinatal and social risk factors. Thus, quantitative evaluation of structural MRI obtained at term equivalent age provides valuable information for clinicians. Because discussion of neurodevelopmental prognosis with families before neonatal intensive care unit discharge is standard of care, and brain abnormality on MRI is the strongest neonatal predictor of long-term outcome, prognostic discussions with families should be informed about MRI findings alongside other clinical indicators.

As should be clear by now, I really disagree with these statements. Doing a screening test with low specificity in a lowish risk population leads to frequent conversations with parents about the fact that, despite these findings, their baby will probably be fine, I still don’t think that we have answered the question of whether this test (alongside many others that we do, such as term equivalent head ultrasound, especially in babies whose US was previously normal) and that conversation, actually helps families.

About keithbarrington

I am a neonatologist and clinical researcher at Sainte Justine University Health Center in Montréal
This entry was posted in Neonatal Research. Bookmark the permalink.

5 Responses to A shiny new test

  1. Gonzalo Mariani says:

    Thank you Keith, extraordinary clarity! I definitely agree. I think that we must choose the tests that help us make decisions for the benefit of our patients. If not, nothing better than an adequate follow-up and an empathic support to the family.
    Best regards

  2. Ruth Davidge says:

    Good morning Keith

    Thanks so much for your blog. I really enjoy it and recommend it to all! I am interested in predischarge MRI practically. We have just had an experience with a healthy newborn born with a coloboma of the eyelid. No other abnormalities apparent. However there has also been a film/scratches on the affected eye apparently unrelated to the coloboma. Anyway it was decided an MRI was required of both eyes to investigate if there was any other abnormality. It has been extremely hard to obtain this as even with sedation the baby couldnt remain asleep. It has now been decided she will need a general anaesthetic in order to under go the MRI and very few centres have capacity to do this. I am interested to know practically how babies are sedated in order to make routine predischarge MRI a viable suggestion? Thank you

  3. William Tarnow-Mordi says:

    Is there enough information in this impressive and stimulating paper for you to calculate the positive and negative Likelihood Ratios for us, Keith? When I attended the Oxford Evidence Based Medicine course in the late 1990s I learned that a positive likelihood ratio >10 (or a negative likelihood ratio of <0.1) for a test indicated a large and often conclusive increase (or decrease) in the likelihood of disease

    http://omerad.msu.edu/ebm/Diagnosis/Diagnosis6.html

    Best wishes to all

    William Tarnow-Mordi
    WINNER Centre for Newborn Research, NHMRC Clinical Trials Centre, University of Sydney

  4. Charles Christoph Roehr says:

    Dear Keith, sorry to trouble you, but I would dearly like to re-recommend one of your previous posts (possibly from 2014) to my trainees. It is the post about the TED talk by the very powerful lady Canadian TV presenter with tale of 2 doctors, where she reports how news was broken to her in 2 very different ways…. It is a great learning piece. By any chance, would you remember and have time to send me the link, please? Thank you so much, Charles 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s