Effect of Out-of-Hospital Pediatric Endotracheal Intubation on survival and neurological outcome

We have recently been asked to review an article published in the JAMA in 2000 concerning the utilization of endotracheal intubation versus bag-valve mask oxygenation in the pre-hospital emergency setting by Paramedics. The article entitled “Effect of Out-of-Hospital Pediatric Endotracheal Intubation on Survival and Neurological Outcome: A Controlled Clinical Trial” is anything but a truly “Controlled Clinical Trial” [1].

The study protocol professes to have a method of randominization based upon whether the pediatric patient was seen on an odd or even day. True randomization requires that entry into a treatment wing requires at a minimum a “blinded” study, if not “double-blinded” the later of which clearly could not be done in this setting. Given the duration of shifts for paramedical personnel being 24-hours in duration, this raises a question regarding fatigue, which could be argued to be balanced with those receiving bag-valve mask (BVM) therapy being equally affected; although the more sophisticated procedure of endotracheal intubation (ETI) would undoubtedly be more affected due to fatigue factors. No, to be truly randomized, the study would and should have required a mechanism where until paramedics arrived upon the scene, they would not know which procedure they would be using.

The second problem was both the training and experience level of both those teaching and those being trained. There is nothing to suggest that the paramedics were trained by someone who provided the level of expertise training a physician would have received; viz. those who are expected to be able to intubate patients. Additionally, while the paramedics had been receiving training in BVM, the training for ETI consisted of “two 3-hours educational sessions.” Hardly the qualification required for ETI expertise when putting someone’s life on the line.

The study goes on to discuss outcomes, outcomes, which don’t match the data. The authors state that the survival of those receiving ETI treatment had “significant(ly)” worse outcomes. They point to the survival differences between those determined to have had “respiratory arrest” and those children who were “mistreated.” While the odds ratios are listed and the authors discuss the importance of p-values being less than 0.05, there are no p-values to support this statement. Neither is there a p-value for the “foreign body aspiration” group the authors discuss.

The p-values, which do exist, provide real insight into the outcome of this study. First, there is one major difference between the demographics of the two groups. Those in the ETI group had a statistically higher rate (P<0.05) number of babies with Sudden Infant Death Syndrome (SIDS) compared with the BVM cohort. Despite the authors comments to the contrary, Table 5 shows there were no statistically increased complications for those undergoing ETI, despite what I would consider to be a lack of sufficient training for this study to have been conducted.

There are p-values comparing “median” time of transport and on scene time for paramedics and total time before arrival to the hospital emergency departments. Purportedly these times were recorded. The authors report “median” not mean + standard deviation times and there are no graphics to compare the two groups. Graphics clearly help us see what is really happening between two or more treatment groups. If you can’t see it, it isn’t real and here I am suspicious because the p-values of <0.01 are reportedly the result of “median” time differences; time differences of up to 120 seconds.

It’s hard to determine the true statistical significance of these 120 “seconds” without some graphic comparison and without mean + standard deviations. Medians are very poor measures of actual events. It would have been very interesting to know the results of capnography in these individuals. Such real life data in these critical settings would have been invaluable.

All of that said and done, you might be inclined to ignore our comments. You might be inclined to say the study is a landmark study and it is being used across the country to support the absence of training paramedics to intubate pediatric patients. You might be inclined to ignore our concerns, unless you were to consider that the primary author helped write the ACLS protocols for the American Heart Association. You might be inclined to ignore our concerns until you realize the primary author has helped as an Experienced Provider – ACLS and AHA Faculty member, train thousands of Physicians (colleagues, attendings, fellows, residents, interns and medical students), Nurses, Emergency Medical Personnel [Emergency Medical Technicians (EMTs) and Paramedics] whose levels of training are substantially different.

You might be inclined to ignore our concerns until you realize that the primary author is a Medical Reviewer for more than 15 peer reviewed medical journals, is on the Editorial Board of two others, has published 100 plus papers in peer reviewed medical journals, presented at more than 60 Conferences, have published 8 chapters in Medical Textbooks and been Editor-in-Chief of one Cardiology Textbook. You might be able to ignore our concerns until you realize the second author is a Nationally Registered Paramedic (NRP) following training at the UCLA Paramedic Program.

Yes, you might even be inclined to ignore our concerns, except for perhaps this one other final concern we have about the paper. The results that are shown are Intent to Treat (ITT) results, not the actual results of treatment. ITT is a cleaver way of not having to take a good hard look at the data and resolve problems with drop out, lost data, et cetera. In essence the author says, “Well they were treated the way they were supposed to be, even though we don’t really know what happened to them, we want to conclude results favorable to our premise!” While this is a cute statistical tool for what I consider sloppy science, it is not a tool that reflects reality.

The reality, based upon what’s really shown in the paper, is that there were really 506 pediatric patients treated with BVM and 187 with ETI, not 410 and 420 respectively. You can’t pretend you did something when you didn’t do it and you can’t pretend to provide best outcomes based medicine based upon what you wanted to happen as opposed to what actually happened. The benefit of a study like this is to say, “If you have a pediatric patient who requires airway assistance, the result of ETI is this and the result of BVM is that.” That is not something we can conclude from this paper.

So the question you need to ask yourself, is if that’s your child, grandchild or the kid next door, what’s the best thing to do to ensure the best outcome? This paper doesn’t even begin to answer that question and based upon the dramatic change this has had on the scope of practice, this is a question that seriously needs to be answered. As a Cardiologist and a Paramedic, when patients arrive in the Emergency Room, we want everything that can be done, done before I see them, which means we are farther along the path of possible success; not farther from it.

It’s clearly time for this type of study to be done and to be done correctly to answer these critical questions regarding the best outcomes treatment in the emergency pre-hospital setting for pediatric patients with compromised airways.

We would like to thank Mr. Brian Wheeler, AGS, NRP, Senior Lecturer, UCLA Paramedic Program for his thoughts, insights and experience into pediatric intubation.

1. Gausche M, Lewis RJ, Stratton SJ (2000) Effect of Out-of-Hospital Pediatric Endotracheal Intubation on Survival and Neurologic Outcome. A Controlled Clinical Trial. JAMA 283: 783-790. Link: https://bit.ly/2Rti2uS

© 2019 Fleming RM, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.