Interpreting Medical Outcome Data

Few things could be more terrifying for a parent than to be told that your child needs to have heart surgery. Unless you happen to be a paediatric heart surgeon it’s very likely that you are going to have to completely put your trust in the team of medical professionals carrying out the surgery; you’re powerless. During the 1990s at the Bristol Royal Infirmary in south west England that trust was abused.

Naturally, there are risks associated with any surgery, especially in children, but far more children were dying than you might expect. The death rate was about double that in other comparable units and perhaps as many as 35 children died that shouldn’t have. A subsequent inquiry into what happened found significant failings: “staff shortages, a lack of leadership, [a] … unit … ‘simply not up to the task’ … ‘an old boy’s culture’ among doctors, a lax approach to safety, secrecy about doctors’ performance and a lack of monitoring by management”.

To add insult to injury, it emerged afterwards that some organs, often the heart, had been removed from the bodies during post mortem examinations but never returned to the body. Once bereaved parents found out they had not buried all of their dead child they were understandably very upset.

This episode became known as the Bristol Heart Scandal and it remains a blot on the healthcare system to this day. The Kennedy Inquiry made nearly 200 recommendations as to how paediatric heart surgery could be improved and better monitored in the UK some of which have been implemented, most of which have not.

For example, it has long been considered a good idea to have fewer, larger centres that carry out heart surgery on children in the UK. The exact number is uncertain but many recommend six, certainly no more than nine. The reason for this is that part of the problem at Bristol was that the team there wasn’t doing enough operations to maintain its quality and standards. If you are doing something every day you tend to get pretty good at it. If you only do it every now and then you can forget some of the finer detail and be less well prepared for when the human body throws up something unexpected.

There remains 13 centres in the UK that carry out heart surgery on children. Whilst everyone seems to agree the need for fewer centres no one wants their local one to be one of the ones to go.

Another recommendation, one that was championed by many surgeons at the time and has been broadly implemented, is to publish survival rates. Whilst this is a good thing care needs to be taken when interpreting the data. Although it seems like the first obvious thing to do, you cannot simply compare the survival rates from one hospital to another like you might when choosing between schools.

To help clarify the confusion around the data the wonderful Sense About Science team has been working together with researchers, patient’s families and medical charities to develop a new website. It is called Child Heart Surgery and helps to illustrate why the basic hospital to hospital comparison is invalid.

Now, I don’t know what might be considered an ‘easy’ heart surgery, or if indeed there is such a thing; but it is certainly the case that there is a range of procedures and some have more risks than others, and some patients are more difficult to operate on than others. Procedure X is likely to be less risky in a strapping 16 year old boy that’s captain of the school rugby team than it is in a prematurely born neonate. All this has to be taken into account when judging a particular hospital.

For example, a hospital that only takes on relatively straightforward surgeries could be expected to have a better survival rate than one that takes on riskier surgeries and so it isn’t fair to compare the two.

The website provides a confidence interval in which you would expect a hospital to perform given the types of patients they see and the procedures they carry out.

Data taken from Child Heart Surgery website

Look at the example data above. There are two predicted ranges for each hospital. The narrower, purple range is the expected survival rate in 19 out of 20 surgeries. The wider, more feint range is the expected range of survival in 998 out of 1000 surgeries. Because every hospital treats different patients for different conditions every hospital will have its own, unique expected survival range. The ranges are calculated based on the procedures carried out by that institution in the last three years and so they must all be calculated afresh each year and each hospital will have a different range from one year to the next.

The black dots represent the actual survival rate. All hospitals in the country fall within their expected range except for Great Ormond Street (not shown in this data) who actually does better than expected but then, well, they’re Great Ormond Street; they’re the best.

The aim of making the data accessible in this way is to help the public interpret currently published survival data on children’s heart surgery and explore recent survival statistics for each hospital. What the data cannot do, something the curators are keen to point out, is help parents to make decisions about their specific child’s specific problem. For that it is necessary to talk to your heart surgeon or cardiologist.

Given the Bristol Heart Scandal it is very necessary to be open about data like this, to make it accessible and understandable to as many people as possible. Hopefully we can reduce the amount of confusion and alarm that parents feel when faced with incredibly difficult decisions. It would be great to see more sites like this for outcomes for a whole host of different procedures and therapies.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s