Read the transcript:
My son had had lots of ear infections and his hearing was suffering. He had grommets when he was two, and when we saw the specialist about his ongoing ear infections and glue ear we were faced with the decision about whether he should have grommets again. The Eustachian tubes in the ears are supposed to develop significantly around the age of 5 to 7 so that grommets shouldn't be necessary after that. So we were not really comfortable with the decision about having grommets a second time at this stage. We just wanted a little bit more information after seeing the surgeon. So we did consult The Cochrane Library.
The information in the background section of a Cochrane review was very useful. The way that reviews are titled is very useful for finding reviews that are relevant, because it gives the intervention, the population, and the PICOT structure. That wasn't something I was aware of as I was reading the titles. It was when they were relevant I thought, "There is a lot in there." The review's population didn't quite fit our son: the population was usually between 6 months and 3 years and he was 5. But it was relevant in terms of outcomes we were interested in: hearing loss, infection, behaviour development. Because he was starting school we wanted to know whether the prognosis afterwards would be for the higher age group, and they were effective in particular for the first 6 months. This informed actually when we would do the operation. The reviews were saying they are of limited effectiveness, given that after 12 months, kids are in the same position as if they hadn't had the grommets done.
So this informed the timing of the operation; we had the grommets done in January as he was starting school and we wanted him to have the "best possible start" with good hearing. It is now 6 months on and he is going well.
I knew what kinds of information there was in Cochrane reviews. I found protocols useful as well because they did cover quite a bit of the background - what is out there already, why they might be trying to look at this, what outcomes you might expect and in what populations. It is interesting to see what populations are included and excluded from the review, as it helps you try and work out whether a child like your son is considered in the review.
We were interested in the overall picture and there were 3 reviews that specifically that fit our type of scenario. The reviews were about grommets (ventilation tubes) for children with otitis media; identification of children in the first 4 years of life for early treatment for otitis media; and grommets for hearing loss.
The reviews helped with side effects of watchful waiting or not doing anything. And the possible negative side effects of doing the operation itself. And having them laid out clearly was also useful in making a decision. We could see them clearly side by side - the types of adverse events were similar whether we continued the watchful waiting or had the operation which actually helped inform our decision.
We would have looked at an individual study if it had had re-grommeting or a higher age group. So because none of them had the age group we were interested in itself we didn't look at any of the studies.
I think at the time when you are wanting to make a decision, you have already weighed up some serious pros and cons so you are looking for something that confirms one or the other. Then in hindsight when you've had an experience and you are going back and having a look at the information, you see something different. I think people look to confirm what they already know and if there's not enough evidence to support what they think they know then hopefully they can reassess it. I think people see what they want to see.
For the searches I Googled "glue ear and grommets" (because glue ear was the condition and grommets was what we were searching), with Cochrane. It took me to The Cochrane Library and it brought up a series of reviews, and some of them weren't Cochrane reviews, because Cochrane is mentioned a fair bit in other people's reviews that are published outside of Cochrane. In The Cochrane Library itself I went into one of the reviews, and searched in the review with the same terms (without Cochrane): just glue ear and grommets, in the same line, and it looked through the titles and abstracts. The structure of the review with the titles helps
with the screening process.
For my search strategy I knew what to do. When I was looking specifically for population etc. to confirm that it was going to be relevant I actually used the find function within the PDF version to see if I could find any of those sort of words that would indicate that there was a sub-analysis to indicate that it would be relevant to my son in the PDF of the full review. But when there wasn't, I mostly looked in the background and the results and the discussion, which is the whole thing! I just skimmed for the outcomes we were interested in and particular words.
I am also knowledgeable about why a Cochrane review over a different type of review is valuable - in that they are randomised controlled trials and you have to know that to begin with; but also about why you would want to know the results of a randomised controlled trial over a surgeon's experience of a situation or what other types of information you might find when talking to other people, consumers stories etc. stories or case studies things like that, which are interesting?
In a review odds ratio and relative risk is presented in a forest plot with a graph of all of the results and a line of null in the middle and you can see whether it's effective or not, based on whether the 95% confidence interval overlaps the null, or is on one side or the other.
I find reviews not as useful as they could be. I think they should be minimally important in the overall decision making. I think it depends on the priorities of the outcomes the patient is interested in. So when an individual person is consulting a review, some statistically significant results might not even be interesting to them because all of the surgeries have the same sort of statistical effectiveness for that. But one particular outcome, for example, in the glue ear for us was the hearing loss, or hearing improvement, and in particular for us for that outcome, if everything else was lined up side by side, that would be the outcome that we would be most interested in. So it is I think important that all of the outcomes are explained in terms of whether it is significant or relevant clinically when it is significant in a way that then a person can make a decision about particular outcomes that they are most interested in.
I think it's just one form of evidence you can use to base a decision on. It's not in and of itself what we based our decision on for whether or not we would have the intervention; it did help inform us of the decision for timing though. I think it is just one form of evidence; it's not the holy grail of evidence, not the only source.
Before we searched for Cochrane review we had already consulted the doctor, and had considered that grommets would probably be the way to go. We just wanted to confirm there were no alternative things that we had overlooked, and that it's effective for the outcomes we were interested in.
So the review informed the timing. When we went back to see the surgeon, we had discussed when to have it and he let us know he could do it any time - six weeks from that date. We said when we read a Cochrane review there seemed to be significant effectiveness in the first 6 months and then it tapers off over the 12 months. So given that our preference would be the beginning of 2009 because our son would hopefully have the first half of the year to shine as well as he can (in the first year of school) without hearing loss influencing his development. And he confirmed with his experience as well that some of them last up to 6 months and they drop out; others can last longer, in terms of hearing loss because he does test hearing when kids come in, as well that they were effective for that length of time. So it was reassuring to know. It wasn't something we had discussed with him in the initial meeting.
We said that Cochrane reviews suggested they are effective up to 6 months and they are of limited effectiveness after that for hearing loss, and he said he could corroborate based on his experience. That was reassuring.
My husband and I read the review side by side and discussed it and teased out bits. It confirmed that we were ok to be considering an intervention and that we weren't just hypercondrialising the situation; and it confirmed as well that the adverse events were equal for waiting or going, aside from the anaesthetic issue, so that gave me confidence that what we were considering was an appropriate course of action despite the fact that the review was suggesting that it didn't have very much confidence. The conclusions said that the intervention was effective for certain things, but it didn't go as far as saying that. It ended with, "The effect is small" and that it wouldn't recommend one intervention over the other. I can't remember the exact words but for us it was enough to make a decision.
I have gone back to The Cochrane Library but would only check regularly if I'm struggling over a particular decision. I would also tell others about systematic reviews and the Library.
Yes I look to see if information online is evidence based. I chose to investigate Cochrane reviews on the subject because I was made aware of the rigorous nature of the research that goes into them, the transparency, and therefore the reliability of the outcomes. I have and do look at individual studies sometimes (that is, search the internet for more information).