Appendix: Bad Science Bingo in the BMA’s “safe cycling” pages

This is just a crude brain dump of a post that comes after the serious series — posts one, two, three, four, five, six, seven and eight.

Sorry, I just can’t get over these extraordinary pages on the BMA’s website. Here’s a very quick run through some of the Bad Science Bingo points that leaped out.

There were the canards, fallacies, and methods of misdirection:

1. Obviously there’s the emphasis on anecdotes and cases, the lowest form of evidence, which are essentially appeals to emotion.
2. Coupled with that the description of the “beliefs” of a few doctors, designed to nudge readers into conformity (acting as a subtle argument from authority for readers who are not doctors, and an argumentum ad populum for those who are).
3. Specifically in a couple of the anecdotes the selective recall of serious injuries in non-helmet wearers and minor injuries in helmet wearers (creating the illusion of control).
4. “Figures from New Zealand show that in 2006 there were 883 cyclists injured and nine killed. This corresponds to 20 people per 100,000 injured and 0.2 people per 100,000 killed. These figures are lower than those reported for 1994 when legislation was first introduced.” Fun factoids, but they don’t actually say anything about helmet efficacy. Lots of things changed between 1994 and 2006. (Post hoc, etc.) Perhaps there is evidence for NZ’s legislation improving safety but the 2006 crude injury statistics aren’t it.
5. Incidentally, while we’re on correlation and causation, the authors even get their statements on cycle tracks subtly wrong: “During the period of 1976 to 1995 Germany almost tripled their mass of cycle networks and this led to a 64 per cent drop in cyclist deaths.” While the evidence of a causative link is much stronger here, it’s a lot more complicated that a simple one “led to” the other. The reference does indeed state that Germany tripled their cycle network and that their death rate fell, but it notes that the later is in part the result — directly and indirectly — of the former.
6. I loved this statement, when discussing the side-effect of reduced rates of cycling: “If legislation were to reduce the rates of serious injury and promote increased public confidence in cycling, the effect might be to make cycling more popular. Clearly, there is a need for further research on this matter.” I don’t know where to begin. After dismissing all the side-effects of helmets as being based on too weak and preliminary evidence, the BMA counter it all with a speculation based on none at all — and tell us that there is a clear need for more research. Well quite.

And there were specific claims or activities that run counter to the cited evidence, or subtly misrepresented it (I did not systematically check references, these are simply things that leaped out as contradicting what I recall of the literature):

1. On page 2 the BMA list the things they are doing in addition to promoting helmets. The first item is “publicity and education campaigns in order to raise drivers’ awareness of more vulnerable road-users, including cyclists”. We know that these don’t really work.
2. The “risk compensation” section on the fifth page cites just one source, the Spanish study described on Monday, whose study design we know can not answer the question that they are asking it to answer.
3. “As noted in Table 2 the Macpherson and Spinks 2007 Cochrane review found no evidence to either support or counter the possibility that legislation may lead to negative societal and health impacts such as reductions in cycling participation.” You would probably read this and think, “studies have been done and they found no evidence for X.” It actually means, “the studies didn’t bother looking at X.”

And there were fun inconsistencies:

1. Kirsty’s story on page 5, “Doctors believe that had she not been wearing a cycle helmet at the time of her crash, she would have died,” and on page 6, “They have been shown to reduce the risk of head injury and its severity should it occur. This does not apply to fatal crashes but in such instances the force of impact is considered to be so significant that most protection would fail.”

The resource is just generally bizarre. It has a very weird set of focusses. On one page it gives a seemingly arbitrary selection of factoids from cyclist demographics (notably absent is any acknowledgement that “cycling” is not a single activity); on another it notes the diversity of cycle helmet standards — but fails to discuss any of the important consequences of this, such as how few helmets these days meet the stricter standards that applied in the past, back when most of the evidence on helmet efficacy was collected. In a table on the fifth page they mention that a study found no evidence of helmets causing or exacerbating rotational injuries — yet this is the only mention they make of the rotational injuries problem. Their inclusion and omission criteria appears to be completely random.

Anyway, enough of this. I don’t want to hog the game — your turn.

How did the BMA get bicycle helmets so wrong?

In 1958, the UK licensed a drug for treating morning sickness. It worked very well. The studies all showed that pregnant women suffering from morning sickness received much relief with the drug. Three years later it was withdrawn, but not before 2,000 babies were born with birth defects — 20,000 worldwide — three quarters of whom would die in infancy. The drug was, of course, thalidomide. It managed to get licensed because too many of the people studying it were focused on very specific aspects of its activity on the disease states that it was thought to treat, and too few were stepping back and looking at the big picture. It prevented morning sickness, therefore it worked — the logic of the day.

Joe’s anecdatum: In 2003, Joe, an 18 year old male, slipped on some wet stairs in a block of flats. His head fell eight feet onto the concrete floor. He was not wearing a bicycle helmet. He had a headache for the rest of the evening. He has never been diagnosed with any long-term ill-effects.

A disaster on the scale of Thalidomide can’t happen these days because the path to drug licensing forces researchers to comprehensively check all effects and outcomes of a new drug. Individual researchers will know in extravagant detail very narrow aspects of how a new drug achieves its desired effect. Some of them will know the exact rate at which it crosses the various barriers into the blood and into organs; others will know the exact chain of activation of molecules and genes within cells, down the individual amino acid residues that are modified and the exact number of seconds after the drug is administered; others will know the exact schedule and mechanism by which the drug is broken down or expelled from the body. They’ll be really excited and enthusiastic about their new drug. But when somebody steps back and points out that the drug causes heart failure, it won’t get anywhere.

But the BMA seems to forget everything it knows about testing interventions when it comes to bicycle helmets. There are some superficial differences between helmets and what we normally think of as “medical intervention”. They are a physical intervention rather than a drug — but medicine deals with and properly tests physical interventions all the time. And it’s supposed to prevent rather than treat injuries — but medicine deals with and properly tests preventative measures, including conventional drugs, all the time. There is no intrinsic reason why bicycle helmets can not be tested properly, in line with the rules that were designed to prevent another thalidomide disaster. We have the methods and the expertise.

Joe’s anecdatum: In 2009, Joe, a 23 year old male, slid on the gravel on the Greenwich Peninsula Thames Path, hitting his head on the concrete path and writing off an £800 camera lens. He was not wearing a cycle helmet. He was unhappy and was bored for several hours waiting for Lewisham Hospital to glue his face back together. He stayed home all next day. He has never been diagnosed with any long-term effects.

And yet the evidence that we have on bicycle helmets is currently in a worse state than the evidence that got thalidomide licensed. There is some (limited) evidence that in people who have had crashes, helmets reduce the rate of specific types of head injury — just as there is undisputed evidence that thalidomide is effective in relieving morning sickness. But there is also (equally limited and disputed) evidence of several different side effects — an increase in other types of injury* and an increased rate of crashes (particularly crashes with vehicles, which are more likely to have negative outcomes). And there is also evidence that helmets discourage many people from cycling* — an activity that adds many quality years to people’s lives by preventing or delaying cardiovascular disease, cancers, diabetes, depression, dementia, and all those other diseases of sedentary lifestyles. Helmets might be an effective intervention for the types of injuries they are claimed to prevent, but that would be irrelevant if, like thalidomide, they cause more problems than they solve.

Joe’s anecdata: In 1991, Joe, a 6 year old male, on separate occasions smashed his head open a door, some concrete steps, and a glass coffee table. On no occasion was he wearing a cycle helmet. He has a scar on his forehead that is almost identical to James Murdoch’s. Unlike James Murdoch, he has never been diagnosed with any other long-term impairment or ill-effects.

I’m not saying that they do. The issue is not that there is overwhelming evidence against helmets. The evidence that they are the cause of crashes and other injuries is no stronger than the evidence that they prevent head injuries. The issue is that the evidence either way is nowhere near good enough to make a recommendation. If helmets were a drug, they would be nowhere close to getting licensed right now.

Which is why British doctors should be embarrassed that the British Medical Association currently lobbies for helmets to be compulsory when riding a bicycle. Imagine if a pharmaceutical company developed a drug which, if administered before receiving a specific kind of traumatic injury, makes that injury easier to treat. Imagine doctors and medical scientists lobbying for it to be compulsory for everybody to take this drug daily, without anybody ever having checked for side-effects.

How has this situation arisen?  The policy decision has largely been made on the insistence of A&E consultants and trauma surgeons.  Consider the anonymous quotations that are scattered through the BMA’s cycling pages:

‘I have seen – in my practice and when working in A/E – quite a number of serious head injuries from children falling off bicycles. I have also seen a number of children who wore helmets who only suffered minor injury. I am convinced that helmets reduce injury.’ — GP

’I would certainly support cycle helmet wearing for cyclists. I have seen far too many young lives ruined by head injuries.’  — Consultant in Emergency Medicine

’I am an Emergency Department Consultant and a keen cyclist. I wholly agree…that we need to move to an environment where cycle helmet wearing is the norm, rather than the exception’  — Emergency Department Consultant

’As a regular commuting cyclist through twelve miles of heavy London traffic and as a Consultant Emergency Physician I whole-heartedly support the BMA’s stance on the introduction of legislation to make the wearing of helmets mandatory.’  — Consultant and Honorary Senior Clinical Lecturer in Emergency Medicine

’Over the [last] 16 years I have worked in A/E. I have dealt with hundreds of head and facial injuries, particularly in children, that could have been avoided had a cycle helmet been worn. I have also had the misfortune to deal with a number of fatalities that I believe would have been avoided by simply wearing a helmet. I firmly believe that legislation making cycle helmet usage mandatory is essential.’  — Emergency Medicine Consultant and Clinical Director

‘I have worked in emergency medicine for the last twelve years. Personally I cycle around two and a half thousand miles each year and my family are rapidly becoming keen cyclists also. Prior to working in emergency medicine, I did not routinely wear a cycle helmet.

I have seen numerous examples of patients sustaining severe head injuries from which they will never recover whilst cycling at low speed without a helmet. I have never seen this pattern of pathology in cyclists wearing helmets under these circumstances.

I am aware of the recent Cochrane review on the subject. I firmly believe that all cyclists should wear helmets. I also believe that the only way to ensure this happens is through legislation. I can see no justification for allowing this entirety predictable pattern of head injuries to persist. I strongly support the BMA position…’  — Consultant in Emergency Medicine

That’s five emergency medics and a GP, all reciting anecdotes from A&E. Nobody who specialises in, say, public health.

Emergency medics and trauma surgeons are obviously very enthusiastic about the potential to put an end to injuries, just as people who were very focused on the problem of morning sickness were excited by thalidomide. But ironically, most doctors and scientists are not very good with complexity. They are good with the intense detail of their own specialism, but when they have a problem to solve they fail to consider that there might be relevant things happening outside of their own field. When emergency medics want to solve the problem of head and brain injury, they look at those injuries in isolation from the rest of medicine. It’s not their job to think about the bigger the picture, or worry about things like side-effects.

Indeed, dare I suggest that for most working emergency medics and GPs, the science of evidence-based medicine is not their job or even a major part of their training: they only need to practice what the scientists amongst them tell them to practice; most working doctors don’t need to understand how we know their interventions work.

Which is fine. But that stuff is somebody‘s job, and somebody isn’t doing it right at the BMA.

This way of thinking about the issue — as an isolated problem of emergency medicine — is reflected all through the BMA’s bizarre “safe cycling” pages, which emphasise these individual anecdotes and opinions of doctors in that field (despite “expert opinion” being frequently out of line with the science and despite everything we know about the ability of anecdotes to lead readers astray), while failing to ever think of the issues around helmets in terms of effects and side-effects or the usual path of research that is demanded for medical interventions.

The authors of the Cochrane review on bicycle helmets say, in dismissing risk compensation, “the fundamental issue is whether or not when bicycle riders crash and hit their heads they are benefited by wearing a helmet.” This is exactly analogous to saying that “the fundamental issue is whether or not when a pregnant woman has morning sickness her symptoms are relieved by thalidomide.” That is not the fundamental issue at all. The fundamental issue with any medical intervention is whether it does more help than harm, whether it improves the length and quality of our lives, whether we are better with it or without. That the authors of a Cochrane review are allowed to get away with saying otherwise is a great failure for evidence-based medicine. That the BMA think there is sufficient grounds not merely to promote this intervention but to enforce it is an epic failure.

* I thought about posting separately on these sets of side-effects too, but those posts would have been much like the rest of this series: there’s a plausible hypothesis, there’s some evidence to support it, but the evidence has limitations. Ultimately the conclusions always are: the evidence base is nowhere near good enough to support helmet promotion, let alone legislation.

The BMA, the BMJ, and bicycle helmet policy

The reason I pick up the bicycle helmet theme again this week is that the BMJ is running a sidebar poll of their readers (or, more accurately, of cycling tweeters and recipients of Robert Davis’s emails ;-)), asking whether it should be compulsory for adult cyclists to wear helmets.

The BMJ is the journal of the British Medical Association, the professional association and trade union of British doctors. Part of the BMA’s remit it to lobby the government on issues that its members believe are important, and it has some clout in this area. These policies are decided by a representative democracy — a group of members elected by region and by field. In recent years, this body has decided that it is BMA policy to support legislation that would make helmets compulsory for cyclists.

Doctors might not even have noticed the adoption of this policy.  To most it is probably an irrelevance — most people will not cycle in the conditions that prevail in this country and doctors are no exception. And I imagine that very few have read the quite astonishing “promoting safe cycling” pages of the BMA website. Readers of Ben Goldacre should get their Bad Science Bingo cards out before clicking the link.

Tomorrow I’ll dissect those pages and ask how they came to be so bad. But there is a more basic issue here. Never mind whether helmets are effective or not, aren’t there more important policies that the BMA should be pursuing?

In 2002, the BMJ polled readers about issues of health and road danger — a slightly more scientific and insightful survey than the free-for-all yes/no question that they ask this week, and one much better targeted to British doctors rather than every joker on the internet.  They asked readers to judge the importance, on a scale of 1 to 6, of various interventions for promoting the health and safety of pedestrians and cyclists. Helmets came out bottom of the doctors’ priority list:

Average ranking	Response
3.25	More and better cycle safety training
2.87	Compulsory cycle helmet wearing
3.42	Separate lanes for bicycles in urban areas
4.04	Traffic calming to reduce vehicle speeds in urban areas
4.04	Reduce car use by better public transport and by encouraging walking and cycling
3.85	Banning motorised vehicles from towns and cities

Interestingly, helmets for cyclists was ranked as only a slightly more sensible solution than helmets for pedestrians. Indeed, the results for pedestrians look much like the results for cyclists.

It’s the most heartening thing I’ve read in a long time. Most doctors get it. They’re not ignoring the bull. Certainly all of the public health doctors and epidemiologists (the people with the most exposure to scientific methods, incidentally) that I know get it. The problem is not that cyclists are taking insufficient measures to protect themselves from danger, it is that they are put in danger by motorists and by the government policies and societal norms that support the mixing of fast-moving motor vehicles, including those driven by people known to be dangerous and incompetent, with cyclists and pedestrians in our towns and cities.

Alongside their policy of lobbying for legislation to compel the use of helmets, the BMA has drawn up a set of recommendations for motor-vehicle reduction. But while the former policy is actively being pursued in Westminster and in the nations, the latter looks to have fallen by the wayside, and is still stuck in 1997. Why?

Second hand; unused

Thinking about how the Cycling Embassy might go about trying to generate political will to progress cycling, I’ve been researching previous failed attempts to advance cycling in this country.  So on Amazon I snapped up a second-hand copy of an out-of-print British Medical Association book written in 1992: Cycling: towards health and safety.

People in Public Health are very interested in the bicycle because it keeps you fit — thus reducing incidence of obesity, cardiovascular disease, cancer, diabetes, dementia, depression, etc, etc — in a way that can be effortlessly integrated into everyday routines.  And because it provides an alternative to transport modes that cause thousands of hideous traumatic deaths and injuries, even more air pollution-related deaths, isolation-related mental ill-health, and so on.

From a quick flick through, I’m expecting all that to be covered, in addition to a section on “barriers to cycling” which looks like it might cause a cardiovascular event itself by chatting about the weather while ignoring the elephant in the room.

But also when I quickly flicked through, I noticed I was breaking the spine.  This second-hand book has never been read, never been opened except to stamp “date of cataloguing 14 May 1992” and “disposed of by authority” on the inside cover.  Where does this never-before-read book make its way to me from?

Perhaps it’s for the best.

Would a helmet help if hit by a car?

This post is part of a series: it starts with the intro to the helmets issue, then the summary of the best evidence on helmets, then a quick diversion into how dangerous cycling is and an attempt to define terms. And there’s more…

Brake, the “Road Safety” charity, say yes:

Helmets are effective for cyclists of all ages, in crashes which do and do not involve another vehicle.

That matters, because if cycling safety is in the news, journalists will go to Brake for an easy quote.

The British Medical Association also say yes:

Helmets provide equal level of protection from cars (69%) compared to other causes (65%)

This is important, because the BMA is a highly trusted organisation with political influence, and their current policy is to endorse the criminalisation of riding a bicycle when not wearing a helmet.

Interestingly, president of the Automobile Association Edmund King, who was giving away free advertising bicycle helmets in London this week, disagrees with the nation’s medics on both issues:

We don’t think helmets should be compulsory but we think there are benefits… Our view is that helmets do not protect against cars but they may protect against some of the 2.2m potholes which often are the cause of smashes into the ground by cyclists.

Carlton Reid adds a little detail:

Most bicycle helmets are designed for falls to the ground from one metre at speeds of 12mph. They offer almost zero protection in collisions between bicycles and fast-moving cars.

The risk reduction provided by helmets in bicycle crashes that do and do not involve motor vehicles is one of the few sub-group analyses that was performed in the case-control studies that are covered by the Cochrane Review, and it’s no surprise that this is the source for the BMA’s claim. In bicycle hospitalisations that did not involve cars it reported nearly 70% fewer head injuries in the helmet wearers. In bicycle hospitalisations that did involve motor vehicles there were nearly 70% fewer head injuries in helmet wearers.  A helmet is equally effective at preventing head and brain injury in crashes with cars as in solo crashes.

What makes Edmund King and Carlton Reid think they know better than the nation’s medics and road safety campaigners?  Indeed, what makes them think that they can go around claiming the opposite of the cold hard corroborated stats of the Cochrane review?

Well actually, they’re not. Not quite. King and Reid are judging helmet efficacy by a slightly different metric to the Cochrane Review.  The Cochrane Review is the looking at the set of bicyclists who have had an accident of a severity that hospitalises but does not kill outright.  The review says nothing about deaths, for example, and as the Cochrane Review itself notes, more than 90% of cyclist deaths are caused by “collisions” involving moving motor vehicles (the same proportion is found again by a separate route in the TRL review and again in NYC).  But only 25% of hospitalisations were caused by motor vehicles.  And while Cochrane suggested a whopping 85% of head injury hospitalisations (which in turn account for around half of all cyclist hospitalisations) could be avoided by wearing a helmet, the TRL review of post-mortem reports found that only 10-16% of all cyclist deaths might have been avoided.  Hospitalisations, of the sort reviewed by Cochrane, are not representative of deaths.  Fall off your bicycle and you might get hurt.  Get hit by a car and you might die.

That’s because when you fall off your bicycle, chances are you are toppling over some way — precisely the sort of simple fall that a helmet is designed for, and the sort of fall that is least likely to cause life-threatening injury to any other part of the body.  When hit by a car the body might be crushed, or thrown up and around at speeds that helmets are not designed for, and so there are many more opportunities to suffer fatal trauma to other parts of the body.

(As an aside, Brake actually get this one the wrong way ’round:

Nearly 50% of cyclist admissions to hospital are for head and facial injuries, and the majority of cyclist deaths and injuries are a result of head injury.

TRL has the answer to this one: around three quarters of cyclist fatalities did indeed involve a serious head injury.  But only about a quarter involved only a serious head injury.  The rest also involved one or more additional life-threatening injury.  The Brake claim is at best misleading.)

This doesn’t mean that the BMA and Brake are all wrong* and King and Reid are completely correct.  A car at speed may be able to cause the sort of multiple trauma that merely falling over doesn’t, but that doesn’t mean that cars aren’t also capable of causing the sort of crashes that helmets are designed for, especially in low speed city traffic.

So Edmund King is wrong**.  But within the untruth he is communicating an important truth: cars are responsible for the most serious injuries and death, and helmets will rarely help in those cases.

Brake and the BMA are correct.  But their strictly truthful statements hide the crucial details, without which they are liable to seriously mislead.

* Indeed, they can’t be wrong.  You can provide a hypothesis for why helmets might be useless in crashes with cars, but no hypothesis can trump the real world stats that say helmets are useful in crashes with cars.

** Carlton Reid is not wrong, because he specified fast-moving cars.

Headline figures

If you haven’t done so already, start from this post and work your way forward.

In rare events like bicyclist injuries, odds ratios can be used as an approximation of relative risk: that is, how much a medical intervention changes the risk of a specific outcome.  An odds ratio of 0.3 is interpreted as a 70% reduction in risk of head injury when wearing a bicycle helmet.

The Cochrane Review looked at five studies, which contained a number of sub-analyses.  There was actually a range of odds ratios found when looking at different types of injury in different groups of cyclists.  In one, an odds ratio of 0.15 was reported.

So now the headline figure is that bicycle helmets protect against a whopping 85% of injuries.  Imagine the lives that could be saved.  Won’t somebody think of the children?  The 85% figure is constantly repeated by “Road Safety” spokesmen, and reported without context by journalists.  It’s cited by the British Medical Association in support for banning people from riding bicycles except when wearing helmets.  The 85% figure matters.

Leaving aside questions over whether the 85% figure represents the real efficacy of helmets, how useful is it as a guide for how to live our lives?  Well, as Ben Goldacre puts it: “you are 80% less likely to die from a meteor landing on your head if you wear a bicycle helmet all day.”  Nobody has ever died from a meteor falling on their head*.

What Ben is saying is that relative risk is only a useful number to communicate to the public if you also communicate the absolute risk.  If you want to know whether it’s worth acting to reduce the risk of something bad happening, you need to know how likely it is to happen in the first place.

In the UK, 104 cyclists died on the roads in 2009, according to DfT stats.  It was 115 in 2008, but the trend has been downwards for a long time.  For simplicity, lets say that in future years we could expect on average 100 cyclist deaths per year.  It’s really difficult to say how many cyclists there are in the UK, because you can define it in several different ways, and even then the data that we have is crap.  You can estimate how many bicycles there are, but these estimates vary, many bicycles might be out of use, and many of us own more than one.  You can take daily commuter modal share — which would give us 1 million cyclists — but there’s more to using a bicycle than commuting, and most people mix and match their modes.  According to the latest National Travel Survey, 14% of people use a bicycle for transport at least once per week.  An additional 4% cycle several times a month, and 4% again cycle at least once a month.  Cumulatively, 32% of the British people cycle at least sometimes, but some of those are too infrequent to be worth counting.  To be generous, and to keep the numbers simple, I’ll round it down to 16%, giving us 10 million on-road cyclists in the UK.  That means one in 100,000 cyclists is killed in cycling incident each year.

To put it another way, there’s a good chance you’ll get killed if you carry on cycling right up to your 100,000th birthday.  (If you do not first die in the inferno caused by the candles on the cake.)  Or, if when Adam and Eve first left Africa 200,000 years ago they had done so on bicycles, there is a good chance that at least one of them would be dead by now.  Alternatively, if you accept that life expectancy is around 80-90, make the unlikely assumption that all cyclists remain cyclists pretty much from cradle to grave, you might die cycling once in over a thousand lifetimes.  Nine-hundred and ninety-nine lifetimes in a thousand, you will die of something much more probable.  Like heart disease, or cancer.

But not everybody who dies on a bicycle dies of head injuries, and not everybody who dies of head injuries sustained while riding a bicycle would be helped by wearing a helmet.  The DfT/Transport Research Laboratory have done their own extensive review of the medical literature on helmets and say: “A forensic case by case review of over 100 British police cyclist fatality reports highlighted that between 10 and 16% of the fatalities reviewed  could have been prevented if they had worn an appropriate cycle helmet.”  This is because, while some form of head injury was involved in over half of cyclist fatalities, the head injury was usually in combination with one or more serious injury elsewhere on the body; and even in those where only the head sustained a serious injury, as often than not, it was of a type or severity that a helmet could not prevent.  There are, of course, many caveats and limitations of such an estimation, which relied on many assumptions, some amount of subjective judgement, and a limited dataset which was biased to the sort of cyclist fatalities that the police are interested in.  So we could be generous and round it up to 20% — that helps keep our numbers simple.

So we’re talking about about 20 lives saved per year, or in terms that matter to you, your life saved if you cycled for half a million years. Of course, a third of British cyclists already wear helmets, so we can add the number of cyclists whose lives are already being saved.  We could be generous again and say 40 lives per year.

That would give you a chance of less than 1 in 2,500 that, as a cradle-to-grave bicycle user, bicycling from nursery school to nursing home, you will die in a crash that a helmet would have protected against.  The chances are 2,499 in 2,500 that you will die of something else.  Like the 4 in 2,500 chances of being killed in a cycling incident where a helmet would not have helped.

Or the 6 in 2,500 chances of death by falling down stairs.  Or the 3 in 2,500 of being run over by a drunk driver.  Or the whopping 30 in 2,500 chances of dying of an air pollution related respiratory disease.**  Unfortunately I couldn’t find the British Medical Association’s policy on legal compulsion for users of stairs to wear appropriate personal protective equipment.

Of course, in addition to the 100 cyclists killed on British roads each year, another 1,000 suffer serious but non-fatal head injury, sometimes involving permanent and life-changing brain damage (as do users of stairs).  The Cochrane Review says that up to 850 of those injuries would be avoided or less severe if a helmet were worn; the more pessimistic TRL review says that perhaps 200 of them might be prevented or mitigated by an appropriate helmet.  Either way, we’re still in the area of many thousands of years spent cycling.

Whether you think those numbers make helmets worthwhile  it is up to you — I don’t think these numbers alone objectively prove that helmets are or are not worth using.  Just don’t be fooled by the stark headline-grabbing figure of 85% risk reduction.  When the absolute risk to begin with is smaller than that for fatally falling down the stairs, and a fraction of one percent of that for cancer and heart disease, consider whether risk reduction matters.

Of course, that might all change once we’ve looked at the next part of the story…

—

* I have not checked this fact, which I just made up, but I would be surprised to hear that it is not true.

** Hastily googled and calculated headline figures for illustrative purposes only; again, I have not thoroughly assessed these.

Final disclaimer: this is a hastily scribbled blog post, not an academic paper.  I’ve checked my zeroes and decimal places, but if I’ve overlooked something or accidentally written something to the wrong order of magnitude, please do point it out.

So what’s the best evidence we have on bicycle helmets?

According to the Cochrane Collaboration — the source that most doctors will go to for their summary of the evidence — it is five studies from the 1980s and 1990s.

The Cochrane Review set out to answer a very specific question: “in the set of people who sought Emergency Room treatment having had a bicycle crash, did wearing a bicycle helmet correlate with the rate of head and brain injuries among the patients?”  These are important details — the question was not “in the entire set of people who ride bicycles, does wearing a bicycle helmet affect mortality, life expectancy, the rate of serious injury, or injury recovery?”  It’s not a bad question that the researchers are asking, but it is a very limited question — the data is restricted to the type of injury that is serious enough to send people to hospital, but not serious enough to kill outright; it does not ask whether helmets correlate with any other types of injury beyond head and brain (more later); and it can say nothing about whether helmet wearers and non-helmet wearers differ in their behaviour or exposure to risk of the type of accident that sends them to hospital in the first place.  The latter possibility turns out to be a very interesting one, which will be explored later.

The Cochrane Review searched the existing medical literature for high quality studies that were theoretically capable of answering their question.  There are several different ways that one could design a study to answer a question like the Cochrane question — some methods more reliable than others.  The Cochrane Review found five studies described in seven papers, all with the same design: case-control studies.  This study design looks at a set of people who have been hospitalised with head injuries while riding a bicycle and examines their records to find out whether whether more or fewer of them were wearing a helmet than a similar set of cyclists who were hospitalised at roughly the same time and place but whose injuries were not head injuries.

Case-control studies have a number of limitations that make them less reliable than other study designs, like the gold-standard randomised controlled trial design.  Principally, the study must merely make the assumption that the “case” population and “control” populations are essentially the same, differing only in the intervention tested (helmets) and potentially in the outcome of interest (head injury as a proportion of all injuries).  The method accepts that there may be other differences between the populations of patients (are helmet wearers on average richer, middle class, more likely to have health insurance, I wonder?), but makes the assumption that those differences are not important to the question being addressed, and so uses statistical methods to attempt to minimise their effect on the results.  For this reason, case control studies are considered to be relatively weak evidence, and when more rigorous trials are conducted, they often find that case control studies exaggerate the effects of interventions.  A good Cochrane review will carefully pick the studies that it includes, eliminating case control studies unless they do everything possible to minimise the limitations of the design, and this review appears to have done that.

The populations in the five studies reviewed were 1040 cyclists hospitalised in Cambridge UK in 1992; 1710 cyclists in Melbourne in the late 1980s; 445 child cyclists in Brisbane in the early 1990s; 668 cyclists in Seattle in the late 1980s; and a further 3390 in Seattle in the early 1990s.  The results therefore apply to the shape, style and construction of the helmets that were on the market in the mid-1980s to early 1990s, and to the types of people who were choosing to wear helmets at that time.  (The Seattle study, completed in 1994, does look specifically at”hard shell”, “soft shell” and “no shell” helmets, finding the same result for all three).  Note that the Cochrane review was assessed as “up to date” in 2006, meaning that the authors do not believe that there is any good quality data newer than the early 1990s.  I’ll let you decide whether these studies are relevant to your own 2010-model helmet or not.

The outcome of the case control study is the odds ratio — a measure of the strength of association between the intervention and the outcome, i.e., how big an affect the intervention appears to be having, and whether it appears to be helping or harming.  It’s literally the ratio of the odds of head injury in hospitalised helmet wearers to the odds of head injury in hospitalised non-helmet wearers.  So an OR of 1 would mean that the odds of head injury were equal, while an OR higher than 1 would mean that hospitalised helmet wearers had a higher rate of head injury than hospitalised non-helmet wearers and an OR lower than 1 would mean that helmet wearers had the lower rate of head injury.

The five studies under review all agreed on odds ratios in the region of 0.3, meaning that hospitalised helmet wearers had considerably fewer head and brain injuries than hospitalised non-helmet wearers.  It’s a significant result.  Not something that often happens by chance — especially repeated in five different studies.  In the set of cyclists who turned up at the Emergency Room, there was a strong correlation between whether one wore a helmet and whether one had a head injury.

That, according to the Cochrane Collaboration, is the best evidence that we have on bicycle helmets.  In the population of hospitalised cyclists in four cities in the late 1980s and early 1990s, there was a significantly higher rate of head and brain injury in those who were not wearing a helmet.  Nothing about mortality or life expectancy.  Nothing about injury recovery.  Nothing about injury and hospitalisation rates in the whole population of cyclists.  That’s not a criticism of the Cochrane Collaboration or it’s review: they are reviewing the best evidence we have.

Evidence that is, apparently, sufficient for the British Medical Association to campaign for compulsory use of a medical intervention.

Those are just the obvious limitations of the question being asked and the study design used to answer it.  The less obvious limitations are where it really gets interesting.