So what’s the best evidence we have on bicycle helmets?

According to the Cochrane Collaboration — the source that most doctors will go to for their summary of the evidence — it is five studies from the 1980s and 1990s.

The Cochrane Review set out to answer a very specific question: “in the set of people who sought Emergency Room treatment having had a bicycle crash, did wearing a bicycle helmet correlate with the rate of head and brain injuries among the patients?”  These are important details — the question was not “in the entire set of people who ride bicycles, does wearing a bicycle helmet affect mortality, life expectancy, the rate of serious injury, or injury recovery?”  It’s not a bad question that the researchers are asking, but it is a very limited question — the data is restricted to the type of injury that is serious enough to send people to hospital, but not serious enough to kill outright; it does not ask whether helmets correlate with any other types of injury beyond head and brain (more later); and it can say nothing about whether helmet wearers and non-helmet wearers differ in their behaviour or exposure to risk of the type of accident that sends them to hospital in the first place.  The latter possibility turns out to be a very interesting one, which will be explored later.

The Cochrane Review searched the existing medical literature for high quality studies that were theoretically capable of answering their question.  There are several different ways that one could design a study to answer a question like the Cochrane question — some methods more reliable than others.  The Cochrane Review found five studies described in seven papers, all with the same design: case-control studies.  This study design looks at a set of people who have been hospitalised with head injuries while riding a bicycle and examines their records to find out whether whether more or fewer of them were wearing a helmet than a similar set of cyclists who were hospitalised at roughly the same time and place but whose injuries were not head injuries.

Case-control studies have a number of limitations that make them less reliable than other study designs, like the gold-standard randomised controlled trial design.  Principally, the study must merely make the assumption that the “case” population and “control” populations are essentially the same, differing only in the intervention tested (helmets) and potentially in the outcome of interest (head injury as a proportion of all injuries).  The method accepts that there may be other differences between the populations of patients (are helmet wearers on average richer, middle class, more likely to have health insurance, I wonder?), but makes the assumption that those differences are not important to the question being addressed, and so uses statistical methods to attempt to minimise their effect on the results.  For this reason, case control studies are considered to be relatively weak evidence, and when more rigorous trials are conducted, they often find that case control studies exaggerate the effects of interventions.  A good Cochrane review will carefully pick the studies that it includes, eliminating case control studies unless they do everything possible to minimise the limitations of the design, and this review appears to have done that.

The populations in the five studies reviewed were 1040 cyclists hospitalised in Cambridge UK in 1992; 1710 cyclists in Melbourne in the late 1980s; 445 child cyclists in Brisbane in the early 1990s; 668 cyclists in Seattle in the late 1980s; and a further 3390 in Seattle in the early 1990s.  The results therefore apply to the shape, style and construction of the helmets that were on the market in the mid-1980s to early 1990s, and to the types of people who were choosing to wear helmets at that time.  (The Seattle study, completed in 1994, does look specifically at”hard shell”, “soft shell” and “no shell” helmets, finding the same result for all three).  Note that the Cochrane review was assessed as “up to date” in 2006, meaning that the authors do not believe that there is any good quality data newer than the early 1990s.  I’ll let you decide whether these studies are relevant to your own 2010-model helmet or not.

The outcome of the case control study is the odds ratio — a measure of the strength of association between the intervention and the outcome, i.e., how big an affect the intervention appears to be having, and whether it appears to be helping or harming.  It’s literally the ratio of the odds of head injury in hospitalised helmet wearers to the odds of head injury in hospitalised non-helmet wearers.  So an OR of 1 would mean that the odds of head injury were equal, while an OR higher than 1 would mean that hospitalised helmet wearers had a higher rate of head injury than hospitalised non-helmet wearers and an OR lower than 1 would mean that helmet wearers had the lower rate of head injury.

The five studies under review all agreed on odds ratios in the region of 0.3, meaning that hospitalised helmet wearers had considerably fewer head and brain injuries than hospitalised non-helmet wearers.  It’s a significant result.  Not something that often happens by chance — especially repeated in five different studies.  In the set of cyclists who turned up at the Emergency Room, there was a strong correlation between whether one wore a helmet and whether one had a head injury.

That, according to the Cochrane Collaboration, is the best evidence that we have on bicycle helmets.  In the population of hospitalised cyclists in four cities in the late 1980s and early 1990s, there was a significantly higher rate of head and brain injury in those who were not wearing a helmet.  Nothing about mortality or life expectancy.  Nothing about injury recovery.  Nothing about injury and hospitalisation rates in the whole population of cyclists.  That’s not a criticism of the Cochrane Collaboration or it’s review: they are reviewing the best evidence we have.

Evidence that is, apparently, sufficient for the British Medical Association to campaign for compulsory use of a medical intervention.

Those are just the obvious limitations of the question being asked and the study design used to answer it.  The less obvious limitations are where it really gets interesting.