Appendix: Bad Science Bingo in the BMA’s “safe cycling” pages

This is just a crude brain dump of a post that comes after the serious series — posts one, two, three, four, five, six, seven and eight.

Sorry, I just can’t get over these extraordinary pages on the BMA’s website. Here’s a very quick run through some of the Bad Science Bingo points that leaped out.

There were the canards, fallacies, and methods of misdirection:

1. Obviously there’s the emphasis on anecdotes and cases, the lowest form of evidence, which are essentially appeals to emotion.
2. Coupled with that the description of the “beliefs” of a few doctors, designed to nudge readers into conformity (acting as a subtle argument from authority for readers who are not doctors, and an argumentum ad populum for those who are).
3. Specifically in a couple of the anecdotes the selective recall of serious injuries in non-helmet wearers and minor injuries in helmet wearers (creating the illusion of control).
4. “Figures from New Zealand show that in 2006 there were 883 cyclists injured and nine killed. This corresponds to 20 people per 100,000 injured and 0.2 people per 100,000 killed. These figures are lower than those reported for 1994 when legislation was first introduced.” Fun factoids, but they don’t actually say anything about helmet efficacy. Lots of things changed between 1994 and 2006. (Post hoc, etc.) Perhaps there is evidence for NZ’s legislation improving safety but the 2006 crude injury statistics aren’t it.
5. Incidentally, while we’re on correlation and causation, the authors even get their statements on cycle tracks subtly wrong: “During the period of 1976 to 1995 Germany almost tripled their mass of cycle networks and this led to a 64 per cent drop in cyclist deaths.” While the evidence of a causative link is much stronger here, it’s a lot more complicated that a simple one “led to” the other. The reference does indeed state that Germany tripled their cycle network and that their death rate fell, but it notes that the later is in part the result — directly and indirectly — of the former.
6. I loved this statement, when discussing the side-effect of reduced rates of cycling: “If legislation were to reduce the rates of serious injury and promote increased public confidence in cycling, the effect might be to make cycling more popular. Clearly, there is a need for further research on this matter.” I don’t know where to begin. After dismissing all the side-effects of helmets as being based on too weak and preliminary evidence, the BMA counter it all with a speculation based on none at all — and tell us that there is a clear need for more research. Well quite.

And there were specific claims or activities that run counter to the cited evidence, or subtly misrepresented it (I did not systematically check references, these are simply things that leaped out as contradicting what I recall of the literature):

1. On page 2 the BMA list the things they are doing in addition to promoting helmets. The first item is “publicity and education campaigns in order to raise drivers’ awareness of more vulnerable road-users, including cyclists”. We know that these don’t really work.
2. The “risk compensation” section on the fifth page cites just one source, the Spanish study described on Monday, whose study design we know can not answer the question that they are asking it to answer.
3. “As noted in Table 2 the Macpherson and Spinks 2007 Cochrane review found no evidence to either support or counter the possibility that legislation may lead to negative societal and health impacts such as reductions in cycling participation.” You would probably read this and think, “studies have been done and they found no evidence for X.” It actually means, “the studies didn’t bother looking at X.”

And there were fun inconsistencies:

1. Kirsty’s story on page 5, “Doctors believe that had she not been wearing a cycle helmet at the time of her crash, she would have died,” and on page 6, “They have been shown to reduce the risk of head injury and its severity should it occur. This does not apply to fatal crashes but in such instances the force of impact is considered to be so significant that most protection would fail.”

The resource is just generally bizarre. It has a very weird set of focusses. On one page it gives a seemingly arbitrary selection of factoids from cyclist demographics (notably absent is any acknowledgement that “cycling” is not a single activity); on another it notes the diversity of cycle helmet standards — but fails to discuss any of the important consequences of this, such as how few helmets these days meet the stricter standards that applied in the past, back when most of the evidence on helmet efficacy was collected. In a table on the fifth page they mention that a study found no evidence of helmets causing or exacerbating rotational injuries — yet this is the only mention they make of the rotational injuries problem. Their inclusion and omission criteria appears to be completely random.

Anyway, enough of this. I don’t want to hog the game — your turn.