Helmet study fails to consider denominator and confuses science with policy

A study is being presented in DC today at the Pediatric Academic Societies (PAS) annual meeting that claims that "bicycle helmets save lives, and their use should be required by law."

We conducted a cross sectional study of all bicyclists aged 0-16 years included in the Fatality Analysis Reporting System (FARS) who died or suffered an incapacitating injury between January 1999 and December 2009. The FARS defines an incapacitating injury as one that prevents a person from walking or normally continuing the activities the person was capable of before the injury. Date law enactment was obtained from several sources including the Insurance Institute for Highway Safety and the Governor's Highway Safety Administration. We compared rates of deaths/incapacitating injuries per age-specific state populations, between states with helmet laws and those without helmet laws. We used a clustered Poisson multivariate regression model to adjust for factors previously associated with rates of motor vehicle fatalities: elderly driver licensure laws, legal blood alcohol limit (< 0.08% vs. ≥ 0.08%), and household income.

Or

To determine if such laws reduce national injury and death rates, Dr. Meehan and his colleagues analyzed data from the Fatality Analysis Reporting System on all U.S. bicyclists younger than 16 years of age who were severely injured or died between January 1999 and December 2009. They compared the injury and death rates in states with mandatory helmet laws to those without.

Of course there are problems with this becauseit just counts raw numbers.

It doesn't consider if states with mandatory helmet laws have less cycling, and of course there is evidence that mandatory helmet laws do reduce cycling. So we have the numerator (injuries/deaths) but not the denominator (cyclists or miles biked). Here is another study that showed the same thing, but considered the change in cycling. 

 In recent years, many states and localities have enacted bicycle helmet laws. We examine direct and indirect effects of these laws on injuries. Using hospital-level panel data and triple difference models, we find helmet laws are associated with reductions in bicycle-related head injuries among children. However, laws also are associated with decreases in non-head cycling injuries, as well as increases in head injuries from other wheeled sports. Thus, the observed reduction in bicycle-related head injuries may be due to reductions in bicycle riding induced by the laws.

In addition it doesn't consider what injury rates were before the laws were enacted. Maybe helmet laws are only enacted in states where there are a lot of cyclists, and states with a lot of cyclists are safer for cycling.

Finally, the second conclusion - that their use should be required by law - is some shocking bit of science. It ignorse the fact that we can determine that helmet use saves lives AND that laws mandating them are wrong. Sunscreen saves lives, we don't mandate it's use. Nor do we mandate condom use or bullet proof vest use. Universal organ donation would save lives, but we don't require it by law because there are other things we value in addition to - and even more than - saving lives.

Scientists should stay clear of these kinds of statements that require considering values in their official work. Saying that helmets save live and helmet laws save lives is something the data can (or not) defend. But saying we should value that over freedom or other concerns is total opinion. Scientists are free to weigh in with their opinion, they just should separate that which they can prove from that which they can not. 

Comments

[ACyclistInTheSuburbs]

the study, by pediatricians, only looked at safety for riders under 16, and looked at helmet laws impacting those under 16 (because AFAIK no US state has laws mandating helmets for adults) Leaving aside any other methodological issues, this cannot be generalized to laws mandating helmets for adults. And thinking about it, it makes sense that that would be different - since the major neg of helmet laws (for those of us who accept that RIDING with a helmet is safer) is the critical mass effect. that effect is likely driven by adult cyclists for many reasons 1. Adult cyclists are more likely to ride in mixed traffic, and so will do more to make drivers aware - 2. Adult cyclsists are more likely to lobby for infrastructure 3. Adult cyclists are more likely to themselves be drivers -one of the benefits of more riders is that more drivers are riders

[JimT]

The 25% seems believable,
though of course the study can not determine what portion is because kids rode their bikes less (due to either secular trends or the laws). Basically, I think they would need to have some states that repealed helmet laws to be sure of attribution.

For all the flaws of case-control studies, this type of study is even less reliable.

[Crickey7]

I'm not getting too worked up about it. It recommends laws mandating use by children. It's silent as to adult use.

[SJE]

The other problem is that mandating a policy makes it harder to reverse that policy once the unexpected side effects occur. For example, they mandated all sorts of laws to improve survivability in a vehicle crash, but had the other result of (a) removing driver incentive to be safer (b) making cars more dangerous for non-cars (c) increasing vehicle weight.

[Smedley Burkhart]

Suppose we were to stipulate that the reduction in incidence were entirely due to a physical protective effect. A 19% reduction of a 2.5 in 1,000,000 risk requires quite a lot of balancing against other factors-- discouragement of physical activity and simple loss of autonomy, to name a couple--before advocating a mandate.

That's the kind of mindless nannyism that's making this old liberal turn red in spots.

[charlie]

helmets are good! laws mandating them are bad!

[oboe]

"Sunscreen saves lives, we don't mandate it's use."

Sure, but normal people use sunscreen. Biking helmets are used by bikers. And bikers are freakish weirdos on the fringe of society who must be kept under control.