Helmet study author says not to consider the raw numbers

Janessa Graves, the author of the bikeshare helmet study showing that the percentage of injuries that were head injuries for cyclists went up in bikeshare cities, responded to criticism of the study. That criticism focused on the fact that the study's raw data showed that total head injuries and total injuries both went down in bike share cities, even though the rate of head injuries went down less than other injuries.

The study’s lead researcher, Janessa Graves of Washington State University’s College of Nursing, said in an email that these numbers don’t tell the whole story, which is why the researchers focused on proportion.

“Evaluating crude numbers alone, without considering the underlying population or denominator (e.g. number of riders in each city), is not entirely appropriate, even when we assume ridership increased,” she writes. “We did not have those numbers for this study, so could not evaluate the NUMBER or RATE [emphasis hers] of injuries. That is why we looked at proportions and risk.”

Graves adds that because her team did not know whether ridership increased, decreased or stayed the same in cities with bike-shares, they were reluctant to extrapolate. The total number of injuries may have gone down, but what of the total biker population?

Some critics of the study, she states “assume that the number of cyclists increased in bike-share cities and likely stayed the same in non-bike-share cities, however, we do not know this for certain. That is why we could not look at this outcome.”

Rachel Dovey concludes that 

Shares would be wise to implement policies based on the higher proportion of brain injuries reported.

I'm not sure that is true, because looking at only the percentage without considering the denominator is just as bad (if not worse) as looking at the numerator without considering the denominator.

Shares should be interested in ways to improve safety. I'm just not sure what the most cost-effective way is to do that, and I don't think this study gives any direction on that. 

Comments

[contrarian]

I can't recall ever reading a bicycle safety study that had any level of statistical rigor.

I'd be interested in seeing who funded the research (the article is paywalled though). It seems there is a never-ending stream of money to fund research of dubious quality, as long as the finding is bike helmets good, bike lanes good, or VC bad.

[DE]

If you don't have the data to come to any meaningful conclusions, then you simply don't have the data. Either get good data, or give it up; it is simply irresponsible to attempt to come to conclusions that the data don't support. It's almost as if they had decided beforehand that helmets improve safety...

[Crickey7]

Facts?

We don't need no stinkin' facts.

[Smedley Burkhart]

J. M. Graves received fellowship support from National In- stitute of Child Health and Human Development during the course of this research (PI: Rivara, T32 HD057822-01A2). This work was also supported by the Harborview Injury Prevention and Research Center, University of Washington, the National Trauma Data Bank, the Quebec Trauma Registry, and the Vancouver Trauma Registry.

[Smedley Burkhart]

Happy to send anyone the article, too.

[Jonathan Krall]

Is she suggesting that the decrease in injuries in bikeshare cities might be caused by a decrease in ridership rather than a decrease in injury rates?!?

In my book, that wouldn't pass the laugh test. Aren't there a gazillion studies out there showing bikeshare increases ridership?

[Jonathan Krall]

Reading the article more carefully, the researcher is saying that instead of bikeshare cities having a decrease in ridership, non-bikeshare cities might have increased ridership faster than bikeshare cities. Again, this doesn't pass a laugh test.

[SJE]

There are so many problems comparing between cities, notably that bikesharing typically comes with other bike-friendly policies (e.g. lanes, enforcement).

[ACyclistInTheSuburbs]

The problem is the decision to derive a policy consequence from the research. The author is correct that we do not know for certain what happened to ridership. Ergo, the proper conclusion is that there is no obvious direct policy application from this research, other than the need for better data on ridership. That does not mean the research has no value, as someone might well find that info on the change in injury mix is useful for some purpose the authors cannot discern. It is unfortunate that some researchers seem to believe that a direct policy conclusion is needed to justify their research - that leads to unjustified recommendations, as in this case.

[washcycle]

+1 ACyclistInTheSuburbs

[Smedley Burkhart]

I agree, too, and would offer one additional comment. To wit, the assumption that helmet use by bikeshare users would have mitigated the population of injuries sampled in this case is invalid, for reasons often discussed. Rather than assuming this to be the case, the investigators should have made it part of their hypothesis.

A better study would have distinguished helmet-mitigable injuries, i.e., skull fractures, from other injuries, independent of severity. This also undermines the validity of the recommendations.

[contrarian]

Smedley, I think you would have a hard time coming up with a rigorous definition of a "helmet-mitigatable injury." The evidence for the effectiveness of helmets is pretty weak.

[DE]

I suppose if you are going 5 mph and fall sideways into a curb. Onto your head. At a 90-degree angle such that the helmet doesn't cause much in the way of extra torsional to be forces exerted. Is that all possible?

[Michael H.]

This just in: 100 percent of cyclists taken to the hospital after an incident were injured!

[Smedley Burkhart]

Contrarian, I think looking at skull fractures and epidural hematomas would at least target what helmets are believed to mitigate. Of course that would likely reduce the sample size to something pretty unreliable.