Criticism of the MSF - A Sanity Check

The MSF is what most of us have available for motorcycle safety and skills training. However, the MSF has its critics, both for its teaching technique (which some educators dislike), for its business methods and for its outcomes.

Criticism of the course includes the effectiveness of the training in reducing crashes, and the injury and fatality rate of course participants.

After being challenged by Gymnast of msgroup.org to read Wendy Moon's blog I read four months of it and sampled the rest.

Moon's blog is huge, but there are some recurring themes, so I will talk about some of them here.

MSF as a Trade Group

(1) The MSF is not a charity, it is a 501 (c) 6 organization. This means it is a trade group associated with the motorcycle manufacturers. MSF confirmed that MSF is a 501 (c) 6 organization by phone on June 25th, 2009. There are a couple of critical differences between 501 (c) 6 trade groups and 501 (c) 3 charitable organizations. The trade group is allowed to lobby government, unlike charities, and it is also required to act in the interests of its members (in this case the bike manufacturers).

Bike Safety Engineering Discounted

(2) The MSF, per Moon, may have been instrumental in the different treatment of the car industry and the motorcycle industry in the 1966 National Traffic and Highway Safety Act, which was at largely due to Ralph Nader's famous 'Unsafe at any speed' public campaign on car safety. The outcome, according to Moon, is that car makers are now responsible for the safety of their product and subject to enforced incorporation of safety features, like 3-point seat belts and ABS, into their vehicles. Motorcycle manufacturers, on the other hand, escaped this control, and, to this day, bikes don't have standard features like dual brake lines and ABS that are mandatory on cars. The result is that motorcycle safety is mostly a function of rider education and enforcement and lacks the engineering component which is an important part of car safety.

My take on this part is that Moon is right about the outcome. Features like ABS, airbags etc are available on some bikes, usually as an option. They could be on all bikes. My problem there is that, although I believe ABS is good (and paid an extra grand to get it on my last bike) a lot of people say it is ineffective on bikes, and though I like this study from Promocycle that is persuasive that ABS increases braking deceleration, I don't think it is actually proven that ABS is a safety benefit. I'd eat my helmet if someone proved it was not a benefit. I'd agree, it would be hard to argue against dual brake controls, especially in the light of the Maids findings on brake failure as one of the major bike failure modes in accidents. I'll accept Moon's argument on the outcomes. Our point is that we don't have the same degree of agreement about what is safe on bikes, compared to cars, due to lack of research. This might also be a factor in the lack of progress on bike engineering safety. Bike riders inherently accept a greater level or risk than cage drivers, and many bikers resent government interference. Motorcycles can't have as much redundancy as cars.

As to the process, I would point out that National Traffic and Highway Safety Act was passed in 1966 and MSF was founded in 1972, so it would be hard to see MSF's hand in the creation of the separate safety climates for cars and bikes. Maybe the motorcycle industry did some lobbying, I'd be surprised if they didn't. The fact of the matter is that motorcycle safety is much more the responsibility of the rider that car safety is of cage drivers. That's the way it is, let's just deal with it.

MSF is required to work on behalf of the manufacturers, who have an interest in avoiding legislated safety engineering improvements, and if that happens they might actually be working against biker safety. I am grateful to Moon for pointing this out. But I would also mention that the industry also has other groups that can lobby on their behalf, including the Motorcycle Industry Council, and while I am not an expert on lobbying, surely it would make more sense for the industry to use a group like this for lobbying on the safety engineering question?

The manufacturers have an interest in keeping their customers alive and able to ride and buy their product, which is why they support MSF. They also have an interest in reducing the perception of excessive risk in motorcycling. These interests are aligned with bikers and much closer to the public mission of MSF. How about we give MSF the benefit of the doubt on motivation and instead judge their actions? I don't see where Moon says they actually lobbied against bike engineering safety improvements. .

If riders wanted and were prepared to pay for safety engineering improvements, the manufacturers would want to sell them. What Nader achieved was a consumer demand for more safety engineering enforced by law. If there is a biker Nader out there in the wings, I'd say, start her up, I'll support him. I might be wrong, but I am not sure that a majority of bikers would be behind such a campaign, as we are are a prickly lot that don't like government interference much.

MSF Training Deaths

(3) This blog entry from March 2009 talks about deaths and serious injuries during MSF training. Apparently, Moon received information from a lawyer representing a woman who was suing MSF for injuries she received as a bystander when a bike went out of control and hit her car in 2006, causing her injury. As part of discovery, her lawyer received some accident reports from MSF. MSF then settled the case for what seems like a large amount and got the accident reports back from the lawyer. Moon is citing the crash report documents without being able to produce them. Her entire article and many other postings have an air of the conspiracy theory, imho, including the implication of cover-up.

My own search found one death of a MSF instructor in 2006, who was not riding a bike at the time. The subject is not search-friendly.

I asked MSF for information on student deaths during training and they responded with this statement, issued June 29th 2009.

Riding, especially learning to ride, has inherent risks. MSF is concerned about any crash that occurs, whether it's on the road or during training. We take safety seriously in creating the best environment to pursue one's dream to ride. A primary goal of the MSF is to ensure a low risk, positive learning environment for beginning students so that they can make the best choices while learning and riding.

Since its founding in 1973, more than 4.6 million students have been trained using Motorcycle Safety Foundation curricula, including approximately 2.5 million since 2002. MSF prides itself on making the highest quality research-based and field-tested motorcycle training curricula available to riders and prospective riders throughout the United States and the world.

MSF is unable to disclose details related to fatalities because of privacy considerations. However, since there has been some misreporting on this subject, MSF welcomes the opportunity to provide factual information.

Since 2002, out of the roughly 2.5 million students trained, there have been six crashes that resulted in the death of students, including one that was caused by a serious medical condition. In the past year, three additional students died from medical conditions while not riding. Every fatality has been thoroughly investigated by law enforcement, insurance investigators, or others. The curricula, and the delivery of the curricula by RiderCoaches, have never been determined to be a factor. MSF employs a stringent quality assurance program as part of its ongoing effort to review and refine policies and practices to minimize the inherent risks associated with training.

We also regret any deaths in training. This entire site is dedicated to safer riding. There are some questions to be answered, especially related to an increase in deaths since 2002. I can't answer them. We recommend that all new riders take a basic training course and obtain their motorcycle operator's license before getting their first bike, and review the suggestions on your first bike in the biker-ed page. These are life-and-death decisions.

Bike Safety Training Effectiveness

(4) Moon writes many articles alleging ineffectiveness of bike safety training and MSF training in particular. There is way too much material to go through it all, so what I did was pick out the first post I came to and checked the sources. This is what I found:

An Australian study from Monash: It concluded

"most of the evaluations of training courses set out to determine whether the courses had any effect on licensing rates, crash involvement, infringements and/or the extent and nature of riding. Methodological deficiencies prevented these aims being achieved in most studies".

This was a literature review paper, which is a summary of other people's work that academics do when they haven't any research of their own to report. This paper was general in nature and found that the research they looked at was basically useless.

What Moon said was that it

"describes and discusses nine of the studies that have been done on MSF curriculum from 1980-1995. None of them found that, adjusting for miles traveled and frequency of riding, gender and general risk-taking behavior in other activities, that the MSF course reduced crashes. At best, untrained riders had 10% more crashes in the first six months of riding. At worst, several studies found that riders trained with MSF curriculum were more likely to have crashes, some found they were more likely to have traffic offenses".

Moon omits to mention that the MSF course studied were the old Motorcycle Rider Course, not the current BRC, and the old MOST skills test. Some of the studies reported good results and some reported bad results from training. She omits to mention that the British, Canadian and Australian training and testing programs studies also produced similar mixed results. And she leaves out completely the study's main conclusion that the studies in general were performed so poorly that the results are useless. She picks out a few sentences that appear to support the point she is making and ignores all contradictory evidence. Read the paper yourself and make your own judgement.

Billheimer's California Study

She also uses Billheimer's study. This was done on the California Motorcyclist Safety Program (CMSP) - the MSF and their training products are not mentioned at all in the article. Moon chose to mention only:

" Rather, one of the greatest benefits Billheimer’s study found was to discourage riders who failed to pass the course from riding".

Billheimer did say this, but also said:

"In the six months following training, the accident rate of trained novices drops significantly to 0.39 accidents per 100,000 miles, less than half of the rate of 0.85 accidents per 100,000 miles attributed to their untrained counterparts",

which Moon does not mention at all. This is accompanied by a prominent graph showing a large drop in overall accidents in the years following introduction of mandatory training. She condemns MSF training using a generally favorable report on someone else's training, which I consider an amzing feat of semantics. The references are there, you can check this for yourself. As the first six months of riding a bike are the most dangerous, anything that would halve the crash rate is huge. I'm not sure of the study methodology here either, although Moon doesn't state a problem with it while ignoring its main conclusions.

Transportation Research Board article

Moon mentions this article from the Transportation Research Board The link is just an abstract, they want money to see the whole thing, which I won't pay. Moon quotes: “those individuals who took beginning rider training courses were more likely to be involved in an accident than those who did not and that those who took the beginning course more than once were much more likely to be involved in an accident.”.

Moon omits to mention the next sentence in the abstract "Although explanations for these findings can range from the use of ineffective course material to changes in risk perception as a result of taking the course, another explanation is that riders who take the course are inherently less skilled than those who do not. The findings underscore the need for a careful and comprehensive study of rider skills and risk perceptions to maximize the effectiveness of motorcycle training courses.". (emphasis is mine).

I agree with this last sentence. The research we have is flawed in many ways and only a comprehensive, well-funded, quality-controlled study will sort this information vacuum. Moon's conclusion "...All the evidence points to minimal if not negative benefits to taking the MSF course..." is not supported by the material she cites. All we can really take from the quoted articles is that most of the existing research in the area is poorly done and doesn't support any conclusions, which in any event were mixed.

A note on course material: one of the arguments made by TEAM Oregon, the winner in a recent copyright infringement case taken by MSF was that both the MSF and Oregon course material originated from 'traditional' material that had been in use for decades before MSF and TEAM Oregon. If the MSF is using ineffective course material it would seem that everyone else probably is also, as they are all legally similar. My the way, slap on the wrist to MSF for prosecuting this losing case and wasting funds that could have been used for training.

Lobbying and State Contracts

(5) There are large chunks of the Moon blog which recounts various shennanigans involving lobbying, approval of course content, interactions with competitors, etc. There is a lot of linking of events and inference of cause and effect without any proof the events are connected. My summary of these parts of the blog is that it is somewhere between gossip and conspiracy theory. I don't believe there is anything to answer here. The general slant is that Moon doesn't like some competitive tactics she alleges MSF is using to obtain work from state governments. If MSF was indeed a charity and involved in lobbying, then there would be a problem, but this is not an issue for a 501 (c) 6 group. I didn't see any evidence of improper practices like bribery or corruption.

We live in a capitalist system which is red in tooth and claw and thrives on competition. If MSF is competing a bit robustly, then it would behoove their competition to get their act together and compete right back. If they do something illegal or unethical, and Moon has evidence, then out with it. I would have a concern if MSF seemed to be getting a monopoly, but my understanding is that they control only four state bike safety administrations, so they are far from triggerng anti-trust sanctions. If a monopoly did arise, then probably the product would suffer, but I don't see any real evidence that is happening.

My conclusion is that this part of the blog has no real safety impact, so I intend to ignore it.

Summary: get some training.

In summary, Moon's training effectiveness postings suggest that we need better research, and we might need to improve the curricula when we have the information to do it with. A point that I totally agree with. The new accident causation study should sort that.

I'm not even sure about posting this, I hate to publicize such 'research', which exemplifies one of the problems of the internet - anyone can publish anything with no fact checking -, but I feel that this stuff needs to be debunked.

I would be horrified if someone read Moon, decided not to do some basic rider training and went out and injured themselves. Moon seems to agree that most of the operational people in MSF are earnest and qualified, and this accords with my personal experience of MSF trainers during the two courses I have taken with them. I haven't see her dispute the generally-acknowledged problems of rider skills and strategy deficits contributing to most bike accidents. She appears to support the efforts of groups like Team Oregon, which employs legally similar course materials to achieve the same objective. But her postings of MSF course effectiveness (or at least the one I dissected in depth) are innaccurate and slanted and have the effect of bringing the MSF into disrepute. If the training is ineffective, I agree it would need to be fixed, but that point is far from proven by this material.

The effect on any new rider, who might skim the material, take the message that basic rider training has no benefit and skip training could be catastrophic or fatal. This writer seems to have no compuction about playing with peoples lives.

If you are a new rider, please get some training, from any qualified provider, and be aware that the presence of trained personnel and a controlled environment does not guarantee your safety doing what is a very risky activity. The training organization might have some flaws, but it is an established fact that lack of skills and defensive riding technique is very likely to kill you, especially in the first six months of riding. Don't add to the already great risk of riding by going out there unprepared.

Perspective

Five training deaths in seven years is an average of less than one, very sad and regrettable, death per year. Every year, over 5000 bikers die in crashes. Lack of skills and training probably contribute to half of these deaths. Your risk of not getting trained is thousands of times the risk from getting trained. Your chances of dying in training is one in half a million.

MSF is not the enemy. They have their faults, but MSF and the many independent training organizations nationwide are a critical resource for every biker, and are, on balance, saving many hundreds of lives every year.

Footnote: In reply to my considered and reasonably argued refutation of Moon's bloggings above, Moon replied in several posts. She did not present any counter-arguments, just ad-hominem attacks such as ET is " ...obscure ... has no qualifications .. ignorant .. an Internet Troll of a particularly offensive nature". Resorting to ad-hominem attack (an attack on a person rather than an argument) is an admission that she has no rational argument to make.

About the MSF's statement, Moon says: "...But note how it describes the three deaths in the past year: “from medical conditions while not riding”.  However, if a student ran into a wall, for example, and suffered head and/or thoraic trauma that could be truthfully described as a “medical condition.” Of course in a crash, a rider ejects from the bike. If the rider did not literally die on the bike...".

Medical conditions while not riding - how could that be clearer?

From henceforth, ET will be known as "Internet Troll, the biker formerly known as ET". It is a badge of honor.