The following article was published in November 2001.

Hawthorne Experiments

Recent copies of our Newsletter have given you some accident statistics that demonstrate that the major emphasis on traffic calming schemes and speed reduction measures (e.g. hundreds of extra speed cameras, lots more speed bumps), seem to have negligible impact on overall accident statistics, which stubbornly refuse to come down.  This is despite the fact that there are studies that clearly appear to demonstrate the effectiveness of these measures based on before/after studies of accident statistics in particular locations.  There are three major reasons why these statistics are misleading:  

Firstly because they often ignore the effect of diverting traffic. To be accurate you need to take account of the changed volume and mix of traffic which is rarely done.  

Secondly they often fall into the common traps of using selective statistics (ie. the bad comparables are ignored and the good ones published), or they don’t allow for extraneous factors such as weather conditions,  or they ignore random statistical variation. Rarely are “confidence levels” attached to the numbers as they then make poor political headlines (in fact the statistics are usually based on such poor experimental design that it would be folly to do so anyway).

Thirdly though they totally ignore the major problem when experimenting on human beings, that predictions tend to be self fulfilling.  This was clearly demonstrated back in about 1930 in a series of research projects in industrial psychology undertaken by Elton Mayo and known as the Hawthorne Experiments (there are several references on the Internet to this work if you want more details as it is a classical study in this field).  One of the things he did was to test the effect of increasing or decreasing lighting conditions in the workplace. With an increase, he expected an improvement in output, and got it. With a decrease, he was expecting a reduction, but got an increase. In other words, any change improved performance. Why was this?  Because the subjects expected the change to improve performance because they knew that was what the experiments were about, and hence it did. Behaviour changed to match peoples expectations.

So let’s take up the analogy with the introduction of speed cameras. People expect the installation of speed cameras will reduce the number of accidents (after all we are told they are only sited at accident black spots), so in fact they might well react accordingly, ie. they will act to match their expectations. How long will this effect last: well quite a long time according to Mayo, but clearly it could not last for ever because otherwise you could simply keep changing the environment and endlessly improve performance.  

One of the clear conclusions is that when experimenting on people you have to be very careful when interpreting the results. This is why medical experiments typically use a double-blind technique where neither the subject not the collector of the statistics knows who is getting the real medicine or who is getting the dummy.  

To really produce proper before/after studies to measure the effectiveness of accident prevention measures, you therefore have to be exceedingly careful. Certainly it must be extended over a long period of time so the Hawthorne effect wears off. Secondly, you should also try removing the change to see what effect that has, or introduce other similar but different measures to see whether any change in the environment stimulates the same change. For example, compare the effect of a real speed camera, with a  sign warning of hazards ahead.  Also you need to separate the collectors of the statistics from the interpreters (in practice they are the same police at present). Unfortunately it is so difficult to do this kind of study in an unbiased and effective manner that in practice it is unlikely ever to be done properly.

Final Comment:  Take any claims for breakthroughs in traffic accident reduction with a pinch of salt.

Back to Road Safety page:  Road Safety