Cause for Effect — The Perils of Forecasting
When developing a predictive model, I take steps to avoid making hasty generalizations. This mistake would occur if I came to a conclusion about a population based on a sample size that is not large enough.
Another type of mistake is false cause. Consider people who decide not to get their car washed because the last several times they did, it rained. This is a false cause and effect relationship. Getting a car washed doesn’t cause rain. It rains because wind brings clouds in.
Closely related to this is coincidental correlation. For example, we know there is a direct relationship between wash volumes and traffic count but not causation. The rise and fall of traffic counts doesn’t cause wash volumes to rise and fall, the behavior of motorists does.
Another mistake can arise if someone infers that something is true of the whole from the fact that it is true of some part of the whole. For example, if I surveyed three charity washes in a city and found all three washing cars without containment mats and wasting water, I may wrongly infer that all charity washes do so.
An error can also occur when everything else is right but the data are wrong. For example, several years ago, the Washington State Department of Ecology (Ecology) estimated the amount of oil flow-ing into Puget Sound equaled roughly an Exxon Valdez spill every two years.
Ecology claimed that, on an average day, the rinsing of vehicles from unregulated car washing allows 140,000 pounds of toxic chemicals to enter the sound including oil, petroleum products, and heavy metals.
Subsequently, the Puget Sound Partnership (PSP), the agency attempting to restore the sound, declared that an Exxon Valdez-size spill of toxic chemicals poured into the sound every two years. PSP explained, every year the rains wash millions of gallons of oil from roadways and parking lots, and it trickles into the sound and kills the fish. However, last April, Ecology admitted to grossly overestimating the amount of oil reaching Puget Sound.
According to Washington State Reporter, Ecology is due to release a report that is expected to show that oil’s contribution to the volume of toxic pollutants in Puget Sound is somewhere between one hundredth and one thousandth of what was estimated four years ago.
Of course, the folks who depended on Ecology’s earlier findings to tell people how bad things were are now in the position of having to tell people it’s not nearly as bad as that.
Hopefully this gaffe will not hamper the fine efforts of agencies and companies that encourage consumers to help reduce pollution and toxic loading by having their vehicles cleaned at a professional car wash or charity wash that uses proper techniques.
In forecasting, all methods depend on an amount of guesswork because no single model can include all variables that influence an outcome. So, forecasters are always looking for ways to reduce variance in a model and improve accuracy through better procedures.
However, Ecology’s error was not a model or the science. Rather it was the assumptions used and the math. In short, the study confused total petroleum hydrocarbons with oil and grease and the calculations assumed too little runoff from land, too much from development.
Arguably, correcting these errors doesn’t make the problem go away. Even at a thousandth of the wildly overestimated amount, a lot of oil and toxic chemicals still washes into Puget Sound.
The actual amount of pollution may be much less, but this doesn’t change the fact that professional car washes and other responsible car washing need to be key components in the equation to mitigate the problem.