If Your Forecasts are Always Wrong, Should You Stop Making Forecasts?

Engineers and planners are regularly tasked with predicting the future. Prediction exercises are useful for thinking about what facilities and investment we need in the future. Prediction exercises are also invariably wrong. Errors occur because trends change, preferences evolve, external forces intervene and/or models are flawed. In transportation and land use planning, these errors lead to permanent changes in our cities, often for the worse, yet no one is held accountable for these errors. Prediction error is not limited to transportation, however. Here is a link to an engineer who laments the limits of his work. (h/t David Levinson)  And here is an op-ed in New Scientist arguing that economic prediction models have much to learn from meteorology. This sounds promising as the weatherman is never wrong.

But here is something we can learn from weather forecasters. Two prominent hurricane forecasters have quit making long range forecasts because the forecasts are always wrong. They will only do near term forecasts. From the Ottawa Citizen:

OTTAWA — Two top U.S. hurricane forecasters, famous across Deep South hurricane country, are quitting the practice of making a seasonal forecast in December because it doesn’t work.

William Gray and Phil Klotzbach say a look back shows their past 20 years of forecasts had no predictive value.

The two scientists from Colorado State University will still discuss different probabilities of hurricane seasons in December. But the shift signals how far humans are, even with supercomputers, from truly knowing what our weather will do in the long run.

Colorado State has been known for decades for forecasts of how many named storms and hurricanes can be expected each official hurricane season (which runs from June to November.)

Last week, the pair made this announcement:

“We are discontinuing our early December quantitative hurricane forecast for the next year … Our early December Atlantic basin seasonal hurricane forecasts of the last 20 years have not shown real-time forecast skill even though the hindcast studies on which they were based had considerable skill.”

The two will still make the traditional forecasts closer to hurricane season.

An earlier version of this story incorrectly said they were stopping all forecasts.

I think this is a promising approach and can be applied to transportation systems. If your long-range forecasts are always flawed, the best solution may be to stop making long-range forecasts.