Tuesday, July 12, 2011

“If the party says there’s no cost overrun, there’s no cost overrun.”

Virginia Postrel writes in Bloomberg about Bent Flyvbjerg's work on the exceptionally poor forecasting used to justify transportation projects. She confuses the meaning of average and median and ignores the substantial variance in forecast error, but the overall message is correct in that there are biases built into the system of financing transport projects that skew forecasts of usage and costs. Flyvbjerg explains it doesn't have to be this way:
He would like to see better incentives -- punishment for errors, rewards for accuracy -- combined with a requirement that forecasts not only consider the expected characteristics of the specific project but, once that calculation is made, adjust the estimate based on an “outside view,” reflecting the cost overruns of similar projects. That way, the “unexpected” problems that happen over and over again would be taken into consideration.

Such scrutiny would, of course, make some projects look much less appealing -- which is exactly what has happened in the U.K., where “reference-class forecasting” is now required. “The government stopped a number of projects dead in their tracks when they saw the forecasts,” Flyvbjerg says. “This had never happened before.”

Or there is the Chinese approach to avoiding cost overruns:
Unfortunately, the world’s biggest infrastructure projects, including the recently opened high-speed rail line between Beijing and Shanghai, are subject to no such checks, or even to scholarly examination. Flyvbjerg has been trying for years to get data on project costs in China, to no avail. “Their data are simply not reliable,” he says. He quotes an unidentified Chinese colleague who said, “If the party says there’s no cost overrun, there’s no cost overrun.”

Others have looked at other reasons for forecast inaccuracy. Here is a paper by Pavithra Parthasarathi and David Levinson in which they consider the effect of demographic forecasts on roadway forecast inaccuracy. There are reasonable as well as unjustifiable reasons for forecast inaccuracy, but ultimately all forecasts are wrong. We can make better predictions but we will never be perfectly right, but we can do better.

No comments: