In the latest New Yorker, James Surowiecki writes about the trouble with gaging the health of the economy only through the two sets of employment figures reported by the government each month. His point is that even though these monthly estimates are well constructed and reasonable, commentators tend to focus on the false precision of the jobs number and ignore the margin or error and other sampling techniques that are also part of the estimate. Unfortunately this problem is not limited to the chattering classes, who can be expected to get basic statistics wrong for whatever reasons. False precision is everywhere, including in fields where people should know better.
Donald Shoup has written about how false precision harms public policy. Here is his abstract:
Transportation engineers and urban planners often
report uncertain estimates as precise numbers, and
unwarranted trust in the accuracy of these precise
numbers can lead to bad transportation and landuse
policies. This paper presents data on parking
and trip generation rates to illustrate the misuse of
precise numbers to report statistically insignificant
estimates. Beyond the problem of statistical insignificance,
parking and trip generation rates typically
report the parking demand and vehicle trips
observed at suburban sites with ample free parking
and no public transit. When decisionmakers use
these parking and trip generation rates for city planning,
they create a city where everyone drives to
their destinations and parks free when they get
there.
In his research he looked at transportation planning and argues that is is better to be roughly right than precisely wrong. I think this holds for employment statistics and financial markets as well.
No comments:
Post a Comment