An Analysis of the Accuracy of Forecasts in the Political Media
"In this paper, we report on the first-ever test of the accuracy of figures who made political predictions. We sampled the predictions of 26 individuals who wrote columns in major newspapers and/or appeared on the three major Sunday television news shows (Face the Nation, Meet the Press, and This Week) over a 16 month period from September 2007 to December 2008. Collectively, we called these pundits and politicians “prognosticators.” We evaluated each of the 472 predictions we recorded, testing it for its accuracy."
"We discovered that a few factors impacted a prediction's accuracy. The first is whether or not the prediction is a conditional; conditional predictions were more likely to not come true. The second was partisanship; liberals were more likely than conservatives to predict correctly. The final significant factor in a prediction's outcome was having a law degree; lawyers predicted incorrectly more often. (R-square of .157) Partisanship had an impact on predictions even when removing political predictions about the Presidential, Vice Presidential, House, and Senate elections."
"We have discovered a number of implications from our regressions and analysis of the data. First, we have discovered that six of the analyzed prognosticators are better than a coin flip (with statistical significance.) Four are worse, and the other 16 are not statistically significant. A larger sample can provide better evidence addressing the question of if prognosticators on the whole are better than a coin flip. We understand that being better than a coin flip is not a high bar to set, but it is a serious indictment of prognosticators if they are, on average, no better than a flipped coin."
FYI, Krugman won.
No comments:
Post a Comment