How Our Primary Forecasts Work

FiveThirtyEight explains How Our Primary Forecasts Work

"One hugely mistaken assumption would be to look at the margin of error associated with the poll. FiveThirtyEight has a database consisting of thousands of primary and caucus polls dating back to the 1970s. Each poll contains numbers for several candidates, so there are a total of about 17,000 observations. How often does a candidate’s actual vote total fall within the theoretical margin of error?

The answer is, not very often. In theory, a candidate’s actual vote total should fall outside the margin of error only 5 percent of the time. In reality, the candidate’s vote total was outside the margin of error 65 percent of the time! Part of this is because the database includes some polls conducted months before the actual voting took place. But even if you restrict the analysis to polls conducted within the final week of the campaign, about 40 percent of the vote totals fell outside the margin of error — eight times more often than is supposed to happen if you could take the margin of error at face value."