EVmodcast provides an apples-to-apples comparison of three leading poll-based models aggregating an Electoral Vote outcome for the 2012 US presidential election: Nate Silver’s FiveThirtyEight@NYT, Simon Jackman’s model in HuffPost Pollster, and Sam Wang’s Princeton Election Consortium.
What do you mean apples-to-apples?
If we only consider Nate Silver’s Now-cast**, it is conceptually reasonable to compare the three models side-by-side as each calculates an Electoral Vote (EV) outcome based on the aggregate of polling information up to the present day, the only difference being the aggregation method and the assumptions underpinning them (which is exactly what we are interested in). However, the models vary not only philosophically but also in presentation, which voids a strict comparison. FiveThirtyEight uses repeated simulations to achieve its EV distribution and relies on the mean as its model call, instead of the median as with Princeton Election Consortium. The HuffPost model does not compute an EV distribution at all, instead relying on the conventional Strong-Lean-Tossup domino format to call the outcome.
Nevertheless, because the three models are structured similarly in that they all track each state separately (though not always independently) and quantify the uncertainty in the outcome of each state via a probability estimate, all three models are amenable to standardized summary statistics which allow for a direct comparison. For each model, I compute the exact electoral vote (EV) distribution based on 51 state win probabilities as ingeniously highlighted by Sam Wang. Now we can use the median of the distribution as the model’s call for a particular day’s data, bracketed by the 5th and 95th percentiles as its uncertainty estimate (which is to say the interval captures 90% of all possible outcomes).
How is the information presented?
The site is dominated by the ModelTracker at the top of the page which adds data points for the day’s three aggregates. The tracker begins on Mon 17 Sep 2012 for all models except for FiveThirtyEight, which begins on 26 Sep 2012 due to data error. In addition, I post daily ModelSnapshots which display what the respective EV distributions actually look like, along with how their median, 5th and 95th percentiles are likely to translate into the Electoral College.
**The FiveThirtyEight Nov.6 Forecast provided on its main page is not comparable with the other models as it is (as the name suggests) a forecast of the EV outcome, not a ‘snapshot’ aggregator like the others. Which is to say that operationally the model incorporates additional assumptions about how the polls are going to move from now till Election Day into its probability estimates, which does not happen with the other two. EVmodcast only compares models which estimates the EV outcome if the election were held today. (Thanks to Professor Drew Linzer for pointing this out, who runs an elegant model at Votamatic but which is also a forecast and thus not amenable to comparison on this site).