PollyVote Panel at the 2013 APSA Meeting in Chicago

The PollyVote team consisting of Scott Armstrong, Alfred Cuzán, Randall Jones and Andreas Graefe organized a panel at the 2013 APSA Annual Meeting in Chicago. Below you can find the slides for each of the talks as well as links to the full papers if available. 

Innovations in Presidential Elections Forecasting: The Pollyvote 

Armstrong, J. S.; Graefe, A.; Jones, R. & Cuzán, A.: The PollyVote: Progress and Plans.

Armstrong, J. S. & Graefe, A.: Index Models for Advising Candidates.

Graefe, A.: Accuracy of vote expectation surveys in forecasting elections. (Full paper available here)

Jones, R. & Cuzán, A.: Expert Judgment in Forecasting American Presidential Elections: 2004-2012.

PollyVote Demonstrates the Value of Combining Forecasts, Again!

With her Election Eve prediction of two party-vote shares of 51.0% for Obama and 49.0% for Romney, Polly, the predicting parrot, almost hit the bull’s eye for these preliminary results. 

The latest vote count has Obama at 50.6% and Romney at 47.9%. This result translates to two party-vote shares of 51.4% for Obama and 48.6% for Romney. That is, Polly missed the final two-party vote share for Obama by 0.4 percentage points. This is similar to Polly’s performance in her two previous appearances. For the 2004 and 2008 elections, the average error of Polly’s final forecast was half a percentage point.

In addition, Polly always predicted Obama to win since her first forecast on January 1st of 2011, more than 22 months prior to Election Day. In comparison, other methods such as polls or prediction markets at times predicted the Republican candidate to win. This is similar to 2004 and 2008, when Polly never strayed from Bush (8 months prior to Election Day) or Obama (14 months). 

The task of predicting U.S. presidential elections is ideal for demonstrating the usefulness of combining forecasts, as there are a number of different methods that use different sources of information, or process it differently. In addition, it is difficult to judge a priori which component forecast is likely to be most accurate at different times in a campaign. The historical track record of single models or methods is usually of little help. A prominent example of this year’s campaign is  the Iowa Electronic Vote Share Market (IEM), which provided the most accurate component forecasts in past elections (Graefe et al., 2012). This time, the IEM was among the least accurate of Polly’s constituent  forecasts, except for its Election Eve forecasts.

The Table here shows the mean absolute error of daily forecasts of the PollyVote and its components across the last 100 days prior to Election Day. On average, the PollyVote missed the actual results by less than half of a percentage point and clearly outperformed its components. The error reductions due to the PollyVote ranged from 41% (compared to combined expert judgment) to 77% (compared to combined index models).

For almost two years, Polly correctly predicted that Obama would be reelected, and with little variation in her forecast. For the general election observer or journalist, such a forecast may not grab headlines, but it beats the competition day-in and day out.

Polly’s final forecast: Obama 51.0% v. Romney 49.0%

After almost two years of collecting and combining forecasts from five different component methods, Polly has finally finished her job for the 2012 election. Polly’s final forecast of the popular two-party vote shares predicts Obama to gain 51.0% (v. Romney 49.0%).

Over the past 22 months since January 1st of 2011, Polly has continuously predicted Obama to win the popular vote. As shown in the graph below, the highest value for Obama (54.1) was reached in May 2011 with the killing of Osama bin Laden. Shortly after, the PollyVote forecast decreased and reached its lowest value in November 2011, predicting Obama to win 50.4% of the popular vote. At that time, Polly could draw only on few component forecasts, as the expert panel was not yet in place and only few forecasts from statistical models were available. This explains the somewhat higher volatility of the early forecasts.

In 2012, with an increasing number of component forecasts available, Polly’s forecast moved in a much narrower range, with a maximum of 52.6% at the beginning of April and a minimum of 50.9% in late October. Thus, range of her forecast was only 1.7 percentage points. 

 

One or more of Polly’s components will of course provide more accurate Election Eve forecasts than Polly, and it is easy to identify the most accurate forecast after the fact. However, at the time of making a prediction, it is difficult to pick the most accurate forecasts, in particular when making forecasts for long time horizons that involve much uncertainty.

The PollyVote is designed for situations that involve much uncertainty. This means that it is most useful months or years prior to the election, not for an Election Eve forecast, when it is usually sufficient to look at the latest poll average.

Polly has a perfect track record in predicting the popular vote winner, also for long time horizons. In 2004, starting eight months in advance, and in 2008 fourteen months ahead, Polly always predicted the correct winner. This time around Polly has continuously predicted Obama to win the two-party vote since her first forecast was released on January 1, 2011, almost two years before the election. With her final forecast of 51.0% for Obama, and all of her five components pointing to an Obama victory, Polly is confident that her perfect track record will remain.  

Big-issue model predicts a virtual tie in the popular vote

{jcomments on}For the first time since its initial prediction in January 2011, the simple big-issue model predicts that Obama will fail to win the popular two-party vote. The model’s latest forecast predicts Obama to win 49.9% of the vote, a virtual tie.

The big-issue model forecasts the election outcome based on information about how voters perceive the candidates’ ability to handle the single most important issue facing the country. It predicted the winner of the past ten elections with an accuracy of 97% (based on an examination of the forecasts on each of the last 100 days prior to each of the last ten U.S. Presidential elections). 

The model is presented in a research paper, which was published in the Journal of Behavioral Decision Making. The full paper can be found here

Reference: Graefe, A. & Armstrong, J. S. (2012). Predicting elections from the most important issue: A test of the take-the-best heuristic, Journal of Behavioral Decision Making, 25(1), 41-48.