Two Very Different Predictive Models for the Presidential Election

by   |   November 5, 2012 4:48 pm   |   0 Comments

Business analysts would do well to pay attention to the scrutiny that election prediction models engender. While the readouts from a customer analytics project won’t rival the passions generated by a close contest for the White House, there are two important takeaways from the political fray. First, the audience for your predictions needs to understand the terms you use to communicate your findings. Second, you need to be ready to defend your model against loud criticism.

Related Stories

Presidential debates provide living laboratory for marketers studying sentiment analysis.

       Read more»

How big data redefines political campaigns’ microtargeting.

       Read more»

Two high-profile models provide a view of how very different approaches to the same predictive problem play out in public. Statistician and writer Nate Silver, of The New York Times’ FiveThirtyEight blog, conducts a meta-analysis of many polls and economic data to derive forecasts of likely outcomes. Allan Lichtman, an American University historian and author of the book Keys to the White House, has developed a model that projects the popular vote winner by examining the incumbent party’s performance in the White House based on 13 conditions, including the U.S. economy, foreign affairs, social unrest, political scandals and the makeup of the challenger.

While both analysts have reason to brag—Silver correctly called voting preferences in 49 of 50 states in 2008, and Lichtman’s model, developed in 1981, has predicted the popular vote winner in every election since 1984—the pair have different communication styles.

Allan Lichtman’s ’13 Keys’

Historian Allan Lichtman’s 13 keys to the White House ask true-false questions and about the nation, the incumbent party and the nature of challengers. When eight or more keys are true—falling in favor of the incumbent—the incumbent’s party wins the popular vote. It has been correct in every election since 1984. (The model predicted Al Gore would win the popular vote in 2000, but does not pertain to the Electoral College.) The answers below reflect Lichtman’s assessment as of Oct. 5.

  1. After the midterm elections, the incumbent party holds more seats in the U.S. House of Representatives than it did after the previous midterm elections. (False. The Republicans won a House majority in 2010.)
  2. There is no serious contest for the incumbent party nomination. (True.)
  3. The incumbent party candidate is the sitting president. (True.)
  4. There is no significant third-party or independent campaign. (True. No Ross Perot or Ralph Nader this time.)
  5. The economy is not in recession during the election campaign. (True. The economy is in a slow recovery.)
  6. Real per-capita economic growth during the term equals or exceeds mean growth during the previous two terms. (False.)
  7. The incumbent administration effects major changes in national policy. (True. Lichtman counts the Affordable Care Act as a major change.)
  8. There is no sustained social unrest during the term. (True. Occupy Wall Street does not compare to the 1960s.)
  9. The incumbent administration is untainted by major scandal. (True.)
  10. The incumbent administration suffers no major failure in foreign or military affairs. (True. Lichtman counts Pearl Harbor and the Vietnam War as major failures.)
  11. The incumbent administration achieves a major success in foreign or military affairs. (True. Lichtman cites the killing of Osama bin Laden as a success.)
  12. The incumbent-party candidate is charismatic or a national hero. (False. Obama falls short of claiming charismatic status that he attained in 2008, Lichtman says.)
  13. The challenging-party candidate is not charismatic or a national hero. (True. Mitt Romney is not charismatic. Dwight Eisenhower was a national hero.)

Lichtman issued his prediction that President Barack Obama would win the popular vote in his reelection bid more than two years before a vote was cast—in July 2010. That was when a broken BP oil well was gushing crude into the Gulf of Mexico, American forces were still fighting in Iraq and the jobless rate was 9.5 percent. In this model, the incumbent party wins when eight or more of the 13 keys fall in their favor. Obama had nine in his favor then, and Lichtman said in a newspaper column on Oct. 5, that his assessment gave the president 10 in his favor.

Silver publishes the daily results of simulations using variations of the data to account for discrepancies between national and state polls, and the effects of economic indicators on the presidential race.  As of Nov. 5, Silver’s model gave Obama an 86.3 percent chance of winning re-election, with a squeaker popular vote victory of 50.6 percent over former Massachusetts Gov. Mitt Romney’s 48.5 percent. (Note that Silver frames his prediction in terms of probability, not certainty.) Silver noted that Obama prospects rose after Hurricane Sandy hit the East Coast, but that the president had been gaining in the polls even before the storm.

The Noise Surrounding the Signals
That an analyst would face scrutiny for the results he presents is a natural condition in the business world; executives should kick the tires on reports. But the controversy around Silver’s projections in the weeks leading up to the election is instructive, and shows why it pays to lay the groundwork for an analysis based on probability and statistics.

For example, Joe Scarborough, the host of “Morning Joe” on MSNBC, took direct aim at Silver’s projections that Obama was favored to win. “Anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes,” Scarborough said. (The TV host is not the only one criticizing Silver’s analysis, as U.S. News points out.)

With his blog receiving widespread media coverage and a new bestselling book The Signal and the Noise to discuss, Silver has had opportunities to defend himself. And he has used these opportunities to suggest that Scarborough, like many people, doesn’t understand the math terms he uses. It helps to know that probability is different than certainty, for one thing. Silver said this on “The Charlie Rose Show” when asked to explain what “separating signal from noise” means:

“It means learning how to work with information to perceive where the real value is there. I think sometimes people tend to overrate the value at the margin of new data that they see. Where they see so many economic statistics, so many polls, people tend to go a little bit haywire and change their views about the economy or the election every few minutes. In fact it’s usually better to realize that any one poll is not all that good an indicator, just like any one economic statistic. But if you take them all together and they all show the same story, that 9 out of 10 of the polls in Ohio, not every one, but 9 out of the last 10, show Obama ahead—that becomes more meaningful.”

One Model Against Another
As the controversy over Silver’s projections shows, analysts have to be ready to defend their methods. It turns out that Lichtman and Silver have gone at it with each other. Last summer, Silver published a critique of Lichtman’s 13 keys model on his blog, calling some of the criteria subjective and noting that it fails to account for margins of victory.

Nate Silver’s Electoral College Model

In June, Nate Silver wrote the first of several articles outlining his methodology. The model runs simulations in the Electoral College to project the likely winner. Data sources include:

  • National and state polls. The model gives different weights to polls based on the historical accuracy of each one.
  • Political fundamentals in each state, including historical voting patterns in the past two presidential elections, local economic conditions, and demographics.
  • Seven economic indicators: job growth, personal income, industrial production, consumption, the Consumer Price Index, the S&P 500 stock market index, and gross domestic product growth forecasts as recorded by a monthly panel at The Wall Street Journal. These economic factors play a lesser role closer to the election.

Lichtman subsequently published a defense on the FiveThirtyEight blog, writing that his  model is not designed to predict the margins of the results, and adding: “The amount of subjectivity is far less than meets the eye, given the careful definition of each key in the published material, the record of how each key was turned in the 38 elections in 1860 to 2008, and the successful predictions from 1984 to 2008.”

His bigger point, Lichtman writes, is that a president’s governing matters more than campaigns: “It suggests that the American electorate makes reasoned, pragmatic decisions in presidential elections and is not manipulated by the pollsters, the admen and the consultants,” he writes.

Michael Goldberg is editor of Data Informed. Email him at


Tags: ,

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>