@NumbersMuncher Eats @fivethirtyeight's Lunch

Josh Jordan, aka NumbersMucher, looks at Nate Silver's much-heralded "The Model" (it's always with a "The" in front of it) and wonders why he's assigning high weights to polls that favor Obama and low weights to polls that favor Romney.

The most current Public Policy Polling survey, released Saturday, has Obama up only one point, 49–48. That poll is given a weighting under Silver’s model of .95201. The PPP poll taken last weekend had Obama up five, 51–46. This poll is a week older but has a weighting of 1.15569.

The NBC/Marist Ohio poll conducted twelve days ago has a higher weighting attached to it (1.31395) than eight of the nine polls taken since. The poll from twelve days ago also, coincidentally enough, is Obama’s best recent poll in Ohio, because of a Democratic party-identification advantage of eleven points. By contrast, the Rasmussen poll from eight days later, which has a larger sample size, more recent field dates, but has an even party-identification split between Democrats and Republicans, has a weighting of .88826, lower than any other poll taken in the last nine days.

There's a lot of this sort of thing in the article, and obviously, I cannot quote it all, as much as I would like to.

But there seems to be a pattern here, at least based on NumberMuncher's data: Polls get weighted more -- judged as "good, professional polls" -- based, it seems, largely on whether they show the results Silver thinks they ought to show (Obama winning, of course). How else to explain this?

This is the type of analysis that walks a very thin line between forecasting and cheerleading. When you weight a poll based on what you think of the pollster and the results and not based on what is actually inside the poll (party sampling, changes in favorability, job approval, etc), it can make for forecasts that mirror what you hope will happen rather than what’s most likely to happen. This is also true of Silver’s dismissal of Romney’s lead in Gallup this week. While Romney is likely not up by seven points nationally, as the poll predicted, you can’t dismiss it while at the same time giving a twelve-day-old Marist/NBC Ohio poll a higher weighting than eight newer polls when Marist has leaned Obama this entire cycle.

The implication is that Nate Silver seems to be cherry-picking polls, perhaps unconsciously, thinking that a poll showing a nice Obama lead must be a better, more professional, more accurate poll simply because it's in line with his hunch (Obama wins). Polls that are incongruent with this belief -- Gallup, Rasmussen, most notably -- are weighted less because they are "bad polls" with a "Republican house effect" or whatever sort of language he employs.

So "good polls" get weighted more heavily -- hey, it's the more accurate, more professional poll, right? -- and "bad polls" less heavily and, surprise surprise, "The Model" winds up saying what Silver subconsciously directed it to say.

I have a problem with the whole "state polls are most important" thing of Silver's.

I see there being three levels of analysis here:

Unsophisticated people just look at the national polls. They don't know any better, and don't realize this is a 50 state election.

Sophisticated people look at the state polls. They understand this is not a race for the national vote, but for 50 separate contests in 50 separate states.

Finally:

Even more sophisticated people do what the unsophisticated people do, which is primarily look at the national polls, because unless the national vote is extremely close (1%, 1.5% separating winner from loser) it is mathematically very unlikely the national vote winner will diverge from the electoral college winner.

What is a swing state, after all? What makes it a swing state? What makes it a swing state is that its "political temperature," for lack of a better term, is very close to the average political temperature of the entire United States.

It's a swing state precisely because it's made up of a balance of Americans which makes it essentially a microcosm of the whole country.

That's why swing states tend to swing together. Because the same pitch that works on swing-voting Iowans will more than likely work on swing-voting Ohioans.

There are, of course, local concerns that can cause a swing state to deviate away from the national trends. In Ohio, it's often noted that the unemployment rate is actually lower than the national average (thought to benefit Obama) and that many jobs in Ohio were "saved" by the GM bailout. So it is possible (moreso than usual, I think) that we could have Ohio swinging away from the broader trend of swing states.

Still, the general rule has been that swing states go the way of the national vote and that's why they decide the election. And it's harder to understand why other swing states would be inclined to depart from broader national trends. If I can see a special case for Ohio, I have trouble seeing the same case for, say, Nevada, which has the highest unemployment in the country.

If Ohio's relatively low (yet still kind of high) unemployment rate means it's likely to break from the national trend and wind up in Obama's corner, why isn't it also likely that Nevada will flip to Romney, no matter who wins the national popular vote?

Are most of the swing states going to diverge from the national vote count? What's the precedent for that?

Ohio and the National Vote: JackStraw offers this nugget, noting that the Ohio is even more closely correlated with the national vote than I thought:

According to Larry Schweikart, the U of Dayton professor who has been studying Ohio voting for years, any politician who carries the national vote by more than 0.5% wins Ohio. This has been true for decades.

posted by Ace at 04:43 PM

| Access Comments