Yesterday's CNN Poll Showing Nine Point Obama Lead Actually Showed Near Tie In Actual Respondent Count

CNN weighted the responses to get a nine point Obama lead, 52-43.

That's what they got when they weighted things. The actual count of people saying they'd vote for Obama versus Romney? 484-476. That's just barely a 1 or 2 point lead (among registered voters).

CAC had a post yesterday defending some media-poll weighting of poll results in favor of Democrats, but only from 0-4 points of Democratic advantage. Not seven points (the Democratic edge in the huge, non-replicable 2008 election) and not the 9, 10, or even 15 points you see sometimes.

One thing is that while we, as partisans, focus on the partisan split and partisan weighting, in fact pollsters are weighting on a lot of other things. Typically, pollsters get far, far too many older voters and far, far too few younger voters, so they have to greatly overweight the younger respondents they get through to.

Now, they're supposed to have quotas for different age groups (they keep calling people until they have the right number of younger voters) so that they don't have to weight. But that gets expensive, and many times they don't bother with the quotas because that drives up cost. Instead they resort to the cheap expedient of weighting.

(I remember a guy defending candidate polls --often dismissed as self-serving-- and noting that at least candidate polls have some money put into them, and tend to be more accurate, if you can get the actual polling data. Media polls are often cheaper and thus feature worse demographics, weighted this way and that to get the right balance.0

But everytime you deliberately weight on one thing (say, weighting up younger voters) you wind up incidentally, and avoidably, weighting up on a host of other things (like Democratic affiliation). As one pollster told me, it's like Wack-A-Mole -- you weight to correct one demographic imbalance, weighting it to make it go away, and due to your weighting another demographic imbalance pops up, which itself needs to be wacked, and that leads to another, and so on.

CNN's poll might not have been engineered to produce a partisan result but when a tie becomes a nine-point blow-out via demographic weighting, well, the numbers have been teased an awful lot, and the results are suspect. Virtually all of Obama's advantage in the CNN poll is not actual respondants giving him their support, but the result of mathematical manipulations. As they say, if you torture the data enough it'll confess to anything.

posted by Ace at 10:57 AM

| Access Comments