Joe Biden Asks an Awful Lot from a Guy in a Wheelchair | Main | Sullivan Taken to the Woodshed (?!!)
Update: Commenter Claiming to from the Atlantic Dishes
September 09, 2008

Weighting on Party: Why It's Dangerous

Conservatives have long complained that polls often oversample Democrats. They get angry when a poll fails to "properly" weight the numbers according to assumed party identification -- reducing the proportions of Dems and increasing the proportions of Reps to the guestimated partisan breakdown.

I used to say this myself. I believe I was wrong, and that those who urge this are wrong now.

Here's the thing: Party identification is not inelastic. It does not remain fixed at 38 Dem/32 Rep/30 Independent (or whatever ratio you might guess at). It changes all the time. Many people are right on the edge of being Democrats or Independents -- when the Democrats please them, they say they're Democrats. When the Democrats piss them off, bang, they're Independents.

Same deal on the Republican side. People kept complaining that 2006 polls didn't include enough Republicans, because the fraction of Republicans was 5% or so lower than longterm historical averages, or what recent (2004) numbers would suggest. But most polls, including Gallup, nailed the 2006 results.

Republicans were not undersampled, mostly. What was going on was that a lot of people who were formerly self-identifying Republicans were disgusted by the Mark Foley scandal, Katrina, and the worsening situation in Iraq and told the pollsters they were Independents. Because, actually, they were. The affinity they once had with the GOP had too much abused of late.

I'm told by someone I trust a great deal that one of the most dangerous things to weight on is Party ID. This is a pollster saying this. Yes, some demographics can be weighted on (age gets weighted a lot, almost always, because there are always too many older people being called and too few young people) and sometimes other demographic designators whose proportional representation in the population we know for a certain fact out to the thirtieth decimal place -- sex, race, economic bracket, education level.

These things do not change. We always know how many Asian people there are in the country. (And thank god for that -- gotta keep tabs. They know karate and ninja skillz.)

But party ID is never known with a great deal of accuracy, and furthermore can change from week to week, sometimes fairly dramatically.

You want to know why Zogby is sometimes very accurate and sometimes seems like he's making up bizarre numbers randomly? Because he weights on party ID. Sometimes his guess as to the current ratio of partisan ID is accurate and he nails a spot-on poll and embarrasses the competition.

But very often his guess is wrong and the numbers he winds up producing seem to have come in from Mars.

It's not so much a bad practice as a dangerous practice. Well, I guess that makes it a bad practice, actually.

Now, here's the weird thing this cycle: You know how you guys have often urged pollsters to weight on party? And correct for oversized samples of Democrats?

The thing is -- now pollsters are getting, it seems, oversized samples of Republicans. And pollsters who do weight on party will knock those self-identifying Republicans down in numbers to get them closer to the long-term average or best guess as to current representation.

But what if Sarah Palin has made many actual new self-identifying Republicans, just as Mark Foley made many self-identifying Independents and Democrats?

Pollsters that weight on party will have less dramatic swings and show closer contests, as they will be massaging down and diminishing numbers that actually reflect the true state of affairs.

Case in point: Rasmussnen. Rasmussen weights on party.

For a variety of reasons, the Rasmussen Reports daily Presidential Tracking Poll is less volatile than some other polls and always shows a somewhat smaller convention bounce than reported by others. This is primarily because we weight our results by party identification (see methodology). Looking at the data before adjusting for partisan identification, the Republican convention appears to have created a larger surge in party identification than the Democratic convention the week before. If this lasts, it could have a significant impact on Election 2008.

Note: "If it lasts." At the moment they're assuming this is either bad numbers, or some other anomaly. "If it lasts" presumably they'll change how they weight the various party IDs and give Republicans a larger fraction of the population -- but only if the trend continues long enough to convince them it's real.

Anyway, I think it's interesting, that. Rasmussen used to show better-than-usual numbers for Republicans, because they would weight on party ID and boost Republican affiliation while diminishing Democratic affiliation, guessing at the right "real" mix.

And now, with Hurricane Sarah blowing a lot of Independents to the Republican side, they're doing the opposite, now correcting for "too many" Republicans. That's why they're showing the race dead even, while Gallup shows a 5 point lead (or even a 10 point one).

I don't think it's established, clearly, that Zogby and Rasmussen are wrong to do this. But how accurate their weighting depends, ultimately, on their guess as to the "real" partisan breakdown in America -- and sometimes they can just be off, and sometimes they can fail to take into account a real shift in partisan ID.

Or at least not take it into account until weeks after the trend has been detected in other polls.

Something to keep in mind when looking at the much-touted WSJ/NBC poll tonight, too.

PS: Ed Morrissey found a poll a few months ago where not only did the jagoffs doing the poll sample far more Democrats than Republicans, but on top of that they weighted the Democrats into an even higher proportion -- up to something like 43% or thereabouts.

So sometimes the partisan weighting is not merely dangerous, but positively dishonest.


digg this
posted by Ace at 06:03 PM

| Access Comments




Recent Comments
Roger Goodell: "Issues, communities, communities, issues, communit ..."

NaCly Dog[/i]: "creeper! How are you doing? It's so good of yo ..."

creeper: "Good heavens!  Dave, you've drained the net.' ..."

Mary Poppins' Practically Perfect Piercing: "[I]More fallout from Horny Harvey (this cartoon fe ..."

Nevergiveup: "Morning all Go Yankees ..."

Case: "Well, he knew where her CAR was. ..."

Colin: "Was with a nephew this weekend and he showed me he ..."

Shep Smith: "Ouch. I really need to reset my alarm dick. ..."

NaCly Dog[/i]: "I always forget about the Gell-Man Amnesia Effect. ..."

Mr. Peebles: "I bought a Commodore tape drive at a thrift store ..."

Hillary Clinton: " *hic* ..."

NaCly Dog[/i]: "I still have two circular slide-rules and one stra ..."

Recent Entries
Search


MuNuvians
Polls! Polls! Polls!
Frequently Asked Questions
The (Almost) Complete Paul Anka Integrity Kick
Top Top Tens
Greatest Hitjobs

The Ace of Spades HQ Sex-for-Money Skankathon
A D&D Guide to the Democratic Candidates
Margaret Cho: Just Not Funny
More Margaret Cho Abuse
Margaret Cho: Still Not Funny
Iraqi Prisoner Claims He Was Raped... By Woman
Wonkette Announces "Morning Zoo" Format
John Kerry's "Plan" Causes Surrender of Moqtada al-Sadr's Militia
World Muslim Leaders Apologize for Nick Berg's Beheading
Michael Moore Goes on Lunchtime Manhattan Death-Spree
Milestone: Oliver Willis Posts 400th "Fake News Article" Referencing Britney Spears
Liberal Economists Rue a "New Decade of Greed"
Artificial Insouciance: Maureen Dowd's Word Processor Revolts Against Her Numbing Imbecility
Intelligence Officials Eye Blogs for Tips
They Done Found Us Out, Cletus: Intrepid Internet Detective Figures Out Our Master Plan
Shock: Josh Marshall Almost Mentions Sarin Discovery in Iraq
Leather-Clad Biker Freaks Terrorize Australian Town
When Clinton Was President, Torture Was Cool
What Wonkette Means When She Explains What Tina Brown Means
Wonkette's Stand-Up Act
Wankette HQ Gay-Rumors Du Jour
Here's What's Bugging Me: Goose and Slider
My Own Micah Wright Style Confession of Dishonesty
Outraged "Conservatives" React to the FMA
An On-Line Impression of Dennis Miller Having Sex with a Kodiak Bear
The Story the Rightwing Media Refuses to Report!
Our Lunch with David "Glengarry Glen Ross" Mamet
The House of Love: Paul Krugman
A Michael Moore Mystery (TM)
The Dowd-O-Matic!
Liberal Consistency and Other Myths
Kepler's Laws of Liberal Media Bias
John Kerry-- The Splunge! Candidate
"Divisive" Politics & "Attacks on Patriotism" (very long)
The Donkey ("The Raven" parody)
News/Chat
Archives
Powered by
Movable Type 2.64