Sponsored Content
« Liberals Apparently in Contest to Write Most Biased Story About Health Care Protesters | Main | Shocker: Networks, CNN Can't Determine William "Cold Cash" Jefferson's Party Affiliation »
August 06, 2009

Defending the CNN Poll, a Little

CNN's poll has Obama dropping 10 points in 100 days.

Ed criticizes the poll, which still puts Obama at an unexpectedly-high 56% support, but he's off-base in this specific criticism, I'm pretty sure:

Interviews with 1,136 adult Americans, including an oversample of African-Americans, conducted by telephone by Opinion Research Corporation on July 31-August 3, 2009. The margin of sampling error for results based on the total sample is plus or minus 3 percentage points.
Did they correct for that oversample? Did they weight the responses? CNN’s polling data doesn’t explain at all, which leads one to the conclusion that they kept the oversample in the final data.

Oversampling does not mean overweighting.

If you want to, say, conduct a broad national poll but also make specific conclusions about what a minority might think, you have to oversample them. I think the minimum -- minimum -- number of respondents you need to put out numbers for any question is around 400. The academic/business organization that sets standards for this sort of thing is the AAPOR, sets 400 as the minimum number for meaningful results for any poll question -- and that includes a sub-segment of the poll's total universe of respondents, too.

So, to get meaningful results from black respondents specifically, you need 400 of them. At a minimum.

Now, blacks are like 12% of the population so if you are polling 1000 Americans you're likely to only get 120 blacks. That's fine, if you don't want to report specific responses from the black population.

But if you want to report those responses, and want them to actually be statistically meaningful, you need to get that sub-cohort up around 400.

Does that mean they didn't weight black responses back down? No, of course they did. Any polling company which didn't would be setting itself up for not only ridicule but loss of business, as its competitors pointed out the gross error of postulating a 37% black US population.

They weighted them back down to 12%, or 13%, or whatever the current Census figures indicate.

As for polling adults rather than voters: Well, it's true that adults are the least predictive sample... if you're predicting election results. If you don't care about who wins an election, but rather only what adults think, whether they vote or not, it's the appropriate sample.

In this case, their decision was likely made not according to what they really wanted to measure -- adult mood or likely future election results -- but according to cost. Anytime you screen out respondents, it costs you money. You spend more on your callers' salaries (paid by the hour, mostly) because they're making many more phone calls than they're actually generating responses (i.e., stuff you can use in your poll).

In this case, CNN had already made one costly decision -- calling lots of people to try to get those 400 black respondents -- and cheaped out on making it more costly by screening for voters (or, even more costly, likely voters).

At any rate, there are a lot of ways that a pollster can manipulate results -- the biggest assumption being about current party identification figures, those being kinda anyone's guess and therefore pretty variable between pollsters -- but grossly overweighting a well-known and easily-checked stat like "percentage of blacks in the US" isn't one of them.

I mention this not to call Ed out, but because it's, well, true, and I've been seeing stray references to an "oversampled CNN poll" in the comments.

By the Way: The minimum 400 rule gets violated all the time.

The media violates it all the time when they report statistically-insignificant stuff like "unmarried Asian females," a group their poll sampled like, what, three of? You asked three unmarried Asian females a question and now you're gonna get all conclude-y about them?

But polling companies themselves aren't supposed to. They too will write lots of analysis about tiny sample-size questions and pass those to clients, to demonstrate they're "getting their money's worth," but they're not supposed to. A lot of those micro-sized break-out questions are absolutely meaningless and the AAPOR says not to report them at all.

One Other Way the Media Biases Polls: They bury polls they don't like and trash their own polls.

I forget which poll it was, but one of the big networks found a lot of support for McCain/Palin, and simply didn't report the poll. When they eventually did have to report it (embarrassed into it), they trashed it.

Now that poll was probably a bad poll.

But when a network finds support for Obama and the Democrats at a suspiciously-high level, they don't bury or trash their poll. They vigorously promote it, claiming it heralds a major come-back or blah blah blah.

If they were being honest and consistent, they'd either report all polls equally enthusiastically, or equally bury/trash polls that were out of kilter with other polls.

They don't. They don't mind when their polls come in overly-optimistically for the Democrats. To them it's win-lose -- hey, it's a garbage poll, but we win, because we inject a little optimism into our Democratic friends.

A poll coming in too high for the GOP is lose-lose, and they trash it.

Heh: Ed's right about this:

One of the questions asked, “How do you personally feel about Barack Obama being president?”, has the following options for response: Thrilled, Happy, Don’t Care, Unhappy, Depressed, No Opinion. Depressed? CNN offers two kinds of responses for happy, one for unhappy, and one that asks people to diagnose themselves with a clinical disease.

Not just a disease, but a sin, isn't it? Don't Catholics teach that allowing yourself to stew in depression is self-indulgent and sinful?

Biased. Why not "very displeased"?

Or maybe CNN can really cook the books and ask,

"How do you personally feel about the President?"

Happy, plus I'm really good-looking

Unhappy, and I've gone all pear-shaped and I smell like funky cheese

Angry, crazed, and racist, and also my dick is the size of watch-battery and/or I've got chronic swamp-ass

digg this
posted by Ace at 01:30 PM

| Access Comments

Recent Comments
Ciampino - always love the ladies in these metal bands: "57 Ah, they're Dutch. Isn't The Gathering Dutch t ..."

Village Idiot's Apprentice: "G'morning, all. Finished the last dose of the 1 ..."

Puddleglum: "[i]56 Joo hater Roger Waters is being investigated ..."

Puddleglum: "Ciampino, Ah, they're Dutch. Isn't The Gatheri ..."

San Franpsycho: "Joo hater Roger Waters is being investigated for p ..."

Wolfus Aurelius, Dreaming of Elsewhere [/i] [/b]: "Re: nightmares, I guess it's the melatonin. I onl ..."

Ciampino --- No bird shit at Monaco?: "44 Evening and morning to all the satellites in sp ..."

Wolfus Aurelius, Dreaming of Elsewhere [/i] [/b]: "[i]Wolfus, you need to smoke more weed. That will ..."

Bruce: "On top of that, I was smoking a pipe out in the ra ..."

Puddleglum: "I stopped taking melatonin after my Spirit Animal ..."

Bruce: "Wolfus, LeMans is 2-3 weeks away and it is a bi ..."

San Franpsycho: "Wolfus, you need to smoke more weed. That will eli ..."

Recent Entries

Polls! Polls! Polls!
Frequently Asked Questions
The (Almost) Complete Paul Anka Integrity Kick
Top Top Tens
Greatest Hitjobs

The Ace of Spades HQ Sex-for-Money Skankathon
A D&D Guide to the Democratic Candidates
Margaret Cho: Just Not Funny
More Margaret Cho Abuse
Margaret Cho: Still Not Funny
Iraqi Prisoner Claims He Was Raped... By Woman
Wonkette Announces "Morning Zoo" Format
John Kerry's "Plan" Causes Surrender of Moqtada al-Sadr's Militia
World Muslim Leaders Apologize for Nick Berg's Beheading
Michael Moore Goes on Lunchtime Manhattan Death-Spree
Milestone: Oliver Willis Posts 400th "Fake News Article" Referencing Britney Spears
Liberal Economists Rue a "New Decade of Greed"
Artificial Insouciance: Maureen Dowd's Word Processor Revolts Against Her Numbing Imbecility
Intelligence Officials Eye Blogs for Tips
They Done Found Us Out, Cletus: Intrepid Internet Detective Figures Out Our Master Plan
Shock: Josh Marshall Almost Mentions Sarin Discovery in Iraq
Leather-Clad Biker Freaks Terrorize Australian Town
When Clinton Was President, Torture Was Cool
What Wonkette Means When She Explains What Tina Brown Means
Wonkette's Stand-Up Act
Wankette HQ Gay-Rumors Du Jour
Here's What's Bugging Me: Goose and Slider
My Own Micah Wright Style Confession of Dishonesty
Outraged "Conservatives" React to the FMA
An On-Line Impression of Dennis Miller Having Sex with a Kodiak Bear
The Story the Rightwing Media Refuses to Report!
Our Lunch with David "Glengarry Glen Ross" Mamet
The House of Love: Paul Krugman
A Michael Moore Mystery (TM)
The Dowd-O-Matic!
Liberal Consistency and Other Myths
Kepler's Laws of Liberal Media Bias
John Kerry-- The Splunge! Candidate
"Divisive" Politics & "Attacks on Patriotism" (very long)
The Donkey ("The Raven" parody)
Powered by
Movable Type 2.64