Polling In 2020 [By - Dave In Florida]

Polls … uh … What are the good for? Absolutely nothing!

I think most of us remember my terrible predictions of a Romney victory in 2012. I literally hid for 6 months after that, out of shame.

I did finally delurk and made a few comments during the 2014 election. I was pretty active during 2016 as one of the few pointing out that Trump could win, by pointing to polls like the LA Times/USC poll that used a different methodology. I was also correct regarding both Rick Scott and DeSantis in 2018.

In 2012 I trusted the information that was contained in polls, and I learned the hard way how bad they can be.

Polling is broken.

There are a number of problems with the polling industry, but the biggest problem they are dealing with is response rate. Prior to 2000, polling was done primarily through random dial landline calling and they would get a response rate of 50%. That means half the people they contacted would answer their questions.

The response rate today is 2%. 98 out of every 100 attempts to reach a voter is rejected. Of those that do respond, there is a very high sampling bias toward young, female, white Democrats. One of the pollsters I admire right now is Rich Baris, and he said, “the very first Republican I get to respond to every poll sounds like a Bill Kristol clone”.

Getting a representative sample of the electorate is HARD.

And that is the key to polling, you are getting a single person to be a proxy for the voting intentions of about 10,000 people. If you don’t get a truly representative sample, then you don’t have an accurate picture, before you even attempt to apply demographic weighting. You need age, ethnicity, class, values, income, genealogy, location, etc etc. A middle-class voter in Bucks County PA has a completely different voting profile than a middle-class voter in Greene County PA.

To compensate for the problems with response rates, polling firms have begun to use multiple “modes” of sampling. They use online polls, text messages, phone calls with a redirect to web sites, emails, direct voice with same ethnicity interviewers, automated response polling, and others I probably don’t even know about. Note that the LA Times/USC poll that I mention above was an interesting attempt at a new mode, since they had a fixed pool for 3200 voters that they sampled every week.

The better pollsters will use multiple modes, because each mode has its own inherent sampling bias. Young millennial women are much more likely to answer an online poll. A good pollster will also do oversampling. They will keep polling until they have enough of the representative demographics, and then will discard the extra responses from demographics that are overrepresented.

This bring us to the next problem. How do you know what a representative sample is? Every pollster works with a voter file. These are databases that are compiled after every election based on voter records. They are correlated with marketing data to produce a profile of the demographic makeup of every precinct, town, city, region, and state in the country. The interesting thing about the voter file is that it can often tell you when a respondent is lying to you. They will tell you their voting history and you can look directly into the voter file to see they are not being truthful.

So with a representative sample and a good voter file, you can get a good idea of how a region or state will vote.

And this almost never happens.

There are many reasons why, but at the core is the inability to get a representative sample. Right now, the one demographic that is brutally hard to sample is working class males. If a poll gets a sample at all, they will get perhaps 1 for every 20 college educated, age 30-45, white, Democrat women.

When polls are wrong it is often because they have over represented a demographic or used the wrong demographic as a proxy. Many polls will consider a hipster working at Starbucks and a factory worker to both be a working-class male. But they don’t vote the same at all.

By the way, this working-class male vote that is so hard to sample? They will vote for Trump 65-30.

And these are just the polls that are getting bad results from sampling errors. There is also tremendous pressure on pollsters to generate results that are favorable in the opinion of their clients. During 2016, the best media pollster, in terms of predicting the correct outcome of the election, was ORC. They were polling for CNN. CNN fired them for being right.

There are legitimate reports of pollsters deliberately targeting a bad sample in order to inflate polls. And there are rumors of some well-known pollsters deliberately faking results. Nate Silver is an enabler in this game by giving poor ratings to polling firms that deviate from the “consensus” while giving provably bad polls an “A rating”. Nate Silver waxed poetic over polls that Nate Cohn did for the New York Times, which included counties in Pennsylvania voting for Biden that have never voted for a Democrat. Meanwhile, last weekend he manufactured an excuse to exclude Trafalgar from his models.

Ironically, it turns out that polling is one of the least accurate indicators of election results. Factors that have historically done a better job of predicting elections include the Norpath model, the Gallup “better off than 4 years ago” number, incumbent vote share during primaries, and new voter registration advantage.

Even the results of Ohio and Florida are more predictive than polling, since other than JFK, no president has ever won without winning at least one of them.

Finally, let me mention that it is often possible to verify if a poll is correct or not just by checking against known quantities. We saw this problem with recent Michigan polls where the polls were showing that Democrats were self-reporting that they had already voted at a 3:1 rate over Republicans. The problem with that poll is that you could check against actual voting records and see that the early voting was at a 1:1 ratio. Obviously, the poll is wrong.

We are seeing the same things in both Texas and Florida right now, where polls are being released that are mathematically impossible given known data from early vote totals reported by the states.

The bottom line is that the polling industry is broken right now, and possibly just as corrupt as our other institutions. You should not be putting faith in public polling, especially when the polling contradicts other historical indicators. I am hoping that this election will finally cause the industry to take a hard look at itself. But I doubt that it will. There is no downside to getting a poll wrong, you still get paid. And probably get paid more.

posted by Misanthropic Humanitarian at 06:52 PM

| Access Comments