The Morning Rant
Polling Incompetence
Dave in Fla [Guest Monkey]

How bad are the polling firms?

We talk on a regular basis about polls and how they are (at best) wrong or (at worst) manipulated. Last week Ace pointed out a new initiative by Real Clear Politics to begin tracking polling firm results. Their effort is intended to point out the single metric that should matter when looking at polling firms. Did they accurately predict the outcome of the election?

It is no surprise that in many cases the answer is a resounding no. But I thought it would be instructive to show some concrete examples.

I am going to pick on Quinnipiac. I’m choosing them, out of the many other bad polling firms I could select (e.g., Siena and Fox) because they are currently polling Georgia, and much of the angst over the Walker/Warnock race is due to their polling, and its impact on the averages.

The following is a short list of their final published polls for recent races:

North Carolina 2016: Tie (actual result Burr +5.7)
Florida Governor 2018: Gillium +7 (actual result DeSantis +0.4)
Florida Senate 2018: Nelson +7 (actual result Scott +0.2)
Ohio Senate 2018: Brown +17 (actual result Brown +6.4)
Ohio President 2020: Biden +4 (actual result Trump +8.2)

These are not small misses within the sampling error. These are huge, consistent, and systemic signs of their methodology being flawed. Their misses in the above examples are 5.7%, 7.6%, 7.8%, 11.6%, and 12.2%. Over time, they have gotten WORSE. In 2016 they ONLY missed North Carolina by 5.7%, but by the time 2020 rolled around, they managed to miss Ohio by 12.2%. At their worst, they should be missing by no more than 3%, and their misses should lean in both directions. But they don’t. Their misses are in one direction only, toward the Democrat candidate.

Despite this abysmal record, the 538 website gives them the same A- rating that they give to Trafalgar and use this weighting in their prediction model. By contrast Trafalgar had the following results in the same polls (only looking where they both polled the same race):

Florida Governor 2018: DeSantis +3 (2.6% miss toward GOP)
Florida Senate 2018: Scott +2 (1.8% miss toward GOP)
Ohio President 2020: Trump +5 (3.2% miss toward Dem)

And these are the final polls, Quinnipiac has had even more outlandish results in the weeks and months leading up to an election, only to settle into being “only” wrong by 7 to 12 points.

What does this mean for the midterms? Here is the current polling out of Georgia:

Quinnipiac: Warnock +7 (52/45)
Trafalgar: Warnock +1 (46/45)
Quinnipiac: Kemp +1 (50/49)
Trafalgar: Kemp +9 (53/44)

Which one do you think is more likely a reflection of the current race? Quinnipiac’s claim that over 50% of the voters have decided they will reelect Warnock in a red year? Or Trafalgar’s claim that the incumbent is under 50% with 9% still undecided? And what of the Governor’s race? Are we supposed to believe that Kemp is clinging to a 1-point lead? Nate Silver says that both are equally likely and plugs them into his vaunted model to spit out talking points for the Democrats and hopium for the Democrat Underground. If we go by track record in the last 3 election cycles, Quinnipiac’s results should be thrown out. There is no indication that their methodology has changed at all, and no reason to believe they are going to be right this year. In fact, the fundamentals of the election; first year incumbent, Biden’s approval rating, resonance on the top voter issues, and the state of consumer economic confidence, makes the notion of Warnock having 52% support for reelection nonsensical.

It isn’t just Quinnipiac either.

Here are few other choice examples of final polling. The media wants you to believe that these firms are completely accurate and show how 2022 will shake out:

2018 Indiana Senate: Fox, Donnelly +7 (actual result Braun +5.9 – 12.9% miss toward Dem)
2020 Wisconsin Presidential: Siena, Biden +11 (actual result Biden +0.7 – 10.3% miss toward Dem)

Siena is considered an A+ pollster according to the Nate Silver, so his model will weigh their results higher than Trafalgar’s. With very few exceptions, the pollsters will miss every race outside the margin of error and the miss is always in the single direction, toward the Democrat. This has been a consistent result for the past 3 election cycles. Yet we are told “there isn’t going to be a red wave”, “the Democrats have a 60% chance of holding the Senate”, and “the Democrats have a chance of holding the House” based upon the current polling results by these same firms.

I don’t buy it.

posted by Open Blogger at 11:00 AM

| Access Comments