Que Será, Será : How I Screwed Up 2012 (and Plan to Fix That)

This is a post that is long-overdue, and one I felt I owed the readers & cobs of Ace of Spades HQ along with the Ewok himself. I warn that it is a very long read, and I implore you to read it as seriously as I wrote it. It's the rare instance of someone publicly admitting how badly they screwed up, so ready your pillowcases-filled-with-soap, and dive in.

I take the election-related work I contribute here seriously. I felt a great sense of responsibility when Ace asked me to co-blog with a focus on elections and polling back in 2010, because I had been given an opportunity to do something I've always had a passion for. Everything I've managed to do since he gave me the keys is a direct result of that opportunity. I have a day job, art, and the stars, but the interest closest to my heart (besides my wife) has been elections. This opportunity- blathering about them and having people, some of whom are very influential, take you seriously- is enormously important to me. So, this has been in the back of my mind for nearly fifteen months, and I touched on it in the comments late last week:

The general election, election night in particular, burned me to the core. I had spent months slowly networking a rag-tag group to do a live coverage thing for the blog intended to rival the AP's, much like what we did for the recall. John Ekdahl created a fabulously simple interactive map, did his computer magic so it would be easy for a hundred volunteers to update it live.

Then the results started trickling in.

And things got bad.

And things got worse.

And I lost my nerve and became a bit unravelled.

A lot of volunteers got very depressed, and I can't blame them.

So I started drinking, and as the numbers continued to grow nastier and nastier, and way from what I had foolishly bought into (that the polls were "skewed", a mistake I will NEVER, EVER repeat)... it just all fell apart.

This was an ambitious project I wanted to pull off- successfully- for Ace, because he gave me the chance to cob for him and I wanted to prove just how "big" I could play.

And in the pit of despair over the sour results, I feel like I failed.

Miserably.

So I want to talk about the election call I completely fucked up.

For one, my partisan leanings took over and blinded me to the numbers right before my eyes, insisting instead to look at anecdotes and the questionable nature of some firms. Some pollsters deserved greater scrutiny, even if some of their final numbers were impressive. Nate Cohn, formerly of The New Republic, eviscerated one such pollster for not having much of a consistent methodology in a great set of reads here, here, here, and here.

But while it is true some pollsters probably posted misleading numbers here and there, and some have questionable methodologies, the averages of the polls aren't generally affected by one or two bad apples. Take a look at the final averages for the swing states in 2012 per RCP:

While most averages deviated from the margin won, all found the correct winner, with the exception of the Florida polls. Using JUST polling averages alone, you could have assumed Romney to gain only North Carolina, Indiana, and Florida as his consolation prizes, losing to Obama 235-303 (he actually lost Florida too, losing the EV by a grand total of 206-332). The simplest method of predicting- sourcing data gathered by various firms- will accurately predict the winner of the overall race. It was true in 2012 as it has been in all general elections in recent memory. Tweeking this, you can weigh polls based on the past-performance of the company releasing them, or the sample size, and perhaps get even closer to the margins (or nail all fifty states, like Nate Silver manged to do).

My biggest mistake was looking at the polls, disbelieving the D+x breakdown, and jumping on the "skewed" bandwagon. If I had simply gone with the tried-and-true, I would have been roared at for not believing in Romney pre-election, and would have been ribbed for missing Florida after, but I would have been right.

When it comes to elections, polls, and all that encircles that stuff, accuracy is the gauge by which we judge forecasters. I deserve every punch I've gotten in comments, on twitter, and in the hundreds of emails from distraught regulars asking me how I could blow it so badly (I really wish I was exaggerating that last bit). To those who referenced my work, who followed along, and who believed it, I'm sorry.

So, here's what I'm keeping in mind to avoid this.

1- Polls Uber Alles.

Polls are the best indicator we have for how A living in B feels about X at time Y. "My cousin says" and "I heard" are as plentiful as cat turds and just as useful. Yes, some firms can be screwy. Yes, others can put their thumb on the scales. But the averages of the polls rarely deviate from the winner of the race that was inquired about. Democrats shoved their fingers in their ears as Governor Scott Walker consistently led in every public poll released, and we did the same as the close of the 2012 cycle approached us.

2- In the absence of polls, consider other data.
What events could a given incumbent or incumbent party be tied to that could hurt them in the election?
How unpopular is this event/string of events?
Is there polling showing an impact on the incumbent?
How well are the incumbent's opponents fundraising?
All of these questions can help determine positioning a race on a scale ranging from solid Republican to solid Democrat.

Let me give you an example:
Oregon is host to a potentially competitive seat. No polling has been publicly released between Dr Monica Wehby and Senator Jeff Merkley, but polling has been released for other potential Republican contenders, and the Senator has enjoyed low double-digit advantages over them, while only in the mid-single digits against a few hypotheticals more well-known to Oregonians.
I have the race currently as moderately Democratic, meaning I do expect that double-digit advantage to drop into the single digits.
How can I call it that way?
Well, we know that his approval ratings took a hit from Obamacare, thanks to data released throughout 2013. He's currently more vulnerable than he was at the start of last year, though he can still expect to win as the race stands today and as the polling pitting him versus other candidates shows.
Republicans are dumping money in near-unknowns, (over $500k to Dr. Wehby in Q4 2013), so it appears the challenger may get enough funds to actually mount an attempt, though not enough as it stands to really, really make the race interesting.
The curveball needed to make a normally slam-dunk race competitive- strong dissatisfaction with a national policy (or several) that the candidate has a direct association with- has been playing out in Oregon media, as the state exchange program has become a 9-figure embarassment.
Without data comparing the match-up I see happening post-primary, Wehby v Merkley, pegging this race as moderately Democratic seems prudent: Merkley is favored to win, and by a considerable though not overwhelming margin, but will face heat over a souring policy within his state, growing if it fails to get fixed fast. If polls start dropping showing him crushing her by 20 points, well, it's a no-brainer which direction I send the race.

3- When polls become plentiful, all other factors will take a back-seat.
Many prognosticators still have Michigan as lean-Democrat, due mainly to the nature of the state and past election results. However, polling has shown the race to be very tight, and now slightly leaning in the Republican's favor. Some have changed their initial calls to reflect this, most notably Charlie Cook. If the polling data keeps showing this, expect all of the polling whizzes to follow suit: once you have data, and a considerable amount of it, running counter to your initial call, your call will change. This doesn't mean completely ignoring all of the on-the-ground details (ad buys, surprising turnout at a rally, etc), but they won't override the winner/loser indicated by the polling.

4- You may be partisan, but the averaged data, piling as the months pass, is not.
If the electorate seems notably partisan across the average in your favor, you'll delight as the numbers look good, if it appears partisan against you, suck time. But guess what? Your personal feelings about a party or race is irrelevant to accurately pegging it. If I am going to be seen as an accurate point of reference on the nature of the current election cycle, I have to put my personal feelings and passions aside. Nothing sucks more than reporting a race is running away from you, but if that's the case, that's the case.

5- Properly weight the polls within the averages, but only if necessary. A lot of guesswork is involved here, because a firm that has a great year in, say, 2008, can have a shit one in 2012 (see Rasmussen). Weighting releases in the averages solely on the track record of the company needs to account regular auditing of polling firms' findings versus results. If a pollster has had a string of bad results after a period of good ones, it's fair to include them in an average, but handicap them. If however they've had a flood of terribly off releases, eliminate them all together. Comparing polls using different kinds of electorates: likely, registered, or "all adults", is a no-no, unless you don't really have a solid quantity of all of the same kind. Lastly, weighing a poll based on the sample size makes sense, so long as the margin of error per release is also taken into account.

6- For This Midterm, Do Not Discount the "Presidential Penalty" (or "Midterm Penalty as it is more commonly known).
My frustrations with the GOP over its ability to constantly lose cannot blind me or any other observers to a demonstrable fact: the President's party never gains Senate seats in the 6th year: the average FDR and since has been a loss of 6.5. Clinton enjoyed no losses at all, Eisenhower a catastrophic 13 seat collapse, and the rest have fallen within that range. But even with the Republicans' incredible skill at losing, they are going to gain something, because history, and the numbers, say so. Harry Enten of 538 brought up this oft-overlooked phenomena on twitter this morning, and it bears more exploration. The best analysis of this, IMO, was by Robert S. Erikson way back in 1988. After exhaustively going through the data, he came to a conclusion that seems extremely generalistic, indicts our mass electorate as fickle, and is of course the only one that fits:

. At midterm, the president's party always performs poorly-even
when the president is popular and the economy is thriving. The one expla-
nation that does fit the data is that of a presidential penalty. At every mid-
term, the electorate turns against the presidential party for being the party
in power.

7. Take in all good points raised by other analysts, rather than rejecting the ones you don't like.
I absorb nearly everything tweeted and written out by Harry Enten, Sean Trende, Nate Silver, and Nate Cohn, not because these people are infallible poll-parsing gods, but because...they raise good points. Take Sean Trende's recent reflection on the performance of a party's senatorial candidates in contested races in relation to the President's job approval. Or Nate Cohn's aforementioned knifing of PPP.

My goal has been to provide accurate, reliable calls. The 2012 Wisconsin Recall was a high point for me, followed just five months later with a low. I'd like to take the time to thank all of those who volunteered and contributed to the AOSHQDD throughout 2012, even on the tough night of hell in November. Lastly, I'd like to thank Ace for continuing to allow me to do my thing here, even when I screwed up royally in the last go-round. There are very, very few accurate "election gurus" on the right. I want to regain the trust of those I lost in last cycle's botch, and realize I'll have to earn it by making accurate calls.

So, here's my promise to all the loyal readers of the AOSHQ: regardless the "good" or "bad" result I am seeing, it will be projected, and as accurately as possible.

Whatever will be, will be.

posted by CAC at 05:50 PM

| Access Comments