Kos' Lawsuit Against R2K Alleges Falsification

Details:

1) The suit alleges that Research 2000 founder Del Ali repeatedly promised to volunteer to Kos the raw polling data behind the surveys, but never came through -- potentially raising further suspicions. The suit quotes multiple emails from Ali to Daily Kos founder Markos Moulitsas, vowing to share the data.

The suit quotes an email from Ali that seemed to ask for more time to furnish the data because of office computer malfunctions, prompting him to go to a Kinkos. "I am at a Kinkos computer because we cannot read any mail from our PCs," Ali emailed to Moulitsas, according to the suit.

2) The suit alleges that Research 2000 had money problems, which could prompt further questions about the firm's business practices. It alleges that Ali asked Moulitsas to make one lump-sum payment in advance, in exchange for some free polling, because it would provide "immense" help for cash flow reasons.
3) The suit alleges that Ali was infuriated when statistics guru Nate Silver recently rated the firm poorly. It says that when Kos reacted to the Silver analysis by terminating their relationship, Ali responded to Kos with a lengthy defense of his polling methods, lashing out at Silver as "nothing more than a fringe blogger."

I don't get that last allegation, really, as it serves to help R2K more than Kos -- because it supplies a motive for Kos to talk smack about the company and weasel out of paying them (if he owed them money, which R2K implies he did but Kos says he didn't).

Nate Silver says that releasing the raw data is R2K's best move... if innocent. And if they don't release the data, it looks pretty shaky for them.

Particularly if the data release occurred as soon as possible, and was as comprehensive as possible, Mr. Ali would be entitled to a strong intellectual (rather than merely legal) presumption of innocence. In addition, the spreadsheet could presumably contain the weights Ali assigned to individual observations, which would help to explain why Research 2000 has produced results that are unusual, even if they are not fraudulent.

But so far, Research 2000 has not made such documentation available, certainly not to the public, nor apparently to its client, Daily Kos, in spite of having had two weeks of warning that Daily Kos was about to publish a study that called the propriety of their polling into question. Perhaps he will take the opportunity to do so now.

Horeover, although it would not be easy to fabricate or reverse-engineer such data, a sufficiently cunning and unscrupulous person could presumably do so, given enough time....

Another form of proof that Ali could offer would be records, such as invoices, from the call center(s) that he used to conduct his polling. Research 2000, which lists only a post office box as its contact information, almost certainly does not have its own call center (nor do most pollsters, for that matter).... The volume of business between Research 2000 and its call center would presumably be extremely high. If the call center could conduct interviews for Research 2000 at a rate of $5 per completed survey -- and that's probably a very aggressive rate for traditional, non-automated telephone polling -- it would be doing about $12,000 worth of business with Research 2000 per week, assuming a 1,200-person national survey and two 600-person state surveys, as is fairly typical for them. That would amount to more than $600,000 worth of business per year, which surely would have produced a long paper trail.

Huh. I assumed R2K did automated (robo) calling, so the costs of surveys would be, if not trivial, pretty darn low, so I thought they'd have no particular motivation to falsify any damn thing. If they were supposed to be conducting person-to-person surveys, that's more expensive then, and then they would have an incentive.

If it was a third-party call-center placing the calls, though: There's the possibility the data was faked but not by R2K, but by whichever company was supposedly doing the calling. It's cheaper to just fake up a bunch of "interviews" than pay 100 people $10 an hour for five nights to actually do them.

More Caution: A pollster tells me the data can get funky when it's weighted -- a lot of this is wack-a-mole stuff where if you, say, weight up a particular subgroup because you didn't get enough respondents there (say young adults), it then starts throwing your other subgroups out of whack, and you then have to weight those down, etc.

A lot of data supplied in polls has been painfully tweaked. Each tweaking introduces new artifacts of manipulation.

The sort of analysis conducted here, showing R2K to have engaged in fraud, is only possible because R2K released detailed crosstabs which most other firms withhold from the public.

Point being: It's quite possible the analysis that showed falsification would have shown falsification in any number of perfectly-legitimate polls, due to all the manipulation/weighting of numbers, but such analyses are never performed because no one else routinely releases detailed crosstabs to the public. So some of the stuff found to be suspicious may only look suspicious because usually the sausage-making ugliness is better hidden from outside eyes.

Anyway, that's one possible theory. I don't know enough to stand by that or do much more than float it.

posted by Ace at 01:01 PM

| Access Comments