#Science: Co-Author of Sociological Study on Attitudes Towards Homosexuality Retracts His Part of the Paper, After "Irregularities" In His Co-Author's Data Collection Are Exposed

I can't swear to it, but this is sounding like a Michael "Arms in America" Belleiles situation, with #Science taking a backseat to political agitation.

This Washington Post story (which I got from Hot Air, of course) makes this all sound much less interesting than I think it actually is.

First, let's talk about the political agenda of the survey, which seems to be designed to push the idea that people's ideas about homosexuality can be fairly easily changed if a gay advocate just talks with them about it.

The study purported to show the ease with which peoples' minds can be changed on the subject of same-sex marriage after short conversations, particularly with gay advocates. It was described as being based on survey research conducted in California after voters passed Proposition 8, the referendum that banned same-sex marriage in the state and that has since been struck down by the courts.

...

The study attracted widespread attention in part because it seemed to fly in the face of conventional wisdom and scholarship about how people cling to their own points of view, sometimes regardless of what they read or hear to the contrary.

"One conversation can change minds on same-sex marriage, study finds," was the headline in The Washington Post reporting the conclusions in December.

"Gay political canvassers can soften the opinions of voters opposed to same-sex marriage by having a brief face-to-face discussion about the issue, researchers reported Thursday," a New York Times report said. "The findings could have implications for activists and issues across the political spectrum, experts said."

The paper got linked and discussed all over the place.

Now, one of the co-authors, Donald P. Green, is retracing the paper, and essentially accuses his co-author of... well, the Washington Post soft-plays it as "irregularities."

Green said two University of California-Berkeley graduate students who had attempted their own research "brought to my attention a series of irregularities that called into question the integrity of the data we present."

The other author, Michael J. LaCour, says, in a Hillary-like way, that he looks forward to addressing these concerns.

If you turn to the paper which discovered these "irregularities," you'll soon see the authors there aren't accusing LaCour of mere "irregularities."

The way I read this, is that they're accusing him of outright fabrication.

Here are some excerpts:

Summary

We report a number of irregularities in the replication dataset posted for LaCour and Green (Science, “When contact changes minds: An experiment on transmission of support for gay equality,” 2014) that jointly suggest the dataset (LaCour 2014) was not collected as described. These irregularities include baseline outcome
data that is statistically indistinguishable from a national survey and over-time changes that are unusually small and indistinguishable from perfectly normally distributed noise. Other elements of the dataset are inconsistent with patterns typical in randomized experiments and survey responses and/or inconsistent with
the claimed design of the study.

A straightforward procedure may generate these anomalies nearly exactly: for both studies reported in the paper, a random sample of the 2012 Cooperative Campaign Analysis Project (CCAP) form the baseline data and normally distributed noise are added to simulate follow-up waves.

I don't know exactly what they mean there, but I think -- think -- they're saying that they suspect the author took an already-available dataset and artificially added some "noise" to make it seem like a new or different study.*

Here's why I think that:

Timeline of Disclosure

• January - April, 2015. Broockman and Kalla were impressed by LaCour and Green (2014) and wanted to extend the article's methodological and substantive discoveries. We began to plan an extension. We sought to form our priors about several design parameters based on the patterns in the original data on which the paper was based, LaCour (2014).

As we examined the study's data in planning our own studies, two features surprised us: voters' survey responses exhibit much higher test-retest reliabilities than we have observed in any other panel survey data, and the response and reinterview rates of the panel survey were significantly higher than we expected.

We set aside our doubts about the study and awaited the launch of our pilot extension to see if we could manage the same parameters. LaCour and Green were both responsive to requests for advice about design details when queried.

• May 6, 2015. Broockman and Kalla launch a pilot of the extension study.

• May 15, 2015. Our initial questions about the dataset arose as follows. The response rate of the pilot study was notably lower than what LaCour and Green (2014) reported. Hoping we could harness the same procedures that produced the original study's high reported response rate, we attempt to contact the survey firm we believed had performed the original study and ask to speak to the staffer at the firm who we believed helped perform Study 1 in LaCour and Green (2014). The survey firm claimed they had no familiarity with the project and that they had never had an employee with the name of the staffer we were asking for. [!!! -- ace] The firm also denied having the capabilities to perform many aspects of the recruitment procedures described in LaCour and Green (2014).

Details omitted. Skipping ahed.

. • May 18-9, 2015. Green conveys to Aronow and Broockman that LaCour has been confronted and has confessed to falsely describing at least some of the details of the data collection. The authors of this report are not familiar with the details of these events.

So there's your #Science for you.

Whenever Science gets into bed with Politics, Science winds up taking it up the ass and then being kicked out the door at 3 am like a dirty smelly whore.

* I think that the original "researcher," LaCour, claimed to have conducted a survey about attitudes about homosexuality in the Los Angeles area, then used that as his baseline for his subsequent see-if-talking-to-people-changes-their-mind test.

However, what the authors of the paper pointing out these "irregularities" are saying, I think, is that LaCour actually conducted no Los Angeles area study to establish a baseline, but instead just took some already-existing national data and claimed it to be his Los Angeles study.

posted by Ace at 02:41 PM

| Access Comments