Saturday Afternoon Chess thread 09-26-2020 | Main | Saturday Overnight Open Thread (9/26/20)
September 26, 2020

Science Isn't Dead, But it Is Ailing! [Jay Guevara]

madscience.jpg

Science faces a crisis of confidence owing to its politicization and the replication crisis. The latter, which intersects with concerns about scientific fraud, is the subject of this post, a matter on which I've previously commented. (OK, ranted.)


Part of the impetus was from a study by Amgen, summarized in Wikipedia as follows:
Out of 49 medical studies from 1990–2003 with more than 1000 citations, 45 claimed that the studied therapy was effective. Out of these studies, 16% were contradicted by subsequent studies, 16% had found stronger effects than did subsequent studies, 44% were replicated, and 24% remained largely unchallenged.[59] The US Food and Drug Administration in 1977–1990 found flaws in 10–20% of medical studies.[60] In a paper published in 2012, Glenn Begley, a biotech consultant working at Amgen, and Lee Ellis, at the University of Texas, argued that only 11% of the pre-clinical cancer studies could be replicated.[61][62]

To try to put some numbers to my visceral impressions, and thereby to test my conjectures regarding dodgy results I went over to Retraction Watch, which features a database of papers that have been retracted by the authors/journals or otherwise had their validity called into question.

One can rummage around in that database for hours, so I narrowed the search to "concerns/issues about results," which yielded 562 hits.

jg01.jpg


The requisite data to normalize for the number of papers published in each field are not to hand, but with that caveat these data bear out the impression that the biomedical literature has some real problems.

Where, geographically, do these problems come from? Querying the database for papers that raised concerns/issues with results (regardless of whether from physical, life, or medical sciences) yielded these results, again not normalized for the number of papers published in each country:

jg02.jpg


Without information regarding the number of papers published, it's impossible to draw hard conclusions, but I would imagine that there are probably a lot more papers published in the U.S. than in China, and yet the two are in the same ball park regarding questionable results.

Interestingly, the database includes data regarding some of the more dubious areas of "science." For this search I included outright scientific fraud, and found effectively none for any of these fields, although a similar search the physical, biological, and health sciences all returned a number of hits. Once again, the caveat regarding total number of papers published applies, this time probably a forteriori.

jg03.jpg

For comparison:

jg04.jpg


My conclusion, subject to the caveat above: the biomedical literature does in fact have grievous problems, and China is punching above its weight in generating lousy results. Last, the Wild West of "scientific research" (climate change, sociology, alternative medicine, and psychology) I suspect may not have fraudulent papers because results in those areas are so squishy that it is impossible to falsify pretty much any assertion made in those literatures.

digg this
posted by Open Blogger at 07:30 PM

| Access Comments




Recent Comments
Blanco Basura - moronhorde.com lives again: "Douchy inbred bad copy of Mitt Romney ..."

Catch Thirty-Thr33: "827 Bernie Sanders @BernieSanders Walter Wallace ..."

Romeo13: "831 Roberts screwed us again with an assist from k ..."

runner: "Both. But consider this. Would you shake hands w ..."

Rodrigo Borgia: "> I have to admit I'm impressed Anon was an actual ..."

Alabaster Jones: "Of insane sock ..."

Zod: ">>Walter Wallace should be alive today, and our th ..."

Berserker-Dragonheads Division: "I have a MoMe related question. Not about any part ..."

The Left: "The Fox producer labeled the package : To: Tuc ..."

Romeo13: "Posted by: Optimizer at October 28, 2020 09:54 PM ..."

Catch Thirty-Thr33: "But do remember this, my TX friends: the blitherin ..."

EdmundBurkesShade: "How does one know at this time how many people of ..."

Recent Entries
Search


Polls! Polls! Polls!
Frequently Asked Questions
The (Almost) Complete Paul Anka Integrity Kick
Top Top Tens
Greatest Hitjobs

The Ace of Spades HQ Sex-for-Money Skankathon
A D&D Guide to the Democratic Candidates
Margaret Cho: Just Not Funny
More Margaret Cho Abuse
Margaret Cho: Still Not Funny
Iraqi Prisoner Claims He Was Raped... By Woman
Wonkette Announces "Morning Zoo" Format
John Kerry's "Plan" Causes Surrender of Moqtada al-Sadr's Militia
World Muslim Leaders Apologize for Nick Berg's Beheading
Michael Moore Goes on Lunchtime Manhattan Death-Spree
Milestone: Oliver Willis Posts 400th "Fake News Article" Referencing Britney Spears
Liberal Economists Rue a "New Decade of Greed"
Artificial Insouciance: Maureen Dowd's Word Processor Revolts Against Her Numbing Imbecility
Intelligence Officials Eye Blogs for Tips
They Done Found Us Out, Cletus: Intrepid Internet Detective Figures Out Our Master Plan
Shock: Josh Marshall Almost Mentions Sarin Discovery in Iraq
Leather-Clad Biker Freaks Terrorize Australian Town
When Clinton Was President, Torture Was Cool
What Wonkette Means When She Explains What Tina Brown Means
Wonkette's Stand-Up Act
Wankette HQ Gay-Rumors Du Jour
Here's What's Bugging Me: Goose and Slider
My Own Micah Wright Style Confession of Dishonesty
Outraged "Conservatives" React to the FMA
An On-Line Impression of Dennis Miller Having Sex with a Kodiak Bear
The Story the Rightwing Media Refuses to Report!
Our Lunch with David "Glengarry Glen Ross" Mamet
The House of Love: Paul Krugman
A Michael Moore Mystery (TM)
The Dowd-O-Matic!
Liberal Consistency and Other Myths
Kepler's Laws of Liberal Media Bias
John Kerry-- The Splunge! Candidate
"Divisive" Politics & "Attacks on Patriotism" (very long)
The Donkey ("The Raven" parody)
Powered by
Movable Type 2.64