Ace of Spades HQ

« Sarah Silverman, Still Occasionally Funny | Main | Economy Powers To 4.3% GDP Growth, Despite Hurricanes »

November 30, 2005

Expert Political Judgment

The New Yorker does a good rundown on “Expert Political Judgment: How Good Is It? How Can We Know?”, a new book by Berkeley academic, Philip Tetlock.

As the title suggests, it’s all about how accurate the predictions the political and economic pundits we see and read actually turn out to be. And Tetlock isn’t just some crank, shooting off his mouth. He’s pretty rigorous:

[H]is conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. …By the end of the study, in 2003, the experts had made 82,361 forecasts

And his conculsion? The ‘experts’ are full of it.

[P]eople who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. …
Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable.

And the big shots are the worst:

Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote.

Tetlock points to a number of explanations, most centering on the psychological foibles inherent in assessing uncertainty, along with simple stuff like the fact there’s really no puditocracy penalty for ‘getting it wrong.’

Anyway, the piece is relatively short, not one of those New Yorker book/articles and well worth the read.

(N.b. My predictions were not part of the study, and remain, of course, rock solid certainties. You can continue to bet the farm on the Good Doctor's forecasts)

(h/t Jane Galt)

posted by Dr. Reo Symes at 01:00 PM

Comments

Yeah, duh. Most bloggers seem to spend their days pointing out how wrong these experts are. The inverse relationship between predictions and accuracies is obvious for most liberal "experts" as they are merely giving voice to their dreams. The wronger they are, the more air they get. That way, Chris Matthews can circle jerk with fellow Bush bashers.

Coulter pointed this out in Slander, how CNN threatened to replay a clip of her being wrong about how long the impeachment trial would last. Gee, who won that battle?

I mean, if you are going to offer your credibility up to pose WP as a WMD, then you have to expect to never be right. Even if WP=WMD will get you NYT cred. BTW, does the WP=WMD mean that Iraq had WMDs? So Bush was right all along!

Posted by: joeindc44 on November 30, 2005 01:19 PM

I predicted those results.

Posted by: polynikes on November 30, 2005 01:34 PM

My guess is that Tradesports is a far better predictor of political events than the prognostications of puffed-up, self important pundits.

I wish there would have been some way to bet on whether a military draft would be installed during the Bush presidency. I would have laid at least a grand against it, and I'm certain not one lefty would have taken the bet. Even they know how full of shit they are.

Posted by: The Warden on November 30, 2005 01:45 PM

He probably has a whole chapter devoted to Paul Krugman and Jim Cramer.

Posted by: Moonbat_One on November 30, 2005 02:23 PM

Or Dick Morris - he seems like the classic case of someone who's rewarded more for being entertaining than being right. He's a smart guy, but his bad predictions could fill a small library. The latest I remember (besides Hillary v. Condi) was that Bloomberg v. Ferrer would be down to the wire.

Posted by: Yaron on November 30, 2005 03:07 PM

I predict the nation is about to be swept by Kusinich fever!! Has anyone seen Dennis' new bride? She's like six foot two. And he's like, five foot nothing, one hundred and nothing.

Like that scene in Deuce Bigalo. "That's a huge bitch!"

Posted by: fugazi on November 30, 2005 03:57 PM

The journal Science had a study on experts several years ago that came to the same conclusion. It focused on experts advising judges on whether a given criminal was likely to commit more crimes if released. Needless to say, they were flipping coins.

The author of the study said that one thing surprised him. He expected that judges actually believed the experts. In fact, he found that judges recognized perfectly well, that the experts were blowing smoke. They went with the expert recommendations because they had nothing better, and it shifted the responsibility off them and on to the "experts."

Posted by: Bob Hawkins on December 1, 2005 02:51 PM

Reminds me of the bio of Jimmy the Greek I saw on ESPN yesterday. He didn't know shit about football, but he sure could call them.

Posted by: Mark V. on December 1, 2005 06:02 PM


Advertise Here! Intermarkets' Privacy Policy Support Donate to Ace of Spades HQ! Contact Ace: aceofspadeshq at gee mail.com Buck: buck.throckmorton at protonmail.com CBD: cbd at cutjibnewsletter.com joe mannix: mannix2024 at proton.me MisHum: petmorons at gee mail.com J.J. Sefton: sefton at cutjibnewsletter.com Recent Entries Another mRNA Vaccine? Really? Sunday Morning Book Thread 6-21-2026 [Sabrina Chase] Daily Tech News 21 June 2026 Saturday Night Club ONT - June 20, 2026 [D Squared] Saturday Evening Movie Thread - 6/20/2026 Hobby Thread - June 20, 2026 [TRex] Ace of Spades Pet Thread, June 20 Gardening, Home and Nature Thread, June 20 Father's Day Weekend, 2026 The Classical Saturday Morning Coffee Break & Prayer Revival Absent Friends Captain Whitebread 2026 Jon Ekdahl 2026 Jay Guevara 2025 Jim Sunk New Dawn 2025 Jewells45 2025 Bandersnatch 2024 GnuBreed 2024 Captain Hate 2023 moon_over_vermont 2023 westminsterdogshow 2023 Ann Wilson(Empire1) 2022 Dave In Texas 2022 Jesse in D.C. 2022 OregonMuse 2022 redc1c4 2021 Tami 2021 Chavez the Hugo 2020 Ibguy 2020 Rickl 2019 Joffen 2014 AoSHQ Writers Group A site for members of the Horde to post their stories seeking beta readers, editing help, brainstorming, and story ideas. Also to share links to potential publishing outlets, writing help sites, and videos posting tips to get published. Contact OrangeEnt for info: maildrop62 at proton dot me Cutting The Cord And Email Security Cutting The Cord [Joe Mannix (not a cop)] Cutting The Cord: It's Easier Than You Think [Blaster] Private Email and Secure Signatures [Hogmartin] Moron Meet-Ups NoVaMoMe 2026: 06/20/2026 Arlington VA Contact Bluebell/Weasel for info Texas MoMe 2026: 10/16/2026-10/17/2026 Corsicana,TX Contact Ben Had for info	« Sarah Silverman, Still Occasionally Funny \| Main \| Economy Powers To 4.3% GDP Growth, Despite Hurricanes » November 30, 2005 Expert Political Judgment The New Yorker does a good rundown on “Expert Political Judgment: How Good Is It? How Can We Know?”, a new book by Berkeley academic, Philip Tetlock. As the title suggests, it’s all about how accurate the predictions the political and economic pundits we see and read actually turn out to be. And Tetlock isn’t just some crank, shooting off his mouth. He’s pretty rigorous: [H]is conclusions are based on a long-term study that he began twenty years ago. He picked two hundred and eighty-four people who made their living “commenting or offering advice on political and economic trends,” and he started asking them to assess the probability that various things would or would not come to pass, both in the areas of the world in which they specialized and in areas about which they were not expert. …By the end of the study, in 2003, the experts had made 82,361 forecasts And his conculsion? The ‘experts’ are full of it. [P]eople who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. … Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable. And the big shots are the worst: Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote. Tetlock points to a number of explanations, most centering on the psychological foibles inherent in assessing uncertainty, along with simple stuff like the fact there’s really no puditocracy penalty for ‘getting it wrong.’ Anyway, the piece is relatively short, not one of those New Yorker book/articles and well worth the read. (N.b. My predictions were not part of the study, and remain, of course, rock solid certainties. You can continue to bet the farm on the Good Doctor's forecasts) (h/t Jane Galt) posted by Dr. Reo Symes at 01:00 PM Comments Yeah, duh. Most bloggers seem to spend their days pointing out how wrong these experts are. The inverse relationship between predictions and accuracies is obvious for most liberal "experts" as they are merely giving voice to their dreams. The wronger they are, the more air they get. That way, Chris Matthews can circle jerk with fellow Bush bashers. Coulter pointed this out in Slander, how CNN threatened to replay a clip of her being wrong about how long the impeachment trial would last. Gee, who won that battle? I mean, if you are going to offer your credibility up to pose WP as a WMD, then you have to expect to never be right. Even if WP=WMD will get you NYT cred. BTW, does the WP=WMD mean that Iraq had WMDs? So Bush was right all along! Posted by: joeindc44 on November 30, 2005 01:19 PM I predicted those results. Posted by: polynikes on November 30, 2005 01:34 PM My guess is that Tradesports is a far better predictor of political events than the prognostications of puffed-up, self important pundits. I wish there would have been some way to bet on whether a military draft would be installed during the Bush presidency. I would have laid at least a grand against it, and I'm certain not one lefty would have taken the bet. Even they know how full of shit they are. Posted by: The Warden on November 30, 2005 01:45 PM He probably has a whole chapter devoted to Paul Krugman and Jim Cramer. Posted by: Moonbat_One on November 30, 2005 02:23 PM Or Dick Morris - he seems like the classic case of someone who's rewarded more for being entertaining than being right. He's a smart guy, but his bad predictions could fill a small library. The latest I remember (besides Hillary v. Condi) was that Bloomberg v. Ferrer would be down to the wire. Posted by: Yaron on November 30, 2005 03:07 PM I predict the nation is about to be swept by Kusinich fever!! Has anyone seen Dennis' new bride? She's like six foot two. And he's like, five foot nothing, one hundred and nothing. Like that scene in Deuce Bigalo. "That's a huge bitch!" Posted by: fugazi on November 30, 2005 03:57 PM The journal Science had a study on experts several years ago that came to the same conclusion. It focused on experts advising judges on whether a given criminal was likely to commit more crimes if released. Needless to say, they were flipping coins. The author of the study said that one thing surprised him. He expected that judges actually believed the experts. In fact, he found that judges recognized perfectly well, that the experts were blowing smoke. They went with the expert recommendations because they had nothing better, and it shifted the responsibility off them and on to the "experts." Posted by: Bob Hawkins on December 1, 2005 02:51 PM Reminds me of the bio of Jimmy the Greek I saw on ESPN yesterday. He didn't know shit about football, but he sure could call them. Posted by: Mark V. on December 1, 2005 06:02 PM Post a comment Name: Email Address: URL: Comments: Remember info?	Now Available! The Deplorable Gourmet A Horde-sourced Cookbook [All profits go to charity] Top Headlines TJM discusses the Martin Scorsese film Hugo Paul Sperry @paulsperry_ NEW: Just heard something extraordinary from a former White House official who worked with former National Security Adviser H.R. McMaster in Trump45's NSC: "McMaster had weekly phone calls with George Soros. We have no idea why." Neither could be reached for comment. Deport...Deport...Deport The F*ing Lot! A new UK anthem? [Hat Tip: S.E.] [CBD] Podcast*: CBD and Sefton dissect the Iran treaty but praise the great U.S. military, decry the deep state's influence on SAVE and FISA, talk marijuana and guns, mock the Northeast's racism, and Go Knicks! Supreme Court Rules Habitual Marijuana Users Can Legally Own Firearms [CBD] Trump: Ukraine War 'Thousands of Miles Away' is 'Nothing to Do' with America Russia isn't threatening to kill Americans!* [CBD] TJM definitively ranks the films of David Lynch. Dougie Jones. Update to Gavin Newsom Under Investigation story: This investigation was begun under Senor Dementia: Adam Housley @adamhousley As I have reported several times and now acknowledged by the Governor of California... Gavin and his wife are under federal investigation... what he failed to tell you... This began during the Biden Admin. Kind of a big detail. SpaceX's IPO sets the stage for the colonization of the solar system -- by private enterprise [CBD] Teen Driver Tayvin Galanakis Wins Jury Trial Against Officers Who Charged Him With DUI Even After He Blew 0.0 on A Breathalyzer And Passed Sobriety Tests. One Officer Accounted For 72% of All DUI Arrests For That PD [dri] Days before the woman was stabbed in the neck by a taxpayer-supported Cultural Enrichment Officer, in the same general area, another taxpayer-supported Cultural Enrichment Officer attacked a boy and bloodied his head with a brick. What is the UK Regime's plan for protecting the citizens from the savage criminals they've foisted on the populace? They offer NONE. They do, however, have a plan for protecting the savage criminals from the citizens: The citizens must STAY CALM and not get angry and not share videos of citizens being attacked by savage criminals. The public keeps saying "protect us from the foreign savages you have imported against our wishes and over our objections" and the UK branch of The Regime keeps proposing plans to protect the foreign savages from the public. Soclose to what the public is demanding, just, you know, the complete opposite. Just a thought: Maybe you wouldn't have to worry about the public attacking the savage criminals if you actually introduced a plan to protect the public from the savage criminals. Maybe they wouldn't feel as if it was necessary for them to protect the public through self-help. Courtney Subramiam, one of the "journalists" who "previewed" her questions for the decrepit and demented Biden so that he could "answer" it with a pre-scripted response, rewarded by promotion to president of the White House Press Corps Bonchie @bonchieredstate hahahahaha This is the lady who gave her question to Biden beforehand, and he had it written verbatim in his notes with her picture. You know what's really terrible? There are Daily Signal reporters in the press room. That's the Real Scandal Here! English racist garbage-person who's on the wrong side of history warns the corrupt Regime that the people cannot take much more of this -- and won't take much more of this The English have rebelled before. All of the various Bob Dylans by Ace-Endorsed Author A.H. Lloyd TJM gets lost in the worlds of David Lynch You might think that movie critics by nature are effeminate and bitchy, but, did you know that grass is green and red peppers are red? "We have black ghetto fatigue" Odd 90s-Retro Susan Collins ad against the Nazi Hotchkiss "hobby farmer" I like the throwback AOL style of the ad. Podcast: Sefton and CBD bounce around from Maine and its pet Nazi, to the cracks in the Democrat messaging, to the failure of California and its effect on the 2028 election, sea drones rescuing Apache crews, and more! Seattle mayor shrugs off millionaire-tax concerns as 44% of business leaders consider leaving It happens in all the blue states, but WA and Seattle will be different! [CBD] Recent Comments Just Some Guy: "And I'm off to fail to do anything constructive to ..." Wolfus Aurelius, Dreaming of Elsewhere [/i] [/b] [/s]: "Anyway! A tip of the chapeau to Sabrina for leadi ..." Wolfus Aurelius, Dreaming of Elsewhere [/i] [/b] [/s]: "Of course the edition of the play script might be ..." Just Some Guy: "A lot of William Goldman's scripts make fun readin ..." Wolfus Aurelius, Dreaming of Elsewhere [/i] [/b] [/s]: "Sometimes reading stage play scripts are disappoin ..." Minotaurs ravaged my cooch!: "Yes, most female bylines now are a turnoff for me. ..." [/i][/s][/b][/u]Oddbob: "[i]Aunt Esther scene I remember: Fred: “wha ..." 13times: "> 208 There were a few other old books, one was ..." Thomas Bender: "The one screenplay I read that was pretty much wor ..." Dr. Pork Chops & Bacons: " It was falling to pieces, the muslin cover loose ..." Just Some Guy: "Reading this week? Decided to put off Bleak House ..." [/i][/s][/b][/u]Oddbob: "[i] Yes, most female bylines now are a turnoff for ..." Bloggers in Arms RI Red's Blog! Behind The Black CutJibNewsletter The Pipeline Second City Cop Talk Of The Town with Steve Noxon Belmont Club Chicago Boyz Cold Fury Da Goddess Daily Pundit Dawn Eden Day by Day (Cartoon) EduWonk Enter Stage Right The Epoch Times Grim's Hall Victor Davis Hanson Hugh Hewitt IMAO Instapundit JihadWatch Kausfiles Lileks/The Bleat Memeorandum (Metablog) Outside the Beltway Patterico's Pontifications The People's Cube Powerline RedState Reliapundit Viking Pundit WizBang Some Humorous Asides Kaboom! Thanksgivingmanship: How to Deal With Your Spoiled Stupid Leftist Adultbrat Relatives Who Have Spent Three Months Reading Slate and Vox Learning How to Deal With You You're Fired! Donald Trump Grills the 2004 Democrat Candidates and Operatives on Their Election Loss Bizarrely I had a perfect Donald Trump voice going in 2004 and then literally never used it again, even when he was running for president. A Eulogy In Advance for Former Lincoln Project Associate and Noted Twitter Pestilence Tom Nichols Special Guest Blogger Rich "Psycho" Giamboni: If You Touch My Sandwich One More Time, I Will Fvcking Kill You Special Guest Blogger Rich "Psycho" Giamboni: I Must Eat Jim Acosta Special Guest Blogger Tom Friedman: We Need to Talk About What My Egyptian Cab Driver Told Me About Globalization Shortly Before He Began to Murder Me Special Guest Blogger Bernard Henri-Levy: I rise in defense of my very good friend Dominique Strauss-Kahn Note: Later events actually proved Dominique Strauss-Kahn completely innocent. The piece is still funny though -- if you pretend, for five minutes, that he was guilty. The Ace of Spades HQ Sex-for-Money Skankathon A D&D Guide to the Democratic Candidates Michael Moore Goes on Lunchtime Manhattan Death-Spree Artificial Insouciance: Maureen Dowd's Word Processor Revolts Against Her Numbing Imbecility The Dowd-O-Matic! The Donkey ("The Raven" parody) Archives June 2026 May 2026 April 2026 March 2026 February 2026 January 2026 December 2025 November 2025 October 2025 September 2025 August 2025 July 2025 June 2025 May 2025 April 2025 March 2025 February 2025 January 2025 December 2024 November 2024 October 2024 September 2024 August 2024 July 2024 June 2024 May 2024 April 2024 March 2024 February 2024 January 2024 December 2023 November 2023 October 2023 September 2023 August 2023 July 2023 June 2023 May 2023 April 2023 March 2023 February 2023 January 2023 December 2022 November 2022 October 2022 September 2022 August 2022 July 2022 June 2022 May 2022 April 2022 March 2022 February 2022 January 2022 December 2021 November 2021 October 2021 September 2021 August 2021 July 2021 June 2021 May 2021 April 2021 March 2021 February 2021 January 2021 December 2020 November 2020 October 2020 September 2020 August 2020 July 2020 June 2020 May 2020 April 2020 March 2020 February 2020 January 2020 December 2019 November 2019 October 2019 September 2019 August 2019 July 2019 June 2019 May 2019 April 2019 March 2019 February 2019 January 2019 December 2018 November 2018 October 2018 September 2018 August 2018 July 2018 June 2018 May 2018 April 2018 March 2018 February 2018 January 2018 December 2017 November 2017 October 2017 September 2017 August 2017 July 2017 June 2017 May 2017 April 2017 March 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 November 2012 October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 March 2012 February 2012 January 2012 December 2011 November 2011 October 2011 September 2011 August 2011 July 2011 June 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 December 0000 Syndicate this site (XML) Powered by Movable Type 2.64