Oh Boy: Big Decline In Childhood Obesity, Lauded By Michelle Obama As Proof of Efficacy of Let's Move Campaign, May Have Just Been... A Statistical Error

We are well and truly in the very best of hands.

In late February, a report from the U.S. Centers for Disease Control and Prevention announced that obesity rates among children aged 2 to 5 had declined by 40 to 43 percent in the past eight years, a dramatic and encouraging finding. But researchers are now saying that the good news may have been a statistical mistake.

...

The CDC’s study relied on a set of government-collected data that’s considered highly reliable, but wasn’t ideal for this comparison: The study looked at over 9,000 Americans, but just 871 were between 2 and 5, and just a small proportion of them are obese. The margin of error, in fact, was wide enough that it’s statistically possible there was no decrease at all.

This Reuters article explains why there's doubt:

If the news last month that the prevalence of obesity among American preschoolers had plunged 43 percent in a decade sounded too good to be true, that's because it probably was, researchers say.

...

In fact, based on the researchers' own data, the obesity rate may have even risen rather than declined.

"You need to have a healthy degree of skepticism about the validity of this finding," said Dr. Lee Kaplan, director of the weight center at Massachusetts General Hospital in Boston.

...

In some research 871 would be considered a large number. But when the obesity rate is fairly low, having a sample of a few hundred makes it easier for errors to creep in through random chance.

"In small samples like this, you are going to have chance fluctuations," said epidemiologist Geoffrey Kabat of the Albert Einstein College of Medicine in New York City.

...

The 13.9 percent obesity rate among preschoolers reported for 2003-2004 had a large enough margin of error that the actual rate could range between 10.8 percent and 17.6 percent, the CDC authors acknowledged. The 8.4 percent rate in 2011-2012 reported could range from 5.9 percent and 11.6 percent.

There are no other studies suggesting what this study claims. A WIC study (with 200,000 kids in it) found a small downward decrease in obesity rates, but nothing like 43%.

As I mentioned yesterday, the scientists involved in finding gravitational waves actually discovered, they thought, evidence of the waves three years ago. They spent the last three years amassing more evidence, and checking their math.

The video here shows one of the researchers coming to a scientist's house to inform him they'd determined the evidence had been proven to be very statistically sound. "5-sigma, 0.2 r," the researcher tells the scientist, which means nothing to me, but it means a lot to the scientist.

We have nowhere near this kind of rigor in the so-called "soft sciences," obviously. People just crank out study after study. Tiny, shaky findings of correlation are trumpeted as being very meaningful.

It's really pretty embarrassing.

A 43% drop in preschool obesity, with no other data supporting such a thing, and no plausible mechanism explaining such a sharp drop, shouldn't really have been reported.

posted by Ace at 05:23 PM

| Access Comments