Friday, January 04, 2013

All Cats Bark and Being Fat Is Healthy

All dogs bark. All dogs have four legs. All cats have four legs, therefore fat cats are healthier than normal sized ones. Wait, somewhere along there the logic broke down.

source
The latest meta-study of poor science to hit the media mainstream drone express shows that being moderately obese (25 ≤ BMI ≤ 30) has 6% less mortality (ie healthier) than being "normal" (18.5 ≤ BMI ≤ 25).

There are a lot of things not to like about this study. The most obvious one is that BMI is a really lousy measure of obesity. Height and weight and that's it, let's completely ignore muscle and bone mass because those things are hard to measure.

Why bother building the Large Hadron Collider to look for the Higgs boson? That's also really hard to measure. Can't we just do a meta-study of all the physics papers and figure it out from there?

Which leads me to the next big problem with this paper, it is a meta-study, and in case it wasn't clear, I HATE META STUDIES!!! Almost every time one of these idiotic pronouncements sweeps through the media drones it is because of a meta-study. Remember the great red meat causes cancer kerfuffle a while back? Another poorly done meta-study.

The great thing about meta-studies is that all a 'scientist' has to do is run a net through the literally millions of scientific papers that have been published in the last few decades, throw all the results into a huge spreadsheet, make a few graphs and BAM, they've made a contribution to science.

Except they haven't.

The problem with normal dietary studies is that there are already way too many variables and very little in the way of hard data (eg self-reported diet data). So when these meta-studies combine studies together they simply add more variables (different nationalities with different diets, genetics, different methodologies, etc).

What could possibly go wrong?

According to the lead author this diversity is a good thing.
CDC researchers analyzed 97 studies involving nearly three million people and 270,000 deaths around the world. "The findings are very consistent across all different ages and continents," said lead author Katherine Flegal, a senior scientist at the CDC. She stressed that the study looks at all causes of mortality, not overall health risks. "This is not meant to suggest that the conventional wisdom is wrong," she said. (Source)
Actually, Kate, the conventional wisdom IS wrong. She does mention that the study looks at overall mortality not health risk which is an important caveat, especially since the study looks at many different countries. In places like China or Taiwan (n = 4), Brazil (n = 2), India (n = 1), and Mexico (n = 1), being moderately obese is likely correlated with higher income and social status, better access to healthcare, better working conditions, etc. And in the richer countries, moderate obesity might be correlated with a different demographic, married people tend to live longer, etc. 

But more importantly, when trying to isolate cause and effect, which is kind of the point of science, diversity is not a strength. 

As for being very consistent across all ages and continents, I find that a pretty dubious claim, and couldn't find any tables in the paper to back it up. All countries and ages showed moderate obesity (as measured by BMI) to be 6% less death-defying than normal weight (as measure by BMI)?

Then there was this little snippet in the comment section of the paper: 
We found a generally lower summary HR (hazard ratio) and less heterogeneity in studies using measured data than in studies using self-reported data. 
In other words, the harder the data, the less correlation and the more uneven were the results.

Followed by this: 
The differences were more pronounced in analyses stratified by sex than in analyses that combined both men and women. Because the errors in self-reported data tend to differ by sex, there may be an offsetting effect when analyses combine men and women.
Wow, this is amazingly disingenuous. Men lie about their weight in self-reported studies, women lie about their weight much more than men. So the "errors" in self-reported studies are less in studies that combine men and women. Really bad data is "offset" by merely bad data. Ladies and gentlemen, this is what passes for science in the world of government sponsored nutritional health! How about we just throw out all the self-reported data studies? Naw, that's just crazy talk.

No comments:

Post a Comment