More Articles

Contact Editor

Newsletter

July 23, 2008

Unhealthy Statistics

By Mark Houston, M.D. and Nadine Taylor, M.S., R.D.

ReZoom Contributors

Unhealthy_Statistics

Should we take medical journals with a grain of salt or let the "experts" lead the way?

A key medical journal reports that an amalgam of studies show antioxidants to be ineffective, but is there a method to the madness?

Do No Harm

Like millions of health-conscious Americans, we were troubled (although not exactly panicked) by the meta-analysis published in the February, 2007 edition of the Journal of the American Medical Association proclaiming that antioxidants were not only ineffective at reducing mortality, but (in the case of vitamins A and E) might actually increase it.

Our first instinct was to be suspicious: A wealth of scientific evidence shows that antioxidants are vitally important disease fighters, immune boosters and promoters of good health. And as we looked more closely at the study, we found that our instincts were correct. The study used flawed premises to produce a flawed finding.

How did the researchers arrive at their startling conclusion? If you studied statistics, you may remember your professor cautioning you that, "If you torture the data long enough, they'll confess to anything." In other words, if you cherry-pick the studies you use so that you can include the favorable ones and exclude the unfavorable ones, if you define "biased" and "unbiased" studies according to your own rules, if you combine the data from wildly differing kinds of studies in the same data pool and otherwise massage the data, you can make your study "prove" whatever you want.

The Healthy Go Uncounted

Such is the case with the JAMA meta-analysis which, by definition, offers nothing new and is just a rehash of findings taken from previous studies. One of its most glaring flaws occurs right up front, with the selection of studies. The team of Danish researchers who authored the analysis collected studies of the effects of five antioxidants – beta-carotene, vitamins A, C and E and selenium – on the death rates of healthy and unhealthy subjects. Yet anyone attempting to make a blanket statement about antioxidants and mortality needs to include a great many other powerful and well-researched antioxidants, such as alpha-lipoic acid, carnitine, ginkgo biloba, grape seed extract, EGCG, lutein, lycopene and co-enzyme Q10, among others. Why limit the focus to these five?

Then, from an original pool of 815 studies, the researchers excluded a whopping 747 of them (91.5 percent) for one reason or another, leaving just eight percent on which to base their conclusions. Of the excluded studies, 405 were rejected simply because none of the participants in these studies had died. Yet how can the researchers possibly prove antioxidants have no effect on mortality if they've eliminated almost half of the studies in the data pool specifically because there was no mortality?

The Cherry Picking

But just for argument's sake, let's imagine for a moment that they were right in eliminating the 747 studies and whittling the data pool down to just 68 studies. The meta-analysis would still be fatally flawed as the included studies were far too diverse in subject, dosage, length and methods to be able to produce a credible conclusion.

For example, several different antioxidants were used in these studies, some singly, some in various combinations, and some as parts of groups that included still other substances, such as lutein or zinc. How could the researchers possibly determine which element or combination of elements (if any) was responsible for the results?

Natural vs. Synthetic

   
If you torture the data long enough, they will confess to anything."
- Dr. Mark Houston.
 
   

The studies also involved differing forms of the supplements. In most cases, synthetic versions of the antioxidants were used, despite the fact that the synthetics behave differently in the body than the natural versions and don't include all the "parts" of a nutrient's natural spectrum. For example, all of the vitamin E studies used in the meta-analysis involved synthetic alpha-tocopherol, even though it is well-established that the natural forms are much better absorbed and retained.

And while the studies of the effects of synthetic vitamin E have often produced disappointing results, emerging evidence suggests that one of the natural forms, gamma-tocopherol, may have greater antioxidant and anti-inflammatory abilities than any other form of E. Still, the researchers only included studies of synthetic E. Similarly, only synthetic versions of beta-carotene were used, even though this form has been shown to increase the risk of lung cancer on smokers and coronary heart disease (although no existing evidence implicates the natural forms of the vitamin). Why, then, only synthetics?

The Data Under the Radar

Dosages and treatment times also ran the gamut from moderate amounts of antioxidants taken over a period of years, to, in one case, a huge dose of 200,000 IU vitamin E taken for just a single day. What conscientious conclusion can be drawn from such diverse data? Was the increased mortality risk the result of taking too much of an antioxidant? Too little? Or neither?

But perhaps the biggest flaw of all in the JAMA study lies in combining studies of healthy people with those of unhealthy people to come up with an overall mortality risk. The study population makes a huge difference in the outcome. Combining these two groups automatically skews the results, as the risk of death in the chronically ill is inherently much higher than it is in healthy people. Plus, no one has ever claimed that antioxidants can prevent death in those who are seriously ill. Lumping together the death rates from both of these groups to come up with a link between antioxidant use and the risk of mortality is simply bad science.

Professor Balz Frei, Director of the Linus Pauling Institute at Oregon State University, may have summed it up best when he said: "All the new study really demonstrates is a bias toward identifying studies or research that show harm caused by antioxidants, and selective removal of research that shows benefits."

Want more? Try our Wellness channel.

Have Something to Say?
Share your comments with other readers... we appreciate your opinion!
(login / or create an account to comment)

0 Comments »

Have Your Say!

Are medical journals biased?
  • Not biased
  • Sometimes biased
  • Generally biased
  • Don't read medical journals