Evidence and Snake Oil

Recently a friend wrote to me and asked what I thought of a presentation she had heard about a product containing beta-glucan: 

Presentation part 1

Presentation part 2

Presentation part 3

I'm not suggesting anyone spend the time to watch all of this, but it is a 30 minute talk at a meeting (of salespeople, I think) of a company planning to market a version of an existing product with a preparation of beta-glucan added to it.

The speaker is excellent, and his audience is clearly impressed to have such a knowledgeable scientist speaking to them, and excited at the chance to market a new healthful product that the speaker promises will be one of the major revolutions in health in this half century or even century.

A claim like that is a bit astounding, and would obviously require high quality evidence to back it up. As Carl Sagan used to say, "extraordinary claims require extraordinary evidence". That someone would make such claims at a sales meeting, such that the attendees were hearing about this revolution in health there first, rather than from the mainstream media reporting on a health breakthrough, might cause the incredulous to suspect they were about to learn about snake oil.

The credulous, on the other hand, might believe that meetings like this are exactly where health breakthroughs are presented because the mainstream media and mainstream doctors would never allow such information to be presented in a medium that they control.

The speaker talks of the astounding immune stimulating virtues of beta-glucan to repair the ravages that modern society has wreaked on our immune systems and points to a number of clinical trials. He goes on to describe the benefits shown in the clinical trials (mainly for upper respiratory tract infections -- URIs) in glowing terms.

Since I'd been asked by a friend for my opinion, I decide to see what I could learn. I searched for clinical trials of the preparation for URIs and found a small randomized trial showing no apparent benefit of a beta-glucan product on URI in athletes.

Not surprisingly, this RCT had not been highlighted in the video. However, a number of other trials were mentioned and were linked on the product's website. Two of the trials listed were apparently only presented in meetings, one was reported as published and I was unable to find the full text (perhaps others will have more luck), but one trial was published in a form I could access.

On the product website, the accessible study was described as showing:

While there were no significant differences in the incidence of symptomatic respiratory infections among the study groups, the duration and severity of symptoms were alleviated in subjects receiving Wellmune WGP.

In the study results, the Wellmune WGP group reported:

• No missed work or school due to colds, compared with 1.38 days of work/school missed for the placebo group. (p = 0.026)

• No incidence of fever, compared with 3.50 incidence in the placebo group. (p = 0.042)

• An increase in quality of life, including physical energy and emotional well-being, as measured by a clinically validated health survey questionnaire (SF-36v-2). (p = 0.042)

• No adverse events were detected and no safety concerns were present.

This actually tracks fairly closely with what the published study states in its abstract. However, if you actually read the paper it becomes quickly apparent that an enormous number of comparisons were made, nearly all of which showed no benefit to beta-glucan (the missed work or school due to colds was one isolated positive finding). The reported effect on fever does not match the data in the text of the article and, even there, no effect on fever was seen in the intention to treat analysis or in the per protocol analysis, though the latter got close to achieving that all important p<0.05 though it didn't make it.

I communicated back to my friend my doubts that anything very interesting had been shown about the product and inquired about possible financial incentives influencing the interpretations of those who were reporting the results to her.

Why is any of this even worth thinking about? After all, the claims made in the video had raised my doubts as soon as I'd heard them, and there are many such claims for many products that are at least as dubious.

The reality, though, is that this beta-glucan product will be marketed and may gain traction. The friend who asked me about it actually currently sells an acai berry product in which she is a true believer. Acai berry products are being marketed all over the Web and doctors are now fielding questions from patients about their value. In my role as a medical editor, I was asked recently by a reader (a skeptical physician) to have our resource include an article discussing the evidence for and against clinical benefits from acai berry supplements.

If we wrote such an article about acai berry, what should it contain? There is no high quality evidence about acai berry and clinical outcomes. Should an article take an agnostic view about its benefits? Should it point out that lots of antioxidant supplements have failed to improve health or have even caused harm? Should it be written at all, or is there some threshold of believability or widespread use that must first be overcome.

The scientific medicine world generally claims it expects some reasonable evidence before administering a new preparation or trying a new procedure (though is not always as good in practice as in theory), and the pharmaceutical industry, for all its problems, can't release a drug without impressive clinical trials.

But when random claims of snake oil salesmen selling some new product spring up, eventually, if the product really does become widespread in the marketplace, the expectation of many in the media and the public seems to be that the burden has shifted to scientific medicine to disprove the benefit or else not take a position about whether a product is good or bad.

At the extreme, you have the most ludicrous ideas of alternative medicine like homeopathy. But even more plausible-sounding ideas when untested do not deserve much credence. That someone who is selling a product can weave a good story about why it is beneficial should not require the resources of the rest of us to test those claims.

We'd all be better off starting out as skeptics, particularly when those promoting an untested product have a financial stake in its success. Snake oil salesmen have been a bane of appropriate medical care for centuries, and there's no reason to think this will change. Those who care about science and evidence, though, must remember that we need not approach entirely untested claims from a position of neutrality. Most things you could imagine doing to improve health do not work, and many are harmful. Though absence of evidence is not evidence of absence, we need not spend time and thought disproving the claims of every snake oil salesman who comes along. It already takes enough resources testing just those claims that have some shred of plausibility as well as the products that have become so widespread that letting them stay untested endangers health.

Comments

[Stephen Ferrara, NP]

Wonderful synopsis of what appears to be the latest and greatest complimentary/alternative treatment for the common cold. Below is the link to the other published article that I was able to find.

http://www.jssm.org/vol8/n4/3/v8n4-3pdf.pdf

I realize that we simply cannot expect to have a systematic review for every treatment that is out there. Heck, as it is, the evidence is thin for many many widely accepted treatments today. I wonder though, can we simply dismiss these claims because there are bigger fish to fry? Our patients look to us as the authority on these treatments and we need to give them an evidence-based answer. I suppose what inevitably will happen (at least for now) is that as long there is no clear harm in the treatment, we will go on record as saying that as long as the perceived benefits outweigh the perceived risks, then its their dime to spend.

[Marilyn Mann]

You are right on!

[Don Cox, Ph.D.]

It was unfortunate that Dr. Rind did not have access to the body of research that supports the health benefits of Wellmune WGP®. This natural ingredient is clinically proven to support the immune system. Biothera, a biotechnology company dedicated to improving immune health, has conducted numerous safety (toxicology), preclinical and human clinical studies with Wellmune WGP. Wellmune WGP is a FDA-GRAS food ingredient that can be found in a growing number of foods, beverages and supplements. We are excited to be an ingredient in Mona Vie’s (M)mūn™, the company’s new addition to its line of acai berry juice–based nutritional and energy drink products.

To aid readers in considering the totality of data available on Wellmune WGP, several abstracts and papers of studies published in peer-reviewed scientific and medical journals can be found at http://www.wellmune.com/scientificliterature.html. These include:

1) Mechanism by Which Orally Administered ß-1,3-Glucans Enhance the Tumoricidal Activity of AntiTumor Monoclonal Antibodies in Murine Tumor Models. The Journal of Immunology. 2004: 173: 797-806. This article details the unique mechanism of action – how Wellmune WGP works in the body. The mechanism is applicable beyond the tumoricidal activity discussed in the paper and includes activity on any non-self cell with complement deposition on its surface.

2) Three peer-reviewed clinical trials with individuals experiencing high physical or lifestyle stress or exposure during cold/flu season (additional clinical publications are planned). These studies demonstrate that Wellmune WGP lowered the incidence of upper respiratory tract infection symptoms and improved the participants’ reported overall health and well-being. We use stress models in our clinical research because increased physical and psychological stress often lead to a higher incidence of upper respiratory tract infections. Using stress models allows us to evaluate the potential benefits of routine consumption of Wellmune WGP in the diet.

• Beta 1,3/1,6 Glucan Decreases Upper Respiratory Tract Infection Symptoms and Improves Psychological Well-being in Moderate to Highly-Stressed Subjects. Agro Food Industry Hi-Tech (2010) 21:21-24. (This paper was published after Dr. Rind wrote his review.)

• Effect of Beta 1, 3/1, 6 Glucan on Upper Respiratory Tract Infection Symptoms and Mood State in Marathon Athletes. Journal of Sports Science and Medicine (2009), 8:509-515.

• Randomized Phase II Clinical Trials of Wellmune WGP® for Immune Support During Cold and Flu Season. The Journal of Applied Research (2009), 9:20-42.

3) Wellmune WGP is GRAS (Generally Regarded As Safe) under regulations of the FDA. In addition, Wellmune WGP has received regulatory approval for use in foods, beverages and supplements in other countries worldwide. The toxicology paper that was part of the Wellmune GRAS dossier submitted to the FDA for review is listed below:

• Toxicological Assessment of a Particulate Yeast (1,3)-ß-D-Glucan in Rats. Food and Chemical Toxicology (2007), 45:1719-1730.

4) In addition, Biothera has completed human clinical biomarker studies that demonstrate Wellmune WGP increases the phagocytic index of innate immune cells, which measures the cells ability to “eat” or destroy foreign challenges. Other human biomarker studies show that Wellmune enhances immune function without increasing pro-inflammatory cytokines. These internal studies have not yet been published (publication is planned).

We encourage interested parties to review the available data on the application of Wellmune WGP to human health. We believe that when one considers the totality of the data available for Wellmune a conclusion that Wellmune WGP supports a healthy immune system is a credible finding. For the record, we do want to clearly state that Wellmune WGP is not a medicine, it is a food ingredient that may provide health benefits with routine consumption. Finally, we routinely encourage customers to seek advice from a medical professional if they have a medical condition and want to include Wellmune WGP in their diet.

Don Cox, Ph.D.
Vice President, Research & Development
Healthcare Group
Biothera

[sanjiva86]

Don Cox: Access to the full texts of the articles listed would be helpful. As Dr. Rind has pointed out, the abstract does not always tell the whole story.

The publication record itself does not also tell the full story, especially if negative trial results are kept in the file drawer. Does Biothera register all clinical trials that it undertakes with databases like clinicaltrials.gov?

[David Walsh]

Providing full text of articles may violate copyright issues. However, the complete articles are available through the publication. Clinicaltrials.gov is for Investigational New Drugs that have been approved for use in clinical trials by the FDA. As a functional ingredient (not a drug) for foods, beverages and supplements, Wellmune research would not be included on this website.

[Ted Smythe]

"The credulous, on the other hand, might believe that meetings like this are exactly where health breakthroughs are presented because the mainstream media and mainstream doctors would never allow such information to be presented in a medium that they control."

Really. Are there not controversies that are only worthy of thought by the credulous?

1. fluoride in water causes health damage
2. chlorine in water causes/contributes to heart disease
3. cholesterol does not cause heart disease
4. homocysteine is the best marker for CVD (folic acid reduces CVD)
5. Vitamin D reduces cancer
6. Vitamin D enhances immune system
7. Synthetic Vitamins not the same as natural (eg: D, K, E)
8. Sunshine (not sunburn) good for your health (via Vitamin D3 production)

Again, must I be credulous to believe the above is worthy of consideration?

Ted Smythe
5.

[Joseph]

See, there's the exploited flaw in our capitalistic society. 'Supplements' are unregulated, in class with ingredients in food... yet not intended to be ingested as such.