« Meta-analysis and New Knowledge | Main | Composite Endpoints and the CREST Trial »

Apr 03, 2010


TrackBack URL for this entry:

Listed below are links to weblogs that reference Rooting for Results:


Feed You can follow this conversation by subscribing to the comment feed for this post.

Too many strikes, in my book:

1. Only one of many subgroups
2. Post hoc analysis
3. Additional uncertainty/confounders regarding dropouts, medication compliance

I'm not sure how we can have any real confidence that this finding is real, in light of how hard one has to dig to find it- even if other studies have suggested a similar finding. Despite the temptations of multiple comparisons/data mining, its seems to me that EBM is not suited to find needles in haystacks, but it is only powered to prove (or more often disprove) hard and concise primary outcomes. Are these subgroup findings ever confirmed in a dedicated RCT addressing these questions?

The tendency to celebrate the negative data of researchers with whom you disagree:


Rightly avoided, but hard to resist.

More seriously, a question: even if you prespecify multiple subgroup analyses, after a certain number of slices and dices, shouldn't you undergo some statistical penalty? If you only have to undergo a penalty for post-hoc analyses, that suggests there is something more statistically pure about having announced your intention to mine the data beforehand. That seems dubious.

Dr. Wright's point is correct -- even a prior hypothesis counts towards issues of multiple testing when looking at subgroups. Still, a prior hypothesis is more believable than one prompted by the data analysis.

There's no perfect solution to this, since sometimes the subgroup is where the truth is. Some would argue that you don't get to look at subgroups if the primary endpoint analysis is negative, but this is rarely if ever honored.

In the Users Guide to the Medical Literature http://www.amazon.com/Users-Guides-Medical-Literature-Evidence-Based/dp/0071590382/ the authors propose a list of things to consider in deciding whether to believe a subgroup analysis including:

Was it a prior hypothesis?
Was it one of a small number of effects tested?
Was it found within rather than between studies?
Was the effect large?
Is the effect consistent across studies?
Was the subgroup effect statistically significant? (Not "was the treatment effect significant in the subgroup?", but that the effects are different from each other in the [usually two] subgroups.)
Is there external evidence supporting the plausibility of the subgroup effect?

I've heard Gordon Guyatt, one of the authors of the Users Guide, give a shorthand approach if you want to skip using the checklist: when in doubt, don't believe the subgroup analysis.

Great post, thanks. I share your scepticism about lipid-lowering treatments other than statins. I liked your post for this reason but also for your honesty about your intellectual biases. We are all afflicted by these biases, but few admit them with such candour.

Conflict of interest is a major concern when deciding whether the very ethos of a particular paper or publication is tainted by vested interests. Particularly when it involves sensitive data that influences the decisions of thousands of doctors around the world, we ought to be strict with deciding if it is influenced by any particular company. This is a much needed decision among the medical fraternity.

Have you seen the recently published meta-analysis in the Lancet which looks at the effect of fibrates on cardiovascular endpoints? While showing an overall reduction in cardiovascular events, they do acknowledge that the size of this effect is modest (10%, p=0.048, 95% CI 0-18) compared to statins and antihypertensives. They also point out that of the 18 trials they included, only the ACCORD trial was studying the strategy of adding a fibrate in patients already on a statin, and that this did not show a statistically significant benefit. They continue: "... the ACCORD trial had insufficient statistical power to detect an effect of the size that we noted. With no evidence that the ACCORD results differ from those of the other fibrate trials, there is no strong rationale for expecting that fibrates might not provide some benefit."

I am not convinced. I consider the addition of fibrate to statins as unproven, and I will continue to use statins first line in patients at a sufficient level of cardiovascular risk that lipid intervention seems warranted. I will consider fibrates in patients intolerant of statins (certainly this review shows they have better evidence than we yet have for ezetimibe). Of the two fibrates available in my country, I will favour gemfibrozil over fenofibrate as, although this review did not find significant heterogeneity, the large gemfibrozil studies (Helsinki, VA-HIT) had larger effect sizes than the fenofibrate studies (FIELD, ACCORD), and the gemfibrozil studies had statistically significant results, while the fenofibrate studies did not.

I agree with the concern that the meta-analysis should not in any way add to our belief in combining fibrates with statins. There was little to support this idea previously, and ACCORD-lipid was not convincing that there was any clinically important benefit to the combination.

I have a different take on the fibrate data overall. Both this meta-analysis, and a prior one from 2007 (PMID: 17967602) found no evidence of an overall benefit on mortality, no reduction in cardiovascular mortality, and an increase in noncardiovascular mortality, such that there was either no effect on, or a trend toward an increase in, total mortality.

I see little point in taking a drug from a class that appears to increase noncardiovascular mortality while not reducing cardiovascular mortality, even if it may somewhat reduce cardiovascular events.

Those are very fair points, and thanks for pointing me to the 2007 meta-analysis. I am not quite as concerned as you are about the mortality statistics. This is because the authors of the 2007 paper found that the significant increase in non-cardiovascular mortality with fibrates seemed to become non-significant once they excluded the clofibrate trials.

But I agree that overall these are not impressive results, and certainly not as good as statins. If I thought a patient was possibly a candidate for a fibrate (as discussed above) I would aim to discuss these uncertainties at a level of detail that my patient could understand, and reach a shared decision on whether to commence one. I suspect many patients would choose not to. In fact I'm pretty sure that none of my regular patients are on a fibrate!

The comments to this entry are closed.