« Video now available | Main | Rooting for Results »

Mar 07, 2010


TrackBack URL for this entry:

Listed below are links to weblogs that reference Meta-analysis and New Knowledge:


Feed You can follow this conversation by subscribing to the comment feed for this post.

It would be interesting to compare the methodology and results of the Lancet meta-analysis with another meta-analysis that was done recently on the same question.

Current Medical Research and Opinion, 2008, Vol. 24, No. 5, Pages 1359-1362
Craig I. Coleman, Kurt Reinhart, Jeffrey Kluger and C. Michael White,

The effect of statins on the development of new-onset type 2 diabetes: a meta-analysis of randomized controlled trials


Objective: To determine the ability of statins to prevent the development of new-onset type 2 diabetes mellitus through a meta-analysis of randomized, controlled trials.

Research design and methods: A systematic literature search through November 6, 2007 was conducted to identify randomized, placebo-controlled trials of statins that reported data on the incidence of new-onset diabetes mellitus. Incidence of new-onset type 2 diabetes mellitus was treated as a dichotomous variable. Weighted averages were reported as relative risk (RR) with associ­ated 95% confidence intervals (CI). A random-effects model was used.

Results: Five prospective, randomized controlled trials (n=39791) were identified. Upon meta-analysis, the use of a statin did not significantly alter a patient's risk of developing new-onset type 2 diabetes mellitus (relative risk, 1.03; 95% confidence interval 0.89–1.19). Subgroup and sensitivity analyses did not significantly change the results. There was statistical heterogeneity that stemmed from pravastatin's tendency towards a reduction in risk and the other statins showing an increase in risk. The funnel plot could not rule out publication bias.

Conclusions: Statins, as a class, do not demonstrate a statistically significant positive or negative impact on a patient's risk of developing new-onset type 2 diabetes mellitus.

I say it would be interesting to compare the two meta-analyses, but I wouldn't attempt to do so myself because I don't have the necessary expertise (nor do I have free access to the article). To note a couple of obvious points:

The first meta-analysis was done before the publication of the JUPITER trial

It looks like they did not correct for the weird definition of diabetes in the pravastatin trial

Even with those factors, the first meta-analysis found a small insignificant increased risk (3 percent), CI 0.89 to 1.19. The 9 percent increased risk found in the Lancet meta-analysis is well within the CI for the first meta-analysis.

Also, I suspect that when they refer to publication bias in the abstract, they really mean reporting bias.

The main difference is that the lancet review is an individual patient data meta-analysis (meaning they had access to data on each patient) and the current medical review is a clinical trial level meta-analysis (meaning they only had access to published results). Therefore, the lancet review is far more exhaustive and allows for much tighter subgroup and sensitivity analyses.

Having said this, the meta-analysis:to:randomized trial ratio in the published literature is far too high - we need more RCTs, not just more meta-analyses.

When journals first appeared online, we heard a lot about the revolution in medical publishing that this change signified. Sure, it means that now, instead of piling up hard copies of unread journals on your desk, you can download only those articles you will actually read. Also, it means that instead of trudging off through the snow to the library to find a journal article, you can stay home and read it in your stocking feet.

But if there were actually to be a revolution in publishing, it would mean that published studies would have online supplements consisting of data files with the individual subject data, so that the reader could check the analyses and run others of interest. For example, if the published articles compared change scores between groups using a t-test, the reader could run an analysis of covariance on the final scores using the baseline score as a covariate and the treatment group as the main effect. In studies using linear regression to analyze the data, the reader could at least run a scatterplot to see if the linearity assumption is halfway reasonable. And the reader could look for homogeneity or the lack thereof between different studies without having to wait for the meta-analyses to be published.

Is there any support for this kind of revolution to be implemented? The ASCII files for an entire journal issue would be no larger than the average video clip of your grandson’s kindergarten graduation. And it would amount to a true revolution in publishing, one that would (pardon the expression) empower the average consumer of the literature.

@Ed Whitney: Wow. Your comment just made my day. thanks

The comments to this entry are closed.