USPSTF, Mammography, and Grading Recommendations

It's hard to have a blog about evidence and medicine without commenting on the changes to the USPSTF's breast cancer screening guidelines.

At a first pass, it's worth pointing out that a lot of academic primary care physicians have been wondering what the USPSTF was thinking when they published their 2002 guidelines recommending mammography for women in their 40s. Yes there may be some small benefits (and lots of harms), but this is just the sort of thing that the USPSTF can usually be relied on to not recommend -- look for instance at what they recommend for prostate cancer screening or lipid screening for women. As a result, there were lots of insinuations that the USPSTF had caved to political pressure in making the mammography recommendation. Perhaps others know more about that one way or the other.

Now they've backed away from that recommendation, going from a favorable to a negative recommendation for mammograms. The overall issues aren't actually that complex: if you imagine 1000 women in their 40's getting mammograms yearly for 10 years, the best estimates are that you will cause more than half of them to need repeat mammograms for concerning findings, and will cause nearly 200 to get breast biopsies, while only preventing 2 deaths from breast cancer.

What is remarkable is the firestorm that has been set off. When I present these data to women patients in their 40s (in a way that I think is fair to both sides of the argument), I've found that approximately half want mammography and half want to wait until they turn 50. Individual patient values and preferences heavily influence the decision. The USPSTF takes this into account by recommending against routine mammograms in the 40s, but states that a decision to screen should take consider an individual woman's values. All in all, not that unreasonable a position as far as I'm concerned.

I think, though, that the USPSTF's way of grading recommendations is awkward in general and may even have contributed to the confusion over their 2002 and 2009 recommendations. The evidence base didn't substantially change over that time frame, but the recommendations did, and that opens them to attack.

I think the USPSTF would be better off using a different grading system, such as GRADE, which they explicitly chose not to do recently.

(Here I need to make one of the only conflict of interest disclosures I should ever need to mention on this blog: one part of my work is as a medical editor, and in our publication we use GRADE.)

That out of the way, the USPSTF grading scheme is unusual in the way that it combines strength of recommendation and quality of evidence into a single value (a letter grade). To me, this overloading causes problems. For many of the recommendations that USPSTF gives a grade of I to (insufficient evidence), GRADE and a number of other schemes would note the low or moderate quality of the evidence and still allow a guideline committee or other recommending body to make a suggestion about what a clinician should do.

When there is enough evidence to make a suggestion (a "weak recommendation" in GRADE parlance), GRADE explicitly deals with the need for considering patient values and preferences. If, instead of being used to this single letter grade from USPSTF, we were used to seeing evidence and recommendation divided, I'm not sure the recommendation change would have caused quite so much trouble.

The USPSTF recommendation, cast into GRADE style, would likely have read:

For most women ages 40 to 50, we suggest not performing yearly mammography (Grade 2B). Women who are willing to accept a high risk of false positive mammograms and a high risk of unnecessary biopsies in return for a small mortality benefit may reasonably prefer annual mammography starting at age 40.

Would a recommendation written this way have caused the same firestorm? I'd be interested in what others think about this. Would it have caused a firestorm if the 2002 recommendation had said the opposite?:

For most women ages 40 to 50, we suggest performing yearly mammography (Grade 2B). Women who are unwilling to accept a high risk of false positive mammograms and a high risk of unnecessary biopsies in return for a small mortality benefit may reasonably prefer annual mammography starting at age 50.

You can see how easy it is to split the unchanged evidence base from the change in what you are suggesting for the majority of women (based on a guess about their values and preferences) when using GRADE. I think such an approach is more transparent than that used by the USPSTF, and keeps the focus where it belongs in a situation like this -- on the benefits, risks, and burdens that influence the decisions for individual patients and for society.

I'll write more about the methods of GRADE, including its strengths and weaknesses, down the road. I do want to say that while I don't like the grading scheme used by USPSTF, I think they mostly do a wonderful job of staying true to the evidence as they present their guidelines. We should all hesitate to recommend screening tests in the absence of solid evidence showing benefit, and the USPSTF can almost always be counted on in this regard.

Comments

[Paul Levy]

Great work and great blog, David. I have linked to it and written about it here: http://runningahospital.blogspot.com/2009/11/evidence-in-medicine.html

[bev M.D.]

Excellent explanation; I found this blog through Paul's link. Don't you think a lot of this firestorm may be from people's fear of how insurance companies may use this information? I don't think insurance companies care a whit about patient's preferences.....

[David Rind]

Bev, I agree that at least some of the fear is about what insurance companies will do, and that the insurance companies don't think in terms of patient preferences. At the same time, I think insurers will keep paying for screening mammograms (and screening PSAs) because they would otherwise lose lots of customers.

It's actually the government that has been slowest to pay for screening procedures/tests through Medicare (even the ones that are clearly indicated), though at this point Medicare pays for most reasonable screening.

[Paul Levy]

A comment on my Facebook posting, David, from someone named Gulay:

"The USPSTF takes this into account by recommending against routine mammograms in the 40s, but states that a decision to screen should take consider an individual woman's values. All in all, not that unreasonable a position as far as I'm concerned."
Ok I can agree with this one.

But I definitely have problems with this one:
"The USPSTF recommends against clinicians teaching women how to perform breast self-examination. (Grade D recommendation)".

And other from Sharon:

Thanks so much for the link. It is refreshing - and helpful! - to read a rational analysis from an informed expert.

[Nicholas Fogelson, MD]

Thanks for the post! I think you have made a good summary of the issue, and really hit the nail on the head with your re-wordings of the USPTF recommendations. Thanks for your post! I am linking to it in my blog, as I think it is solid!

Nicholas Fogelson, MD

[Michael Kirsch, M.D.]

You write about the evidence behind the USPSTFs revised guidelines. Fair minds can hash this out. I was reviled by the suffusion of politics over the issue. This is a glimpse of the obstacles that comparative effectiveness research will face, and they may be insurmountable. While the USPSTF may be imperfect, they're a lot more objective than those who have attacked them. Many of these groups have transparent conflicts of interest on the issue. More thoughts at http://bit.ly/8R0wAx Best of luck on the blog.

[james ruiz, md]

The USPSTF states "given that the age groups (40 to 49 years and 50 to 59 years) are adjacent, the Task Force elected to emphasize the mortality outcomes." However, the decrease in mortality cited by the task force is nearly identical (15% and 16%) for these two age groups. Compounding their error, there is no “statistics of adjacency” and the USPSTF fails to describe how "adjacency" of the two age groups leads one to conclude that mortality outcomes is a better measure than life-years gained.
A primary source motivating the USPSTF to alter guidelines states that "conclusions about the optimal starting ages for screening depend more on the measure chosen for evaluating outcomes. If program benefits are measured in life-years, the measure most commonly used in cost- effectiveness analysis, then our results suggest that initiating screening at age 40 years saves more life-years... Starting annual screening at age 40 years (vs. 50 years) and continuing annually to age 69 years yields a median of 33 life-years gained per 1000 women screened. If the goal of a screening program is to efficiently maximize the number of life-years gained, then the preferred strategy would be to screen biennially starting at age 40 years (1)."
Nelson's review, upon which the USPSTF relied to change their recommendations, states that "The ages at which this tradeoff becomes acceptable to individuals and society are not clearly resolved by the available evidence (2)." However, the USPSTF chooses to emphasize the adverse consequences for most women and classifies harms for all age groups as "moderate."
The task force identifies false positives as the major harm in the younger age group. The task force indicates that it requires 565 more mammograms to save a life in the 40-49 year age group as compared to the 50-59 year group. Schell, et al, in evaluating the recall rate for 545,505 examinations in women aged 40-49, found that the rate is on the order of 9% which would lead to a "harm" of approximately 51 instances of additional views and/or ultrasound, 5 percutaneous biopsies under local anesthesia and the discovery of one cancer (at a rate of 3 cancers/1000 mammograms)(3). In order to conclude that the harm overshadows the life saved, the task force must find that the value of a human life is something less than the combined negative value of 616 imaging exams, 5 biopsies under local anesthesia, and the discovery of one new cancer.
In the newly passed Senate health reform bill, the USPSTF has been given a prominent role in determining what kind of care will and will not be allowed in a health care system. Under the new administration, this first effort by the USPSTF does not inspire confidence.
References
1. Mandelblatt JS, Cronin KA, Bailey S, Berry DA, de Koning HJ, Draisma G, et al. Effects of mammography screening under different screening schedules: model estimates of potential benefits and harms. Ann Intern Med. 2009;151:738-47.
2. Nelson HD, Tyne K, Naik A, Bougatsos C, Chan BK, Humphrey L. Screening for breast cancer: an update for the U.S. Preventive Services Task Force. Ann Intern Med. 2009;151:727-37.
3. M. J. Schell, B. C. Yankaskas, R. Ballard-Barbash, B. F. Qaqish, W. E. Barlow, R. D. Rosenberg, and R. Smith-Bindman Evidence-based Target Recall Rates for Screening Mammography Radiology, June 1, 2007; 243(3): 681 - 689.

[Cheap Computers Canada]

The USPSTF takes this into account by recommending against routine mammograms in the 40s, but states that a decision to screen should take consider an individual woman's values. All in all, not that unreasonable a position as far as I'm concerned.

[deceptions7]

The USPSTF may be imperfect, they're a lot more objective than those who have attacked them. Many of these groups have transparent conflicts of interest on the issue. More thoughts at rheumatic fever Best of luck on the blog.

[Breast cancer brachytherapy]

Given the economic reasoning and health care recommendation it is clear that more attention should be paid to further genetic research in order to limit the amount of required mammography and biopsies.