Center for Medical Consumers

Working to help you make informed decisions

  • Categories

Posts Tagged ‘misleading use of statistics’

What MDs don’t know about cancer screening

Posted by medconsumers on March 8, 2012

Most primary care physicians are keen on cancer screening. In fact, sending symptom-free patients for regular tests is central to their practice. Yet an understanding of cancer screening statistics is critical to informed decision-making, whether you’re the doctor sending people for tests or a patient just following orders. A new survey of U.S. primary care physicians shows the majority accept misleading statistics as proof that screening works.

Four hundred and twelve physicians took the online survey, which was designed by an American and German research team with a history of trying to improve understanding of health statistics by health professionals as well as the general public. “Most physicians incorrectly equated improved survival and early detection as evidence of lives saved,” concluded the researchers led by Odette Wegwarth, PhD, Max Planck Institute for Human Development, Berlin, Germany. “Few correctly recognized that only reduced mortality in a randomized trial constitutes evidence of the benefit of screening.” The survey results were published this week in Annals of Internal Medicine.

The survey presented physicians with two ways of expressing the effect of a hypothetical screening test which was described as improved 5-year survival and increased early detection in one scenario and as decreased cancer mortality and increased incidence in the other. Though the type of cancer was not identified, the hypothetical test scenarios were based on real-life data from the European prostate-cancer screening randomized trial. And the 5-year survival statistics and the percentage of stage I prostate cancers came from the U.S. database of cancer statistics collected in 1975. To be safe, that year was chosen because it predates the introduction of any organized screening program for prostate cancer.

The physicians were more impressed with what the survey authors called “irrelevant evidence,” for example, a test with a large 5-year survival rate. Here’s why this is irrelevant: The older we get, the more cancers we have in our bodies; many will never become life-threatening. Prostate cancer, for example, is in the overwhelming majority of cases a slow-growing or non-progressive cancer.

Therefore, prostate cancer’s 5-year survival rate will look like a clear justification for early cancer detection because most men will die of something else. Conversely, they  could die six years after a diagnosis of prostate cancer and still be counted as a “survivor”. Furthermore, screening often moves up the time of diagnosis (and treatment) without moving back the time of death.  (By the way, we can thank the American Cancer Society for its long-time use of this extremely misleading measurement of a cancer screening test’s benefit. In the not-so-distant past, the ACS actually used the word ‘cure’ interchangeably with 5-year survival, thus making generations of cancer patients think that making it to five years meant something.)

Now for the other worrisome finding:  The surveyed physicians were less impressed with a test described as having “reduced mortality”. And they were more impressed with a test that finds lots of cancer. But screening for cancer will always increase the number of cancer cases diagnosed, compared with the number of cancers found in people who seek medical attention only after symptoms appear. That’s because screening detects many more cancers that do not progress, which falsely inflates the apparent benefit of a screening tests (a phenomenon that the survey authors describe as overdiagnosis). This is why careful researchers will—after many years of follow-up—-compare the overall death rate of both the screened and unscreened groups. It is the only way to sort out the people who actually achieved a life-saving benefit from those who were treated unnecessarily for a cancer they didn’t need to know about.

This comparison is also a way  for researchers to determine screening’s “cost” in terms of harms. Here’s what the European prostate cancer screening trial found:  For every one prostate cancer death avoided in the PSA screened men, 48 men suffered severe complications from unnecessary treatment of a non-progressive cancer.

What to do

If you are going for any cancer screening test, inform yourself first at the National Cancer Institute’s website.  And be sure to use the “health professional” version which is more honest and in-depth than the patient version. If you get most of your medical information from the media, plan on regular visits to this media fact-checking website ( See its recent excellent critique of the media’s take on the latest colon cancer screening research, particularly The New York Times’ erroneous portrayal of it as definitive proof for colonoscopy as the best screening method. Click here

Maryann Napoli, Center for Medical Consumers©

Related posts:
Most drug don’t work (This is about understanding drug trial statistics.)
PSA screening for prostate cancer
Cancers that do not kill
Reduce your risk of breast cancer: Avoid mammograms (unless you have a breast symptom)

Posted in Cancer, colon cancer, Doctors, Screening, testing | Tagged: , , , , , , , , , | 1 Comment »

Most drugs don’t work…

Posted by medconsumers on May 15, 2011

“Most drugs don’t work on most patients.” When I first read this statement many years ago in the British Medical Journal, a light bulb went off. I’d been a medical writer for a long time and knew it to be true. Yet I had never seen prescription drug effectiveness summed up so accurately. In a medical journal, no less! Unfortunately, drug study results are routinely reported in statistical terms known to make the drug’s benefit appear larger than it really is. More on that later.

The trigger for my light bulb moment was a 2003 editorial entitled, “Drug Don’t Work” by Richard Smith, MD, who explained, “This is of course no news to doctors. Anybody familiar with the notion of ‘number needed to treat’ (NNT) knows that it’s usually necessary to treat many patients in order for one to benefit. NNTs under 5 are unusual, whereas NNTs over 20 are common.”

The latest research, however, indicates that Dr. Smith may have overestimated physician understanding of medical statistics. A new Cochrane review of all relevant studies found that the most common statistical terms for expressing drug study results—in medical journals and the media—are misunderstood by doctors and consumers alike. In fact there was no difference in understanding between physicans and the lay public.

For the Cochrane review entitled, “Using different statistical formats for presenting health information,” the authors identified the 35 best studies designed to evaluate how well people understood the different ways of expressing the same study results. This review came down in favor of absolute risk reduction rather than relative risk reductions (RRR), as described below:

“You read that a study found that an osteoporosis drug cuts the risk of having a hip fracture in the next three years by 50%. Specifically, 10% of the untreated people had a hip fracture at three years, compared with 5% of the people who took the osteoporosis drug every day for three years. Thus 5% (10% minus 5%) less people would suffer a hip fracture if they take the drug for 3 years. In other words, 20 patients need to take the osteoporosis drug over 3 years for an additional patient to avoid a hip fracture. ‘Cuts the risk of fracture by 50%’ represents a relative risk reduction. ‘Five per cent less would suffer a fracture’ represents an absolute risk reduction. ‘Twenty patients need to take the osteoporosis drug over 3 years for an additional patient to avoid a hip fracture’ represents a number needed to treat.”

To people without statistical training—and that includes most physicians—the “cuts your risk by 50%” RRR example appears more impressive. And too often this is the sole way drug study results are described in medical journals, as well as ads aimed at physicians and the general public. Rarely is it explained that the 50% simply represents the difference between the treated and the untreated.

Here’s a real world example: Your doctor says, “You have mild hypertension and should go on a blood pressure lowering drug for the rest of your life because it will cut your risk of stroke by 50%.” Sounds good, maybe even worth the risk of sexual dysfunction and depression—two common side effects of anti-hypertensives. But here’s the other side of the story: A person with mild hypertension has only an infinitessimal chance of having a stroke, so the drug will reduce that infinitessimal risk by 50%

The common usage of RRR comes from the biostatisticians and the methodologists whose job it is to determine whether studies are conducted rigorously and results reported accurately. The fact that the Cochrane Collaboration’s own biostatisticians and methodologists are concerned enough to have produced this new review signals some recognition that it’s time for a change. (Long overdue, given that the sole use of RRR serves the pharmaceutical industry’s interest.) The Cochrane Collaboration itself has produced too many drug reviews with results expressed solely as RRR. Here’s one improvement under consideration because it is better understood: 100 of 1,000 untreated people will have a hip fracture in the next three years; 50 of 1,000 people taking an osteoporosis drug will have a hip fracture in the next three years.

The most easily understood way of expressing “most drugs don’t work on most people” can be found at this website: Say, you are suffering from acute sinusitis but wary of taking an antibiotic for this distressing condition. You want to know how well the drug is likely to work and the likelihood of harm. shows that 93% will not benefit from antibiotics; nearly 7% saw a faster resolution of their symptoms; and just over 1% were harmed by the antibiotics (e.g., diarrhea, vomiting). Even more interesting, four out of five people (in the placebo group) got better without antibiotics.

Maryann Napoli, Center for Medical Consumers(c)

Posted in Drug ads, Drugs, hypertension | Tagged: , | 6 Comments »