The forward momentum of medical progress is manifest, it could be argued, in the $50 billion spent in 2008 on pharmaceutical research and development in the quest to bring new drugs to market. But little scientific or governmental infrastructure exists to ensure that each new treatment is actually an improvement over existing therapies—and to tease out what therapies are best for which patients.

People facing tough medical decisions, such as cancer patients, often have to work with their doctors to decide what combination of surgery, chemotherapies, radiation, lifestyle changes or other treatments will likely be the most effective choice. The array of options can be dizzying and most patients, and even some doctors, are ill-prepared to do the background research to glean an answer. And in many cases there is often no data available that compares the effectiveness of various drugs and other treatment options.

Support for this sort of work, known as comparative effectiveness research (or CER, which is defined by the federal government as comparison of "the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions"), has begun to spread from sectors of the scientific world into the public and political realms—thanks in part to health care reform debates and $1.1 billion in funds from the 2009 American Recovery and Reinvestment Act (ARRA).

But a new review of recent literature reveals that medical research has shied away from direct comparisons of therapies. In fact, less than a third of recent studies published in the top six medical journals could be classified as CER, according to the new report, published in the March 10 edition of JAMA The Journal of the American Medical Association.

Comparative assessment might not be as sexy as discovering the next blockbuster drug—and is not likely to ever supplant development—but it is a crucial aspect of patient care. "As we develop more and more treatments, it's important that we know we can use them in an effective, rational way," says Michael Hochman of the Keck School of Medicine of the University of Southern California and lead author of the report. He found that many of the published studies failed to assess "what we considered to be fundamental therapeutic decisions."

The knowledge gap between new discoveries and best treatments "is frustrating for patients, who too often undergo trial and error medicine," Patrick Conway and Carolyn Clancy, both of the U.S. Department of Health and Human Services (HHS), wrote in an editorial accompanying the new study. "The United States leads the world in biomedical science," they noted, but it "now must implement a framework for research that advances patient-centered care"—care that is based not just on more options but a better understanding of these options, they asserted.

Comparing the comparative
When a new treatment hits the market, it is often accompanied by a wave of promotion (aimed toward doctors—and sometimes patients themselves) in addition to the trial results published before approval by the U.S. Food and Drug Administration (FDA). But what is often missing is formal, scientific research about how that new treatment compares with existing ones, the authors of the new report discovered.

After assessing studies published between June 2008 and September 2009 in the six most-cited medical journals (Annals of Internal Medicine, Archives of Internal Medicine, the British Medical Journal (BMJ), JAMA, The Lancet and The New England Journal of Medicine), Hochman and his co-author Danny McCormick (of the Cambridge Health Alliance and Harvard Medical School) found a paucity of solid comparative effectiveness studies.

For the sake of clarifying the often-murky field of CER, the two researchers focused only on research that evaluated medications. Of the 328 studies that purported to do this, 104 met the formal definition of CER from the HHS's Federal Coordinating Council for Comparative Effectiveness Research. Among the 104 studies, 43 percent compared two or more medications with each other, 16 percent examined different dosing approaches, and 11 percent compared medications with nonpharmacologic treatments.

"We found that only a minority of medication research focuses on helping doctors use them in an effective, rational way," Hochman says. "The rest of research seems to be devoted to finding novel therapies."

Two percent of the studies evaluated cost effectiveness and 19 percent focused on medication safety, which, notes Hochman, "probably isn't enough."

Hochman and McCormick also found that even those studies that qualified as CER had some "serious shortcomings," Hochman notes. Part of that might be the nebulous nature of CER itself. Although the term CER has been around for years, the Federal Coordinating Council only established the working definition in June 2009, when the papers reviewed for this new study had already been either published or submitted to journals. "It's actually relatively difficult to define studies that are CER—it's sort of a gray area," Conway says, noting that he still thought the new report was "a good overview and estimation of the CER literature."

6 Comments

1. 1. JamesDavis 12:07 PM 3/10/10

   And the tax payer, not the pill builder, is paying for this inadequate information and, most often, dangerous medicines and therapies. Just getting this CER system straightened out should save the tax payer and medical bill payer billions in wasted funds. That, in a couple of years, should cover the cost of the new revised medical bill before the Senate now. Who was it that screwed up the medical system so bad like this, and when did it become this screwed up???

   Reply | Report Abuse | Link to this

2. 2. Bill Case 12:38 PM 3/10/10

   CER should be the function of, and in the interest of, patients, doctors and payers. Payers would include Medicare, Medicaid AND Health Insurance Companies. There are several ways that Insurance Companies should be trying to control costs besides just removing at-risk patients from their roles. One major way, is checking the effectiveness of treatments they are being billed for on behalf of insured patients. When I read articles about how the insurance companies never question bills from hospitals and doctors that are less than $15,000, I see why American Health Care costs so much.

   Reply | Report Abuse | Link to this

3. 3. dtrach 06:10 PM 3/10/10

   The sickening thing is that much of the data needed for comparison is already held within the FDA, that refuses to release it to the Country.

   Reply | Report Abuse | Link to this

4. 4. abyssalmystery 10:16 PM 3/10/10

   Although I think capitalism is the best economic system there is, it fails in some ways. Healthcare is one of them. The development of new drugs and therapys is driven by profits not on what is most effective. There is an enormous amount of medicines that are prescibed that have questionable merit. Good for the drug companies, not so go for patients. Of course some medicine is very effective. We could probably replace 75% of it with diet and excercise.

   Reply | Report Abuse | Link to this

5. 5. jgrosay 04:37 AM 3/11/10

   Comparative drug studies are done everyday as part of the drug approval process, this is called Phase IIIclinical trials (Phase I is the first dosing in humans, formerly done in concentration camps inmates, then in imprisoned persons, and currently in paid volunteers). Once the drug is approved, and has found a niche in the market, pharmaceutical companies just stop investing money in that kind of studies, they are no longer necessary, and once the product is marketed, doctors themselves will find new applications for it, that sometimes are too expensive to formally develop and get HRA approval. The current situation in many medical fields is that there are so many new compounds and so many new indications, that it is often very hard finding patients adequate and willing to join clinical trials

   Reply | Report Abuse | Link to this

6. 6. wilberforce 06:44 AM 3/11/10

   How can one write on this subject with no mention of the Cochrane Collaboration?

   Reply | Report Abuse | Link to this