The forward momentum of medical progress is manifest, it could be argued, in the $50 billion spent in 2008 on pharmaceutical research and development in the quest to bring new drugs to market. But little scientific or governmental infrastructure exists to ensure that each new treatment is actually an improvement over existing therapies—and to tease out what therapies are best for which patients.

People facing tough medical decisions, such as cancer patients, often have to work with their doctors to decide what combination of surgery, chemotherapies, radiation, lifestyle changes or other treatments will likely be the most effective choice. The array of options can be dizzying and most patients, and even some doctors, are ill-prepared to do the background research to glean an answer. And in many cases there is often no data available that compares the effectiveness of various drugs and other treatment options.

Support for this sort of work, known as comparative effectiveness research (or CER, which is defined by the federal government as comparison of "the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions"), has begun to spread from sectors of the scientific world into the public and political realms—thanks in part to health care reform debates and $1.1 billion in funds from the 2009 American Recovery and Reinvestment Act (ARRA).

But a new review of recent literature reveals that medical research has shied away from direct comparisons of therapies. In fact, less than a third of recent studies published in the top six medical journals could be classified as CER, according to the new report, published in the March 10 edition of JAMA The Journal of the American Medical Association.

Comparative assessment might not be as sexy as discovering the next blockbuster drug—and is not likely to ever supplant development—but it is a crucial aspect of patient care. "As we develop more and more treatments, it's important that we know we can use them in an effective, rational way," says Michael Hochman of the Keck School of Medicine of the University of Southern California and lead author of the report. He found that many of the published studies failed to assess "what we considered to be fundamental therapeutic decisions."

The knowledge gap between new discoveries and best treatments "is frustrating for patients, who too often undergo trial and error medicine," Patrick Conway and Carolyn Clancy, both of the U.S. Department of Health and Human Services (HHS), wrote in an editorial accompanying the new study. "The United States leads the world in biomedical science," they noted, but it "now must implement a framework for research that advances patient-centered care"—care that is based not just on more options but a better understanding of these options, they asserted.

Comparing the comparative
When a new treatment hits the market, it is often accompanied by a wave of promotion (aimed toward doctors—and sometimes patients themselves) in addition to the trial results published before approval by the U.S. Food and Drug Administration (FDA). But what is often missing is formal, scientific research about how that new treatment compares with existing ones, the authors of the new report discovered.

After assessing studies published between June 2008 and September 2009 in the six most-cited medical journals (Annals of Internal Medicine, Archives of Internal Medicine, the British Medical Journal (BMJ), JAMA, The Lancet and The New England Journal of Medicine), Hochman and his co-author Danny McCormick (of the Cambridge Health Alliance and Harvard Medical School) found a paucity of solid comparative effectiveness studies.

For the sake of clarifying the often-murky field of CER, the two researchers focused only on research that evaluated medications. Of the 328 studies that purported to do this, 104 met the formal definition of CER from the HHS's Federal Coordinating Council for Comparative Effectiveness Research. Among the 104 studies, 43 percent compared two or more medications with each other, 16 percent examined different dosing approaches, and 11 percent compared medications with nonpharmacologic treatments.

"We found that only a minority of medication research focuses on helping doctors use them in an effective, rational way," Hochman says. "The rest of research seems to be devoted to finding novel therapies."

Two percent of the studies evaluated cost effectiveness and 19 percent focused on medication safety, which, notes Hochman, "probably isn't enough."

Hochman and McCormick also found that even those studies that qualified as CER had some "serious shortcomings," Hochman notes. Part of that might be the nebulous nature of CER itself. Although the term CER has been around for years, the Federal Coordinating Council only established the working definition in June 2009, when the papers reviewed for this new study had already been either published or submitted to journals. "It's actually relatively difficult to define studies that are CER—it's sort of a gray area," Conway says, noting that he still thought the new report was "a good overview and estimation of the CER literature."

Translational medicine
More research is not the only way to increase the sway of science in evaluating medical treatments. "It's not just about doing the studies, but making sure that the studies lead to changes in doctors' practices," Hochman says. He argues that government CER funds should also be used for research that examines the best ways to translate comparative effectiveness findings into clinical practice.

Expanding CER and the adoption of its conclusions can also be improved "from a workforce training perspective," Conway says. He notes that if more researchers and doctors are trained with this aspect of medicine in mind, the patients will ultimately be better off.

Not everyone is on board with boosting CER, however. "People who are concerned about health care rationing have opposed some of the comparative effectiveness funding," Hochman points out. But the researchers and others close to the data say that this is not the goal of the field. "The goal of CER is to inform clinicians and patients," Conway notes, but often, "the message gets lost."

Highlighting some of the personalized promises of CER is another way to increase its broader appeal. Comparative effectiveness studies can not only help doctors and patients know which therapy (prescription or lifestyle changes) is most effective overall, but also, Conway says, eventually what combination of treatments are best for a particular individual. "I think CER is very in line with personalized medicine. If done well, it should actually identify which patient would benefit from which therapy," he notes.

Effecting change
Most of the comparative effectiveness research (87 percent) that the JAMA report assessed had been backed by noncommercial funding. The field currently holds little appeal for companies, which rely largely on new products to boost their earnings, and evaluating a new drug against existing ones—or nonpharmacological interventions—is risky, as a negative result would hardly be good for sales. Additionally, the researchers noted, comparative studies funded by private companies are not always as rigorous as those supported by the government or nonprofits. Pharmaceutically funded trials often test a treatment against an inactive placebo, whereas noncommercial studies are more likely to use an active, competing treatment for comparison. The authors posited that this situation that might help to explain why industry-funded research has historically been more likely to return positive results of a test drug.

But with the bulk of CER thus falling to the government and nonprofits, funding will likely continue to be a major issue.

The $1.1 billion from ARRA has been a big boost to the field. The HHR's Agency for Healthcare Research and Quality (AHRQ) received a budget increase for 2011 about 11 times greater (to $286 million) from the previous year. "I definitely think this is a good thing," Hochman says. "But it may quickly become apparent that this is not nearly enough."

Down the road, if medical cost effectiveness becomes as big a factor as many health care reform proponents have argued it should be, CER could even become a profitable line of research, the HHS's Conway notes. "The goal is to eventually pay for the value of services," he says. And if that goal comes to pass, "there should be incentives in the private sector, as well: to demonstrate that your product has a better value."

Some in this research field have proposed integrating CER into the FDA approval process—and requiring labeling of an approved product's proven effectiveness. Such a change would "improve clinicians' and the public's understanding of the role of new treatments, reduce the amount of taxpayer-funded comparative effectiveness research needed, and may even reduce health care costs," Alec O'Connor of the University of Rochester School of Medicine and Dentistry wrote in a commentary in the same issue of JAMA.

Others are also optimistic about the increasing attention to CER. "This is an opportunity to reexamine and reevaluate how we approach patient-centered research," Jean Slutsky and Carolyn Clancy, both of AHRQ, wrote in a March 8 commentary in Archives of Internal Medicine. The task might seem daunting, they noted, but "the payoff will be fantastic."

Any changes, however, are not likely to come quickly. With medical research in particular, years often intercede between study design and published findings, so even with the recent government funding increases the general public is not likely to see a flood of concrete CER findings for some time.