That original researcher could not foresee the specific harm he contributed to, and it’s hard to blame him as an individual, because he operated in an academic culture where leaving data unpublished was regarded as completely normal. The same culture exists today. The final report on TGN1412 concluded that sharing the results of all first-in-man studies was essential: they should be published, every last one, as a matter of routine. But phase 1 trial results weren’t published then, and they’re still not published now. In 2009, for the first time, a study was published looking specifically at how many of these first-in-man trials get published, and how many remain hidden. They took all such trials approved by one ethics committee over a year. After four years, nine out of ten remained unpublished; after eight years, four out of five were still unpublished.
In medicine, as we shall see time and again, research is not abstract: it relates directly to life, death, suffering and pain. With every one of these unpublished studies we are potentially exposed, quite unnecessarily, to another TGN1412. Even a huge international news story, with horrific images of young men brandishing blackened feet and hands from hospital beds, wasn’t enough to get movement, because the issue of missing data is too complicated to fit in one sentence.
When we don’t share the results of basic research, such as a small first-in-man study, we expose people to unnecessary risks in the future. Was this an extreme case? Is the problem limited to early, experimental, new drugs, in small groups of trial participants?
In the 1980s, U.S. doctors began giving anti-arrhythmic drugs to all patients who’d had a heart attack. This practice made perfect sense on paper: we knew that anti-arrhythmic drugs helped prevent abnormal heart rhythms; we also knew that people who’ve had a heart attack are quite likely to have abnormal heart rhythms; we also knew that often these went unnoticed, undiagnosed and untreated. Giving anti-arrhythmic drugs to everyone who’d had a heart attack was a simple, sensible preventive measure.
Unfortunately, it turned out that we were wrong. This prescribing practice, with the best of intentions, on the best of principles, actually killed people. And because heart attacks are very common, it killed them in very large numbers: well over 100,000 people died unnecessarily before it was realized that the fine balance between benefit and risk was completely different for patients without a proven abnormal heart rhythm.
Could anyone have predicted this? Sadly, yes, they could have. A trial in 1980 tested a new anti-arrhythmic drug, lorcainide, in a small number of men who’d had a heart attack—less than a hundred—to see if it was any use. Nine out of forty-eight men on lorcainide died, compared with one out of forty-seven on placebo. The drug was early in its development cycle, and not long after this study it was dropped for commercial reasons. Because it wasn’t on the market, nobody even thought to publish the trial. The researchers assumed it was an idiosyncrasy of their molecule, and gave it no further thought. If they had published, we would have been much more cautious about trying other anti-arrhythmic drugs on people with heart attacks, and the phenomenal death toll—over 100,000 people in their graves prematurely—might have been stopped sooner. More than a decade later, the researchers finally did publish their results, with a mea culpa, recognizing the harm they had done by not sharing them earlier:
When we carried out our study in 1980, we thought that the increased death rate that occurred in the lorcainide group was an effect of chance. The development of lorcainide was abandoned for commercial reasons, and this study was therefore never published; it is now a good example of ‘publication bias’. The results described here might have provided an early warning of trouble ahead.10
See what we're tweeting about
18 Comments
Add CommentThis is an interesting commentary and it certainly has some merit in pointing out deficiencies in our current system of clinical trials. However, I think the author shows a clear bias against pharmaceutical research in his tone and style and would be surprised to see a conclusion other than this is a "problem".
Reply | Report Abuse | Link to thisIt is likely that the trials sponsored by pharmaceutical companies are better funded, larger, and more carefully controlled. These trials are often admittedly designed to show efficacy - sometimes in a narrow population - as opposed to effectiveness. You simply have to do this to get a drug approved. There is nothing evil or inappropriate in this. If it doesn't work in a focused population, you might as well stop there. So, success and failure, independent versus pharma, need to be studied in similar trials otherwise you are comparing apples to oranges and committing the same sins of bias alluded to here. This is very difficult to do as pharma trials tend to be very different than independent trials in design, scope, and patient population. As to meta-analysis, namely, the analysis of a compilation of studies, this is extremely error prone and subject to bias and selection error as the author well knows.
Independently funded research is often conducted with an agenda no less biased than that of pharmaceutical companies. There are many people who set out with a goal, conscious or unconscious, of proving that a drug does not work and design their trials accordingly. For some, it seems good sport to be a thorn in the side of industry.
The truth is that nothing is perfect and perfect is the enemy of the good. There have been staggering R&D cuts in the pharmaceutical industry throughout the world due in large part from regulatory hurdles that demand clinical trials of unsupportable size, cost, and duration. This can chew up so much patent life and capital that companies simply give up.
Considering the poor trajectory of the pharmaceutical industry today, the pundits may get what they want; few industry funded trials because there will be little financial incentive and even less research.
@deneb
Reply | Report Abuse | Link to this"I think the author shows a clear bias against pharmaceutical research in his tone and style and would be surprised to see a conclusion other than this is a "problem""
It's clear he has the right kind of evidence to support his conclusions so I don't worry about tone and style.
Hi there,
Reply | Report Abuse | Link to thisI wrote the book that's extracted above.
"I think the author shows a clear bias against pharmaceutical research in his tone and style and would be surprised to see a conclusion other than this is a "problem"."
There is a discussion at the end of each section, and the end of each chapter, and then at the end of the book, about what can be done to remedy these problems.
There is also a campaign at www.alltrials.net for the results of all trials on all currently used treatments to be reported in full, that has been signed by over 30,000 people, 80 patient groups, research funders, medical bodies, and the drug company GSK, who have signed up to our commitment:
http://www.badscience.net/2013/02/this-is-excellent-and-amazing-gsk-have-just-signed-up-to-alltrials-net/
Do please sign yourself:
www.alltrials.net
"Considering the poor trajectory of the pharmaceutical industry today, the pundits may get what they want; few industry funded trials because there will be little financial incentive and even less research."
I wouldn't want that, nor do I ask for it. I simply ask that all results for all trials on all treatments used by patients are made available. It is impossible for doctors and patients to make informed decisions when results are withheld. Industry should be able to make money from good treatments (as they often do) without hiding information on them. I don't see that this is unreasonable, and you are also incorrect to assume that these issues only affect industry: academic studies are also frequently withheld, as I have discussed at length here and elsewhere.
Like many of the issues in the book, these issues are structural, and they are long overdue fixing.
at bengoldacre
Reply | Report Abuse | Link to thisThank you.
Most of normal us have this altruistic view, big pharma et al, like to hide everything so people don't get to see the real picture.
I signed your petition. :)
Shame we cant edit our posts to remove grammar errors though, as we race to type things.
Reply | Report Abuse | Link to this@Ben
Reply | Report Abuse | Link to thisThough I appreciate your campaign for openness, how are you planning to address the significant intellectual property rights aspect of funded research? It is a sticky issue, but don't you think that investors have the right to discover valuable information without giving it away to everyone, including their competitors?
Though patients and doctors need to make informed decisions, they are buying products from companies that have invested a great deal of money and skill into developing treatments that are often very easy and materially inexpensive to copy. Patents provide a limited amount of protection against the unfair competition that results when copycats have equal access to information.
I admit that I am troubled by the current situation and wish that information was more freely available. As taxpayers, we have the ability to ask our government to spend more of our money on publicly funded research, but our constitution is based on the principle that we do not have the right to take other people's property for public good without fair compensation.
Rod Adams
Publisher, Atomic Insights
Drug companies also undermine their own research by not disclosing what's in their placebos (and active placebos). They're allowed to manufacture their own placebos, some of which produce dry mouth and other symptoms (by design) and not report the ingredients. I have written about this recently in my blog at http://asserttrue.blogspot.com/2013/01/are-placebos-really-sugar-pills.html.
Reply | Report Abuse | Link to thisThank you for a great article.
Possible corrections to my comment:
Reply | Report Abuse | Link to this>However, I think the author shows a clear bias against pharmaceutical research in his tone and style and would be surprised to see a conclusion other than this is a "problem".<
1) It's clear he has the right kind of evidence to support his conclusions so in this case I don't worry about tone or style.
2) I agree tone and style are important, but in this case they are so tame and the evidence is so strong there really is nothing to complain about.
3) I think the author does not show a 'clear bias against pharmaceutical research' but rather he shows 'clear and solid evidence against certain research practices in the pharmaceutical industry'.
Are you insane? Do you read your comments before posting them? When was the last time you checked your ethimometer?
Reply | Report Abuse | Link to thisI agree that all drug or therapy research has an agenda regardless of its funding source. However the gravy train of industry funded research is greased by mining for positive results regardless of how inane. While it is not fraud, it does create a lean to research literature that researchers, clinicians and others trained in the business recognize and defend against. This is one of the reasons pharmaceutical companies moved to direct consumer advertising. It is easier to create demand with images of happy families running through flower draped fields than by producing research papers that are going to be reviewed with a jaundiced eye by the prescriber.
Reply | Report Abuse | Link to thisAh, so! Those industry trials and surveys, are just as biased, as the ones, that I took, and often did not complete, because despite my answers, I was being led towards a conclusion, I could not support.
Reply | Report Abuse | Link to thisDitto political ones that tend to lead to a conclusion, the paid surveyor, promised his client.
No surprise here. The pharmaceutical industry (and chemical industry as a whole) have been cherry picking results of studies for decades. As far back as the early 1980s I wrote a series of stories about "cherry picking" that was so blatant that it was considered fraud in the testing of hundreds of food additives, drugs, pesticides, and industrial chemicals.
Reply | Report Abuse | Link to thisIndustrial Bio-Test Laboratories (IBT) was a particularly prolific source of toxicological testing. It also was involved in massive misconduct that resulted in the indictment and conviction of its president and a number of top executives in 1983. An audit by the FDA found that 71% of 867 studies were invalid because of "numerous discrepancies between the study conduct and data".
However, IBT was not an isolated case as the pharmaceutical and chemical industries claimed. Information I gathered from the FDA in 1980-1982 revealed that more than two dozen other labs had also engaged in significant levels of misconduct.
Three decades have passed and not much has changed when it comes to ensuring the integrity of safety studies on a massive number of products that we are exposed to.
As we move into an era in which biotechnology and nanotechnology will have an increased impact on our lives, the integrity of safety studies needs to be beyond question. Unfortunately, the lack of transparency makes it all to easy to continue to high unfavorable results.
AllTrials is a good start, but we need to ensure that not only results, but all of the data and study documentation are made available online so that many eyes can verify independently that "cherry picking", selective reporting, or outright fraud and fakery are not happening.
I'd say this has bias, I've seen an antidepressant drug research discontinuation seriously considered, don't konw what happened in the end, because more suicide attempts, although not reaching statistical signification, were in the experimental than in the comparator group. For pre-market trials, aimed to obtain a drug registration, this simply can't happen, Health Regulatory Authorities watch over all the process, and no fake data can be presented. Post-marketing studies are a different thing, as long as an study has not a Registered Trial Number, you cannot trust it, but Drug Surveillance data about unexpected side effects, not known in the early phases, when not too many patients had received the drug are even harder to hidden, as the economical consequences of hidding data are so enormous, that no company can seriously thinking it can be afforded, and a double registry, of drug producer and HRAs exists. There was another book on this more tan 20 years ago: "Dangerous medicine", I received it as a tip gift from a very nice person, but haven't found yet the time to read it, reality is often much more interesting than fiction.
Reply | Report Abuse | Link to thisA good read and well presented. Sadly, not very surprising findings given what I know about business and humans. You really can't help wanting to find positive results for the company you work for, where as an independent comes in with a skeptical eye.
Reply | Report Abuse | Link to thisThe details about the consequences of unpublished research are clear and important. It's unfortunate that the author starts with the statistics about how many studies support the drug being tested, because this is exactly the result you would expect if drug companies were doing what we want them to do in a highly ethical way.
Reply | Report Abuse | Link to thisLook, studies are expensive, so your "ideal" drug company is going to to do large, repeated studies on drugs they believe actually work. That is, they will re-test, in larger populations, the drugs that showed promising reults in small tirals. Unless drug companies are quite incompetent, the later trials will tend to confirm the ealier trials, producing exactly the result that the author presents as evidence of misconduct: most trials deonstrate the effectiveness of the drug being tested.
This lapse does not impugn the importance of the story he tells later about missing information in unpublished studies.
Clearly the past 'industry study' negative bias towards pharma companies has stopped with this article; Not. I agree that a case appears to have been made for greater transparency, but how would we know for sure with such blatant negative bias. Clearly there are countless more successful drugs and trials than there are questionable one. Lets not cost yet more lives by dramatic sensationalism and spooking the herd into a stampede of not taking drugs that do actual prolong and vastly improve people's lives.
Reply | Report Abuse | Link to thisI think a good start-up read to this book, one that I've already read and as a student of the pharmaceutical sciences, thoroughly enjoyed, was "Body Hunters" by Sonia Shah.
Reply | Report Abuse | Link to thisIt would seem to me that the high rate of positive results from pharma drug trials is simply because they are only willing to invest in a trial when they are already confident of a positive result.
Reply | Report Abuse | Link to this