In a few pages, we will walk through the literature that demonstrates all of this beyond any doubt, showing that “publication bias”—the process whereby negative results go unpublished—is endemic throughout the whole of medicine and academia; and that regulators have failed to do anything about it, despite decades of data showing the size of the problem. But before we get to that research, I need you to feel its implications, so we need to think about why missing data matters.
Evidence is the only way we can possibly know if something works—or doesn’t work—in medicine. We proceed by testing things, as cautiously as we can, in head-to-head trials, and gathering together all of the evidence. This last step is crucial: if I withhold half the data from you, it’s very easy for me to convince you of something that isn’t true. If I toss a coin a hundred times, for example, but only tell you about the results when it lands heads-up, I can convince you that this is a two-headed coin. But that doesn’t mean I really do have a two-headed coin: it means I’m misleading you, and you’re a fool for letting me get away with it. This is exactly the situation we tolerate in medicine, and always have. Researchers are free to do as many trials as they wish, and then choose which ones to publish.
The repercussions of this go way beyond simply misleading doctors about the benefits and harms of interventions for patients, and way beyond trials. Medical research isn’t an abstract academic pursuit: it’s about people, so every time we fail to publish a piece of research we expose real, living people to unnecessary, avoidable suffering.
In March 2006, six volunteers arrived at a London hospital to take place in a trial. It was the first time a new drug called TGN1412 had ever been given to humans, and they were paid £2,000 each. Within an hour these six men developed headaches, muscle aches, and a feeling of unease. Then things got worse: high temperatures, restlessness, periods of forgetting who and where they were. Soon they were shivering, flushed, their pulses racing, their blood pressure falling. Then, a cliff: one went into respiratory failure, the oxygen levels in his blood falling rapidly as his lungs filled with fluid. Nobody knew why. Another dropped his blood pressure to just 65/40, stopped breathing properly, and was rushed to an intensive care unit, knocked out, intubated, mechanically ventilated. Within a day all six were disastrously unwell: fluid on their lungs, struggling to breathe, their kidneys failing, their blood clotting uncontrollably throughout their bodies, and their white blood cells disappearing. Doctors threw everything they could at them: steroids, antihistamines, immune-system receptor blockers. All six were ventilated on intensive care. They stopped producing urine; they were all put on dialysis; their blood was replaced, first slowly, then rapidly; they needed plasma, red cells, platelets. The fevers continued. One developed pneumonia. And then the blood stopped getting to their peripheries. Their fingers and toes went flushed, then brown, then black, and then began to rot and die. With heroic effort, all escaped, at least, with their lives.
The Department of Health convened an Expert Scientific Group to try to understand what had happened, and from this two concerns were raised. First: can we stop things like this from happening again? It’s plainly foolish, for example, to give a new experimental treatment to all six participants in a ‘first-in-man’ trial at the same time, if that treatment is a completely unknown quantity. New drugs should be given to participants in a staggered process, slowly, over a day. This idea received considerable attention from regulators and the media.
Less noted was a second concern: could we have foreseen this disaster? TGN1412 is a molecule that attaches to a receptor called CD28 on the white blood cells of the immune system. It was a new and experimental treatment, and it interfered with the immune system in ways that are poorly understood, and hard to model in animals (unlike, say, blood pressure, because immune systems are very variable between different species). But as the final report found, there was experience with a similar intervention: it had simply not been published. One researcher presented the inquiry with unpublished data on a study he had conducted in a single human subject a full ten years earlier, using an antibody that attached to the CD3, CD2 and CD28 receptors. The effects of this antibody had parallels with those of TGN1412, and the subject on whom it was tested had become unwell. But nobody could possibly have known that, because these results were never shared with the scientific community. They sat unpublished, unknown, when they could have helped save six men from a terrifying, destructive, avoidable ordeal.