ADVERTISEMENT
See Inside Scientific American Volume 310, Issue 5

The “Crisis” in Scientific Results Is a Matter of Biology

Biology is making it harder for scientists to reproduce one another's experiments
test tubes, patient in test tube



Sam Falconer

Science works by iteration.' Scientists repeat their peers' work and build on their findings. The literature of peer-reviewed scientific papers is the record of this step-by-step process. In recent years, however, prominent reports have suggested that many scientists are not able to replicate others' published results. Is scientific progress going wrong on an unprecedented scale? Before we jump to that conclusion, it would help to consider the changing nature of science itself—particularly biology.

Basic biomedical research and its translation into therapeutic interventions to cure diseases are at the center of this issue. In an ideal world, academic scientists identify targets for drugs—typically proteins involved in disease—and industry scientists look for agents that interfere with those targets' function. In reality, more often than not, industry scientists find that they cannot replicate the effects seen by academics in a sufficiently robust way to justify drug development. Worse, many promising drug candidates fail in phase II clinical trials when their efficacy is put to the test.

The world seemed simpler in the 1970s, when molecular biology brought us concepts such as “gene A leads to protein B, which leads to function C.” Thinking this way, scientists uncovered amazing mechanistic insights and, sometimes, designed effective drugs—the cancer drug Gleevec is the poster child of that reductionist approach. Wouldn't it be nice if drug discovery always went this way?

Those first drugs, however, were low-hanging fruit. Biology is much more complicated than simple schematics. Biological processes do not work in linear ways independently of one another but in tightly interconnected networks. In each branch of these networks, layers of regulatory controls constantly change the nature and abundance of the molecular players. We know little about the inner workings of human cells.

To illustrate how little, consider how genes are controlled. The modern study of gene regulation started in the 1950s, but researchers only started to unravel the complex array of histone modifications that fine-tune chromatin control of gene expression 20 years ago. The fact that RNA interference, another mode of gene regulation, is pervasive has only been realized in the past 10 years. What else don't we know yet?

Laboratory biologists deal with complexity on a daily basis. Mice bred with identical DNA behave differently. Two cells growing side by side in a petri dish cannot be considered identical. In the variable environment of the cell, it is difficult to distinguish a change that is meaningful to a process from one that is unrelated. Working in a modern lab also entails using sensitive apparatuses, rare technical skills and biological reagents—antibodies and enzymes, for example—which are themselves variable.

In such noisy systems, it is easy to mistake a chance observation for a robust, biologically meaningful effect. Biologists have to undertake large studies that can guarantee the statistical significance of observations, and they need self-critical analysis to avoid inadvertent biases. Scientists cannot be too careful to avoid falling prey to their own enthusiasm.

In that regard, they need the support of their institutions and the journals that publish their results. Some journals, such as Nature, have introduced checklists to ensure that scientists consider and report key information about experiments. (Scientific American is part of Nature Publishing Group.) Still, research institutions should provide more training and supervision of younger scientists. Institutions and funders should manage their incentive systems to limit undue pressures on researchers and promote best practices.

The need for replicating results is as important as ever. But it is inevitable that results obtained in one cell line might not exactly match those in another. They in turn might not be completely predictive of the observations in animal models, let alone human beings. The literature of published results is still strong. To keep it that way, the scientific community cannot afford to be complacent. It must pay attention to the professionalism of researchers and take into account the complexity of biology.

This article was originally published with the title "Eureka Once, Eureka Twice."

Rights & Permissions
Share this Article:

Comments

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Scientific American Holiday Sale

Black Friday/Cyber Monday Blow-Out Sale

Enter code:
HOLIDAY 2014
at checkout

Get 20% off now! >

X

Email this Article

X