The famous opening words, “It was the best of times, it was the worst of times,” of Charles Dickens’s 1859 novel A Tale of Two Cities referred to the period of French Revolution. But he could equally well have been describing his contemporary Charles Darwin’s experience with his theory of evolution by natural selection. Darwin was born at the best of times in 1809 when conditions were highly conducive for his theory to flourish, but he died in 1882 at the worst of times because there was a real danger that it might soon be killed off. Darwin's nemesis was the eminent physicist Lord Kelvin, and the weapon used against him was the age of the Earth.

Various theories of evolution predated Darwin, but whatever version one favored, one thing was clear: it needed a very long time for its consequences to work itself out. Precisely how long was hard to pin down, but it was believed to require tens or hundreds of millions of years. From 1650 on, the dominant theory about the age of the Earth, based on the work of Bishop Ussher, Isaac Newton and many other scholars who used various textual sources, was that it was about 6,000 years.

Theories of geology and biology had to accommodate themselves to this short timeframe. The geological theory known as catastrophism postulated that major features such as the Grand Canyon, Himalayas, etc. had emerged as a result of sudden and violent upheavals. When it came to biological diversity, the idea of special creation, that organisms were immutable and had been specially designed to fit the biological niches in which they found themselves, was the prevailing view.

Such a short window of time would have made it impossible for anyone to credibly propose the emergence of new species by a process of slow evolution. But around 1785, ideas of the age of the Earth began to undergo a radical change as geologists and paleontologists started using the features of the Earth itself, such as erosion, sedimentation and the layers of strata and the fossils embedded in them, to estimate its age. The theory of uniformitarianism, that major geological features were caused by the very slow accumulation of tiny changes, gained ground, culminating in Charles Lyell’s epic three-volume work Principles of Geology in 1830 that cemented the idea that the Earth had been around for hundreds of millions of years and possibly much longer, so long that it seemed impossible to fix an actual age.

Darwin was also a keen student of geology and was familiar with Lyell's work (they later became close friends) and had the first two volumes of his book with him as he made his five-year voyage around the globe on the HMS Beagle during 1831–1836, where the ideas for his theory of evolution germinated as he observed the patterns of species in the various locations that he visited. Darwin knew from his work with pigeons that even deliberately breeding for specific characteristics took a long time to produce them. But how much time was necessary? He felt that it required at least hundreds of millions of years. The work of Lyell and other geologists gave him the luxury of assuming that sufficient time existed for natural selection to do its work.

Darwin also came of age at a time when the idea that species were immutable had begun to crumble. While Darwin was taught and accepted the still dominant special creation theory, he was also familiar with the general idea of evolution. His own grandfather Erasmus Darwin had in 1794 published a book Zoonomia that explored protoevolutionary ideas. Jean-Baptiste Lamarck had published his own model of how evolution worked in 1802. The various models proposed for the mechanism of evolution, such as Lamarckian evolution, orthogenesis and use-disuse, all implied some level of teleology: that there was a directionality inherent in the process.

The extremely long window of time opened up by the new geology allowed Darwin and his co-creator Alfred Russell Wallace, working independently, to develop the central new insight of natural selection that differed from other models of evolution by showing how diversity could arise naturally by a process of Malthusian population selection pressures, without any kind of mystical agency directing the process towards any specific ends.

Their theory held that evolution was contingency-driven in that how organisms evolved depended on chance factors and a changing environment, which suggested that if we ran the clock again, we could get very different outcomes in which human beings might never appear. Their work was first presented in a joint paper in 1858. Darwin's major work On the Origin of Species, published a year later, was a closely argued compilation of a massive amount of evidence that helped establish evolution as a fact.

The radical change in the status of human beings implied by their model, from being specially created in the image of God to just another accidental byproduct of the evolutionary process just like all other species, was an immediate source of controversy because it challenged a key religious tenet that human beings were special. This was why natural selection aroused such opposition even while evolution was accepted. Many scientists of that time were religious and believed in theistic evolution that said that a supernatural agency was guiding the process to produce the desired ends.

While all models of evolution required long times, natural selection required much longer times than any guided selection process. Hence the younger the age of the Earth, the more likely it was that natural selection could not be the mechanism. Physicists, led by the eminent Kelvin, himself a theistic evolutionist, were leaders of the charge for a young Earth, though it must be emphasized that “young” at that time meant around 100 million years or less. Even religious scientists had abandoned the idea of the Earth being just 6,000 years old.

Beginning around 1860, Kelvin and other physicists started estimating the ages of the Earth and sun using the nebular hypothesis proposed around 1750 by Immanuel Kant and Pierre-Simon Laplace. This model treated the Earth and the sun as starting as rotating clouds of particles that coalesced under gravity to form molten balls, with the Earth subsequently solidifying and cooling. Kelvin used the laws of thermodynamics and other physics principles to arrive at estimates of 20–400 million years. By 1879 the upper limit had been lowered to about 100 million years for the Earth and an even shorter upper limit of 20 million years for the Sun, much less than the 200 million years or so believed to be required for natural selection to work. Since physics was considered to be the most rigorous of the sciences, things looked bad for natural selection.

When Darwin died in 1882, he was mourned as a great scientist who had radically changed our understanding of how the vast diversity of organisms we see around us came about, dethroning the idea that they were immutable. But because of the shrinking age of the Earth, he died with a major cloud hanging over his mechanism of natural selection. Darwin's final words on the topic, written in 1880 just two years before his death, expressed a plaintive hope that future developments might reconcile the needs of natural selection with physics calculations:

“With respect to the lapse of time not having been sufficient since our planet was consolidated for the assumed amount of organic change, and this objection, as urged by [Lord Kelvin], is probably one of the gravest as yet advanced, I can only say, firstly that we do not know at what rate species change as measured in years, and secondly that many philosophers are not yet willing to admit that we know enough of the constitution of the universe and of the interior of our globe to speculate with safety on its past duration.”

It turned out Darwin was prescient that improved knowledge of the interior of the Earth might change the calculations in his favor, but at the time he seemed to be grasping at straws. In fact, in the near term the problem got even worse because Kelvin and others produced new calculations that resulted in the age of the Earth being reduced even more, so that by 1895 the consensus physics view was that the age of the planet lay in the range 20–40 million years. Natural selection appeared to be doomed.

But physicists were now encountering stiffer opposition from other disciplines. Geologists were adamant that their models based on the accumulating evidence on sedimentation and erosion, while not as rigorous as the physics models, were well enough established that they were confident of their lower limit of 100 million years. Paleontologists were also arguing that the fossil record was not consistent with the physicists’ shorter ages. Both groups argued that the physicists must have gone awry somewhere, even if they could not point out the specific flaws.

Beginning in 1895, this impasse began to be broken when physicist John Perry, a former assistant of Kelvin's, challenged the latter's assumption that the Earth was a rigid and homogeneous body, saying there was little evidence to support it. By introducing inhomogeneity and convective flow in the Earth’s interior, he found that Kelvin’s estimates for the age of the Earth could change by as much as a factor as 100, shifting the upper limit into the billions of years. Other physicists also chimed in with similar upward shifts, and this encouraged geologists, paleontologists and biologists to ignore the physicists' arguments for a young Earth.

It was the discovery of radioactivity that decisively changed the picture; it led to an entirely new way of measuring the age of the Earth, by allowing scientists to calculate the ages of rocks. Since the oldest rock that could be found set a lower limit for the age of the Earth, the race was on to find older and older rocks using this method; records fell rapidly, leading to ages of 141 million years by 1905, 1.64 billion years by 1911, 1.9 billion years by 1935, 3.35 billion years by 1947, and to 4.5 billion years by 1953, which is where the current consensus lies. 

When Kelvin died in 1907 at the age of 83, it was not clear if he had accepted that his estimates were no longer valid. But Darwin's hope that Kelvin would be proven wrong, and that eventually it would be shown that sufficient time existed for natural selection to work, was realized, 30 years after Darwin’s death. His theory now has all the time it needs.

This is an opinion and analysis article; the views expressed by the author or authors are not necessarily those of Scientific American.