Editor's Note: This story was originally printed in the February 2004 issue of Scientific American. We are reposting this story because author Adam Riess was selected as a MacArthur Fellow in 2008 by the MacArthur Foundation.

From the time of Isaac Newton to the late 1990s, the defining feature of gravity was its attractive nature. Gravity keeps us grounded. It slows the ascent of baseballs and holds the moon in orbit around the earth. Gravity prevents our solar system from flying apart and binds together enormous clusters of galaxies. Although Einstein’s general theory of relativity allows for gravity to push as well as pull, most physicists regarded this as a purely theoretical possibility, irrelevant to the universe today. Until recently, astronomers fully expected to see gravity slowing down the expansion of the cosmos.

In 1998, however, researchers discovered the repulsive side of gravity. By carefully observing distant supernovae—stellar explosions that for a brief time shine as brightly as 10 billion suns— astronomers found that they were fainter than expected. The most plausible explanation for the discrepancy is that the light from the supernovae, which exploded billions of years ago, traveled a greater distance than theorists had predicted. And this explanation, in turn, led to the conclusion that the expansion of the universe is actually speeding up, not slowing down. This was such a radical finding that some cosmologists suggested that the falloff in supernova brightness was the result of other effects, such as intergalactic dust dimming the light. In the past few years, though, astronomers have solidified the case for cosmic acceleration by studying ever more remote supernovae.

But has the cosmic expansion been speeding up throughout the lifetime of the universe, or is it a relatively recent development— that is, occurring within the past five billion years or so? The answer has profound implications. If scientists find that the expansion of the universe has always been accelerating, they will have to completely revise their understanding of cosmic evolution. But if, as cosmologists expect, the acceleration turns out to be a recent phenomenon, researchers may be able to determine its cause—and perhaps answer the larger question of the destiny of the universe—by learning when and how the expansion began picking up speed.

Battle of Titans
ALMOST 75 YEARS AGO astronomer Edwin Hubble discovered the expansion of the universe by observing that other galaxies are moving away from ours. He noted that the more distant galaxies were receding faster than nearby ones, in accordance with what is now known as Hubble’s law (relative velocity equals distance multiplied by Hubble’s constant). Viewed in the context of Einstein’s general theory of relativity, Hubble’s law arises because of the uniform expansion of space, which is merely a scaling up of the size of the universe.

In Einstein’s theory, the notion of gravity as an attractive force still holds for all known forms of matter and energy, even on the cosmic scale. Therefore, general relativity predicts that the expansion of the universe should slow down at a rate determined by the density of matter and energy within it. But general relativity also allows for the possibility of forms of energy with strange properties that produce repulsive gravity. The discovery of accelerating rather than decelerating expansion has apparently revealed the presence of such an energy form, referred to as dark energy.

Whether or not the expansion is slowing down or speeding up depends on a battle between two titans: the attractive gravitational pull of matter and the repulsive gravitational push of dark energy. What counts in this contest is the density of each. The density of matter decreases as the universe expands because the volume of space increases. (Only a small fraction of matter is in the form of luminous stars; the bulk is believed to be dark matter, which does not interact in a noticeable way with ordinary matter or light but has attractive gravity.) Although little is known about dark energy, its density is expected to change slowly or not at all as the universe expands. Currently the density of dark energy is higher than that of matter, but in the distant past the density of matter should have been greater, so the expansion should have been slowing down then.

Cosmologists have other reasons to expect that the expansion of the universe has not always been speeding up. If it had been, scientists would be at a loss to explain the existence of the cosmic structures observed in the universe today. According to cosmological theory, galaxies, galaxy clusters and larger structures evolved from small inhomogeneities in the matter density of the early universe, which are revealed by variations in the temperature of the cosmic microwave background (CMB). The stronger attractive gravity of the overdense regions of matter stopped their expansion, allowing them to form gravitationally bound objects—from galaxies such as our own to great clusters of galaxies. But if the expansion of the universe had always been accelerating, it would have pulled apart the structures before they could be assembled. Furthermore, if the expansion had been accelerating, two key aspects of the early universe—the pattern of CMB variations and the abundances of light elements produced seconds after the big bang—would not agree with current observations.

Nevertheless, it is important to look for direct evidence of an earlier, slowing phase of expansion. Such evidence would help confirm the standard cosmological model and give scientists a clue to the underlying cause of the present period of cosmic acceleration. Because telescopes look back in time as they gather light from far-off stars and galaxies, astronomers can explore the expansion history of the universe by focusing on distant objects. That history is encoded in the relation between the distances and recession velocities of galaxies. If the expansion is slowing down, the velocity of a distant galaxy would be relatively greater than the velocity predicted by Hubble’s law. If the expansion is speeding up, the distant galaxy’s velocity would fall below the predicted value. Or, to put it another way, a galaxy with a given recession velocity will be farther away than expected— and hence fainter—if the universe is accelerating.

Supernova Hunting
TO TAKE ADVANTAGE of this simple fact requires finding astronomical objects that have a known intrinsic luminosity—the amount of radiation per second produced by the object—and that can be seen across the universe. A particular class of supernovae known as type Ia are well suited to the task. These stellar explosions are so bright that ground telescopes can see them halfway across the visible universe, and the Hubble Space Telescope can view them from even farther away. Over the past decade, researchers have carefully calibrated the intrinsic luminosity of type Ia supernovae, so the distance to one of these explosions can be determined from its apparent brightness.

Astronomers can deduce the recession velocity of a supernova by measuring the redshift of the light from the galaxy in which it lies. Radiation from receding objects is shifted to longer wavelengths; for example, light emitted when the universe was half its present size will double in wavelength and become redder. By gauging the redshift and apparent brightness of a large number of supernovae located at a variety of distances, researchers can create a record of the universe’s expansion.

Unfortunately, type Ia supernovae are rare, occurring in a galaxy like the Milky Way only once every few centuries on average. The technique used by supernova hunters is to repeatedly observe a patch of sky containing thousands of galaxies and then compare the images. A transient point of light that appears in one image but not in a previous one could be a supernova. The 1998 results showing evidence of cosmic acceleration were based on the observations of two teams that looked at supernovae that exploded when the universe was about two thirds of its present size, about five billion years ago.

Some scientists wondered, though, whether the teams had correctly interpreted the data from the supernovae. Was it possible that another effect besides cosmic acceleration could have caused the supernovae to appear fainter than expected? Dust filling intergalactic space could also make the supernovae appear dim. Or perhaps ancient supernovae were just born dimmer because the chemical composition of the universe was different from what it is today, with a smaller abundance of the heavy elements produced by nuclear reactions in stars.

Luckily, a good test of the competing hypotheses is available. If supernovae appear fainter than expected because of an astrophysical cause, such as a pervasive screen of dust, or because past supernovae were born dimmer, the putative dimming effects should increase with the objects’ redshift. But if the dimming is the result of a recent cosmic speedup that followed an earlier era of deceleration, supernovae from the slowdown period would appear relatively brighter. Therefore, observations of supernovae that exploded when the universe was less than two thirds of its present size could provide the evidence to show which of the hypotheses is correct. (It is possible, of course, that an unknown astrophysical phenomenon could precisely match the effects of both the speedup and slowdown, but scientists generally disfavor such artificially tuned explanations.)

Finding such ancient and far-off supernovae is difficult, however. A type Ia supernova that exploded when the universe was half its present size is about one ten-billionth as bright as Sirius, the brightest star in the sky. Ground-based telescopes cannot reliably detect the objects, but the Hubble Space Telescope can. In 2001 one of us (Riess) announced that the space telescope had serendipitously imaged an extremely distant type Ia supernova (dubbed SN 1997ff) in repeated observations. Given the redshift of the light from this stellar explosion—which occurred about 10 billion years ago, when the universe was one third its current size—the object appeared much brighter than it would have been if the dusty universe hypothesis were true. This result was the first direct evidence of the decelerating epoch. The two of us proposed that observations of more high-redshift supernovae could provide definitive proof and pin down the transition from slowdown to speedup.

The Advanced Camera for Surveys, a new imaging instrument installed on the space telescope in 2002, enabled scientists to turn Hubble into a supernova-hunting machine. Riess led an effort to discover the needed sample of very distant type Ia supernovae by piggybacking on the Great Observatories Origins Deep Survey. The team found six supernovae that exploded when the universe was less than half its present size (more than seven billion years ago); together with SN 1997ff, these are the most distant type Ia supernovae ever discovered. The observations confirmed the existence of an early slowdown period and placed the transitional “coasting point” between slowdown and speedup at about five billion years ago. This finding is consistent with theoretical expectations and thus is reassuring to cosmologists. Cosmic acceleration was a surprise and a new puzzle to solve, but it is not so surprising as to make us rethink much of what we thought we understood about the universe.

Our Cosmic Destiny
THE ANCIENT SUPERNOVAE also provided new clues about dark energy, the underlying cause of the cosmic speedup. The leading candidate to explain dark energy’s effects is vacuum energy, which is mathematically equivalent to the cosmological constant that Einstein invented in 1917. Because Einstein thought he needed to model a static universe, he introduced his “cosmological fudge factor” to balance the attractive gravity of matter. In this recipe, the constant’s density was half that of matter. But to produce the observed acceleration of the universe, the constant’s density would have to be twice that of matter. Where could this energy density come from? The uncertainty principle of quantum mechanics requires that the vacuum be filled with particles living on borrowed time and energy, popping in and out of existence. But when theorists try to compute the energy density associated with the quantum vacuum, they come up with values that are at least 55 orders of magnitude too large. If the vacuum energy density were really that high, all matter in the universe would instantly fly apart and galaxies would never have formed.

This discrepancy has been called the worst embarrassment in all of theoretical physics, but it may actually be the sign of a great opportunity. Although it is possible that new attempts to estimate the vacuum energy density may yield just the right amount to explain cosmic acceleration, many theorists believe that a correct calculation, incorporating a new symmetry principle, will lead to the conclusion that the energy associated with the quantum vacuum is zero. (Even quantum nothingness weighs nothing!) If this is true, something else must be causing the expansion of the universe to speed up.

Theorists have proposed a variety of ideas, ranging from the influence of extra, hidden dimensions to the energy associated with a new field of nature, sometimes called quintessence. In general, these hypotheses posit a dark energy density that is not constant and that usually decreases as the universe expands. (But the suggestion that dark energy density is actually increasing as the universe expands has also been put forth.) Perhaps the most radical idea is that there is no dark energy at all but rather that Einstein’s theory of gravity must be modified.

Because the way the dark energy density varies is dependent on the theoretical model, each theory predicts a different time for the transition point when the expansion of the universe switched from slowdown to speedup. If the dark energy density decreases as the universe expands, then the switch-over point occurs earlier in time than it would for a model assuming constant dark energy density. Even theoretical models where gravity is modified lead to a discernible signature in the switch-over time. The latest supernova results are consistent with theories positing a constant dark energy density, but they also agree with most of the models that assume a varying dark energy density. Only theories stipulating large variations in dark energy density have been ruled out.

To narrow the range of theoretical possibilities, the Hubble Space Telescope is continuing to gather supernova data that could pin down the details of the transition phase. Although the space telescope remains the only means to probe the early history of cosmic expansion, more than half a dozen ground-based programs are trying to improve the precision of the measurement of recent cosmic speedup enough to reveal the physics of dark energy. The most ambitious project is the Joint Dark Energy Mission (JDEM) proposed by the U.S. Department of Energy and NASA. JDEM is a two-meter, wide-field space telescope dedicated to discovering and accurately measuring thousands of type Ia supernovae. Supernova hunters hope to see JDEM launched at the start of the next decade; until then, they will have to rely on the Hubble telescope to detect the most distant stellar explosions.

Solving the mystery of cosmic acceleration will reveal the destiny of our universe. If the dark energy density is constant or increasing with time, in 100 billion years or so all but a few hundred galaxies will be far too redshifted to be seen. But if the dark energy density decreases and matter becomes dominant again, our cosmic horizon will grow, revealing more of the universe. Even more extreme (and lethal) futures are possible. If dark energy density rises rather than falls, the universe will eventually undergo a “hyper speedup” that would tear apart galaxies, solar systems, planets and atomic nuclei, in that order. Or the universe might even recollapse if dark energy density falls to a negative value. The only way to forecast our cosmic future is to figure out the nature of dark energy.