How Astronomers Discovered the Universe's Hidden Light

Why is the night sky dark? After all, if the universe is filled with billions of galaxies, every one of them swirling with billions of stars that have been emitting photons of light for billions of years, why would the universe not be awash with light? German astronomer Wilhelm Olbers pondered that question in the 1820s, and the riddle became known as Olbers's paradox. By then, astronomers and philosophers had wondered for centuries why the sky was dark and what the darkness implied about the nature of the universe. It turns out that these scholars were on to something truly profound.

More light is out there than we can easily see. Even from deep space, far away from the lights of Earth and the stars of the Milky Way, the sky of intergalactic space is not absolutely black. It glows with what is called the extragalactic background light (EBL). The EBL consists of all the photons of light radiated by all the stars and galaxies that have ever existed, at all wavelengths from the ultraviolet through the far infrared, during all of cosmic history to the present. The EBL from distant galaxies is faint because extragalactic space is vast compared with the number of galaxies that glow (or have ever done so). Because the universe is expanding, the photons emitted by galaxies over the history of the cosmos have spread throughout the cavernous volume of space and become dilute. And because of the expansion, light from distant galaxies undergoes a “redshift”—wavelength increases, pushing the light toward the red side of the electromagnetic spectrum, outside the visible realm.

Astronomers have realized for a while that this extragalactic background light should exist but were unable to measure it accurately. Between 2012 and 2013, for the first time, researchers (including two of us, Domínguez and Primack) were able to unambiguously quantify the extragalactic background light using gamma-ray data from the Fermi Gamma-ray Space Telescope and ground-based very high energy gamma-ray detectors called atmospheric Cherenkov telescopes. Intriguingly, because stars contribute most of the EBL either directly as starlight or through heating dust that radiates at longer wavelengths, the background preserves the “memory” of star formation at different epochs throughout the history of the universe. Indeed, measurements of the EBL are allowing us to explore the evolution of galaxies from ancient times to the present. Eventually it may let us study the very first generation of galaxies from more than 13 billion years ago, whose light is too faint to see directly with current telescopes.

The cosmic backgrounds
Olbers's paradox was primarily a philosophical question until the 1960s, when phenomenal astronomical discoveries across the entire electromagnetic spectrum were transforming cosmology from speculation to a hard observational science. Researchers were beginning to discover a menagerie of bizarre galactic and extragalactic objects. The universe, it was becoming clear, is filled with a rarefied “gas” of photons zooming every which way through extragalactic space. These photons come in many wavelengths—and equivalently, in many energy ranges (shorter wavelengths correspond to waves with higher frequencies and thus greater energies; long wavelengths have lower frequencies and thus smaller energies). That gas includes the EBL, as well as several other radiation fields seen in all directions. The brightest is the cosmic microwave background (CMB), which originated from the explosive big bang. In 1965 Arno Penzias and Robert W. Wilson discovered the CMB while at AT&T Bell Laboratories, for which they received the 1978 Nobel Prize in Physics. Another radiation field, an extragalactic diffuse x-ray background, was discovered in the 1960s with sounding rockets. In the late 1960s an orbiting solar observatory found yet another background of more energetic gamma rays.

The EBL—the cosmic background encompassing the near-ultraviolet, visible and infrared wavelengths—is second in energy and intensity to the CMB. Unlike the CMB, however, the EBL was not produced all at once. Instead it has been growing over billions of years, beginning with the formation of the first stars in the first galaxies roughly 200 million years after the big bang. Indeed, the EBL is still being added to today as new stars are born and begin to shine.

Directly measuring the EBL by collecting its photons with a telescope is akin to trying to observe the dim band of the Milky Way at night from among the brightly lit theaters and skyscrapers in New York City's Times Square. The EBL has a lot of competition at the same visible and infrared wavelengths. Earth is inside an extremely bright galaxy with billions of stars and immense clouds of glowing gas that outshine the extragalactic background light. Even worse for directly measuring the EBL, Earth resides in a very well-lit solar system: sunlight scattered by all the dust near Earth's orbit around the sun creates the zodiacal light—sometimes so luminous that from a dark site at the right time of year it can be mistaken for early dawn—that shines in similar wavelengths to the EBL.

How could astronomers ever hope to isolate, capture and identify faint EBL photons when they are swamped by a much brighter glow from the solar system and Milky Way? They cannot. Ground- and space-based telescopes have not succeeded in reliably measuring the EBL directly. In 2000 Piero Madau of the University of California, Santa Cruz, and Lucia Pozzetti of the Bologna Astronomical Observatory added up the light from galaxies detected by the Hubble Space Telescope. (Remember, the EBL is all the light emitted from near-ultraviolet through infrared wavelengths, including all the light from bright galaxies, which is easy to measure, plus galaxies too faint for telescopes to see.) But that count did not include faint galaxies or other possible sources of light, which means it gave only a lower limit for how bright the EBL could be at various wavelengths.

In 2011 Domínguez and Primack and our observational collaborators placed stronger lower limits on the EBL by adding up the amount of infrared and visible light observed from ground- and space-based telescopes from nearby galaxies out to about eight billion years ago—what astronomers call a redshift of 1, a little more than halfway back in time to the big bang. (Looking great distances out into space is equivalent to looking eons back in time because one sees objects as they looked when the light now reaching telescopes first departed on its journey—billions of years ago, in the case of truly distant galaxies.) We measured the changing patterns of wavelengths emitted by galaxies at different distances—that is, at various cosmic eras. This method allowed the best EBL determination yet based on observations. We calculated upper and lower estimates for the EBL from even more distant, older galaxies at redshifts greater than 1.

To move beyond limits, however—to truly measure the brightness of the extragalactic background light—astronomers would need to take another tack.

Colliding light
As far back as the 1960s, researchers started thinking about looking for the EBL through its interactions with other, more easily visible, forms of light.

Photons, it turns out, can collide with other photons. Specifically, high-energy gamma rays may collide with lower-energy photons, such as visible starlight, and mutually annihilate to create an electron and its antiparticle, the positron. Several astronomers began to wonder: What might happen if high-energy gamma rays from a distant cosmological source heading toward Earth collided with lower-energy EBL photons along the way? Would the EBL photons effectively waylay gamma rays, weakening the apparent brightness of the gamma-ray source as seen from Earth? If scientists could detect this attenuation of gamma rays, they reasoned, it might reveal the composition of the EBL.

That question remained purely a matter of theoretical speculation until 1992, when NASA's EGRET (Energetic Gamma Ray Experiment Telescope) detector onboard the orbiting Compton Gamma Ray Observatory discovered the first of a new class of gamma-ray sources that came to be called blazars: galaxies with central supermassive black holes emitting gamma rays in strong jets that happen to be pointed toward Earth like flashlight beams. The gamma rays in such jets have phenomenal energies of billions of electron volts—that is, giga-electron volts (abbreviated GeV). Indeed, some blazars, such as Markarian 421 (Mrk 421 for short), are emitting gamma rays at mind-boggling energies as high as 20 trillion electron volts (TeV), or about 100 million times as much energy as medical x-rays.

At about 400 million light-years away, the blazar Mrk 421 is relatively nearby as extragalactic distances go. But finding such a powerful gamma-ray source in the 1990s made Primack wonder whether similar TeV-energy blazars might exist at far greater distances—and thus be useful for detecting the EBL. Indeed, over the following years other TeV-energy gamma-ray blazars were discovered at increasingly greater distances. And figuring out how to harness blazars to measure the EBL began to occupy Domínguez in 2006, when he started Ph.D. research at the University of Seville in Spain, where he studied blazars with the MAGIC gamma-ray observatory.

In 2012 Domínguez was among nearly 150 co-authors led by Marco Ajello, now at Clemson University, who made the first measurement of how much blazar light gets absorbed by the EBL. The team pored over data from NASA's orbiting Fermi Gamma-ray Space Telescope, analyzing observations of 150 blazars at different distances to measure how much their gamma rays were attenuated with increasing distance—that is, after traveling through greater thicknesses of the EBL. The observations extended out to a redshift of 1.6, corresponding to light emitted almost 10 billion years ago.

To improve on that measurement, astronomers needed a way to better understand blazars' intrinsic nature and thus to know how many gamma rays of various energies a blazar actually produced before some of those gamma rays were absorbed by collisions with EBL photons across billions of light-years of extragalactic space.

The best way of estimating a blazar's initial output is to combine theoretical models of how blazars work—especially how they generate higher-energy gamma rays—with telescope observations of blazar's lower-energy gamma rays and x-rays, which are not absorbed by the EBL as often. The high-energy gamma rays in many blazars are thought to originate in a process called synchrotron self-Compton (SSC) scattering. In the blazar jet, an energetic beam of electrons and positrons interacting with magnetic fields emits x-rays. Some of those x-rays are then hit—Compton-scattered is the technical term—by the same energetic electrons, kicking them to much higher energies to become gamma rays. The SSC models allow us to predict the unattenuated intensity of the high-energy gamma rays by comparing them with the low-energy gamma-rays we can observe.

Finally, in 2013, Domínguez, Primack, Justin Finke of the Naval Research Laboratory, Francisco Prada of the Institute of Astrophysics of Andalusia, and three others collated nearly simultaneous observations of 15 blazars at different cosmological distances made by half a dozen NASA spacecraft and several ground-based telescopes operating at different wavelengths. We compared the Fermi Gamma-ray Space Telescope findings with the intensity of x-rays from the same blazars measured by the x-ray satellites Chandra X-ray Observatory, Swift, the Rossi X-ray Timing Explorer and XMM-Newton, plus optical and radio wavelengths measured by ground-based observatories.

By comparing these observations in various wavelengths with SSC models of the blazars' output, we were able to calculate the original unattenuated gamma-ray brightness emitted at the highest TeV energies by nine of the blazars. We then compared those calculations with direct measurements by ground-based telescopes of the actual attenuated gamma-ray light received at Earth from those same blazars. Thus, at long last, we measured the EBL through its imprint on the gamma rays of various energies received from blazars located at different redshifts.

Window to the past
The detection of the EBL was one of the toughest measurement challenges in observational astronomy—perceiving such a faint and diffuse signal required coordinating telescopes and researchers around the world to make simultaneous observations of extremely distant objects. It has given us a powerful new tool for studying cosmic history. Almost as soon as astronomers realized that blazars might be useful for studying the EBL, back in the 1990s, Primack and Donn MacMinn—then a brilliant college senior at the University of California, Santa Cruz—began to explore whether such measurements might reveal something about the evolution of galaxies. We still have many basic questions about galaxy formation, such as how common massive stars were in galaxies at various stages of development, how dust absorbed starlight and reemitted the energy at longer wavelengths, and how the number of stars that formed in galaxies varied during different epochs in the universe. MacMinn and Primack wondered whether studying gamma rays from blazars at different distances—gamma rays that traveled through different amounts of the EBL—might help answer some of those fundamental questions by providing windows on different eras of star formation in the universe.

For example, we know that distant galaxies in the early universe look significantly different from nearby galaxies: instead of being smooth spheroids or magnificent spirals, they are compact and distorted. Their distorted shapes were partly caused by collisions among these early galaxies because the young universe was much denser than it is today. The early galaxies also emit much more of their light at long infrared wavelengths than nearby galaxies do. That fact means that the EBL light created by long-ago galaxies at great distances has a different wavelength spectrum than the EBL light emitted by recent galaxies at closer range.

Thus, the pattern of gamma-ray energy absorbed by EBL photons from great distances out in space—that is, far back in time—should also differ from the pattern of gamma-ray energy absorbed by EBL photons nearby. Indeed, by 1994 MacMinn and Primack had done enough preliminary theoretical modeling to assert that the dominant factor influencing the characteristics of the EBL would be the epoch of galaxy formation at which the photons were emitted. We predicted how the gamma-ray attenuation by the EBL would have evolved over time based on several different cosmological assumptions. Eventually we showed that it would be possible to use measurements of the absorption by EBL photons of gamma rays from TeV sources at different distances to distinguish among competing theories of galaxy evolution.

Now that we have the first measurements of the EBL from blazar attenuation, we are starting to dig into our data to build a picture of star and galaxy formation throughout the cosmic timeline. For example, the wavelength spectrum of our EBL measurements gives a view of what was happening during the peak of star formation—a “cosmic high noon”—between eight billion and 12 billion years ago. The EBL spectrum shows two bumps: one representing ultraviolet and visible light shining from stars and another, larger bump in longer-wavelength far-infrared light. This second bump appears to come from dust. We know that exploding stars produce dust (made of heavier elements such as carbon, oxygen and iron) that envelops and obscures star-forming regions and that during cosmic high noon, dust absorbed much of the starlight and reradiated it in the infrared. The EBL gives us a way to study just how common such dust-obscured galaxies (nicknamed “DOGs”) were during this era—an important factor in understanding how rocky planets such as Earth formed because these planets contain large quantities of cosmic dust.

Looking into the future
What a satisfying experience, to realize a dream extending back more than two decades, where observations from many instruments confirm predictions. Moreover, how exciting it is to explore these new data—indeed, this brand-new cosmological instrument—to begin to discern what the evolution of the EBL reveals about the evolution of the universe.

Future EBL research could likewise tell us about earlier periods in the universe's history. If we could extend our observations of the EBL to include a few additional gamma-ray sources at higher redshift, astronomers could study how the universe was reionized (when ultraviolet light from the first stars knocked the electrons off hydrogen atoms) during the first billion years after the big bang. That goal is a major motivation for the huge new international Cherenkov Telescope Array now being designed with installations in both the Northern and Southern Hemispheres, which will consist of different instruments tailored to be sensitive to gamma rays from low to very high energies. And once we better understand and quantify the EBL, we can subtract its attenuation from the observations of blazars and gamma ray bursts to more fully describe the nature of those exotic objects themselves.

Meanwhile the EBL intensity measured indirectly by our gamma-ray attenuation technique is compatible with the EBL intensity estimated independently from observed galaxies over earlier cosmic epochs. This agreement means that the light emitted from galaxies at optical and near-infrared wavelengths appears to explain the EBL observations via gamma-ray attenuation, and it helps us close the accounting books.

As observations improve, agreement between these different types of measurement will either get tighter—powerfully constraining alternative sources of light in the universe (for example, the decay of hypothetical relic particles in the early universe)—or else discrepancies will emerge that will point toward new astrophysical phenomena (for example, exotic hypothetical particles converting to gamma rays). Better gamma-ray observations should come from continued use of existing facilities and the planned Cherenkov Telescope Array. Additionally, improved galaxy observations—including from future observatories such as the James Webb Space Telescope, the Large Synoptic Survey Telescope and 30-meter-class ground-based telescopes—will help scientists understand galaxy formation better.

We now know the answer to Olbers's paradox: the night sky is not dark; rather it is filled with the glow of all the galaxies that ever existed, even if that glow is difficult to detect. And all the time, supernovae are going off, gas clouds are glowing and new stars are being born to add their light to the pervading background that fills every cubic inch of the cosmos.