The ancient Greeks believed atoms were the smallest bits of matter in the universe. Then scientists in the 20th century split the atom, yielding tinier ingredients: protons, neutrons and electrons. Protons and neutrons, in turn, were shown to consist of smaller particles called quarks, bound together by “sticky” particles, the appropriately named gluons. These particles, we now know, are truly fundamental, but even this picture turns out to be incomplete.

Experimental methods for peering inside protons and neutrons reveal a full-fledged symphonic orchestra within. These particles each consist of three quarks and varying numbers of gluons, along with what are called sea quarks—pairs of quarks accompanied by their antimatter partners, antiquarks—which appear and disappear continuously. And protons and neutrons are not the only particles made of quarks found in the universe. Accelerator experiments in the past half a century have created a host of other particles containing quarks and antiquarks, which, together with protons and neutrons, are called hadrons.

Despite all this insight—and a good understanding of how individual quarks and gluons interact with one another—physicists, to our dismay, cannot fully explain how quarks and gluons generate the full range of properties and behaviors displayed by protons, neutrons and other hadrons. For example, adding up the masses of the quarks and gluons inside protons does not begin to account for the total masses of protons, raising the puzzle of where all this missing mass comes from. Further, we wonder how exactly gluons do the work of binding quarks in the first place and why this binding seems to rely on a special type of “color” charge within quarks. We also do not understand how a proton's rotation—a measurable quantity called spin—arises from the spins of the quarks and gluons inside it: a mystery because the smaller particles' spins do not easily add up to the whole.

If physicists could answer these questions, we would finally begin to comprehend how matter functions at its most fundamental level. Identifying the main enigmas surrounding quarks and gluons, which we will detail below, is itself a key step toward discerning the physics of matter at its finest levels. Ongoing and future work, including studies focused on exotic configurations of quarks and gluons, should help demystify the puzzles. With a little luck, we will soon be able to break out of the fog.

Where does proton mass come from?

The mystery of mass is one of physicists' most vexing questions and offers a good sense of why the workings of quarks and gluons are so perplexing. We have a pretty good grasp of how quarks and leptons—a category of particles that includes electrons—get their mass. The mechanism arises from the Higgs boson—the particle discovered to much fanfare at the Large Hadron Collider (LHC) at CERN near Geneva in 2012—and from its associated Higgs field, which pervades all of space. When particles pass through this field, their interactions with it imbue them with mass. The Higgs mechanism is often said to account for the origin of mass in the visible universe. This statement, however, is incorrect. The mass of quarks accounts for only 2 percent of the mass of the proton and the neutron, respectively. The other 98 percent, we think, arises largely from the actions of gluons. But how gluons help to generate proton and neutron mass is not evident, because they are themselves massless.

A clue to resolving this conundrum is provided by Albert Einstein's famous equation relating a particle's mass at rest to its energy. By inverting the equation to read m = E/c2, we see that the mass (m) of the proton at rest can be said to arise from its energy (E), expressed in units of the speed of light (c). Because the energy of a proton is mostly contributed by gluons, in theory, one would only need to figure out the net energy of gluons to calculate the proton's mass.

Calculating the energy of gluons is difficult, however, in part because their total energy arises from several contributing factors. The energy of a free particle (unattached to others) is its energy of motion. Yet quarks and gluons almost never exist in isolation. They survive as free particles only on unimaginably small timescales (less than 3 × 10−24 seconds) before they are bound up into other subatomic particles and literally screened from view. Moreover, in gluons, energy comes not merely from motion; it is inseparable from the energy they expend in binding themselves and quarks together into longer-lived particles. Solving the mystery of mass therefore requires a better understanding of how gluons “glue.” But here, too, gluons throw up roadblocks to deciphering their mysteries.

How do gluons bind?

On one level, the answer to how gluons glue is simple: they wield the strong force. But this force is itself puzzling.

The strong force is one of the four fundamental forces of nature, along with gravitation, electromagnetism and the weak force (the last responsible for radioactive decay). Of these four, it is the most powerful by far (hence its name). In addition to binding quarks together to form hadrons, the strong force is what binds protons and neutrons into atomic nuclei, overcoming the enormous electromagnetic repulsion that exists between like-charged protons in a nucleus. Each of nature's fundamental forces appears to be tied to a particle, a so-called force carrier. Just as the photon—the fundamental unit of light—is the force carrier of electromagnetism, the gluon is the carrier of the strong force.

So far so good. But the strong force sometimes acts in surprising ways. According to quantum mechanics, the distance range of a force is inversely proportional to the mass of its force carriers. The electromagnetic force, for example, has an infinite range; a free electron on Earth will, in principle, experience a slight repulsion from an electron on the other side of the moon. Photons, which carry the force between the electrons, are therefore massless. In contrast to electromagnetism, the range of the strong force does not extend outside the nuclei of atoms. This fact would imply that gluons are very massive. Gluons, however, appear to be massless.

The strong force is also peculiar in that it seems to pull on quarks more strongly the farther away they get. In contrast, the electromagnetic force between two magnets is strongest when they are close and weaker when they are apart. Physicists first observed quarks in the 1960s, in experiments at the Stanford Linear Accelerator Center (now the SLAC National Accelerator Laboratory) that collided energetic electrons off proton targets. Sometimes the electrons passed right through, but other times they hit something solid and bounced back. Their rebound speed and direction revealed the presence and arrangement of the quarks inside protons. These so-called deeply inelastic scattering (DIS) experiments showed that quarks attract one another weakly at short distances; at larger separations, though, no free quarks are seen, which implies that they must pull on one another strongly.

To visualize how the strong force works, imagine two quarks tied together by strings. When they are close to each other, the tension in the string goes slack, and the quarks appear to experience no force. When they move farther apart, the tension in the string holds them together. This force between quarks corresponds to a weight of 16 metric tons at separations that are roughly the size of a proton. But what happens if an outside force pushes against the strong force's pull? The string snaps. Just how string breaking occurs is another mystery we cannot fully explain, and it is central to the story of how gluons glue inside atomic nuclei but not beyond them.

Why do some particles have colors?

In the 1970s physicists devised a theory called quantum chromodynamics (QCD) that mathematically describes the strong force. Just as the electromagnetic force revolves around particles' electrical charge, the strong force, according to QCD, revolves around a property known as color charge. The concept of color does help clarify why the strong force behaves so differently from the electromagnetic force, but it also raises a bevy of new conundrums—such as why some particles have color and others lack it and are thus “color-blind.”

According to QCD, both quarks and gluons carry color charge. All colored particles interact by exchanging gluons, implying that not only do quarks pass gluons back and forth but that gluons, too, exchange gluons with one another. This implication of quantum chromodynamics is a big leap from electromagnetism—photons do not interact with one another, as transparently demonstrated by crisscrossing light beams in a dusty room. Physicists think, however, that the self-interactions of gluons are central to the reason why the strong force gets weaker at close range. A gluon can temporarily become either a quark-antiquark pair or a gluon pair before turning back into a single gluon. The quark-antiquark fluctuation makes the interaction strength between color charges stronger, whereas the gluon-pair fluctuation makes it weaker. Because such gluon oscillations are more prevalent than the quark exchanges in QCD, they win. (Physicists David J. Gross, Frank Wilczek and H. David Politzer won the 2004 Nobel Prize in Physics for this discovery.)

In the decades since the advent of QCD, experiments worldwide have confirmed the theory's claim as one of the pillars of the Standard Model of physics. Yet many details of QCD remain elusive. Curiously, for instance, although the three quarks in a proton individually carry one of three, say, red, green and blue color charges, the proton does not have a net color charge. Likewise, the quark and the antiquark in a hadron known as a pi-meson (often called pion) carry color charges, but the pion is colorless. Color neutrality of hadrons is analogous to the electrical charge neutrality of atoms. But whereas the zero net charge of atoms is a clear consequence of the canceling out of the proton's positive charge and the electron's negative charge, how colored quarks and colored gluons combine to make colorless hadrons is not clear in QCD.

QCD should also explain how protons and neutrons overcome the powerful electromagnetic repulsion between protons to stick together inside atoms. But despite some progress, deriving nuclear physics from QCD is challenging. This impediment persists because the equations of QCD are fiendishly difficult to solve at the large distances where the interaction strength between quarks and gluons becomes very strong. And we lack a mathematical proof of how the QCD equations ensure that colored quarks and gluons are confined within colorless hadrons. Confinement is literally a $1-million problem—it is one of six outstanding puzzles identified by the Clay Mathematics Institute, whose solution will result in the award of $1 million to anyone who provides the answer.

Why do gluons not multiply forever?

A striking consequence of QCD is that the number of gluons and quarks inside the familiar proton can vary considerably. In addition to the basic three quarks, changing numbers of gluons flit around like fireflies, flickering in and out of existence, and pairs of quarks and antiquarks form and dissolve; the result is a “quantum foam” of appearing and disappearing particles. Physicists think that when protons and neutrons reach extreme speeds, the gluons inside the protons split into pairs of new gluons, each with slightly less energy than the parent. The daughter gluons, in turn, produce more daughter gluons, with even less energy. This splitting of gluons resembles an out-of-control popcorn machine. Theory suggests it could go on forever—yet we know it does not.

If gluons continued to procreate, the lid would blow off the popcorn machine—in other words, the proton would become unstable and collapse. Because matter is obviously stable (we exist), it is clear that something must rein in the runaway cascade—but what? One idea is that nature manages to put up a maximum occupancy sign when gluons become so numerous that they begin to overlap within the proton. Strong self-interactions cause them to repel one another, and the less energetic gluons recombine to form more energetic gluons. When the growth in gluon number tapers, the gluons reach a steady state of splitting and recombination called gluon saturation, bringing the popcorn machine under control.

This conjectured saturated gluon state, often called a color glass condensate, would be a distilled essence of some of the strongest forces in the universe. So far we have only hints of its existence, and its properties are not fully understood. By probing this state using more powerful DIS experiments than are currently available, physicists will be able to examine gluons closely in their densest, most extreme form. Is the force field limiting the amount of gluons that can build up inside the color glass condensate the same as the confining field that holds protons together in the first place? If so, observing the same field in different contexts could give us new insights into how gluons create it.

Where does proton spin come from?

Yet another mystery about quarks and gluons is how their spin contributes to the overall spin of their parent particles. All hadrons have spin, which is analogous to the rotational energy of a top spinning about its axis. Hadrons with differing spins precess and curl in different directions in the fields of powerful magnets.

Experiments probing a proton's spin show that quarks generate approximately 30 percent of the total. Wherein lies the rest of these hadrons' spin? The many-body picture of the proton as a seething sea of quarks and gluons immediately suggests that the rest of the spin might be contributed by gluons. But experiments smashing polarized protons (with their spins aligned with or opposite to their motion) into other polarized protons indicate that gluon spin constitutes only about 20 percent of a proton's spin—meaning 50 percent of the spin is still missing.

A celestial analogy illustrates a possible solution. The angular momentum of the solar system consists of the sum of the spin of planets about their axes as well as their orbital motion around the sun. Quarks, antiquarks and gluons confined within protons also undergo orbital motion. To understand how significant this orbital motion is, we must map out both the velocities and positions of quarks and gluons inside a proton. One of us (Ent) is involved in performing DIS experiments with very high intensity electron beams to do so. In level of detail, we are moving from snapshots toward 3-D movies of matter at subfemtometer (below one quadrillionth of a meter) distances.

Exotic states of matter

To understand the true nature of quark and gluon interactions, we must study them not just in the familiar configurations of protons, neutrons and other well-known particles but in all their possible forms. QCD allows for the existence of exotic hadron states beyond the familiar proton and neutron. Simulations suggest that additional colorless hadrons may exist, such as “glueballs” (which contain gluons exclusively), “molecules” composed of two quark-antiquark pairs or entities that are hybrids classified as quark-antiquark-gluon-bound states. Experimental evidence for these exotic hadrons is limited, with a few tetraquark molecule candidates identified so far. This situation may be about to change significantly, however, thanks to a number of experimental searches going on worldwide. Notably, a dedicated facility called GlueX is starting operations at the Thomas Jefferson National Accelerator Facility in Newport News, Va.

Physicists have recently discovered another extreme state of matter known as a quark-gluon plasma. It forms when atomic nuclei collide at close to light speed. Theorists suspect that when the speeding neutrons and protons of the two nuclei smash into one another, their color glass condensates shatter, breaking the confinement of quarks and gluons and releasing the condensates' energy to create a feverish swarm of quarks and gluons. This plasma is the hottest matter ever created on Earth, with a temperature of more than four trillion degrees Celsius. Strikingly, this material flows with almost no resistance—at least 20 times less than that of water.

The quark-gluon plasma bears a strong similarity to the early universe. Scientists at laboratories that have created such plasmas—Brookhaven National Laboratory's Relativistic Heavy Ion Collider and CERN's LHC—are now observing the world's smallest, most perfect fluids. By watching such plasmas as they cool, two of us (Ullrich and Venugopalan) and others are gaining insights into how the universe evolved. And by engineering the destruction of protons and neutrons into plasma in this way, researchers can study confinement in reverse in hopes of uncovering the secrets to how quarks and gluons stick together.

The way forward

Ideally, physicists would like to fully map out the locations, motions and spins of gluons and quarks within protons and neutrons. Such maps would help us calculate the contributions that quarks and gluons make to their parent particles' total mass and spin. These maps will provide unprecedented insight into the quark and gluon activities that bind protons and neutrons together. Constructing these images requires a quark-gluon femtoscope—a DIS tool akin to a microscope that would peer inward at the universe at scales as small as 1,000th the radius of a proton. In the U.S., Jefferson Lab and Brookhaven are seeking funding and approval for a femtoscope that would collide electrons with polarized protons and lead nuclei. In contrast to previous experiments that smashed fast-moving electrons at stationary nuclear targets, both types of particles will accelerate to near light speed in this machine before crashing head-on.

The electron-ion collider (EIC) project would achieve unparalleled levels of intensity, meaning the particles in the colliding beams would be packed together so tightly and in such high numbers that crashes would occur with higher frequency than ever before. The increase in crashes, up to 1,000-fold more than a previous DIS collider, would allow investigators to generate many individual snapshots of the innards of protons and neutrons.

Over the past four decades since the formulation of quantum chromodynamics, physicists have made some strides in explaining why the strong force behaves as it does and in understanding where the gaps in our knowledge of quark and gluon dynamics lie. Yet we have not filled in the missing pieces to create a simple and coherent story of how gluons glue. The technologies being developed today give us hope that by the time another 40 years roll around, we will have finally cracked the essential mystery of how matter, at its most fundamental level, is made.