A WORLD WITHOUT COLOR appears to be missing crucial elements. And indeed it is. Colors not only enable us to see the world more precisely, they also create emergent qualities that would not exist without them. The color photograph at the left, for example, reveals autumnal leaves in the placid water of a fountain, along with the reflections of trees and of a dark-blue afternoon sky behind them. In a black-and-white picture of the same scene, the leaves are less distinct, the dark-blue sky is absent, the reflections of the light are weak, the water itself is hardly visible, and the difference in apparent depth among the sky, trees and floating leaves is all but gone.

Yet this role for color, and even the true nature of color, is not well recognized. Many people believe that color is a defining and essential property of objects, one depending entirely on the specific wavelengths of light reflected from them. But this belief is mistaken. Color is a sensation created in the brain. If the colors we perceived depended only on the wavelength of reflected light, an object’s color would appear to change dramatically with variations in illumination throughout the day and in shadows. Instead patterns of activity in the brain render an object’s color relatively stable despite changes in its environment.

Most researchers who study vision agree that color helps us discriminate objects when differences in brightness are insufficient for this task. Some go so far as to say that color is a luxury and not really needed: after all, totally color-blind people and many species of animals seem to do well without the degree of color perception that most humans have. The pathway in the brain that serves navigation and movement, for example, is essentially color-blind. People who become color-blind after a stroke appear to have normal visual perception otherwise. Such observations have been taken as support for the insular nature of color processing, suggesting it has no role in processing depth and form—in short, that color is only about hue, saturation and brightness.

But the study of illusory colors—colors that the brain is tricked into seeing—demonstrates that color processing in the brain occurs hand in hand with processing of other properties, such as shape and boundary. In our decade-long attempt to discern how color influences perceptions of other properties in objects, we have considered a number of novel illusions, many created by us. They have helped us understand how the neural processing of color results in emergent properties of shape and boundary. Before we begin our discussion of these illusions, however, we need to recall how the human visual system processes color.

Pathways to Illusions

Visual perception begins with the absorption of light—or, more precisely, the absorption of discrete packets of energy called photons—by the cones and rods located in the retina [see box on next page]. The cones are used for day vision; rods are responsible for night vision. A cone photoreceptor responds according to the number of photons it captures, and its response is transmitted to two different types of neurons, termed on and off bipolar cells. These neurons in turn provide input to on and off ganglion cells that sit side by side in the retina.

The ganglion cells have what is called a center-surround receptive field. The receptive field of any vision-related neuron is the area of space in the physical world that influences the activity of that neuron. A neuron with a center-surround receptive field responds differently depending on the relative amount of light in the center of the field and the region around the center.

An on ganglion cell fires maximally (at a high rate) when the center is lighter than the surround, firing minimally when the receptive field is uniformly illuminated. Off cells behave in the opposite way: they fire maximally when the center is darker than the surround and minimally when the center and surround are uniform. This antagonism between center and surround means that ganglion cells respond to contrast and in this way sharpen the brain’s response to edges and borders.

Most of the ganglion cell axons (fibers) relay their signals to the brain, specifically to the lateral geniculate nucleus of the thalamus (near the center of the brain) and from there to the visual cortex (at the back of the brain). Different populations of ganglion cells are sensitive to somewhat different features of stimuli, such as motion and form, and their fibers conduct signals at different velocities. Color signals, for example, are carried by the slower fibers.

About 40 percent or more of the human brain is thought to be involved in vision. In the areas stimulated early in visual processing (parts of the visual cortex called V1, V2 and V3), neurons are organized into maps that provide a point-to-point representation of the visual field. From there, visual signals disperse to more than 30 different areas, interconnected by more than 300 circuits. Each of the areas has specialized functions, such as processing color, motion, depth and form, although no area mediates one perceptual quality exclusively. Somehow all this information is combined, in the end, into a unitary perception of an object having a particular shape and color. Neuroscientists do not yet understand the details of how this comes about.

Interestingly, bilateral damage to certain visual areas leads to deficits in the perception of form as well as color, which offers another piece of evidence that color is not disembodied from the other properties of an object. The intermingling of color signals in the brain with signals carrying information about the form of objects can result in perceptions not expected from an analysis of the wavelengths of light reflected from those objects—as our illusions make startlingly clear.

The Watercolor Effect

One of our early experiments with illusory color illustrates how important color can be in delineating the extent and shape of a figure. Under certain conditions, color changes in response to the surrounding color; it can become more different (called contrast) or more similar (called assimilation). The spreading of similar color has been described only over rather narrow areas, in agreement with the finding that most connections among visual neurons in the brain are relatively short range. We were therefore surprised to find that when an uncolored area is enclosed by two differently colored boundary contours—with the inner contour lighter than the outer contour—tint emanates from the inner contour, spreading across the entire area, even over rather long distances [see illustration on page 83].

Because the color resembles a faint veil such as that seen in watercolor painting, we call this illusion the watercolor effect. We found that the spreading requires the two contours to be contiguous so that the darker color can act as a barrier, confining the spreading of the lighter color to the inside while preventing it from spreading to the outside. The figure defined by the illusory watercolor appears dense and slightly elevated. When the colors of the double contour are reversed, the same region appears a cold white and slightly recessed.

The watercolor effect defines what becomes figure and what becomes ground even more powerfully than the properties discovered by the Gestalt psychologists at the turn of the 20th century, such as proximity, smooth continuation, closure, symmetry, and so on. The side of the double contour that has the lighter color fills in with watercolor and is perceived as figure, whereas the side that has the darker color is perceived as ground. This asymmetry thus helps to counteract ambiguity. The phenomenon is reminiscent of the notion of Edgar Rubin, one of the pioneers of figure-ground research, that the border belongs to the figure, not the ground.

A possible neural explanation for the watercolor illusion is that the combination of a lighter contour flanked by a darker contour (on an even lighter background) stimulates neurons that respond only to a contour that is lighter on the inside than the outside or to a contour that is darker on the inside than the outside, but not to both. Border ownership most likely is encoded at early stages of processing in the visual cortex, such as in brain areas V1 and V2. In experiments with monkeys, neurophysiologists have found that approximately half the neurons in the visual cortex respond to the direction of contrast (whether it gets lighter or darker) and therefore could delineate the border. These same neurons have a role in depth perception that might contribute to figure-ground segregation.

Our investigations showed that wiggly lines produce stronger watercolor spreading than straight ones do, probably because the undulating borders engage more neurons responsive to orientation. The color signaled by these uneven edges must be propagated across regions of cortex that serve large areas of the visual field, continuing the spread of color until border-sensitive cells on the other side of the enclosed area provide a barrier to the flow. Color and form are thus bound together inextricably in the brain and perception at this level of cortical analysis.

Radial Lines

The radial line illusion offers further evidence of the role color plays in distinguishing figure from ground. In 1941 German psychologist Walter Ehrenstein demonstrated that a bright circular patch conspicuously fills the central gap between a series of radial lines. The patch and the circular border delineating it have no correlate in the physical stimulus; they are illusory. The bright illusory surface seems to lie slightly in front of the radial lines [see top illustration on preceding page].

The length, width, number and contrast of the radial lines determine the strength of this phenomenon. The spatial configuration of the lines necessary for the illusion to take effect implies the existence of neurons that respond to the termination of a line. Such cells, called end-stopped neurons, have been identified in the visual cortex, and they may explain this effect. These local signals combine and become inputs to another (second-order) neuron, which fills in the central area with enhanced brightness.

In our studies of the Ehrenstein illusion, we evaluated variations in the number, length and width of the radial lines, and the examples we present in this article use the most striking combination that we found [see lettered illustrations]. We show four copies of each pattern, arranged as a quartet, to enhance the effects. Once we determined the characteristics for the radial lines that produced the brightest central circle (a, on preceding page), we experimented with variations in the chromatic properties of the central gap. First we added a black annulus, or ring, to the Ehrenstein figure, and the brightness of the central gap disappeared entirely—the illusion was destroyed, as Ehrenstein had already noticed. We suspect that this effect arises because the ring silences the cells that signal line terminations.

If the annulus is colored, however, other cells may be excited by this change. When we added a colored annulus, the white disk not only appeared much brighter (self-luminous) than it did in the Ehrenstein figure, it also had a dense appearance, as if a white paste had been applied to the surface of the paper (b). This phenomenon surprised us; self-luminosity and surface qualities do not ordinarily appear together and have even been considered to be opposing, or mutually exclusive, modes of appearance. We call this phenomenon anomalous brightness induction. As occurs with the watercolor effect, cells in early cortical areas are candidates for causing this illusion.

Next we inserted a gray disk into the central gap of an Ehrenstein figure (c). Another phenomenon, called scintillating luster, arose, in which illusory brightness gives way to the perception of a glossy shimmer that occurs with each movement of the pattern or of the eye. The scintillation, or flashing, may come about by a competition between the on and off systems: line-induced brightness (illusory increment) competes with the dark gray of the disk (physical decrement). When we replaced the central white disks within the colored ring with black disks and used a black background (d), the disks looked even darker than the physically identical surrounding area. Instead of appearing self-luminous, as white disks do, blackness seems to generate a void, or a black hole, that absorbs all the light.

When the central disk within the chromatic ring was gray instead of white or black, the disk appeared to become tinted with the complementary color of the annulus—for example, greenish-yellow when the surrounding ring was purple (e). Furthermore, the disk appeared to flash with each eye movement, or when the pattern moved back and forth, and to move in relation to its surround. Flashing anomalous color contrast depends on radial lines and a chromatic annulus the way the other effects do, but it also has unique qualities that do not appear to be a simple combination of the other known effects. In this illusion, the induced color appears both self-luminous and scintillating. Strikingly, it appears to float above the rest of the image. The surface color and the self-luminous color do not mix; instead one belongs to the disk on the page, and the other emerges from a combination of the other characteristics of the stimuli.

In flashing anomalous color contrast, the radial lines may activate local end-stopped neurons, as has been proposed for the filling in of gaps by illusory contours, but activity by those cells does not account completely for the combined flashing and complementary color. It is not clear whether the radial lines have a direct effect on color contrast or whether the vividness of the color is derived indirectly from the luster and scintillation caused by the combination of radial lines and the gray center.

Current understanding of the brain cannot explain all the things going on in this illusion. The complexity of the illusion suggests that it is unlikely to result from a single unitary process but may represent an attempt by the brain to reconcile competing signals from multiple specialized pathways. Scientists clearly have much more to learn about how the brain perceives the physical world. Fortunately, ongoing work on illusory colors will continue to offer a tantalizing portal into the complexities of the human visual system.