ILLUSIONS ARE ANOMALIES that can reveal clues about the mysterious workings of the brain to neuroscientists in much the same way as the fictional Sherlock Holmes can solve a crime puzzle by homing in on a single out-of-the-ordinary fact. Think of the phrase “the dog that did not bark” (in Sir Arthur Conan Doyle’s short story “Silver Blaze”) or of the missing dumbbell (in Conan Doyle’s novel Valley of Fear).

Perhaps the most famous examples of such visual tricks are the geometric optical illusions. In the Ponzo illusion (a), first demonstrated by Italian psychologist Mario Ponzo in 1913, one horizontal line looks shorter than the other one, although they are identical. In the Müller-Lyer illusion (b, on page 46), created by German psychiatrist Franz Müller-Lyer in 1889, the line bounded by the diverging arrowheads looks shorter than the one with converging arrowheads—although they, too, are identical.

These illusions are very familiar yet powerful; knowledge of true line length does not stop or diminish their effect. Do we have any idea what causes them? Why would the visual system persist in committing an error, in perceiving incorrectly something so simple even when we consciously know it is a trick? Before we explore those questions, let us introduce two more eye puzzles.

In d, on page 47, we have a field of shaded disks that are seen as eggs dispersed among cavities. The disks that are light on top look like bumps or eggs, the others like cavities. This sense of depth comes from a built-in tendency for your visual system to assume that light shines from above (after all, we evolved on a planet with a single sun overhead). So the brain interprets the disks that are lighter on top as rounded like eggs and the light-on-bottom ones as cavities (because a hollow would be light on its bottom if lit from above). In e, on page 47, the shading gradient changes from left to right, and the depth is far less compelling (the tokens seem flatter) and more “bistable” (individual disks are equally likely to be seen as convex or concave, and the light source can be seen as arising from either side).

So far so good. But we also noticed that the perceived gradient of lightness—the apparent difference in brightness between the lightest and darkest parts of each disk—seems shallower for the spheres than for the craters. The brightness gradient also appears less steep for the light-on-top disks than for the light-on-side disks. Why? The physical gradient is exactly the same for each of the shaded disks (to convince yourself, rotate the paper).

Constancy Connection

These two sets of illusions, the geometric optical illusions and the gradient-steepness type, seem completely unrelated. But both reveal a basic principle in vision called perceptual constancy. This effect is the tendency to observe correctly an object as having constant physical attributes (size, shape, color, lightness, distance, and so on) despite tremendously variable retinal images that may occur for that object, which arise from changes in vantage point, distance, illumination and other variables. This point is not trivial. Unlike a video camera, our brains do not merely “read out” the retinal image to perceive an object. Rather we interpret it based on knowledge and context. For instance, constancy guides us despite changes in lighting. Believe it or not, the black ink of a newspaper has a higher absolute luminance (the physical light intensity measured by a photometer) when viewed in sunlight than white paper does when viewed in a well-lit room at night. Yet we recognize the true character of the objects and their comparative brightness: despite lighting conditions, we experience it as black type on white paper and do not—in fact, cannot—perceive the absolute luminance.

Another example, more relevant to our geometric illusions, is size constancy, or the tendency to identify an object as being constant in size whether it is near or far. If you watch a person running toward you, his image on your retina enlarges, although you do not see him expanding. Your brain unconsciously takes into account the distance and interprets size correctly. Similarly, if a person is lying on the ground with his feet extended toward you, the retinal image of his feet is twice the size of his head, but you do not see a microcephalic with giant feet. You see a normally proportioned person with his feet closer to you than his head.

But how does size constancy explain our geometric illusions? The phenomenon arises from a depth cue, called linear perspective, with which every visual artist is familiar. An object of constant size will throw a smaller image on your retina as it moves farther away. This shrinkage is just a simple consequence of optics; it has nothing to do with perception. Now consider what happens when you stand in the middle of parallel railway tracks and cast your gaze along their length. The rails remain parallel and the ties between them a constant size along their length, yet the resulting retinal image or indeed any 2-D projection of the tracks, such as a photograph or line drawing, shows the space between the rails and the corresponding size of the ties shortening with increasing distance. Again, this result is from simple optics, not perception. In the perceptual world, our brain largely corrects for this linear perspective, and we interpret the railroad as straight and parallel and the ties as being of a constant size. You correctly attribute the size changes to distance, not to changes in size.

Coming Together

Now take another look at the Ponzo illusion. Consider the converging lines; like railroad tracks, they suggest parallel lines extending far into the distance. Like the railroad ties, the horizontal segments are interpreted in the context of these converging lines and thus are seen to exist at different distances. In the Ponzo illusion, however, the two horizontal segments are drawn to be exactly the same length (unlike railroad ties, which get smaller with distance). Because they are interpreted in the context of converging lines and appear to lie at different distances, the brain applies a constancy correction, so that the top line looks longer than the bottom one. It is as if the brain reasons: “One horizontal line is farther away, so if it is the same physical length as the other horizontal line it should cast a smaller image in my eye. But because the image is the same size, it must be produced by a longer line that is farther away.” This correction occurs even though the viewer may not have any sense of depth from the converging lines.

Because the top line is deliberately drawn to be the same length as the bottom one, the brain misapplies this constancy rule, and you perceive it as looking abnormally long. The exact converse happens for the bottom line; it looks artificially short. Neuropsychologist Richard L. Gregory of the University of Bristol in England refers to this phenomenon as inappropriate constancy scaling. Your visual modules, concerned with depth, distance and size, perform the task on autopilot, without your conscious cogitation. Even if I use a ruler to show you that the two lines are the same, this high-level, conscious knowledge cannot “correct” what is signaled from the bottom up by constancy mechanisms.

Gregory has also proposed a delightful size- constancy explanation for the Müller-Lyer illusion (b). He points out that the contours of this illusion are identical to the contours one encounters when viewing the outside edge of a building or the inside corners of a room (c). In this two-dimensional projection of a three-dimensional world, the inside corner of the room is seen as farther away; size scaling is triggered and produces the misperception of different line lengths. As with the Ponzo illusion, whereas depth is implied by this figure, it need not be consciously experienced. The perspective lines, Gregory proposes, directly set constancy scaling, so judgments of distance are unnecessary.

Let us now return to the eggs and cavities. We have explained the illusion of depth as being based on a built-in assumption that the light is shining from above. But why do the top-lit eggs look more uniform in surface reflectance (lightness) compared with the side-lit disks or the bottom-lit cavities? Here we need to invoke the analogous phenomenon of lightness constancy—the ability of the brain to extract the true reflectance of an object’s surface, instead of variations in luminance caused by illumination.

First, consider a light-on-top egg. The brain assumes the sun is above you, and a real egg would convey exactly this pattern of luminance variation—a gradient of luminance decreasing gradually from top to bottom. So you see it as an egg or bump rather than as a flat, shaded disk; it is the “best-fit” hypothesis. But then the brain says, in effect: “The variation in luminance—light on top—is obviously not from the object itself but because of the way it is illuminated from above, so I will see it as uniform in reflectance.” This effect of lightness constancy implies that if you did not see depth in the display, there would be no lightness constancy and you would in fact see the top as being much lighter and the bottom much darker than they seem now.

Now why does not the same argument apply to the light-on-side eggs seen in e, especially given that the luminance gradient is exactly the same? It is because the brain is not used to sideways illumination. Consequently, the impression of depth is weaker, and the correction for luminance variation (lightness constancy) is correspondingly weaker. The gradients of perceived lightness therefore appear steeper than they do for the top-lit eggs in d.

The same reasoning applies to the cavities. Because of the phenomenon of interreflection (light bouncing off the walls of the interior of a true cavity, partially nulling the gradient produced by illumination), the brain “expects” a smaller illumination gradient in cavities than in eggs. So it only weakly applies the constancy correction to the former. This milder correction would be sufficient in the real world, but the shading of the artificial cavities in d is physically identical (though inverted) to that of the eggs. Thus, the perceived gradient of lightness is higher than it is for the eggs. A second reason is that cavities are less common than bumps, and therefore the visual system is less adept at this constancy correction.

We have presented these complex arguments to emphasize that even extraordinarily subtle aspects of the statistics of the world are built into the visual system as rules. We can devise extremely simple displays from which we can use clues—like Sherlock Holmes—to help solve the mystery of visual perception.