ALL PRIMATES, including humans, have two eyes facing forward. With this binocular vision, the views through the two eyes are nearly identical. In contrast, many other animal groups, especially herbivores such as ungulates (hooved animals, including cows, sheep and deer) and lagomorphs (rabbits, for example), have eyes pointing sideways. This perspective provides largely independent views for each eye and an enormously enlarged field of view overall. Why did primates sacrifice panoramic vision? What benefit did they gain?

We know binocular vision evolved several times independently in vertebrates. For example, among birds, predatory species such as owls and hawks have forward-pointing eyes. One theory is that the feature conferred a statistical advantage—two eyes are better than one—for detecting and discriminating objects, such as prey, in low light levels. But whatever the original reason for its emergence, the evolutionary novelty afforded a huge advantage: stereoscopic (literally, solid) vision.

Shifting Views
How does it work? Even though both your eyes point forward, they are separated horizontally so that they look at the world from two slightly different vantage points. It follows that each eye receives a slightly different picture of the three-dimensional scene around you; the differences (called retinal disparities) are proportional to the relative distances of the objects from you. Try this quick experiment to see what we mean: hold two fingers up, one in front of the other. Now, while fixating on the closer finger, alternately open and close each eye. You’ll notice that the farther the far finger is from you (don’t move the near finger), the greater the lateral shift in its position as you open and close each eye. On the retinas, this difference in line-of-sight shift manifests itself as disparity between the left and right eye images.

A simplified example shows this effect clearly. When you look at the pyramid, the right eye sees more of the right side than the left eye does, and vice versa; it is a simple consequence of geometric optics. Notice that the images in the two eyes are correspondingly different; the inner square is shifted right or left. This retinal disparity is proportional to the height of the pyramid. The brain measures the difference and experiences it as stereoscopic depth.

Although this explanation seems patently obvious today, it wasn’t elucidated until the 19th century. Leonardo da Vinci attempted to explain it several hundred years earlier and correctly observed that because the eyes normally receive different views of a 3-D scene, it is impossible, even in principle, to convey a full sense of 3-D on a 2-D canvas. Leonardo puzzled over how we can see a single world of solid objects given the different eye views (now known as Leonardo’s paradox), but he failed to grasp the critical point that retinal disparity is not a problem but is the basis for stereopsis.

This fact was finally made clear in 1838 by English physicist Charles Wheatstone, who published an elegant series of experiments on binocular vision. Recognizing the difference in perspective of the left and right eyes, he began by making line drawings of each eye’s view of simple objects. Then, employing a device he invented, called a mirror stereoscope, he presented these line drawings together to the viewer: left view to left eye alone; right view to right eye alone. Imagine his astonishment—and delight!—when he saw the skeletal outline of the object spring into 3-D relief, looking like he could almost reach out and grab it. It must have been the same sense of wonder every child experiences when playing with a stereo viewer such as the familiar View-Master. It seems like magic.

But how exactly does the brain blend the two eyes’ slightly different pictures harmoniously into a single fused picture? And how does it measure and extract the differences to allow for seeing in stereo? On one hand, it needs to unify the pictures; on the other hand, it needs to preserve and measure their differences.

Consider what happens when you fixate on an image, such as a letter—the X on this page—with both eyes. Images of the letter project to the central part of each retina (the fovea), and the brain fuses them into one. You see one X, not two. English physiologist Charles Sherrington suggested in the early 20th century that this blending was a mysterious psychological process occurring in the mind, requiring no actual confluence of messages into a single brain area. We must not confuse mental fusion with physiological fusion, he urged.

We now know he was wrong: binocular fusion is a physiological process. The X, or any point on which you fixate, falls on what is, functionally and geometrically, termed corresponding retinal points. In fact, any point from an entire plane (or, strictly speaking, from a slightly curved surface), centered on the fixation point, would stimulate corresponding retinal points and be seen as a single object (any letter on this page, not just the one on which you fixate, appears singly). As neuroscientists David Hubel and Torsten Wiesel of Harvard University discovered in a series of groundbreaking experiments in the 1960s, individual cells in the visual cortex, so-called binocular cells, receive input from both eyes, specifically from corresponding retinal locations, thus providing a mechanism for perceptual fusion.

Yet if binocular neurons were only excited when identical input arrived from both eyes, you would have trouble perceiving real 3-D objects. John “Jack” Pettigrew, then a young medical student in Canberra, Australia, noted this fact in the mid 1960s, reasoning that the neural mechanism for stereopsis must entail another set of binocular neurons, ones that signal retinal disparity by processing noncorresponding retinal points.

What Pettigrew (along with his colleagues Horace B. Barlow, Colin Blakemore and Peter Bishop) found was that Hubel and Wiesel’s description was only partially correct. Sure enough, corresponding points from the retinas send signals that converge on single neurons in the visual cortex. It is as if there is a map of each eye’s image in the brain, and these maps are in registration (speaking anatomically); that arrangement makes sense overall. But many noncorresponding points also converge on and activate binocular cells. It is these neurons that signal stereo depth because they are, in effect, measuring the horizontal scatter between the left and right eye images. As a consequence, what you have even at this early stage is not a flat 2-D map of the world on the cortex but a 3-D map. This fact was probably the most important discovery about binocular vision since Wheatstone’s insight.

Of course, we have progressed much since Wheatstone’s days. Instead of drawings, we can mimic the two eyes’ views using a camera. Look at any 3-D scene and take one picture from the left eye’s vantage point. Then shift the camera to the right eye’s location and take a second picture. Print the two photographs, place a vertical partition so that each eye gets only its own image and, lo and behold, the image transforms into a 3-D scene. (See the example at a.) Such stereograms were highly popular in Victorian drawing rooms (they were carefully stashed away if they were pornographic, proudly passed around at family gatherings if they were travel series).

The best way to view them is through a stereoscope, which incorporates lenses and prisms or mirrors for more natural accommodation and convergence. But you can try the rudimentary partition method just discussed. With some practice, you can get the eyes aligned to fuse the images and see stereo depth. It is well worth the effort.

Pendulum Play
Another stereo illusion you can construct and experience is the Pulfrich effect, described, ironically, by the famous one-eyed scientist Carl Pulfrich in 1922 (experimenting on others, of course). Hang a weight on the end of an 18-inch string and set it in motion like a pendulum, moving back and forth horizontally in a single plane (its speed gradually accelerates as it approaches the center and decelerates again as it reaches the top at the other end). Now put a filter (sunglasses will do) in front of one eye alone. Astonishingly, you will see the pendulum making an elliptical 3-D excursion toward and away from you! With a left eye filter, motion will be clockwise, as seen from above; counterclockwise with a right eye filter. And the darker your glasses, the greater the depth of the ellipse you will see. Remove the filter, and it goes back to the 2-D swing of a regular pendulum.

The effect occurs because the filter reduces the luminance of the pendulum on the one retina, producing a slight delay in transmission to the binocular cells in the visual cortex. This delay means the pendulum’s dim image is “assumed” by the brain to lag behind spatially—as if noncorresponding points were stimulated—thereby fooling the brain into thinking the pendulum is moving in 3-D. The greater the velocity of the pendulum (for instance, during midflight), the greater the three-dimensionality experienced, hence its elliptical path in 3-D.

It has been a long journey from Leonardo, Wheatstone and Victorian parlor toys to modern physiology and psychophysics, but we have barely begun to understand the subtleties of binocular vision. In the next issue we will explore this theme further.

Note: This article was originally printed with the title, "Seeing in Stereo."