THE LENS IN YOUR EYE casts an upside-down image on your retina, but you see the world upright. Although people often believe that an upside-down image in the eyeball gets rotated somewhere in the brain to make it look right side up, that idea is a fallacy. No such rotation occurs, because there is no replica of the retinal image in the brain—only a pattern of firing of nerve impulses that encodes the image in such a way that it is perceived correctly; the brain does not rotate the nerve impulses.

Even leaving aside this common pitfall, the matter of seeing things upright is vastly more complex than you might imagine, a fact that was first pointed out clearly in the 1970s by perception researcher Irvin Rock, then at Rutgers University.

Tilted View

Let us probe those complexities with a few simple experiments. First, tilt your head 90 degrees while looking at the objects cluttering the room you are in now. Obviously, the objects (tables, chairs, people) continue to look upright—they do not suddenly appear to be at an angle.

Now imagine tipping over a table by 90 degrees, so that it lies on its side. You will see that it does indeed look rotated, as it should. We know that correct perception of the upright table is not because of some “memory” of the habitual upright position of things such as a table; the effect works equally well for abstract sculptures in an art gallery. The surrounding context is not the answer either: if a luminous table were placed in a completely dark room and you rotated your head while looking at it, the table would still appear upright.

Instead your brain figures out which way is up by relying on feedback signals sent from the vestibular system in your ear (which signals the degree of head rotation) to visual areas; in other words, the brain takes into account head rotation when it interprets the table's orientation. The phrase “takes into account” is much more accurate than saying that your brain “rotates” the tilted image of the table. There is no image in the brain to “rotate”—and even if there were, who would be the little person in the brain looking at the rotated image? In the rest of the essay, we will use “reinterpret” or “correct” instead of “rotate.” These terms are not entirely accurate, but they will serve as shorthand.

There are clear limits to vestibular correction. Upside-down print, for instance, is extremely hard to read. Just turn this magazine upside down to find out. Now, holding the magazine right side up again, try bending down and looking at it through your legs—so your head is upside down. The page continues to be difficult to read, even though vestibular information is clearly signaling to you that the page and corresponding text are still upright in the world compared with your head's orientation. The letters are too perceptually complex and fine-grained to be aided by the vestibular correction, even though the overall orientation of the page is corrected to look upright.

Let us examine these phenomena more closely. Look at the square in a. Rotate it physically 45 degrees, and you see a diamond. But if you rotate your head 45 degrees, the square continues to look like a square—even though it is a diamond on the retina (the tissue at the back of the eye that receives visual inputs); vestibular correction is at work again.

The Big Picture

Now consider the two central red diamonds in b and c. The diamond in b looks like a diamond and the one in c looks like a square, even though your head remains upright and there is obviously no vestibular correction. This simple demonstration shows the powerful effects of the overall axis of the “big” figure comprising the small squares (or diamonds). It would be misleading to call this effect “context” because in d—a square surrounded by faces tilted at 45 degrees—the square continues to look like a square (though perhaps less so than when isolated).

You can also test the effects of visual attention. The figure in e is a composite. In this case, the central red shape is ambiguous. If you attend to the vertical column, it resembles a diamond; if you view it as a member of the group forming the oblique line of shapes, it seems to be a square.

Even more compelling is the George W. Bush illusion, a variant of the Margaret Thatcher illusion, which was originally developed by psychologist Peter Thompson of the University of York in England. If you look at the upside-down images of Bush's face on this page (f), you see nothing odd. But turn the same images right side up, and you see how grotesque he really looks. Why does this effect happen?

The reason is that despite the seamless unity of perception, the analysis of the image by the brain proceeds piecemeal. In this case, the perception of a face depends largely on the relative positions of the features (eyes, nose, mouth). So Bush's face is perceived as a face (albeit one that is upside down) just as an upside-down chair is readily identified as a chair. In contrast, the expression conveyed by the features depends exclusively on their orientation (downturned corners of the mouth, distortion of eyebrows), independent of the perceived overall orientation of the head—the “context.”

Your brain cannot perform the correction for the features; they do not get reinterpreted correctly as the overall image of a face does. The recognition of certain features (downturned mouth corners, eyebrows, and so on) is evolutionarily primitive; perhaps the computational skill required for reinterpretation simply has not evolved for this capability. For the overall recognition of the face simply as a face, on the other hand, the system might be more “tolerant” of the extra computational time required. This theory would explain why the second upside-down face appears normal rather than grotesque; the features dominate until you invert the face.

This same effect is illustrated very simply in the cartoon faces (g). Upside down, it is hard to see their expressions even though you still see them as faces. (You can logically deduce which is smiling and which is frowning, but that is not the result of perception.) Turn them right side up, and the expressions are clearly recognized as if by magic.

Finally, if you bend over and look between your legs at f, the expressions will become strikingly clear, but the faces themselves continue to look upside down. This effect is because the vestibular correction is applied selectively to the face but does not affect perception of the features (which are now right side up on the retina). It is the shape of the features on the retina that counts—independent of vestibular correction—and the “world-centered” coordinates that such corrections allow your brain to compute.

Depth Cues

Vestibular correction also fails to occur when we perceive shape (and depth) from clues provided by shading. In h, you see what appears to be a 550-foot-tall mound in the desert. The brain centers involved in computing shading make the reasonable assumption that the sun usually shines from above, so hills would be light on top and concave areas would be light on the bottom. If you rotate the page, you will see that this is actually a photograph of Arizona's Meteor Crater.

You can verify this effect by repeating the experiment of looking between your legs while the page is right side up in relation to gravity. Once again, the mound and crater switch places. Even though the world as a whole looks normal and upright (from vestibular correction), the modules in the brain that extract shapes from assumptions about shading cannot use the vestibular correction; they are simply not hooked up to it. This phenomenon makes evolutionary sense because you do not normally walk around the world with your head upside down, so you can afford to avoid the extra computational burden of correcting for head tilt every time you interpret shaded images. The result of evolution is not to fine-tune your perceptual machinery to perfection but only to make it statistically reliable, often enough and rapidly enough, to allow you to produce offspring, even if the adoption of such heuristics or “shortcuts” makes the system occasionally error-prone. Perception is reliable but not infallible; it is a bag of tricks.

Bobbing Heads

One last point: Next time you are lying on the grass, look at people walking around you. They look like they are upright and walking normally, of course. But now look at them while you are upside down. If you can manage yoga, you might want to try your downward dog or another inversion. Or just lie sideways with one ear on the ground. The people will still look upright as expected, but suddenly you will see them bobbing up and down as they walk. This motion instantly becomes clear because after years of viewing people with your head held straight, you have learned to ignore the up-down bobbing of their heads and shoulders. Once again, vestibular feedback cannot correct for the head bobbing, even though it provides enough correction to enable seeing the people as upright. You might be bending over backward to understand all this, but we think it is worth the effort.