HUMANS, LIKE ALL PRIMATES, are highly visual creatures. Most of the back of our brain is devoted to visual processing, and half of the cortex is involved with sight. In addition, when visual inputs conflict with clues from other senses, vision tends to dominate. This supremacy is why, for example, ventriloquists are so compelling. We see the dummy talking, and we are fooled into hearing the voice coming from it—a case of what scientists call “visual capture.” (With eyes closed, however, we can correctly localize the dummy voice to the ventriloquist.)

If information from vision and touch are incompatible, visual dominance may cause us to actually feel things differently than if we relied only on touch (without looking).

Curved Touch

In a simple but striking demonstration by James Gibson in the 1930s, a subject is first presented with a short, straight metal rod and asked to feel it with his eyes closed. Of course, he correctly feels it is straight. He then lets go of the rod and is asked to open his eyes and look down at it. Unbeknownst to him, it is the same rod but viewed through a wedge prism, which causes the rod to appear curved rather than straight. Not surprisingly, he now reports seeing a curved rod. But what happens when he reaches out and touches the rod while looking at it? Subjects reported nothing unusual: they noticed no rivalry, instability or averaging between the senses; the rod that they saw as curved they simply also felt as curved.

In short, vision redirects the tactile perception so that no conflict is experienced. Similarly, perception researcher Irvin Rock, then at Yeshiva University, showed in the 1960s that when shape or size perception for single simple objects was made to conflict between the senses (by the introduction of distorting lenses), perception conveyed by active touch was modified to conform to visual perception.

Yet another example of vision influencing touch occurs in patients with phantom limbs. After amputation of an arm, the vast majority of patients continue to feel vividly the presence of the missing arm, a phenomenon termed phantom limb in the late 1800s by physician and author Silas Weir Mitchell. Many people report that their phantom limb is frozen, paralyzed in a constant or fixed position, and that this experience is sometimes painful.

We wondered whether touch sensations in the phantom arm could be influenced by visual input. We positioned a mirror on the table in front of a patient, along his midline, and asked him to position his intact arm and stump/phantom hand symmetrically on either side of the mirror (a). When he looked at the reflection of his normal hand in the mirror, he experienced the phantom being visually resurrected. Remarkably, if the patient moved his normal hand while looking at its reflection in the mirror, the previously frozen phantom seemed to become animated; he not only saw the hand but also felt it move. In some cases, this sensation seemed to alleviate the pain associated with the phantom.

The visual-capture effect also indicates our need for a single, sensible narrative of the world. That is, we (our brains) tend to reinterpret or discard some information, even when doing so may produce errors or illusions (as with the ventriloquist). This influence of vision has resulted in a kind of vision chauvinism in research, leading scientists to pay less attention to the other senses.

Touched in the Head?

The neural basis of these intermodality illusions has not been studied in detail. Recent work by Krish Sathian of Emory University and Alvaro Pascual-Leone of Harvard University suggests that somatosensory signals (those having to do with touch) may be seen in the primary visual cortex under certain circumstances—for example, in blind Braille readers. The tactile signals processed in the somatosensory centers of the brain may actually send feedback all the way to the very early stages of visual processing instead of being merely combined at some higher level. Studies on visual capture suggest that the converse may also be true—namely, that visual input may project to what is traditionally considered primary somatosensory cortex. These interactions between the senses, in addition to educating us about brain mechanisms for information processing, may also provide a useful tool in rehabilitation for neurological disorders.

We would like to consider here some tactile illusions that bear a striking similarity to visual illusions. Try the following experiment. Place two coins in your freezer till they are chilled (maybe 20 minutes). Remove them and place them on a table flanking a similar coin that has been kept at room temperature, so that the three coins now form a row. Now place the tips of the index and ring finger of one hand on the two cold coins and the middle finger on the middle coin. Amazingly, the middle finger feels equally cold. Perhaps the temperature-sensing pathways of the brain simply do not have the resolving power to discern two discrete sources. Yet the middle finger does not feel cold unless it is in contact with a neutral coin; if there are no tactile sensations emerging from it, the brain is reluctant to “fill in,” or ascribe cold to, this region.

But how clever is this filling-in mechanism? What if the middle finger pressed against velvet or sandpaper rather than a coin? Does it have to be similar to what is being touched by the index and ring fingers? If so, how similar? And does this interpolation of cold occur early in sensory processing—for example, in the spinal cord or thalamus (the “gateway” for sensory in-puts to the brain)? Or does it take place “higher up,” in later processing stages in the brain?

One way to find out is to see what happens if you simply bend the middle finger upward and then put the middle finger of the other hand in its place. The illusion now disappears, suggesting that the filling in occurs at an early stage of tactile information processing, not at the higher level of space representation in the brain. (We know this occurs at an early stage because the sensory signals from two hands project to two separate hemispheres in the brain; information from them can be compared only at a relatively late stage of processing.)

What if the two outer coins were very hot and icy cold, respectively; would the middle coin take on the average temperature, or would it alternate between the two? What about an intermediate case? Say you crossed the index finger under the middle digit so that you formed a row with the index between the ring and middle fingers, the middle and ring fingers resting on the cold coins. Would the index finger now feel cold because of its intermediate location in space?

The reader might wish to dream up his or her own experiments: that is what makes the study of perception so much fun. You do not need to be an expert to do experiments that have far-reaching implications. If you attempt such an experiment, we would love to hear from you.

Let us try something different. Cross your left middle finger over your left index finger, making a small V at the end. Now close your eyes and place the V formed by the fingers on your nose (b). Astonishingly, many people who perform this “Aristotle illusion” maneuver report a distinct feeling of having two noses! How is this effect possible?

One way to interpret the phenomenon is to realize that given the normal, habitual spatial arrangement of the fingers, the only way the left side of your left middle finger will be stimulated simultaneously with the right side of your left index finger is when they are touching two objects. So the brain interprets the tactile experience as “I must have two noses.” According to psychologist Stuart M. Anstis of the University of California, San Diego, the nose is not the only appendage in which perceptual doubling can be produced.

Last, look at the visual illusion above (c). Believe it or not, the middle disk in the left panel of circles is the same size as the one on the right, but the left looks larger because it is surrounded by small disks. This optical trick is a powerful demonstration of the contextual nature of perception. (The skeptical reader may make a cardboard occluder with two holes to directly compare the two.) Is there an equivalent of this effect for touch?

Jelly or Velvet?

The following demonstration may be a related effect. Get some coarse chicken-cage mesh, preferably mounted in a wooden frame. Then hold the mesh between the palms of your hands. Nothing peculiar so far. Now start rubbing your palms against each other with the wire between them. Remarkably, your palms will feel like jelly or velvet. The cause of this striking illusion has yet to be determined. One possibility is that it has something to do with sensing and signaling the contrast between the sharp wire and the “neutral” touch sensations on the skin—the opposite of sharp being velvety or jellylike. A version of this illusion can be found in many science museums.

You can even get your hands to “float”—a well-known trick, sometimes called the Kohnstamm effect, reintroduced to us by our son, Jayakrishnan Ramachandran. Stand in the middle of an open doorway and use your arms to apply outward pressure on the two sides as if you were pushing them away from your body. After about 40 seconds, suddenly let go and relax, stand normally and just let your arms hang by your sides. If you are like most of us, your arms will involuntarily rise up as if pulled by two invisible helium balloons. The reason? When you apply continuous outward force, your brain gets used to this as the “neutral state”—so that when the pressure suddenly disappears, your arms drift outward.

This simple demonstration shows that the sensory areas of your brain are not the passive recipients of signals from your sense organs. Instead we should think of them as being in a state of dynamic equilibrium with the outside world, an equilibrium point that is constantly shifting in response to a changing environment.