The serial number of a human specimen is the face, that accidental and unrepeatable combination of features.
— Milan Kundera,
Immortality,1988
Faces are the glue that holds us together and that gives us our identity. All of us but the visually impaired and blind are experts at recognizing people's identity, gender, age and ethnicity from looking at their faces. First impressions of attractiveness or competence take but a brief glimpse of somebody's face. Newly born infants already tend to fixate on faces. This bias also turns up in art. Paintings and movies are filled with faces staring at the viewer. Who can forget the endless close-ups of the feuding husband and wife in Ingmar Bergman's Cimmerian masterpiece Scenes from a Marriage?
Because recognizing a face is so vital to our social lives, it comes as no surprise that a lot of real estate in the cerebral cortex—the highly convoluted region that makes up the bulk of our brain—is devoted to a task crucial to processing faces and their identity. We note whether someone looks our way or not. We discern emotional expressions, whether they register joy, fear or anger. Indeed, functional brain imaging has identified a set of adjacent regions, referred to as the fusiform face area (FFA), that are situated on the left and the right sides of the brain, at the bottom of the temporal lobe of the cerebral cortex. The FFA turns up its activity when subjects look at portraits or close-ups of faces or even when they just think about these images.
Two just published studies of the brain's visual networks, including the FFA, enlarge what we know about the physical basis of face perception. Both explore the unique access to the brain afforded by patients whose epileptic seizures have proved resistant to drugs. A surgical treatment finds the locations in the brain where the hypersynchronized activity that characterizes a seizure begins before spreading from its point of origin to engulf one or sometimes both hemispheres. If a single point—a focus where the seizure begins—can be found, it can be removed. After this procedure, a patient usually has significantly fewer seizures—and some remain seizure-free. To triangulate the location of the focus, neurosurgeons insert electrodes into the brain to monitor electrical activity that occurs during a seizure.
This clinical setup is the starting point for these two related but quite different studies that provide fascinating new details about whether the brain, like a camera, captures a literal rendition of a face or whether that image is synthesized in the brain by neurons in the cortex.
.jpg)
Prez 42 Morphs into Prez 43
To describe the first experiment, it is best to re-create what happened to the subjects. Keep your eyes steady on the red square in the top panel of the figure at the right for a fraction of a minute. Out of the corner of your eyes, you will see Bill Clinton on the left and his successor on the right. Now quickly shift your gaze to the bottom red square and note what you see. Don't hesitate. Just go for it! Most people see George W. Bush in the image on the left and his predecessor on the right. Yet when you compare the two photographs, you will realize that they are the same, a morphed image of the two presidents. Call this hybrid Clintush, the 42nd and a half president. This illusion is an instance of a general class of phenomena, called sensory adaptations, that are a hallmark of the mind. As you stare at the face, the neuronal mechanisms supporting its perception undergo a process of recalibration. The longer you stare at the same image, the more it changes. So when you look for a while at Clinton and then quickly glance at Clintush, you will perceive Bush, although this illusory perception quickly dies away, and the picture becomes ambiguous again.
How do the myriad nerve cells that make up the visual brain respond to such images? Neurons early on, say, in the eye, will respond to the chiaroscuro patterns of the photographs no matter what the brain the eye is attached to sees. That is, they register an image of the outside world. But somewhere in the upper reaches of the brain, there must be neurons that actively construct what the mind's eye sees when looking at Clintush. And depending on circumstances, that can be a picture of Bush or of Clinton.
The study was carried out by Rodrigo Quian Quiroga of the University of Leicester in England and Alexander Kraskov and Florian Mormann, all then members of my laboratory at the California Institute of Technology, where I was a professor. The project was supervised by neurosurgeon and neuroscientist Itzhak Fried of the University of California, Los Angeles, and myself. Fried implanted hair-thin wires in the brains of patients at the David Geffen School of Medicine at U.C.L.A. The wires enabled the researchers to monitor the electrical activity of individual nerve cells in the medial temporal lobe and to detect neurons that would respond more to the perception of one person than to that of another.
Years earlier, while using this very setup, we had discovered individual neurons that responded only to various pictures or drawings of specific people that the patient was familiar with—politicians, celebrities or family members. One of the first such neurons to be characterized in this manner fired only when the patient saw images or cartoons of Clinton. Another cell became excited by photographs of Hollywood actor Jennifer Aniston, no matter what her dress or hairstyle was. Indeed, this class of neurons now bears her name. Some neurons not only respond to pictures of a particular, familiar individual but also to his or her written or spoken name.
For the present experiment, the investigators recorded from a cell that fired selectively to a picture of Clinton. When they next asked the patient to look for four seconds at a photograph of Bush, the cell remained nearly silent. That is, the cell preferred the sight of Clinton over that of Bush. Immediately afterward, a blended picture of Clintush was flashed on the screen, and the patient had to push one of two buttons: either “I saw Clinton” or “I saw Bush.” As probably happened when you performed this experiment earlier, the scientists found that the patient was more likely to see Clinton when previously exposed to four seconds of Bush, and vice versa.
How do the recorded neurons act? Do they care only about what is in front of the eyes out in the world, or do they tap into the image consciously formed in the patient's mind? If the former is true, they should fire equally strongly to the ambiguous, blended picture, no matter whether the patient perceives Clinton or Bush. But if these cells follow the patient's perception—whether looking at the defined images on top or Clintush on the bottom—they should respond only when their preferred stimulus is actually experienced in the mind's eye. That is, a Clinton cell should not fire when the morphed picture is perceived as Bush but should be active when the identical morph is seen as Clinton. And this is what happened.
In 62 neurons from 10 patients, the response to the morphed picture was significantly stronger when the patient recognized the neuron's preferred face, compared with when the patient reported the face that the neuron did not care about. Indeed, statistically, the strength of the neuron's response to a pure Clinton could not be distinguished from the response to Clintush, as long as the subject reported seeing Clinton (and the reverse for the other image—here Bush). That is, these neurons, located in a part of the brain that received input from the FFA, either directly participated in the mental decision “Clinton” or “Bush” or actually generated the conscious experience of the face.
To distinguish between these two possibilities, it would be necessary to entice these neurons to fire by some artificial means—remember the technology used in The Matrix—and then to ask the subjects if they saw something. Another approach would interfere with the neuronal firing activity to determine whether this perturbation would affect the patient's experience of faces.
Moving from Correlation to Causation
This second option inspired neurologist Josef Parvizi, psychologist Kalanit Grill-Spector and their colleagues at Stanford University to conduct a study of 10 epileptic patients. One of those patients, Ron Blackwell, came to Parvizi's clinic at Stanford when the drugs he had been taking since childhood could not control his seizures anymore. To localize the source of his seizure and identify which nearby tissue could be surgically removed without major loss of function, Parvizi's team implanted so-called subdural intracranial electrodes into Blackwell's brain that would not only monitor neuronal activity but could also apply electric current and so stimulate the adjacent part of the brain.
Both electrical mapping with the implanted electrodes and the more conventional functional whole-brain imaging in a magnetic scanner identified a cluster of regions in the FFA in both cortical hemispheres of Blackwell's brain that responded strongly to faces. Knowing the location of these face-selective regions gave the clinicians a unique opportunity to test what Blackwell would experience when the current from the electrode interfered with normal electrical activity connecting these face-selective networks of neurons in his brain.
Parvizi can be heard in a video, talking to the patient, “Look at my face and tell me what happens when I do this.” On the first trial, the physician pretends to inject current, and Blackwell just shakes his head and mutters, “Nothing.” But when a four-milliampere current is sent through the electrodes, he says, “You just turned into someone else. Your face metamorphosed. Your nose got saggy and went to the left. You almost looked like somebody I'd seen before but somebody different. That was a trip.” (Go to www.jneurosci.org/content/32/43/14915.full to watch the video of Blackwell and Parvizi.)
These perceived facial distortions occurred in all seven real trials but not in the four sham trials. They were specific for faces and did not happen when Blackwell was told to look at a television screen. Also, not much happened when electrodes in a neighboring region were stimulated. “Only your face changed. Everything else was the same,” Blackwell emphasizes. Repeating this procedure in nine other patients revealed the same result.
Yet something else became more and more apparent—a striking left-right asymmetry was operating. Only stimulation of electrodes underneath the right fusiform gyrus—never its left counterpart—induced distortions in face perception. Electrical stimulation of the left fusiform gyrus caused either no perceptual change or more low-level changes such as twinkling and sparkling, traveling blue and white balls, flashes of light—so-called phosphenes—but no change in the character of the perceived face.
The faces retained their identity even though functional brain imaging and electrical recordings had pinpointed both left and right FFA circuits as responding more to faces over nonfaces. Thus, the second lesson this beautiful experiment teaches involves the potential pitfalls and perils of homing in on a brain region or a nerve cell and inferring that because it correlates with seeing faces or with recalling bad experiences or with making decisions, it must therefore be involved in face perception, in memory or in decision making. There is a reason scientists constantly preach that “correlation is not causation.”
Untangling the tightly woven neuronal tapestry to discover what is real is one of the challenges scientists confront when crossing the brain-mind divide, linking the physics of excitable matter to ephemeral subjective, conscious experience, the most real thing there is.
