Julesz’s demo inspired a young medical student, Jack Pettigrew (then at the University of California, Berkeley), to look at the physiology of binocular nerve cells in the earliest stage of binocular processing. Until then, the problem of stereoscopic vision seemed intractable because, if von Helmholtz were right, researchers would have had to tackle the physiology of form perception first—about which no one had the foggiest idea how to proceed. Pettigrew found, however, that his hunch was right—these cells were extracting the horizontal shifts and signaling stereo (as we discussed in our previous column).
That is the simple story, but the picture got more complicated when a student (Ramachandran) from India found that in some circumstances form perception preceded stereo, showing the flexibility of the brain’s visual centers. He created a stereogram that had a texture-defined square in each eye. He then shifted this entire square instead of shifting the dots that defined the textures.
He had two random-dot patterns, one in each eye. But this time there is a square visible in each eye separately—unlike Julesz patterns. It is still made of random dots yet, because of a difference of texture, a square is visible separately in each eye. The dots that constitute the left eye’s image (including S) are completely different in the two eyes; unlike Julesz’s pictures, they are uncorrelated. This stereogram is the converse of Julesz’s—a square is visible in each eye, but the dots that constitute it (and its background) is unrelated in the two eyes.
Ramachandran found that when he viewed this image through a stereoscope, the central square floated out. Because the dots defining the squares were uncorrelated in the two eyes, he and his colleagues concluded that, in this case, form perception occurred prior to stereo. The square was recognized separately in each eye before the shift across the eyes was measured. The Julesz rule could be violated. The brain often uses multiple tricks to achieve the same goal. In a noisy camouflaged environment, it makes sense to use both strategies.
The second display he invented makes the same point. It takes advantage of a curious visual effect dubbed illusory contours. Four “pacmen” are made of four black disks with pie-shaped wedges cut out from each. What you see, though, is not pacpeople facing each other; you see an opaque illusory white square occluding four black disks in the background. The brain says, in effect, “What is the likelihood that an evil scientist has precisely aligned these disks? More likely it is an opaque square, so that is what I will see.” You hallucinate the edges, called image segmentation.
Now can these illusory edges provide an input for stereo? Begin with the left eye’s picture and shift the illusory square to the left to create the right eye’s image. (This shift entails taking bigger bites out of the pie.) When you view the images through a viewer—lo and behold—the illusory square floats out! Again, form processing and image segmentation occur prior to stereo.
It gets better. Let us take a template of this stereogram and paste it on repeating wallpaper made of columns of dots. The dots are identical in the two eyes; they convey no disparity information. Yet amazingly, the dots inside the illusory square float out along with it—an illusion we call stereo capture; the dots are captured by the illusory square and dragged forward even though they themselves are not shifted.
This result suggests that Julesz’s claim was not entirely correct: stereo involves more than comparing pixels across the two eyes. Even if you consider Pettigrew’s disparity cells, they must be extracting tiny oriented clusters (not points) and “looking for” identical clusters to match. But the experiments of Ramachandran (and very similar results from psychologist Lloyd Kaufman of New York University) showed that the mechanism was even more sophisticated than that; it could segment the image based on implied occlusion and “hallucinate” illusory contours that can serve as tokens for stereoscopic matching. Once this information has been extracted and disparity measured, the brain constructs a 3-D illusory surface. The fact that the enclosed dots are dragged forward implies that the 3-D surface feeds back to be applied to the dots.
See what we're tweeting about
3 Comments
Add CommentThankyou for a very informative article, which deals with vision rather than perception. The brain is aware of what is behind it, even if we are not looking there. In the same way blind people have mental images of what is around them. Similarly we know how objects are moving when we hear them. The brain has to combine this and all other sensory stimuli into one concious image extending sometimes for miles around us and memorise it all as we move! Stereo vision is only one small part of perception...
Reply | Report Abuse | Link to thisI think that peripheral vision has a lot to do with our perception of depth, and dimension. The ability to "see" up, down, to the sides and forward all at once is the true mechanism of sight. This article would be very on target if it was "focus" at it's crux.
Reply | Report Abuse | Link to thisVery interesting. I've had little sight; am able to see light or darkness, but can not focus in my right eye since I was a little tyke. So, don't know how this affects my brain.
Reply | Report Abuse | Link to this