# Blurring the Boundary Between Perception and Memory

Can you trust your lying eyes—or any of your other senses and memory? Not really

Image: Andrei Nacu

• ### Perv

“As a sex writer, Jesse Bering is fearless—and peerless.” —Dan Savage “You are a sexual deviant. A pervert, through and through.” We may not want to admit...

Perception is mathematically impossible.

This might seem like a bold statement—after all, you are perceiving these letters right now—but it’s nonetheless true. Imagine a black-and-white line drawing of a cube on a sheet of paper. Although this drawing looks to us like a picture of a cube, there are actually an infinity of other three-dimensional figures that could have produced the same set of lines when collapsed on the page. But we don't notice any of these alternatives. Happily for all of us, our visual systems have more to go on than just bare perceptual input. They use heuristics and short cuts, based on the physics and statistics of the natural world, to make the “best guesses” about the nature of reality. Just as we interpret a two-dimensional drawing as representing a three-dimensional object, we interpret the two-dimensional visual input of a real scene as indicating a three-dimensional world. Our perceptual system makes this inference automatically, using educated guesses to fill in the gaps and make perception possible.

It turns out that our brains use the same intelligent guessing process to reconstruct the past, in addition to using it help perceive the world. Memory itself is not like a video-recording, with a moment-by-moment sensory image. In fact, it’s more like a puzzle: we piece together our memories, based on both what we actually remember and what seems most likely given our knowledge of the world. Just as we make educated guesses in perception, our minds’ best educated guesses help “fill in the gaps” of memory, reconstructing the most plausible picture of what happened in our past.

The most striking demonstration of the minds’ guessing game occurs when we find ways to fool the system into guessing wrong. When we trick the visual system, we see a “visual illusion”—a static image might appear as if it’s moving, or a concave surface will look convex.  When we fool the memory system, we form a false memory—a phenomenon made famous by researcher Elizabeth Loftus, who showed that it is relatively easy to make people remember events that never occurred. As long as the falsely remembered event could plausibly have occurred, all it takes is a bit of suggestion or even exposure to a related idea to create a false memory.

In the Blink of an Eye
In past literature, visual illusions and false memories have been studied separately. After all, they seem qualitatively different: visual illusions are immediate, whereas false memories seemed to develop over an extended period of time. A surprising new study blurs the line between these two phenomena, however. The study, conducted by Helene Intraub and Christopher A. Dickinson, both of the University of Delaware, reveals an example of false memory occurring within 42 milliseconds—about half the amount of time it takes to blink your eye.

Intraub and Dickinson’s study relied upon a phenomenon known as “boundary extension”, an example of false memory found when recalling pictures. When we see a picture of a location—say, a yard with a garbage can in front of a fence—we tend to remember the scene as though more of the fence were visible surrounding the garbage can. In other words, we extend the boundaries of the image, believing that we saw more fence than was actually present. This phenomenon is usually interpreted as a constructive memory error—our memory system extrapolates the view of the scene to a wider angle than was actually present.

The new study, published in the November issue of the journal Psychological Science, asked how quickly this boundary extension happens. The researchers showed subjects a picture, erased it for a very short period of time by overlaying a new image, and then showed a new picture that was either the same as the first image or a slightly zoomed-out view of the same place. They found that when people saw the exact same picture again, they thought the second picture was more zoomed-in than the first one they had seen. When they saw a slightly zoomed-out version of the picture they had seen before, however, they thought this picture matched the first one. This experience is the classic boundary extension effect. So what was the shocking part? The gap between the first and second picture was less than 1/20th of a second. In less than the blink of an eye, people remembered a systematically modified version of pictures they had seen. This modification is, by far, the fastest false memory ever found.

Although it is still possible that boundary extension is purely a result of our memory system, the incredible speed of this phenomenon suggests a more parsimonious explanation: that boundary extension may in part be caused by the guesses of our visual system itself. The new dataset thus blurs the boundaries between the initial representation of a picture (via the visual system) and the storage of that picture in memory.

So is boundary extension a visual illusion or a false memory? Perhaps these two phenomena are not as different as previously thought. False memories and visual illusions both occur quickly and easily, and both seem to rely on the same cognitive mechanism: the fundamental property of perception and memory to fill in gaps with educated guesses, information that seems most plausible given the context. The bottom line? The work of Intraub and colleagues adds to a growing movement that suggests that memory and perception may be simply two sides of the same coin.

Are you a scientist? Have you recently read a peer-reviewed paper that you want to write about? Then contact Mind Matters editor Jonah Lehrer, the science writer behind the blog The Frontal Cortex and the book Proust Was a Neuroscientist. His next book, How We Decide, will be available in February 2009.

### ABOUT THE AUTHOR(S)

Timothy Brady is a Ph.D. candidate in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology. Adena Schachner is a Ph.D. candidate in the Department of Psychology at Harvard University.

Rights & Permissions

View
1. 1. federica 09:49 AM 12/16/08

Be careful, not all cognitive scientists believe that our visual system makes inferences in order to "fill in the gaps" and obtain veridic visual perceptions...
There may some misunderstanding in this article, as it seems to state that in order to have a realistic picture of the world our brain interprets the visual scene using its prior konwledge.
There are indeed some theories of visual perception stating this, but there are also other, influent, ones that reject the idea of "inferences and hypothesis" made by the brain to see the world in a proper way.
There is some evidence that prior knowledge may influence perceptions, but there is also a lot of evidence that visual heuristics are in some ways hardwired in the brian, in the form of low-level rules.
And more, maybe the proposition "fill-in-the-gap" may be misleading too. The brain rather than filling-in poor information, has to reduce the overload of data that it recieves in every single moment. Usually visual information is too much, rather than too less.
Sorry for my bad english

Reply | Report Abuse | Link to this
2. 2. TimothyBrady 11:20 AM 12/16/08

federica, I think you are misunderstanding the claim of the article. It is an absolute certainty, doubted by nobody in the vision science community, that the light hitting your retina does not provide enough unambiguous information for your brain to figure out the exact state of the world. Our minds are therefore filling things in somehow, since we do see the world. This is why we see illusions in some circumstances as well.

The question that is up for some debate in the field is whether the mind is 'stuck' with low-level, evolved over millions of years, heuristics for filling in the missing information (you call this 'low-level rules'), or whether it is making more high-level inferences based on the surfaces and objects present in the scene. Twenty years ago there was a lot more debate about this point than there is now, though. There are now many convincing illusions that demonstrate very high-level effects even on brightness illusions or 3D perception. It is still an open question to what extent low-level vision is encapsulated from high-level knowledge; but certainly even very simple illusions depend on the organization of surfaces in 3D space.

The question of whether we get too much or too little information is also too dichotomous. We get both too little information to unambiguously determine the state of the world AND too much information to process, which is why we must attend to specific areas of a visual scene for further processing (rather than just processing it all). The fact that we get too much information to process does not imply that the information we get is enough to figure out the state of the world.

Reply | Report Abuse | Link to this
3. 3. eco-steve 01:13 PM 12/16/08

Our perception is determined by the configuration of our sensory organs. We only see in front of us, but can hear around 360°, so we do have a 3D awareness of our surroundings. We have to turn our heads for more details though. Rabbits see through 360°, so their mental image is far more detailed than ours... So every creature has its own specific illusion of reality.

Reply | Report Abuse | Link to this
4. 4. John Turri 03:42 PM 12/16/08

Timothy, you say,

"It is an absolute certainty, doubted by nobody in the vision science community, that the light hitting your retina does not provide enough unambiguous information for your brain to figure out the exact state of the world. Our minds are therefore filling things in somehow, since we do see the world."

This is a plausible inference. But I have a question. Doesn't it assume that _light hitting the retina_ exhausts the information relevant to visual perception? If so, what evidence is there for that assumption?

Reply | Report Abuse | Link to this
5. 5. JHSibal 05:46 PM 12/16/08

I have read this and I am reminded that whatever we see--how we categorize it and even the colors we see it with--it not determined by us as individuals, but as a member of cultural group which tells us how to "see". As you know, according to some, there are two, five, seven and eleven color cultures. In the first, members see only light and dark and don't see as modern Westerners do. Their perceptions are filtered by their culture. In ancient art--my field--we always must remember that they saw differently than we do. Are you accounting for any of this in your research.

Reply | Report Abuse | Link to this
6. 6. stookie2738 03:05 PM 12/21/08

I have come across many articles in relation among this subject evolving. I bring memories past / forward and strangely recall to myself, these thoughts of my history are complete illusion my mind has formed among my desire. Crazy as it sounds, WOW, an individual needs no drugs to hallucinate. Sounds as if our minds create what we want to see. To unlock the doors, and walk along the wall of our history. May be because we could not handle the results. I also was informed how we as society are taught not to notice what's in between, that space our nose occupies. Something along that arena. That sounds like we were deprived much of our true nature. Any how the years ahead, will create more theories to consider. To each there own.
Peace out

Reply | Report Abuse | Link to this
7. 7. stookie2738 03:07 PM 12/21/08

I LUV SCIENCE.....

Reply | Report Abuse | Link to this

### Add a Comment

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Click one of the buttons below to register using an existing Social Account.

## More from Scientific American

• Octopus Chronicles | 6 hours ago

### Octopus In Space: Why the Latest U.S. Spy Rocket Is More Appropriate Than the NRO Knows

• TechMediaNetwork | 6 hours ago | 5

### Radiation on Mars 'Manageable' for Manned Mission, Curiosity Rover Reveals

• News | 6 hours ago | 2

### Powder Rive Basin Coal on the Move

• TechMediaNetwork | 6 hours ago | 1

### Formula 1 Racing Loud Enough to Damage Hearing

• News | 9 hours ago | 1

### Strategies for Aging Vary Widely across 46 Species with Some Scarcely Deteriorating

See what we're tweeting about

More »

## Latest from SA Blog Network

• ### Octopus In Space: Why the Latest U.S. Spy Rocket Is More Appropriate Than the NRO Knows

STAFF
Octopus Chronicles | 6 hours ago
• ### Oh the Horror! Best of the Blogs for November...

Video of the Week | 9 hours ago
• ### Best of the Blogs for November: Oh the Horror!!

PsiVid | 9 hours ago
• ### 4 Billion Years of Martian History in 2 Minutes

Life, Unbounded | 11 hours ago
• ### My Story Collider Story - Working Twice as Hard

The Urban Scientist | 11 hours ago

## Science Jobs of the Week

Blurring the Boundary Between Perception and Memory

X

### Give a Gift & Get a Gift - Free!

Give a 1 year subscription as low as \$9.99

X

X

###### Welcome, . Do you have an existing ScientificAmerican.com account?

Yes, please link my existing account with for quick, secure access.

No, I would like to create a new account with my profile information.

X

Are you sure?

X

### Institutional Access

It has been identified that the institution you are trying to access this article from has institutional site license access to Scientific American on nature.com. To access this article in its entirety through site license access, click below.

X

X