Next, they applied the same Bayesian framework to decode fMRI signals. They used 5,000 hours’ worth of short clips pulled at random from YouTube to establish a baseline of “natural movies.” The same three subjects were tested by watching movies in the magnet they had not previously seen and that were not drawn from the natural movies data set. The decoder estimates the most likely clip based on the response of many voxels in the visual cortex of each volunteer. It is a very sophisticated form of hedging one’s bets based on prior experience, widely used in a variety of applications—such as predicting that your credit card is being misused by somebody who has very different purchasing patterns.

Reconstructing the movie in the head leads to some stunning results. (I urge the reader to visit Gallant’s Web site, where a movie highlights the side-by-side comparison between viewed and decoded movies.) The method is far from perfect—the reconstructed clips are slow and lack details. After all, the fMRI signal is read out only once every second, whereas the underlying movies are much more dynamic (with a 15-hertz frame rate). Yet the net result is astounding, even for an old hand like me.

What Does the Future Hold?
As our measurement tools become more precise and our algorithms more sophisticated, the quality of the reconstructed movies will improve. Indeed, it is not inconceivable that the kind of visual daydreaming we all engage in—sexual fantasies, the crux of the climb where I keep on falling, what I should have told my boss—will one day yield to these tools (provided that I engage in imagery while lying completely immobile in a magnetic scanner). And who’s to say that dreams might not also be accessible to Gallant’s reconstruction techniques?

Functional brain imaging is perfectly safe and requires nothing more than reclining on one’s back uncomfortably for a few hours in a tight metal cylinder. Yet the fundamental spatiotemporal limits of fMRI remain. It does not access the atoms of perception, individual neurons. At the moment, only intrusive microelectrodes that are implanted in the brains of some patients, as was described in my May/June 2011 column, can access the substrate out of which our most fleeting experiences, thoughts and conscious memories arise. For now these remain safe from prying eyes.

1 Comments

1. 1. sjfone 09:01 AM 12/30/12

   Yeah man, voxels, semantic space, but when I watch television there's no neural activity, like totally.

   Reply | Report Abuse | Link to this