Image: Hayden Bird
As a favor to friends in my academic department, the MRC Cognition and Brain Sciences Unit in Cambridge, UK, I’ve frequently been a guinea pig in the fMRI scanner. Normally I fight valiantly against slumber as the stimuli flash on the small screen in front of me and the hypnotic, high-pitched beeps of the scanner reverberate around me. This time, though, it was very different. This time, my colleague Martin Monti was going to read my mind. As the bed I lay on robotically slid into the giant donut shape of the scanner, I had a strange sense that I was about to be mentally naked.
The task was simple: for each upcoming scan in the session, Martin would ask me either to imagine that I was playing tennis, or that I was moving around the rooms of my home. These two mental tasks activate very similar regions to actually carrying out the activity – mainly motor regions for tennis and navigation regions for roaming about a building. And because the brain scans of these two different tasks look totally different, Martin was going to guess my thoughts from my neural activity alone. When each scan was finished, and Martin accurately ascertained the contents of my imagination, the experience was thrilling and unnerving in equal measure.
In 2006, Adrian Owen, of the same department, has already used a similar technique to prove that a patient with a diagnosis of persistent vegetative state was in fact conscious. One day soon, it should be possible even to communicate with such patients, where no other means are available, by linking, say, imagining tennis, with a yes answer, and imagining moving around a building with a no answer. Similar methods are being used to train patients with motor neurone disease (also known as amyotrophic lateral sclerosis, or ALS), whose motor functions are slowly failing, to control a communication interface by thought alone.
This feat of scientific telepathy was largely unheard of a decade ago. But now, in various guises, it is taking over the field. What has caused such a revolution in brain-scanning? The simple answer is that over the last five years, the way that many researchers analyze their brain-scanning data has radically shifted.
fMRI datasets can be vast. There might be 100,000 3D pixels, called voxels, in each brain activity image, with a new image being taken every 2 seconds – for up to an hour. Then there will be around 20 subjects in a study, which equates to perhaps 4 billion different voxels to examine in total. The traditional, common way of breaking this problem down is to go back to just one of those 100,000 voxels in each image, in one location across subjects, and see whether it will rise or fall in activity over time, in accord with the mental fluctuations under study.
But some scientists have recently suspected that analyzing brain-scans in this way involves throwing away vast amounts of useful data, because we are ignoring how these voxels might be working together, in a pattern of activity, to represent information. The old, bog-standard, method is like being handed a fuzzy photo and concluding only that the bright regions are the most important, because they have the most light. Instead, the new method would look at the same photo, at the textures and contrasts, and how the shapes are built up, before recognizing the depiction of Marilyn Monroe in her prime.
This new, far more sensitive, method, known as multivariate pattern analysis (MVPA), effectively works like a form of AI. The program will learn to link some mental event with a particular pattern of brain activity, and then make predictions about how new brain data relates to mental states, based on these prior lessons. It is these predictions that now allow neuroscientists potentially to read minds.
The main early successes were in the tricky, subtle field of studying how brain activity generates consciousness. If competing images are presented to each eye, using a technique called binocular rivalry, we consciously perceive only one image at a time, even though our eyes are viewing both images. Geraint Rees and colleagues at University College London demonstrated via MVPA that the pattern of primary visual cortex activity has little to with our conscious image, but instead reflects the raw input from the eyes. To uncover brain regions that actually do reflect consciousness, they found, you have to center on later, more complex visual regions. Standard brain-imaging analysis methods simply lacked the power to detect such results.
More intriguingly, Geraint Rees and his colleague John-Dylan Haynes followed up these findings by using MVPA centered on the primary visual cortex to decode the orientation of lines, even though the volunteer was completely unaware of this visual detail. Again, these results paint a picture of our primary visual cortex as little more than the brain’s copy of what our eyes see, with the information being divided up to be processed in more interesting, conscious ways in later visual brain regions.
It wasn’t long before these powerful MVPA methods branched out into territory far removed from perception. Although ethically contentious progress is being made using MVPA to predict when a person is lying, considerably more profound results are appearing in another field. Last year, John-Dylan Haynes, now at the Bernstein Center for Computational Neuroscience in Berlin, carried out a very simple task – just choose whether to press the left or right button while in the fMRI scanner. When Haynes set his MVPA algorithm to learn which patterns corresponded with this decision, astoundingly he found strong signals in the prefrontal and parietal cortex up to 10 seconds before the volunteer had consciously intended to act. Does this mean we have no free will? Or does free will only kick in for more complex decisions? More research is required before these questions can be adequately answered.
One drawback of many fMRI studies is that the stimuli are so artificial as to limit their generalization to the real world. Because of the increased flexibility and power of MVPA methods, though, showing any old photos or movies in the scanner is quite feasible. For instance, Eleanor Maguire from University College London, and co-workers, have recently used a combination of MVPA and high resolution imaging to connect patterns in certain sub-regions of the hippocampus with long-term memory acquisition for naturalistic videos.
So far, studies have involved using an algorithm to guess one of a small handful of alternative mental states, when presented with a certain brain pattern. This is a far cry from genuine brain-reading, where you simply look at the neural activity and know what the person is thinking, without being given the vital clue of the shortlist of possibilities. One lab, though, seems to be edging ever closer to this more impressive aim. Jack Gallant, of the University of California at Berkeley, published results last year showing that his own flavor of pattern recognition programs can guess which one of a thousand pictures the person just viewed. And at the Society of Neuroscience conference two months ago he presented data that went many steps further – actually reconstructing what the volunteer was seeing from their visual cortex activity, after they viewed a series of movie trailers. For instance, the program would spit out an outline of a white torso just when a man in a white shirt was shown. The data hasn’t yet been published in a peer reviewed journal, and the reconstruction is at a very crude stage, so caution should be exercised. But nevertheless such provisional progress suggests tantalizing possibilities in future years, such as the ability to “read off” a crime witness’s memories, or visual imagery in dreams.
Other scientists are skeptical of the extent of progress that can be made with MVPA, though. Although all these studies demonstrate significant predictions, this regularly means that the computer’s guess is a hair’s breadth above chance. Many studies that rely on MVPA to pick between two alternatives score around 60% accuracy, for instance, when a blind guess would give 50% - a useful improvement over chance, but hardly telepathy. Although the experiment I mentioned at the start of this article is far more robust, partly because of far more data for each guess, if I were mischievously to imagine playing baseball instead of tennis, or navigating around my childhood home instead of my current one, neither the prediction program, nor the experimenter, would have had a clue that I was breaking the rules.
In the end, what the fMRI scanner is picking up is a very noisy, indirect measure of neural activity, and this creates inherent limits on what’s possible. But even if it were direct, a single voxel would still represent the collective activity of many tens of thousands of neurons. Technological advances in MRI physics may be on the horizon, allowing more reliable, higher resolution measurements to be made. Until then, the ability of fMRI, via pattern analysis, to make fascinating discoveries about the brain is flourishing, even if true mind-reading technology remains – largely – the realm of science fiction.
Wholesale Computation
A New Robot Rolls, and a New Prize Is Set
16 Comments
Add CommentI believe that the more intractible one's biases, and the more structured, the more predictable one is. Ideally a purely open mind would be a mind which could not be read beyond reading it to say, "I am listening, and am willing to hear all valid perspectives, and therefore am more interested in what YOU might provide by way of allowing me to update, correct, or upgrade my working hypotheses. Some say this old thinker is a genius; some, an old fool. But, whatever the actual case, if he err in his thinking, his errance is on the side of refusal to believe any philosophy or any science yet has availed all there is that can become known, nor has access to all factors lying above and beyond human grasp, nor has the capacity to conceive of any benefit to measurement of which man is not the ultimate unit of that measure. Even the greatest of human synthesists merely copes, merely finds a more efficient mode of muddling. Pure objectivity not only is unattainable by humans qua humans but, also, countervails mankind's ultimate coping success. But... to the immediate issue... of whether human minds can be read. If we define that as taking stock of clues to a mind's bias set, then yes there are such clues. They are only proxies, but they are discernable in mechanical ways. Beyond that, the very limits of mechanics itself, are self-constraining. Let those who are fools merely agree. Let those with open, and hence less readable minds, ponder. And please do be aware these words are protected by copyright, though surely none would find in them any value worth plagiarizing. (:>)
Reply | Report Abuse | Link to thisPlease forgive if I return too soon. But no evaluation of any abstract thing is logically valid, I humbly submit, so long as any operant be less than defined with absolute rigor and/or any operation be less than absolutely limited and consistent; and no actual object or phenomenon under contemplation is valid unless all definitions of players and play are described in terms of certainties. Even scientists must admit on occasion that certainty is unreachable. And this is why we only cope and muddle. In the January 2010 issue of SciAm appears an article estimating what a quantum computer might provide, if an when a functional one is made available to us. From amongst the ideas in that fine article, let me pluck just one and dangle it for our mutual musing. The notion of super-position (whether one subscribe on faith in the Copenhagen interpretation or nay) is not confined to a sub-molecular level at all. Let us allow that, for all practical human coping purposes, there is enormous reliance in moment to moment decision making, or preference making (increments of coping processes generally) which must proceed for practical and unavoidable reasons, in abscnce of certainty as to many of the subsumptive variables of any given constellation of factors participating in what is to be the veritable next tick of the mind-clock. To bring this down a few notches in abstractivity, perhaps it can be argued that we never have either all the essential data involved in an incremental mindcope moment, nor assurance that what we do know covers all the bases of essentials needed to 'get it right,' nor any certainty that the data we presume is necessary, sufficient and exhaustive is correct as to every twie, even, of that. So, as the expression is borrowed from football, we mentally "punt;" whereupon we are confronted by a necessity to proceed to the next succeeding "think increment." Now weigh this questioo carefully: Where, for ontology's sake, do we humans have the luxury -- outside our wonderfully gorgeous abstract models of pure mathematics -- have the luxury of calculating with nothing uncertain? We do not. We cope, we muddle, by processing information that allows for the postulation (whether tacitly or naively) of many, many factors which either are in super-position per the Copenhagen stance, or simply in a state unknown to us at that moment. What if you should ask this old thinker, where his dog is. In addition to being hyper-active relative to his assigned yard, he sometimes gets out, and is on a spinning planet orbiting...etc
Reply | Report Abuse | Link to thisWhat about the radiation?Is not the subject at risk?I suggest the reading of electomagnetic fields produced by the brain,as a safe way of looking at whats going on inside in real time,and in the end may be more accurate.Although no research has been done,but it should be,if anybody out there knows of such please respond about who and where, I believe that when two people are close together their fields can interact with each other at a subconscious level.A form of telepathy if you will,and it could also explain divining.In the search for water and oil,as the subjects fields and that of the Earths interact.And who knows in some cases maybe, a certain talented person could read minds.
Reply | Report Abuse | Link to thisExperiments into telepathy scored positive results because of there being a limited sample size and the experimenters throwing away their negative results and only publishing positive results. Only when larger studies were done was it concluded that telepathy was no more effective than random chance.
Reply | Report Abuse | Link to thisHow much of this brain scanning and number crunching down to statistical fluke? A 60% accuracy result from a sample size of 20 subjects is statistically insignificant, unless each subject is repeating the test a number of times, which isn't stated in the article.
There could equally have been tests that only provided, say 40% accuracy, but it's unlikely these results would have been published in a peer reviewed journal.
Already a member.The palm print of brain line by its swing could be decoded
Reply | Report Abuse | Link to thisMcGroo - yes, it's standard policy for each test to be repeated many times. So 60% would actually be a highly significant result in these papers.
Reply | Report Abuse | Link to thisMcGroo - yes, it's standard policy in these studies for there to be many trials per subject, so that 60% accuracy would be highly significant, even though numerically it doesn't sound very impressive.
Reply | Report Abuse | Link to thisIn fact, all fMRI studies I can think of involve multiple trials per subject (normally dozens if not hundreds), in order to combat noise in the data, and make the result more reliable.
Wouldn't want to be the subject receiving all that radiation,hundreds of times?Ouch.
Reply | Report Abuse | Link to thisDaniel Bor confuses two distinct meanings of the term consciousness. One might be called the hard-question concept of consciousness and the other the easy-questions concept.
Reply | Report Abuse | Link to thisBor reports a brain researcher at Cambridge used fMRI to discover that a patient in a persistent vegetative state was still conscious and could even communicate. In this case, conscious means having awareness, and indeed Bor uses that term as a synonym in the same sentence.
But in other paragraphs he describes using fMRI to study:
how brain activity generates consciousness. If competing images are presented to each eye, using a technique called binocular rivalry, we consciously perceive only one image at a time, even though our eyes are viewing both images. & The pattern of primary visual cortex activity has little to with our conscious image, but instead reflects the raw input from the eyes. To uncover brain regions that actually do reflect consciousness, they found, you have to center on later, more complex visual regions.
Here Bor writes about something that is better called attention or focus. The brain function is not awareness per se. Bor explains that the fMRI scan showed the primary visual cortex included all the visual data. But more complex visual regions attended to only one of two possible images.
Consider the famous illustration that alternately appears as a goblet or two profiled faces. You can shift between the perceptions but cant see both at once. It stretches the point to say that your consciousness, awareness, encompasses only one. Better to say you shift your focus or attention back and forth.
Perhaps this distinction is mere definition or semantics, but I think its more.
Bors first use of conscioussynonymous with awarepoints to what philosophers call the hard question of consciousness. Its about subjectivity: How does the brain, a material organ, generate the immaterial contents of subjectivitythe redness of red, the sweet taste of watermelon, the rough feel of sandpaper.
Bors second use of the wordregarding binocular rivalrypoints to a relatively easy question, how does the brain attend or focus on an aspect of what is in awareness.
I would have difficulty answering the hard question about subjectivity. But I can guess what the answer to the easy question will be.
When viewing the goblet/faces illustration, besides the visual cortex, different parts of the brain light up on fMRI, depending on which image is attended to. One area lights when we see or think of things like gobletsglasses, cups, etc. A different region lights when we perceive facesfamiliar faces, strange ones, mens or womens, etc. The findings will be similar with binocular rivalry.
Using fMRI to detect which image is in attention will be an amazing feat technically, but the fact that scientists will do it one day wont be surprising.
Mark Pine - thanks for the long comment you've made. It's true that whether someone is conscious or not, and what they are conscious of are two separate questions. But I don't recall confusing them at any point - just talking about patients in the wakeful context, and the contents of consciousness in normal volunteers. I also think that both questions are relevant to the easy/hard problem - not just one (easy being what brain areas or processes correlate with consciousness, and the hard problem being exactly how the brain generates consciousness).
Reply | Report Abuse | Link to thisYou're right that I use "consciousness" and "awareness" interchangeably, as do many people in the field.
With the goblet/faces, you don't actually consciously see both at once. The same with binocular rivalry - only one image at a time is consciously seen, and the percepts flip pretty regularly, which makes it a very useful paradigm for experiments.
Some researchers suggest that attention and consciousness are one and the same, in fact. A very interesting question.
For more details about these and many other questions and experiments relating to consciousness, look out for my book on the subject (published by Basic Books - due out early 2011). In the meantime, perhaps follow me on twitter for thoughts/articles on consciousness and related topics.
Reading any proxy is a problem of interpretation, and no interpretation is any more reliable than the unmeasurable, unfalsifiable assumptions feeding the interpretive model. Of course electromagnetic resonance imaging and other kinds of marker monitorings can be made; and some degree of accuracy can determine whether a person is INTENDING to be illusive or not, and whether, say, pleasure centers or psychic pain centers are activated, or correlations can be found between how one responds to one, or one kind of, stimulus as relating to another. But is the content of a mind no more than brain-mapping and tracing of responses. What are the chances that two brain maps are identical, and hence readable in terms of what one subect is thinking when a particular area is excited as a guide to what another is thinking when the same vicinity lights up. As to "reading" electromagnetic impulses, what of response stimulus and response cascades that wax from electromagnetic, to chemical, to other chemical, involving other veritable cellular receptor types, where two individuals having different DNA and different epigenetic "conditioned response" variables, different sex, different age... get interpreted in accordance with translation of a an analogical anatomical location of brain stimulation get interpreted. Or, to put it another way, what "standardized" brain geography pathway will provide certainty that what one brain is experiencing when a given locale lights up, is what another brain is experiencing -- whether consciously or unconsciously? Is it not possible that reading a mind by MRI is comparable to finding a point in two different day's newspapers, exactly nine inches down from top, and exactly fourteen inches right of left marging, at that grid intersection will locate THE SAME PRINTED WORD there? This is not to say approximate comparisons and analyses are useless. They may shed light on many useful analogies between subject A and subject B. But, unless we define "mind reading" as comparable to taking apart and analyzing how a wristwatch works, by using a jack hammer to disassemble it, we suggest a substantial degree of false accuracy by such terminology, do we not?
Reply | Report Abuse | Link to thisHaving lived through the time in the 1960's and 1970's, and recalling the hell days of pre-frontal lobotomies by brain surgeons, and efficiency control of factory workers, on basis of behavioral psychology studies on rats and dogs, I cannot help but wonder if poor George Orwell is turning over in his grave about now.
Reply | Report Abuse | Link to thisI fear no open-ended research, and no data it might provide. However, many human lives have been made hell on Earth, by well-intended data crunchers who perceived themselves on the verge of ringing in a new error of peace and prosperity and happiness based on some interpretations of results that they deemed themselves capable of "reading" in such a way as to -- in some cases -- know even more about an individual than he knows for himself, in such a way that the subject individual could be "helped" or rendered better off by some application of that enlightened science.
Whether we are speaking of Alfred Nobel's TNT or barbed wire or the machine gun, the APPLICATIONS of these technological advancements either were a mixed blessing, or merely carried competition (in industry, war, society) to a new level of complexity.
The thought of anyone's being convinced he has a veritable window into the mind of another, brings shivers of remembrance of such things as attempted brain-washing, and refinements of effective propaganda techniques to persuade, for example, millions of people to smoke tobacco who might not have otherwise.
Again, I have no fear of any pure research, nor any data that might be derived from it. But the prospect of someone's THINKING he can read my mind is more horrifying to me than if I knew he really could. The thinking that pre-frontal lobotomies were a lesser evil than psycho-pharmeceutical protocols, was not only mistaken, it was misbegotten. The thinking that workers would be both happier and more productive if manipulated like rats in a skinner box, or dogs conditioned to salivate at the ring of a bell, failed to work out as predicted.
Hopefully those who continue to pursue a refining of a presumed possibility of "reading human" minds will examine thefor applications othe than, perhaps improving interrogation techniques of suspected criminals or terrorists, will take great care to think make sure their hypotheses will not convict innocent suspects of crimes, or cause undue prolongation of detainment of terrorism suspects, for reasons of misinterpretation. This research poses a veritable mine field. So the very least it can do is exercise an abundance of interpretive caution.
On Wednesday I wrote about the hard question of consciousness and the easy ones. The hard question asks how subjectivity can arise from material objects. Philosophers use the term "qualia" for the "introspectively accessible, phenomenal aspects of our mental lives" (plato.stanford.edu) - i.e., subjectivity.
Reply | Report Abuse | Link to thisSome easy questions ask how consciousness gets directed to particular elements of it, for example a musical tune.
Answers to such questions make use of the brains wiring diagram. For example, hearing a tune probably involves neuronal activities in the auditory cortex, some other brain loci and the axon trunks connecting those areas. Although it may require intricate, difficult technical achievements, there is no towering conceptual hurdle to jump to understand the mechanisms.
I also wrote about awareness and attention. In my view, a synonym for consciousness as subjectivity is "awareness." In contrast, "attention" means directing consciousness to a particular element - a sensation or idea.
I believe there cannot be wiring-diagram answers to the hard question about subjectivity, for the reason that qualia and neurons are different kinds of things. But I dont take a Cartesian view that mind and matter are separate universes. Rather, I think subjectivity (qualia and ideas) and objectivity (neurons and other material things) are two sides of one coin. But when we examine one side, we cannot hope to find answers by using concepts appropriate to the other side.
A similar duality exists in quantum mechanics. Everything that exists, according to QM, has a particle form and a wave form. But the duality of QM is also not of the Cartesian type. Rather, particle and wave are two ways of describing something - anything. When you use one description, however, you cant contaminate it with concepts from the other. (Contd in the next posting.)
In response to my Wednesday post, the articles author, Daniel Bor, kindly replied, "You're right that I use 'consciousness' and 'awareness' interchangeably, as do many people in the field. & Some researchers suggest that attention and consciousness are one and the same, in fact. A very interesting question."
Reply | Report Abuse | Link to thisIn my view, the word 'awareness' points to the how the concept of consciousness differs from the concept of neuron. In contrast, the word 'attention' points to the particular operation of neurons and the wiring diagram.
I think QMs wave-function descriptions can more easily explain awareness than QMs particle descriptions, which can more easily explain attention.
For one thing, wave functions and awareness are both immaterial mental phenomena. Like can better explain like.
A second thing is that wave functions better explain the integrated coherence of awareness. Whether one investigates the figure-ground alternation of images of the goblet-faces illustration or binocular rivalry experiments, what makes the phenomena remarkable isnt that they involve the activities of neuronsbut that the alternating percepts are coherenttaking one form, while excluding the other.
Wave functions behave this way. As wave function, an entity exists as a set of eigenvalues. When it is observed (in consciousness), it appears in the form of one eigenvalue or another of that set, but not both.
A third thing that wave functions explain better is the unity awareness. A conscious phenomenon is perceived as unified experience. With the wave function explanation it is easier to understand how a group of neurons can generate a unified percept.
In QM, when material objects interact with each other, the wave functions of the objects become entangled and combine into a single unified wave function. In the case of the interaction of a group of neurons, the wavefunctions of the neurons become entangled in a single wavefunction of the group. Thus, they can generate a unified conscious experience.
If I can 'speak' to my unconscious self while in dream stage, am I awake sleeping? Or sleeping awake?
Reply | Report Abuse | Link to thisIf I am speaking to my sleeping self, is my consciousness active? If my sleeping unconscious responds to my conscious questions, am I dreaming?
Reply | Report Abuse | Link to this