February 17, 2009 | 2 comments

In Defense of the Value of Social Neuroscience

A social neuroscientist responds to the controversial critique that the results of imaging studies are routinely overstated.

 


Courtesy of Matthew Lieberman

e-mail print comment

Matthew Lieberman is associate professor of social neuroscience at the University of California, Los Angeles.  In recent weeks, he’s also rebutted the claims of a recent paper, “Voodoo Correlations in Social Neuroscience,” which explored the high correlations between measures of personality or emotionality in the individual—such as the experience of fear, or the willingness to trust another person—with the activity of certain brain areas as observed in an fMRI machine. Mind Matters editor Jonah Lehrer chats with Lieberman about why most fMRI correlations aren’t false, the “reward” of intense grief and why accepting unfair offers seems to activate brain areas involved with self-control.

LEHRER: Your field of research has come under fire in a recent paper titled "Voodoo Correlations in Social Neuroscience."  What's the authors' argument and have they identified a significant problem in this field?

LIEBERMAN: In their paper, Vul and colleagues suggest that brain-personality correlations in many social neuroscience studies depend on invalid methods and thus are “implausibly high,” “likely…spurious” and “should not be believed.” These claims are incorrect. These analyses use standard procedures for drawing inferences and protecting against false positives. The correlation estimates will tend to be somewhat higher than the true value, but there is no evidence to suggest that these correlations are meaningless or “voodoo” science.

The argument that Vul and colleagues put forward in their paper is that correlations observed in social neuroscience papers are impossibly high. There’s a metric (the product of the reliabilities of the two variables) that determines just how high of a correlation can be observed between two variables. They suggest that because, on average, this metric allows correlations as high as 0.74, that social neuroscientists should never see correlations higher than that.

Given the gravity of the claim, it’s important to get this [figure] right, but they do not. Here’s their mistake: it’s not the average of this metric that determines what can be observed in a study, but rather the metric for that particular study or at the very least, the metric estimated from prior use of the actual measures in that study. Just because the average price of groceries in a supermarket is $3 does not mean you cannot find a $12 item. In fact, a study that I’m an author on (and is a major target in the Vul et al. paper) is a perfect example. The reliability of the self-report measure in our study is far higher than the average they report allowing for higher observed correlations. They knew this [fact], but presented our study as violating the “theoretical upper bound” anyway.

Their second major conceptual point is that numerous social neuroscience authors were making a non-independence error. Ed Vul gives a nice example of what he means by the non-independence error in a chapter with [Massachusetts Institute of Technology neuroscientist] Nancy Kanwisher. They suggest that we might be interested in whether a psychology or a sociology course is harder and assess this [question] by comparing the grades of students who took both courses. In a comparison of all students, we find no difference in scores. But what if we began by selecting only students who scored higher in psychology than sociology and then statistically compared those? If we used the results of that analysis to draw a general inference about the two courses, this [strategy] would be a non-independence error, because the selection of the sample to test is not independent of the criterion being tested. This [practice] would massively bias the results. 



Read Comments (2) | Post a comment 1 2 3 4 5 Next >


Share
Propeller    Digg!  Reddit delicious  Fark 
Slashdot    RT @sciam In Defense of the Value of Social NeuroscienceTwitter Review it on NewsTrust 
sharebar end

You Might Also Like


Discuss This Article


Click here to submit your comment.

VIEW:

2,573 characters remaining
 
  Email me when someone responds to this discussion.
 

risk free issue 

Sciam - cover Email:
Name:
Address:
Address 2:
City:
State:  
spacer




Editor's Pick

  • Adapting to the Freshwater CrisisForward-thinking experts are getting a better handle on the growing global water shortage and coming up with innovative approaches to ensuring the security, safety and sustainability of this resource

Newsletter

Mind & Brain Newsletter

Get weekly coverage delivered to your inbox


 Podcasts

  • 60-Second Psych     RSS  · iTunes The Roots of Language
    click to enable

    Download

  • 60-Second Science     RSS  · iTunes Plants Share Light If Neighbor Is Related
    click to enable

    Download





ADVERTISEMENT
 
 


Also on Scientific American


© 1996-2009 Scientific American Inc. All Rights Reserved. Reproduction in whole or in part without permission is prohibited.
ADVERTISEMENT