Early in his new book, Everything Is Obvious: *Once You Know the Answer (Crown Business, 2011), Duncan Watts tells a story about the late sociologist Paul Lazarsfeld, who once described an intriguing research result: Soldiers from a rural background were happier during World War II than their urban comrades. Lazarsfeld imagined that on reflection people would find the result so self-evident that it didn't merit an elaborate study, because everyone knew that rural men were more used to grueling labor and harsh living standards. But there was a twist: the study he described showed the opposite pattern; it was urban conscripts who had adjusted better to wartime conditions. The rural effect was a pedagogical hoax designed to expose our uncanny ability to make up retrospective explanations for what we already believed to be true.

Though Lazarsfeld was writing 60 years ago, 20/20 hindsight is still very much with us. Contemporary psychologists call this tendency to view the past as more predictable than it actually was "the hindsight bias." Watts, a Yahoo! Labs scientist best known for his research on social networks and his earlier book, Six Degrees: The Science of a Connected Age (W. W. Norton, 2003), argues that this tendency is a greatly underappreciated problem, one that not only causes us to make up just-so stories to explain any conceivable outcome—but to delude ourselves that we can predict the future by learning from the past. (Just because we can create a plausible account of why a book became a bestseller doesn't mean we can tell which new book will be a hit.)

Predictability is elusive because randomness holds much more sway than most of us would like to believe. Drawing on his own research, Watts shows that messages on Twitter don't spread through a predictable set of influential hubs. Similarly, when you ask large numbers of people to relay an e-mail to a stranger through someone they know, there turn out to be no star intermediaries through whom most e-mails find their way. "When we hear about a large forest fire, we don't think that there must have been anything special about the spark that started it," Watts wrote. "Yet when we see something special happen in the social world, we are instantly drawn to the idea that whoever started it must have been special also."

It's interesting that you include a story about hindsight from Paul Lazarsfeld, since he also co-wrote a book about influence. Would you speculate on what Lazarsfeld would think of your ideas about influencers?
That book [Personal Influence: The Part Played by People in the Flow of Mass Communications], with Elihu Katz (Free Press, 1955)] was a brilliant work, and probably the best book on social influence ever written. They had a fairly sensible view of what they called "opinion leaders". The problem with what they said is it got conflated with this other theory of diffusion and with what got labeled the "law of the few," which is not what they were talking about at all. Lazarsfeld and Katz define opinion leaders as the subset of society who consume a larger than average amount of media, and who make decisions about what's interesting and should be passed on, so they act as filters between the media and the rest of society. But even if you're talking about 10 or 20 percent of society, in America that's tens of millions of people, so it's not three or four people or a hundred people. So what I'm saying I think is consistent with what they said.

A big part of your book deals with the problem of ignoring failures—a selective reading of the past to draw erroneous conclusions, which reminds me of the old story about the skeptic who hears about sailors who survived a shipwreck supposedly because they'd prayed to the gods. The skeptic asked, "What about the people who prayed and perished?"
Right—if you look at successful companies or shipwrecked people, you don't see the ones who didn't make it. It's what sociologists call "selection on the dependent variable," or what in finance is called survivorship bias. If we collected all the data instead of just some of it, we could learn more from the past than we do. It's also like Isaiah Berlin's distinction between hedgehogs and foxes. The famous people in history were hedgehogs, because when those people win they win big, but there are lots of failed hedgehogs out there.

Other scholars have pointed out that ignoring this hidden history of failures can lead us to take bigger risks than we might had we seen the full distribution of past outcomes. What other problems do you see with our excessive focus on the successful end of the distribution?
It causes us to misattribute the causes of success and failure: by ignoring all the nonevents and focusing only on the things that succeed, we don't just convince ourselves that things are more predictable than they are; we also conclude that these people deserved to succeed—they had to do something right, otherwise why were they successful? The answer is random chance, but that would cause us to look at them in a different light, and changes the nature of reward and punishment.

But, as you say in the book, history plays out only once, and the world is complex, so it's impossible to know how big a role luck played in a real-world outcome like the success of the Mona Lisa, the example you give. Can you think of cases where we can know with any certainty why something succeeded, and therefore predict what will succeed in the future?
We can show that certain things are predictive on average, which is exactly the point that comes up in the Twitter study. I'm willing to bet $100 right now that books about boy wizards are more likely to become bestsellers than books about differential equations; we're reasonably good at identifying factors that might be relevant to success or to some outcome of interest. What we're bad at is weighing those factors relative to each other, and intuition is not necessarily a very good guide to that. So it's not that we can't predict anything—the problem is that the predictions we most want to make are precisely the ones we can't make.

Given your background, how did you come to write this book, which ranges far beyond network theory?
I started out life in physics and then mathematics, and at some point I switched over to become a sociologist—and in the process of transitioning, I noticed this interesting phenomenon: When people perceived me as a mathematician, and I would describe my research, they would say, "Wow, that's really fascinating. How do you figure these things out? It's complicated and difficult." But when a few years later I was describing the same work in terms of social phenomena and the behavior of people, fads and historical events, success and failure, and so on, people would say, "That sounds kind of obvious. Don't we all know that?" My initial reaction was frustration; I thought, "What the hell? I spent years studying this stuff, and it's not obvious, so why are they reacting this way?" But I eventually switched on my sociologist brain and realized that there's something about social science that's different in how people who are not social scientists perceive it. When someone tries to explain to us how electrons behave, we think it's amazing and completely unintuitive, but when we explain how people behave, it always seems trivial. The whole book is a dialectic between these two related questions—what is it that makes the world complicated, and then, if it's really that complicated, why do we keep behaving as if it's not that complicated?

The most provocative and counterintuitive part of your book for me was your chapter "Special People," based on your research suggesting that buzz about cultural products is not spread through a predictable set of influencers.
Do you think that, really?

I do, particularly because you're responding to the by-now conventional wisdom that if you go through influential people, they will spread your message far and wide. And one of the pieces of evidence you muster against what Malcolm Gladwell in The Tipping Point [Little, Brown and Co., 2000] calls "the law of the few" is your small-world experiment, in which you replicate Stanley Milgram's famous letter-passing experiment on a massive scale using e-mail and show that there are really no hubs for transmitting messages, and that anybody in a network is as likely as anybody else to be an intermediary. But given that people might be reluctant to tap the most influential members of their network for a trivial favor like relaying an e-mail in an experiment, is the small-world study really a good model of how influence spreads?
No, I don't think the small-world experiment is a model of influence, and I don't claim that it is. But people think the small-world experiment works because of hubs, and I'm rebutting that claim on its own territory.

Along these lines you also describe your Twitter study, which, contrary to popular beliefs about viral marketing, showed that the overwhelming majority of messages don't get retweeted at all, and that previously influential Twitter users aren't reliably influential in the future. But Twitter seems as much a broadcast medium as mass media is. On Twitter you can follow somebody like Oprah directly just as you can tune into Oprah on TV, so I don't understand the distinction you make between Twitter and mass media.
I have a whole paper on this that didn't make it into the book. Twitter is interesting for studying influence because it spans the whole gamut—from entities that are true mass media, like CNN and The New York Times, through celebrities who would previously have had to use mass media instead of reaching people directly, all the way down to ordinary individuals. A lot of the debate about influencers has been mired in ambiguity because it's not clear who or what we're talking about, and the nice thing about Twitter is you can measure influence, and it has all these types of people in it, so you can compare them in an apples-to-apples manner. It does matter, on average, how many followers you have and how successful you've been in spreading your messages in the past, but it's a lot more random than intuition suggests. And then you can ask the real question, which is how to maximize the impact of your marketing dollar.

So how is what you know about the spread of cultural influence affecting how you're marketing your own book? How, for example, are you and your publisher allocating your review copies?
It's pretty random [laughs]. On average, you've got the people with the largest span—the editors of major publications—which is what people do anyway. What I can't really do is get the data on all of this. What is the effect of getting into The New York Times Book Review versus a getting a Q and A in Scientific American? I can't measure any of that—and because I can't, I can't optimize my strategy. So I take the blanket approach. And people already do that, which raises the question of why do they believe in the influencers stuff anyway?

I always thought the problem with targeting influencers—assuming we all know who they are—is that everybody else is targeting them.
We're not even at that problem yet; the popular books about influence, like The Tipping Point and Ed Keller and Jon Berry's book, The Influentials [Free Press, 2003], are very clear that the influencers are not the celebrities and not the CEOs and not the politicians. They say it's ordinary people. Sure, Oprah's great, but it's hard to get on Oprah. So if there's an ordinary person out there who does the same thing in a magical, word-of-mouth way, that's brilliant! That's where the appeal comes from—it's the free lunch theory.