Neil deGrasse Tyson, the renowned science communicator, earned his Ph.D. in Astrophysics from Columbia University in 1991. About 4,000 astrophysicists resided in the country at the time. Tyson brought the total number of African-Americans among them to a paltry seven. In a convocation address, he spoke openly about the challenges he faced:

“In the perception of society ... my academic failures are expected and my academic successes are attributed to others,” Tyson said. “To spend most of my life fighting these attitudes levies an emotional tax that is a form of intellectual emasculation. It is a tax that I would not wish upon my enemies.”

Tyson's words speak to a broad truth: negative stereotypes impose an intellectual burden on many minorities and on others who think that the people around them perceive them as inferior in some way. In many different situations—at school, at work or in sports stadiums—these individuals worry that they will fail in a way that affirms derogatory stereotypes. Young white athletes fear that they will not perform as well as their black peers, for example, and women in advanced math classes worry that they will earn lower grades than the men. This anxiety—Tyson's “emotional tax”—is known as stereotype threat. Hundreds of studies have confirmed that stereotype threat undermines performance, producing the very failure they dread. Sometimes people become trapped in a vicious cycle in which poor performance leads to more worry, which further impedes performance.

In recent years psychologists have greatly improved their understanding of how stereotype threat affects individuals, why it happens and, most important, how to prevent it. Although the threat is real, some researchers question how well some of the relevant laboratory studies mirror anxiety in real-world settings; they also note that it is just one of many factors that contribute to social and academic inequality. Yet it is also one of the factors that can be easily changed. In studies conducted in actual schools, relatively simple interventions—such as self-esteem-boosting writing exercises completed in less than an hour—have produced dramatic and long-lasting effects, shrinking achievement gaps and expelling stereotype threat from the classroom and students' minds. Some educators are working on ways to scale up these interventions to statewide education programs.

Identifying the Threat

Two psychologists, Claude Steele of Stanford University and Joshua Aronson, then also at Stanford, coined the term “stereotype threat” in 1995. Then, as now, black students across the U.S. earned worse grades on average than their peers and were more likely to drop out early at all levels of education. The various explanations for this gap included the pernicious idea that black students were innately less intelligent. Steele and Aronson were not convinced. Instead, they reasoned, the very existence of this negative stereotype might impair a student's performance.

In a now classic experiment, they presented more than 100 college students with a frustrating test. When they told the students that the exam would not measure their abilities, black and white students with comparable SAT scores did equally well. When Steele and Aronson told the students that the test would assess their intellectual ability, however, the black students' scores fell, but those of their white peers did not. Simply asking the students to record their race beforehand had the same effect.

The study was groundbreaking. Steele and Aronson showed that standardized tests are far from standardized. When presented in a way that invokes stereotype threat, even subtly, they put some students at an automatic disadvantage. “There was a lot of skepticism at first, but it's reducing with time,” Aronson says. “In the beginning, even I didn't believe how strong the effects were. I thought, ‘Somebody else has to replicate this.’”

Many researchers have. To date, hundreds of studies have found evidence of stereotype threat in all manner of groups. It afflicts students from poorer backgrounds in academic tests and men in tasks of social sensitivity. White students suffer from it when pitted against Asian peers in math tests or against black peers in sports. In many of these studies, the strongest students suffer the greatest setbacks. The ones who are most invested in succeeding are most likely to be bothered by a negative stereotype and most likely to underperform as a result. Stereotype threat is nothing if not painfully ironic.

Exactly how pervasive stereotype threat is in real-world settings remains somewhat unclear, however, largely because the relevant studies face the same problems that plague much of social psychology. Most were conducted with small numbers of college students—which increases the chances of statistical flukes—and not all studies found a strong effect. Some critics also note that laboratory experiments are often a poor substitute for the real world. Paul Sackett of the University of Minnesota has argued that outside the lab, stereotype threat could be less common and more easily overcome. Last year Gijsbert Stoet, then at the University of Leeds in England, and David C. Geary of the University of Missouri–Columbia examined every study that looked for stereotype threat among women taking math tests—a phenomenon that Steele and his colleagues first identified in 1999. Out of 20 that repeated the 1999 experiment, only 11 concluded that women performed worse than men. Geary is not ready to discount stereotype threat, but he thinks it may not be as strong as it is sometimes portrayed.

Ann Marie Ryan of Michigan State University has identified some plausible reasons for such inconsistent conclusions. In 2008 she and Hanna-Hanh Nguyen, then at California State University, Long Beach, compared the results of 76 different studies on stereotype threat in high schoolers and undergraduates. They found that in the lab, scientists are able to detect the threat only under certain conditions, such as when they give volunteers an especially difficult test or when they work with people who strongly identify with their social group.

In the past decade psychologists have shifted from showing that stereotype threat exists to understanding how it works. Researchers have demonstrated that the threat operates in the same way across different groups of people. Anxiety arrives; motivation falls; expectations lower. Building on these findings, Toni Schmader of the University of British Columbia surmised that the threat preys on something fundamental. The most obvious culprit was working memory—the collection of cognitive skills that allows us to temporarily hold and manipulate information in our mind. This suite of skills is a finite resource, and stereotype threat can drain it. Individuals might psychologically exhaust themselves by worrying about other people's prejudices and thinking about how to prove them wrong. To test this idea, Schmader gave 75 volunteers a difficult working memory test, during which they had to memorize a list of words while solving mathematical equations. She told some volunteers that the test would assess their memory skills and that men and women may have inborn differences in their abilities. Sure enough, women who were told of this supposed discrepancy kept fewer words in mind, whereas their male colleagues had no such problems.

This depletion of working memory creates various stumbling blocks to success. People tend to overthink actions that would otherwise be automatic and become more sensitive to cues that might indicate discrimination. An ambiguous expression can be misread as a sneer, and even one's own anxiety can become a sign of imminent failure. Minds also wander, and self-control weakens. When Schmader stopped women in the middle of a math test and asked them what they were thinking of, those under stereotype threat were more likely to be daydreaming.

Expelling Stereotypes

Most recently, researchers have moved the study of stereotype threat out of the lab and into schools and lecture halls, where they try to dispel or prevent the threat altogether. “I see three waves of research,” Schmader says. “The first was identifying the phenomenon and how far it travels. The second was looking at who experiences the effect and its mechanisms. The third wave is now to translate these results into interventions.”

Geoffrey Cohen, also at Stanford, has achieved particularly impressive results. His method is disarmingly simple: he asks people to consider what is important to them, be it popularity or musical ability, and write about why it matters. The 15-minute exercise acts like a mental vaccine that boosts students' self-confidence, helping them combat any future stereotype threat.

In 2003 Cohen visited racially diverse middle schools in California and put his exercise through a randomized controlled trial—the gold-standard test in medicine that checks if an intervention works by pitting it against a placebo. Cohen administered his exercise to seventh graders: half wrote about their own values, and the rest wrote about things that were unimportant to them. The trial was double-blinded, meaning that neither Cohen nor the students knew who was in which group.

At the end of the term, black students who completed the exercise had closed a 40 percent academic gap between them and their white peers. Best of all, the students at the bottom of the class benefited most. Over the next two years the same students took two or three booster versions of the original exercise. Only 5 percent of the poorest students who wrote about their values ended up in remedial classes or repeated a grade, compared with 18 percent of those in the control group. Ultimately, the black students' grade point averages rose by a quarter of a point and by 0.4 point among the worst performers.

A few fractions of a point here and there might not seem like a huge improvement, but even small changes in confidence—whether positive or negative—have a cumulative effect. Children who do poorly at first can quickly lose self-confidence or a teacher's attention; conversely, signs of modest progress can motivate far greater success. By intervening early on, Cohen asserts, educators can turn vicious cycles into virtuous ones.

Cohen's task is so simple that Ryan and others are not entirely convinced by his results. “It was hard for us to believe, but we've replicated it since,” Cohen says. In the past five years he has used his exercise to swing the fortunes of black students in three different middle schools and to largely close the gender gap in a college-level physics class. Skeptics, though, still hope that independent researchers will try to replicate these studies.

Meanwhile Cohen is seeking new ways to help students. He has collaborated with Greg Walton, also at Stanford, to counter a kind of isolation that stereotype threat often induces. Many minorities worry that their academic peers will not fully accept them. Walton combated these worries with survey statistics and quotes from older students showing that such feelings are common to everyone regardless of race and that they disappear with time. “It makes them reframe their own experiences through the lens of this message, rather than of race,” Walton explains.

Walton and Cohen tested their hour-long exercise with college students in their first spring term. Three years later, when the students graduated, the achievement gap between blacks and whites had been halved. The black students were also happier and healthier than their peers who did not take part in Walton's exercise. In the past three years they had made fewer visits to the doctor. Walton acknowledges that such a simple exercise may look trivial to an outsider. But, he says, for a students who are “actively worried about whether they fit in, the knowledge that those concerns are shared and temporary is actually very powerful.”

Cohen and Walton are now scaling up their simple and inexpensive interventions from individual schools to entire states. The pair—as well as Carol Dweck and Dave Paunesku—both also at Stanford, created PERTS (the Project for Education Research That Scales), which allows them to rapidly administer their interventions online. They can also combine the programs or pit them against one another to see which have the greatest effects.

Even if the programs work as planned, researchers who study stereotype threat admit that undoing it is not a panacea against inequality. Cohen, for example, tested his initial writing exercise only in schools with mixed ethnicities, and he is unsure if it would work in predominantly minority schools. “There are many reasons why we have achievement gaps—inequality of resources, bad schools, less well-trained teachers,” Walton adds. “There doesn't seem to be much hope of addressing these structural barriers. What's exciting about stereotype threat is that we can make headway in the face of those things.”

Recent work on the phenomenon not only offers realistic hope for alleviating some truly tenacious problems—it also upends pervasive beliefs. By thwarting stereotype threat, researchers have shown that the stereotypes themselves are unfounded. Performance gaps between black and white students or between male and female scientists do not indicate differences in ability; rather they reflect prejudices that we can change. “The things we thought were so intractable 15 years ago aren't,” Aronson says, “and that's a hugely positive message.”