When humans live together, they work together—whether they are building cities or just trying to snag some grub for dinner. But if you can still reap the benefits of contributions to the greater good without lifting a finger yourself, why would you not choose that option? Chances are you tow the line out of fear of being punished.

"The problem with punishment—why it's an interesting question for evolutionary biologists and anthropologists—is [that] it's not clear how this behavior would evolve," says Christoph Hauert, a research associate in evolutionary dynamics at Harvard University and lead author of a study on punishment in the new issue of Science.

Hauert and co-author Karl Sigmund, a mathematician at the International Institute for Applied Systems Analysis in Laxenburg, Austria, devised a mathematical model that determines how groups fare when collaborating on an activity. They reasoned that within any such community, there would be three types of individuals: cooperators, who do their fair share and are rewarded for their effort; defectors, who beg off but still benefit from the others' work; and punishers, who penalize the defectors.

After running multiple versions of their algorithm, with different variables adjusted in each trial, the researchers discovered that the punishers must dominate to sustain the success of a collective activity. But, ironically, they discovered that the effort—be it cutting carbon emissions or hunting bison—must be voluntary if punishers are to rule the roost.

Hauert and Sigmund explain their puzzling findings by citing what happens in hunter-gatherer communities. Initially, each community member takes care of his or her own needs, subsisting on, say, mushrooms found on the ground. Eventually weapons are introduced, at which point members turn to hunting to survive. But after a few weeks of cooperating and sharing the kill, one member of the tribe figures out that he can reap the benefits of the hunt without participating by hiding behind a tree while the others do all the work.

In the researchers' first simulation, participation in the hunt was mandatory. Under those conditions, Hauert and Sigmund say, other members of the community began to notice the freeloaders and realized that the biggest payoff for the least amount of work came from also being a defector. Soon, the so-called defectors came to dominate the group, which, in this instance, would likely result in the failure of hunting as a means of survival and force everyone to revert to subsisting on mushrooms.

This was not the case when a fourth type of individual was introduced into the model: the nonparticipant, who, in the hunter-gatherer example, would neither hunt nor be allowed to eat the kill. According to the model, the collective would then became dominated by punishers, who would keep everyone in line; penalizing defectors and by doing so keeping others from adopting their lazy ways, thus sustaining the success of the hunt.

Essentially, the researchers detail an oscillating cycle: Some cooperators may emerge from a group of nonparticipants, who increase their bounty relative to the group and make their cooperative practice the norm. Then, the group of cooperators may become dominated by defectors, who ruin everything for everyone. Punishers would be unwelcome when defectors take over, as they would have to police the entire group—at high cost to themselves. But what the algorithm has pointed out is that if punishers become dominant before defectors take over a group, they can ensure long-term cooperation … at least until a new innovation comes along—say, processed food—and the cycle begins anew.

In an editorial in the same issue of Science Robert Boyd and Sarah Mathew, anthropologists at the University of California, Los Angeles, write that the new study "provides the first cogent mechanism that can jump-start the evolution of punishment."

"Essentially, we are looking here at a game between cops and robbers," says Sigmund. When the activity is compulsory, he explains, the robbers win; when it is optional, the cops win. "If there is a possibility for being a nonparticipant, then everyone becomes a cop," he says. Sigmund is now working to see if these theoretical results can be replicated in actual human trials in which students participate in public goods games.