In many animals, relatives tend to stay close, either sharing the same territory or living in neighboring ones. By sticking together, individuals can defend food, mates and other resources, thereby working to perpetuate the family genes, even if not all manage to breed or raise young.

One particular mammal, however, turns this general observation on its head, and experts in behavioral ecology do not quite understand why. Striped hyenas (Hyaena hyaena), which live in Africa and parts of Asia, demonstrate so-called protosocial tendencies. Although little direct interaction occurs among individual hyenas, a closer look reveals that they actually form spatial groups living on exclusive and stable ranges just like species that display obvious social behavior. One would expect relatives to share the same territory or to inhabit territories close together. This is not always so, however, claims Aaron P. Wagner, now at Michigan State University.

Wagner and his colleagues trapped an entire population of striped hyenas in Kenya’s Laikipia District, about 135 miles north of Nairobi, and collected its members’ DNA. The scientists also radio-tracked the animals to understand their movements. “As far as we are aware, the patterns of relatedness and space use we found in these hyenas are unseen in any other carnivore,” says Wagner, who published some of the results of the four-year study in the October 2007 Molecular Ecology.

Wagner’s team found that the spatial groups formed by this species always consist of just a single female defended by up to three males. Similar coalitions occur in other carnivores, but in those cases the males usually guard several females so that each male has a chance to mate. With only one female available, however, some male striped hyenas are unlikely to breed at all. Such sacrifice and cooperation could be understandable if all males were related, but genetic analysis showed that the hyena coalitions often include unrelated males.

Even more remarkable, for both sexes, hyenas living in neighboring territories are less closely related than nonneighbors. Wagner found this arrangement puzzling because individuals that disperse and cross territorial boundaries are at great risk of being attacked and killed by others of their species. “Therefore, you would expect to find relatives living near one another and for relatedness to decline with distance,” Wagner says.

No single theory can explain the overall patterns recorded. Wagner speculates that for males, any form of coalition—even with nonrelatives—might provide a temporary safe haven when the alternative of roaming alone is too hazardous or the chance of single-handedly defending a female is too low.

But given the dangers, why would striped hyenas leave their kin, risking death to travel to distant nonneighboring territories? Perhaps relatives settle in nonadjacent territories to avoid cross-border competition and costly fights with one another. In the spotted hyena (Crocuta crocuta), male and female group members sometimes engage in aggressive disputes with neighbors over food or mates—even battling with their own kin. Zoologist Kay E. Holekamp, also at Michigan State, questions the evidence that striped hyenas actually engage in these boundary conflicts at all and instead suggests that inbreeding avoidance might account for their odd patterns of dispersal. According to Wagner, however, the need to prevent inbreeding would not explain why low levels of relatedness were found among neighbors of the same sex.

Maybe, at the time of the study, there was nowhere else to go, suggests ecologist Hans Kruuk of the University of Aberdeen in Scotland. “Dispersing animals might not have found an empty space in a neighboring territory,” meaning that individuals would have had to venture farther into several territories. Wagner doubts this saturated-environment scenario, arguing that it could explain why some relatives end up far away but does not explain why almost none ever live close by.

Calling the results “tantalizing,” biologist Gus Mills of South Africa’s Tony and Lisette Lewis Foundation, which funds nature conservation, appeals for longer-term studies. “Large carnivores are long-lived and reproduce quite slowly. Getting large samples over extended periods is very difficult and so is interpretation of results.”

Wagner agrees and argues that research has been too skewed toward species with well-developed social behavior, such as lions or spotted hyenas. “Up to 90 percent of carnivores are, like striped hyenas, behaviorally solitary. We must learn more about the solitary majority before claiming we know why and how groups form and how sociality evolved.”