By Nicola Nosengo of Nature magazine

If you are a small animal, it is useful to know whether there is anything around that might want to eat you. Stephen Liberles from Harvard Medical School in Cambridge, Massachusetts, and his colleagues have analyzed urine samples from a variety of zoo inhabitants, including lions and bears, and discovered how rodents can use smell to do just that.

In a research published today in the Proceedings of the National Academy of Science, the team identifies a chemical found in high concentrations in the urine of carnivores that makes mice and rats run for cover.

Chemicals have already been identified that allow prey to recognize a known predator. But this is the first example of a generic clue that allows an animal to detect any potential predator, irrespective of whether the two species have ever come into contact.

The researchers started by analyzing an enigmatic group of olfactory receptors discovered in 2001 called trace amine-associated receptors (TAARs). They are found in most vertebrates, in varying numbers. Mice, for example, have 15, rats 17 and humans have just 6. Very little is known about what chemicals bind to them.

Liberles and his colleagues found that one member of the receptor family, TAAR4, is strongly activated by bobcat urine, which is sold online and used by gardeners to keep rodents and rabbits away. They managed to extract the molecule responsible for activating the receptor, called 2-phenylethylamine.

They then wondered whether the molecule was specific to the bobcat. But the urine from other animals cannot always be bought as easily. "Also, commercial products may be contaminated, whereas we wanted to be sure we were studying only natural substances," says David Ferrero, a graduate student in Liberles's lab and first author of the study.

So the researchers collected urine samples from a range of sources, including zoos in New England and South Dakota. Their collection covered 38 species from predators such as lions, snow leopards and servals to herbivores including cows, giraffes and zebra. They also tested humans, cats and various rodents.

The operation was not trivial. A giraffe had to be trained to urinate in a cup, and Ferrero had a nose-to-nose encounter with an uncooperative jaguar when the animal jumped against the bars as he approached its cage.

Top of the list

Carnivores had by far the greatest concentration of 2-phenylethylamine in their urine, with the highest levels in lion, serval and tiger. Levels in the herbivores' urine were up to 3,000 times lower. The chemical might be a by-product of digesting meat proteins, although the researchers have yet to confirm this idea.

Liberles and his team double-checked the role of 2-phenylethylamine by placing a few drops of it - on its own, or within lion urine - in a cage. They found that mice and rats stayed away from that part of the cage. But when they used an enzyme to remove the chemical from lion urine, the drops no longer caused any reaction.

"The role of TAAR receptors is still a bit of a mystery," says Anna Menini, a physiologist at the International School for Advanced Studies in Trieste, Italy, and president-elect of the European Chemoreception Research Organization in Paris. "Here we have the first convincing evidence that they might control instinctive behavior."

She adds that the study questions a dogma in olfactory studies: that the olfactory receptors that trigger instinctive responses are found only in the vomeronasal organ, a part of the olfactory system that humans have lost. TAARs are in the olfactory epithelium--specialized tissue on the roof of the nasal cavity--which humans have, although they do not have an active gene for TAAR4 itself.

The researchers are still missing the smoking gun for proving that TAAR4 directly controls the animal's behavior: a mouse in which this receptor has been knocked out should be fearless when faced with a carnivore's urine. Liberles says he is working on this, as well as studying what brain circuits are activated by the receptor.

"That is the big black box in neuroscience" he says. "We know a lot about perception and we can observe behavior, but we need to find the circuits in the brain that bridge the two. TAAR4 offers a way to do that."

This article is reproduced with permission from the magazine Nature. The article was first published on June 20, 2011.