The global obesity epidemic is one of the greatest health challenges facing humanity. Some 600 million, or 13 percent, of the world's adults were obese in 2014—a figure that had more than doubled around the globe since 1980. At present, 37 percent of American adults are obese, and an additional 34 percent are overweight. If current trends continue, health experts predict that half of all Americans will be obese by 2030.

If fad diets, reality television programs and willpower could make a dent in the problem, we would have seen a change by now. Obesity (characterized by excess body fat and measured as 120 percent or more of ideal weight) is much too complex to be solved with quick fixes, however. Figuring out why we eat what we eat, how the body controls weight and how best to get people to change unhealthy habits is not easy. Our laboratory has spent the past two decades trying to develop, with all the rigor that science allows, more effective methods for treating obesity and maintaining a healthy weight.

Much of our work has challenged common dogmas and opened doors for new approaches. We have shown, for example, that exercise is not the most important thing to focus on when you want to lose weight—although it has numerous other health benefits, including maintaining a healthy weight. As many experts have suspected and as we and others have now proved, what you eat and how much you eat play a substantially greater role in determining whether you shed kilograms. But our research has gone much deeper, showing that different people lose weight more effectively with different foods. This realization allows us to create personalized weight-loss plans for individuals that work better than any one-size-fits-all advice.

We believe this new understanding could improve the health of millions of people around the world. Obesity increases the risk of all the major noncommunicable diseases, including type 2 diabetes, heart disease, stroke and several types of cancers—enough to decrease a person's potential life span by as much as 14 years. Research shows that excessive weight also interferes with our body's ability to fight off infections, sleep deeply and age well, among other problems. It is long past time for us to understand how to combat this epidemic.

Fuel-Efficient

Losing weight can be reduced to a simple mathematical formula: burn more calories than you consume. For decades health experts figured that it did not matter too much how you created that deficit: as long as you got the right nutrients, you could safely lose weight with any combination of increased exercise and reduced consumption of food. But this assumption does not take into account the complexities of human physiology and psychology and so quickly falls apart when tested against real-world experience. As it happens, sorting out the details and putting weight management on a more scientific footing have taken much longer and have required a wider range of expertise than anyone had expected.

Our first step, beginning in the 1990s, was to determine a base requirement: How much energy does it take to fuel the average human body? This straightforward question is not easy to answer. People get their energy from food, of course. But for individuals to use that energy, the food must be broken down or metabolized to become the equivalent of gasoline for a car. The oxygen we breathe helps to burn that fuel, and whatever is not used right away is stored in the liver as glycogen (a form of carbohydrate) or fat. When no more space is available in the liver, the excess is stored elsewhere in fat cells. In addition, metabolism creates carbon dioxide, which we exhale, as well as other waste products that are excreted as urine and feces. The process runs at different levels of efficiency in different individuals and under different circumstances in the same individual.

For a long time the best way to measure people's energy expenditure was to have them live for two weeks in a specialized lab, such as ours, where researchers could measure everything subjects eat and track their weight. Another way was to put volunteers in a sealed room (called a calorimeter) and measure the oxygen they breathe and the carbon dioxide they exhale. From these measurements we could assess the body's basic energy requirements. Neither method is terribly convenient, and neither does a good job of replicating the conditions of everyday life.

A much easier approach uses so-called doubly labeled water, which contains tiny amounts of deuterium (2H) and oxygen 18 (18O), both harmless, nonradioactive isotopes. For one to two weeks after a person drinks doubly labeled water, the body excretes the deuterium and some of the oxygen 18 in urine. (The rest of the oxygen 18 is exhaled as carbon dioxide.) Investigators take urine samples and compare how quickly these two isotopes disappear from the body during that time. With these data, they can calculate the number of calories an individual burns without interrupting his or her daily routine.

The method was developed in the 1950s, but for decades doubly labeled water was too expensive to use in people. By the 1980s prices had dropped, and the technique had become more efficient, although there were times when our lab had to spend as much as $2,000 to perform a single measurement. As a result, it took more than 20 years to accumulate enough data to figure out how much energy the body needs to avoid weight gain or loss.

These experiments—conducted by our group and others—helped us determine that humans do not need a lot of calories to stay healthy and active. And any excess consumption quickly results in weight gain. In this respect, we are much like other primates, including chimpanzees and orangutans. An adult male of healthy weight and typical height living in the U.S. today requires about 2,500 calories per day to maintain his weight, whereas the average nonobese adult female requires around 2,000 calories. (Men tend to use more calories because, on average, they have larger bodies and greater muscle mass.)

In contrast, studies show that species as diverse as red deer (Cervus elapus, average weight 100 kilograms for the six-year-old females in one experiment) and gray seals (Halichoerus grypus, average weight 120 kilograms for three adult females) require two to three times more calories, kilogram per kilogram, than primates to maintain their size.

It is tempting to assume that Americans have low calorie requirements because they lead sedentary lives, but researchers have documented similar calorie needs even in indigenous populations leading very active lives. Herman Pontzer of Hunter College and his colleagues measured the calorie requirements of the Hadza people in northern Tanzania, a group of hunter-gatherers, and found that the men needed 2,649 calories on average per day. The women, who—like the men—tend to be smaller than counterparts in other regions, needed just 1,877. Another study of the indigenous Yakut people of Siberia found requirements of 3,103 calories for men and 2,299 for women. And members of the Aymara living in the Andean altiplano were found to require 2,653 calories for men and 2,342 calories for women.

Although our calorie requirements have not changed, government data show that, on average, Americans consume 500 more calories (the equivalent of a grilled chicken sandwich or two beef tacos at a fast-food restaurant) each day than they did in the 1970s. An excess of as little as 50 to 100 calories a day—the equivalent of one or two small cookies—can lead to a gain of one to three kilograms a year. That easily becomes 10 to 30 kilograms after a decade. Is it any wonder, then, that so many of us have become overweight or obese?

Complicated Calories

The formula for maintaining a stable weight—consume no more calories than the body needs for warmth, basic functioning and physical activity—is just another way of saying that the first law of thermodynamics still holds for biological systems: the total amount of energy taken into a closed system (in this case, the body) must equal the total amount expended or stored. But there is nothing in that law that requires the body to use all sources of food with the same efficiency. Which brings us to the issue of whether all calories contribute equally to weight gain.

Research in this area is evolving, and understanding why it has taken so long to get definitive answers requires a trip back in history to the late 1890s and the tiny community of Storrs, Conn. There a chemist by the name of Wilbur O. Atwater built the first research station in the U.S. designed to study the production and consumption of food. In fact, Atwater was the first to prove that the first law of thermodynamics holds for humans as well as animals. (Some scientists of his day thought people might be an exception to the rule.)

The experimental design of metabolic labs has changed remarkably little since Atwater's day. To determine how much energy the body can derive from the three major components of food—proteins, fats and carbohydrates—he asked a few male volunteers to live, one at a time, inside a calorimeter for several days. Meanwhile Atwater and his colleagues measured everything each human guinea pig ate, as well as what became of that food, from the carbon dioxide the volunteer exhaled to the amounts of nitrogen, carbon and other components in his urine and feces. Eventually the researchers determined that the body can extract about four calories of energy per gram from proteins and carbohydrates and nine calories per gram from fat. (These numbers are now known as Atwater factors.)

Food does not come to us as pure protein, carbohydrate or fat, of course. Salmon consists of protein and fat. Apples contain carbohydrates and fiber. Milk contains fat, protein, carbohydrates and a lot of water. It turns out that a food's physical properties and composition play a greater role in how completely the body can digest and absorb calories than investigators had anticipated.

In 2012, for example, David Baer of the U.S. Department of Agriculture's Beltsville Human Nutrition Research Center in Maryland proved that the body is unable to extract all the calories that are indicated on a nutritional label from some nuts, depending on how they are processed. Raw whole almonds, for example, are harder to digest than Atwater would have predicted, so we get about a third fewer calories from them, whereas we can metabolize all the calories found in almond butter.

Whole grains, oats and high-fiber cereals are also digested less efficiently than we used to think. A recent study by our team looked at what happened when volunteers consumed a whole-grain diet that included 30 grams of dietary fiber versus more typical American fare that contained half as much fiber. We detected an increase in the number of calories lost to the feces, as well as a bump in metabolism. Together these changes amounted to a net benefit of nearly 100 calories a day—which can have a substantial effect on weight over a period of years.

And so we and others have proved that not all calories are equal—at least for nuts and high-fiber cereals. As scientists learn more about how efficiently different foods are digested and how they affect the body's metabolic rate, we will likely see some other examples of such disparities that are just large enough to influence how easy—or hard—individuals find managing their weight.

Energy Expenditure

So much for what we put in our mouth. What our body does with the food we eat brings us to the other side of the energy balance equation—energy expenditure. Researchers are discovering a surprising deal of variability here as well.

One of the most common pieces of advice that people get when they are trying to lose weight is that they should exercise more. And physical activity certainly helps to keep your heart, brain, bones and other body parts in good working order. But detailed measurements conducted in our lab and others show that physical activity is responsible for only about one third of total energy expenditure (assuming a stable body weight). The body's basal metabolism—that is, the energy it needs to maintain itself while at rest—makes up the other two thirds. Intriguingly, the areas of the body with the greatest energy requirement are the brain and certain internal organs, such as the heart and kidneys—not the skeletal muscle, although strength training can boost basal metabolism modestly.

Credit: Brown Bird Design (food illustrations); Amanda Montañez (graphs); Sources: U.S. Department of Agriculture Food Composition Databases (food data); “High-Protein Weight-Loss Diets: Are They Safe and Do They Work? A Review of the Experimental and Epidemiologic Data,” by Julie Eisenstein et al., in Nutrition Reviews, Vol. 60, No. 7; July 2002 (thermic effect data); Human Energy Requirements: Report of a Joint FAO/WHO/UNU Expert Consultation. World Health Organization, Food and Agriculture Organization of the United Nations and United Nations University. FAO, 2001 (basal metabolism data); “High-Glycemic Index Foods, Overeating, and Obesity,” by David S. Ludwig et. al., in Pediatrics, Vol. 103, No. 3; March 1999 (glycemic index data)

In addition, as anyone who has ever reached middle age understands all too well, metabolism changes over time. Older people need fewer calories to keep their body running than they did in their youth. Metabolic rate also differs among individuals. One study published in 1986 measured the metabolic rates of 130 people from 54 families. After accounting for differences in age, gender and body composition, investigators reported variability among families of around 500 calories a day. The inescapable conclusion: when it comes to metabolic rate—and your ability to lose or maintain your weight—parentage makes a difference.

But let us suppose that you have started to lose some weight. Naturally, your metabolic rate and calorie requirements must fall as your body becomes smaller, meaning that weight loss will slow down. That is just a matter of physics: the first law of thermodynamics still applies. But the human body is also subject to the pressures of evolution, which would have favored those who could hold on to their energy stores by becoming even more fuel-efficient. And indeed, studies show that metabolic rate drops somewhat more than expected during active weight loss. Once a person's weight has stabilized at a new, lower level, exercise can help in weight management by compensating for the reduced energy requirement of a smaller body.

Hungry Brains

Variations in Atwater factors and metabolic rates are not the end of the story. A growing body of research has demonstrated that our brain plays a central role, coordinating incoming signals from a wide range of physiological sensors in the body while alerting us to the presence of food. The brain then creates sensations of hunger and temptation to make sure that we eat.

In other words, the role of hunger has long been to keep us alive. Thus, there is no point in fighting it directly. Instead one of the keys to successful weight management is to prevent hunger and temptation from happening in the first place.

Single-meal feeding tests by several labs, including our own, show that meals higher in protein or fiber or those that do not cause a sudden spike in blood sugar (glucose) levels are generally more satisfying and better at suppressing hunger. (Carbohydrates are the most common source of glucose in the blood, but proteins can generate it as well.) A summary one of us (Roberts) published in 2000 indicated that calorie consumption in the hours following a breakfast with a so-called high glycemic index (think highly processed breakfast cereals) was 29 percent greater than after a morning meal with a low glycemic index (steel-cut oatmeal or scrambled eggs).

In fact, our team recently obtained the first preliminary data showing that it is possible to reduce hunger during weight loss by choosing the right foods. Before assigning 133 volunteers to one of two groups, we asked them to answer a detailed questionnaire about how often, when and how intensely they were hungry. Then we randomly assigned subjects to either a weight-loss program that emphasized foods high in protein and fiber and low in glycemic index (fish, beans, apples, vegetables, grilled chicken and wheat berries, for example) or to a “waiting list,” which served as the control group.

Remarkably, over the course of six months members of the experimental group reported hunger levels that decreased to below the values measured before the program began. We noticed a difference on the scales as well. By the end of the study, they had also lost an average of eight kilograms, whereas the control group had gained 0.9 kilogram.

Just as interesting, the intervention group experienced fewer food cravings as well, which suggests that what their brains perceived as pleasurable had changed. We then scanned the brains of 15 volunteers as they viewed pictures of a wide range of foods. The results showed that the reward center of the brain became more active over time in the intervention group in response to pictures of grilled chicken, whole-wheat sandwiches and fiber cereal. Meanwhile that group's brains became less responsive to images of french fries, fried chicken, chocolate candies and other fattening foods.

Personalized Diets

Differences in the hunger-reducing properties of foods, the efficiency with which they are absorbed and the real, though limited, ability of our metabolism to adapt to changes in energy intake make weight management a complex system. We keep finding special circumstances that affect various people differently. For example, it has been well established that the majority of individuals who are obese secrete proportionately higher levels of insulin, the hormone that helps the body to metabolize glucose. This so-called insulin resistance leads to a host of other metabolic problems, such as increased risk of heart attack or developing type 2 diabetes. When we placed such people on a six-month weight-loss program featuring more protein and fiber, fewer carbohydrates and a low glycemic index, we found that they lost more weight than they could on a high-carbohydrate diet with a high glycemic index. People with low insulin levels, in contrast, did equally well on diets that were higher or lower in the ratio of proteins and carbohydrates, as well as in glycemic index.

Today we regularly help our study volunteers lose weight and keep it off. Despite the fact that our 133-volunteer investigation, described earlier, was six months long and required participants to attend weekly meetings and reply to e-mails during most of that time, only 11 percent dropped out. Some even cried at the research team's final visit because they did not want to say goodbye. Not only had they lost weight, but they had been so much more successful than they expected that they felt transformed psychologically as well as physically. In the words of one participant, “the science worked.”