To spank or not to spank? This age-old parenting question elicits fierce debate among parents, psychologists and pediatricians. Surveys suggest that nearly half of U.S. parents have spanked their children as a disciplinary tactic, but many experts argue that this form of punishment—hitting a child on the bottom with an open hand—increases the risk that kids will develop emotional and behavioral problems. Other scientists counter that research on the issue is fraught with problems, making it impossible to draw black-and-white conclusions. A new meta-analysis addresses several of the most contentious points in the debate and concludes that spanking does pose risks, but differences of opinion persist.

In the meta-analysis, researchers Elizabeth Gershoff and Andrew Grogan-Kaylor from the University of Texas at Austin and the University of Michigan, respectively, evaluated 75 published studies on the relationship between spanking by parents and various behavioral, emotional, cognitive and physical outcomes among their kids. They found that spanking was associated with 13 out of a total of 17 negative outcomes they assessed, including increased aggression and behavioral and mental health problems as well as reduced cognitive ability and self-esteem.

The meta-analysis was not simply an attempt to synthesize studies—Gershoff and Grogan-Kaylor also wanted to address two concerns often raised about the body of research linking spanking to childhood problems. The first is that much of it has evaluated the effects of physical punishment in general, without homing in on the effects of spanking specifically—and because physical punishment can include tactics such as hitting with objects, pinching and biting, this “lumping problem” may ultimately exaggerate spanking’s risks. The second concern is that many published studies are “cross-sectional,” which means that they evaluate the effects of spanking by collecting data at a single point in time, making determinations of cause and effect difficult. A cross-sectional study might, for instance, find that aggressive 10-year-olds were more likely than docile 10-year-olds to have been spanked as toddlers, but that does not mean that spanking made them aggressive. They may have been spanked because they were acting out back then, too.

To address these issues, Gershoff and Grogan-Kaylor did several things. First, they limited their meta-analysis to studies that evaluated the effects of spanking, slapping and hitting children without the use of objects, and found that spanking is still associated with negative outcomes. They also compared the results from cross-sectional studies with results from longitudinal studies, which track the kids’ behavior over time and are better able to tease out cause and effect. Gershoff and Grogan-Kaylor found that spanking is associated with negative outcomes in both types of studies, which strengthens the argument that spanking poses risks.

Yet some researchers remain skeptical. First, although the new analysis did attempt to separate the effects of spanking from those of physical tactics that are considered harsher, research has shown that many parents who spank also use other forms of punishment—so “you’re still not really isolating spanking from overall abusiveness,” explains Christopher Ferguson, a psychologist at Stetson University in Florida. In other words, the negative effects associated with spanking could still be driven in part by parents’ use of other tactics.

The new analysis also did not completely overcome the lumping problem: It considered slapping and hitting children anywhere on the body as synonymous with spanking but these actions might have distinct effects. Some research also suggests that the effects of spanking differ depending on the reasons parents spank, how frequently they do so and how old children are at the time—so the conclusion from the meta-analysis that spanking itself is dangerous may be overly simplistic. “I think it’s irresponsible to make exclusive statements one way or another,” Ferguson says.

Finally, the associations reported in the meta-analysis between spanking and negative outcomes did not control for the potential mediating effects of other variables, which raises the chicken-or-egg question: Are kids spanked because they act out or do they act out because they are spanked—or both? (Even longitudinal studies don’t completely resolve this problem, because behavioral problems may worsen over time regardless of spanking’s effects.) To rule out the possibility that spanking is only associated with bad outcomes because poorly behaved kids are the ones getting spanked, researchers can use statistical methods to control for the influence of temperament and preexisting behavioral characteristics—but these methods are difficult to employ in meta-analyses, and the new analysis did not attempt such a feat. Ferguson did try to control for the effects of preexisting child behavior in a 2013 meta-analysis he published of the longitudinal studies on this issue; when he did, “spanking’s effects became trivial,” he says. As a further demonstration of the importance of careful statistical controls, Robert Larzelere, a psychologist at Oklahoma State University, and his colleagues reported in a 2010 study that grounding and psychotherapy are linked just as strongly to bad behavior as spanking is but that all the associations disappear with the use of careful statistical controls. It makes sense that disciplinary tactics used as responses to bad behavior will be associated with such behavior, Larzelere says, unless care is taken to control for children’s preexisting characteristics and temperaments.

Still, a number of individual studies have found associations between spanking and negative outcomes, even after controlling for preexisting child behavior. So Gershoff says that in spite of the lingering controversy, the safest approach parents can take is not to spank their kids. “Studies continue to find that spanking predicts negative behavior changes—there are no studies showing that kids improve,” she says. And although Ferguson is not convinced that spanking is categorically bad, he is “certainly not an advocate of spanking.” Furthermore, there is a worrying body of research suggesting that parents who spank will later use harsher forms of punishment. “If spanking is not working, and spanking is all the parents are doing, then they’re going to escalate,” Gershoff says.