Intuitively, it makes sense Splatterhouse and Postal 2 would serve as virtual training sessions for teens, encouraging them to act out in ways that mimic game-related violence. But many studies have failed to find a clear connection between violent game play and belligerent behavior, and the controversy over whether the shoot-‘em-up world transfers to real life has persisted for years. A new study published on October 1 in Proceedings of the National Academy of Sciences tries to resolve the controversy by weighing the findings of two dozen studies on the topic.

The meta-analysis does tie violent video games to a small increase in physical aggression among adolescents and preteens. Yet debate is by no means over. Whereas the analysis was undertaken to help settle the science on the issue, researchers still disagree on the real-world significance of the findings.

This new analysis attempted to navigate through the minefield of conflicting research. Many studies find gaming associated with increases in aggression, but others identify no such link. A small but vocal cadre of researchers have argued much of the work implicating video games has serious flaws in that, among other things, it measures the frequency of aggressive thoughts or language rather than physically aggressive behaviors like hitting or pushing, which have more real-world relevance.

Jay Hull, a social psychologist at Dartmouth College and a co-author on the new paper, has never been convinced by the critiques that have disparaged purported ties between gaming and aggression. “I just kept reading, over and over again, [these] criticisms of the literature and going, ‘that’s just not true,’” he says. So he and his colleagues designed the new meta-analysis to address these criticisms head-on and determine if they had merit.

Hull and colleagues pooled data from 24 studies that had been selected to avoid some of the criticisms leveled at earlier work. They only included research that measured the relationship between violent video game use and overt physical aggression. They also limited their analysis to studies that statistically controlled for several factors that could influence the relationship between gaming and subsequent behavior, such as age and baseline aggressive behavior.

Even with these constraints, their analysis found kids who played violent video games did become more aggressive over time. But the changes in behavior were not big. “According to traditional ways of looking at these numbers, it’s not a large effect—I would say it’s relatively small,” he says. But it’s “statistically reliable—it’s not by chance and not inconsequential.”

Their findings mesh with a 2015 literature review conducted by the American Psychological Association, which concluded violent video games worsen aggressive behavior in older children, adolescents and young adults. Together, Hull’s meta-analysis and the APA report help give clarity to the existing body of research, says Douglas Gentile, a developmental psychologist at Iowa State University who was not involved in conducting the meta-analysis. “Media violence is one risk factor for aggression,” he says. “It's not the biggest, it’s also not the smallest, but it’s worth paying attention to.”

Yet researchers who have been critical of links between games and violence contend Hull’s meta-analysis does not settle the issue. “They don’t find much. They just try to make it sound like they do,” says Christopher Ferguson, a psychologist at Stetson University in Florida, who has published papers questioning the link between violent video games and aggression.

Ferguson argues the degree to which video game use increases aggression in Hull’s analysis—what is known in psychology as the estimated “effect size”—is so small as to be essentially meaningless. After statistically controlling for several other factors, the meta-analysis reported an effect size of 0.08, which suggests that violent video games account for less than one percent of the variation in aggressive behavior among U.S. teens and pre-teens—if, in fact, there is a cause-and effect relationship between game play and hostile actions. It may instead be that the relationship between gaming and aggression is a statistical artifact caused by lingering flaws in study design, Ferguson says.  

Johannes Breuer, a psychologist at GESIS–Leibniz Institute for the Social Sciences in Germany, agrees, noting that according to “a common rule of thumb in psychological research,” effect sizes below 0.1 are “considered trivial.” He adds meta-analyses are only as valid as the studies included in them, and that work on the issue has been plagued by methodological problems. For one thing, studies vary in terms of the criteria they use to determine if a video game is violent or not. By some measures, the Super Mario Bros. games would be considered violent, but by others not. Studies, too, often rely on subjects self-reporting their own aggressive acts, and they may not do so accurately. “All of this is not to say that the results of this meta-analysis are not valid,” he says. “But things like this need to be kept in mind when interpreting the findings and discussing their meaning.”

Hull says, however, that the effect size his team found still has real-world significance. An analysis of one of his earlier studies, which reported a similar estimated effect size of 0.083, found playing violent video games was linked with almost double the risk that kids would be sent to the school principal’s office for fighting. The study began by taking a group of children who hadn’t been dispatched to the principal in the previous month and then tracked them for a subsequent eight months. It found 4.8 percent of kids who reported only rarely playing violent video games were sent to the principal’s office at least once during that period compared with 9 percent who reported playing violent video games frequently. Hull theorizes violent games help kids become more comfortable with taking risks and engaging in abnormal behavior. “Their sense of right and wrong is being warped,” he notes.

Hull and his colleagues also found evidence ethnicity shapes the relationship between violent video games and aggression. White players seem more susceptible to the games' putative effects on behavior than do Hispanic and Asian players. Hull isn’t sure why, but he suspects the games' varying impact relates to how much kids are influenced by the norms of American culture, which, he says, are rooted in rugged individualism and a warriorlike mentality that may incite video game players to identify with aggressors rather than victims. It might “dampen sympathy toward their virtual victims,” he and his co-authors wrote, “with consequences for their values and behavior outside the game.”

Social scientists will, no doubt, continue to debate the psychological impacts of killing within the confines of interactive games. In a follow-up paper Hull says he plans to tackle the issue of the real-world significance of violent game play, and hopes it adds additional clarity. “It’s a knotty issue,” he notes—and it’s an open question whether research will ever quell the controversy.