Are men more impressed with their own scientific research than women? Or are women warned off “overstating” their work?
A new analysis suggests it might be a little of both.
Women were 12.3% less likely than men to frame their work with positive words like “novel” or “excellent” in abstracts, according to a new study of 15 years of clinical research publications. In the case of only top-tier journals—there are numbered rankings for this in the arcane world of scientific publishing—the gap widened to 20.4%.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
These differences may be more than semantic, according to the authors of the study, which was published Monday in BMJ. Publication in one high-impact journal leads to citations in articles appearing in other journals, eventually snowballing into a measure of a researcher’s influence. And not just in theory: Cumulative citations count explicitly in recruitment, promotion, pay, and funding.
“Promotion is primarily measured by impact in your field, so the words that are listed here—“novel,” “unique,” “promising”—these are the type of things in letters people write that determine whether someone gets promoted or gets tenure,” said Dr. Vineet Arora, assistant dean of scholarship and discovery at the University of Chicago’s Pritzker School of Medicine. She studies gender differences in the way that physicians in training are evaluated but was not involved in the research. “This is more than just a random assortment of words. These are words that actually matter in terms of career potential.”
To reach their conclusions, three researchers from Harvard, Yale, and the University of Mannheim, in Germany, scanned titles and abstracts from more than 100,000 clinical research articles found among more than 6 million life sciences papers indexed by the archive PubMed from 2002 to 2017. They looked for the first and last authors of a study, typically a junior scientist and a mentor, respectively. After running those names through an algorithm called Genderize to assign male or female to the names, they used natural language processing to comb through abstracts, searching for 25 positive words taken from a standard list vetted in prior research.
What if the findings really were novel (by far the most common positive word) or “excellent” (middling on the list)?
The researchers accounted for that possibility by comparing apples to apples, adjusting for specific areas, keywords, and types of research in similar journals. A gold-standard randomized clinical trial, for example, would be more likely than an observational trial to be groundbreaking, so they used trial design to narrow comparisons. They also compared publications in similar time frames, because there were fewer female authors—and fewer papers overall—in 2002 than in 2017.
They concede there is still one black box. The BMJ article’s authors could see only the published abstract, not the draft a scientist submitted or suggestions made during the peer review and editing process. Marc Lerchenmueller of the University of Mannheim and first author of the BMJ paper, noted that recent research in economics shows journal editors and reviewers comment on manuscripts written by women more often than they remark on work by men. That can make the review process longer and change what eventually appears in the paper.
“If women are held to different standards and editors or reviewers opine in a gendered way, then you end up with an end product that is more timid,” he said.
There is ample research to support this interpretation. Not only are women less likely than men to promote themselves, they are also less likely to consider themselves qualified for a position unless they meet every criterion for it, said Dr. Esther Choo of Oregon Health & Science University. She said the latest study adds to a growing body of research that captures the many barriers, large and small, for women advancing in health care and research.
“Savvy women learn to rein it in compared to men in order to satisfy stereotypical communal traits,” she said. “I think this phenomenon probably applies to how people are free or not free to tout their own work, and wonder what this study reflects in terms of women pulling back from stronger language out of conscious or unconscious acknowledgement of this censure, or the reviewers or decision editors being quicker to accuse female authors of overstating their study or being overly assertive in the title and censuring the messaging.”
In previous research, Dr. Anupam Jena, senior author of the BMJ paper and professor of health care policy at Harvard Medical School, reported that patients treated by female physicians were more likely to survive than patients treated by male physicians. Mortality, of course, can be easy to measure. Bias in science and medicine is not, but analyzing positive language in scientific abstracts is one way to address a lack of evidence.
“There’s an indication in this one specific behavior that we can measure that language is being used in this way,” he said. “We don’t have any insight into what’s happening in social media yet.”
In the BMJ abstract, none of the 25 positive words makes an appearance.
“We were obviously very careful not to, in our own abstract,” Jena said. “That irony would not be lost.”
Republished with permission from STAT. This article originally appeared on December 16, 2019
