Researchers at Google (GOOGL), working alongside experts at Northwestern University and three British medical institutions, have created an AI model that appears capable of more accurately spotting breast cancer in mammograms than human experts, according to a new study.
The AI model both detected cases of breast cancer at higher rates than radiologists and reported fewer false positives, according to the study, which was published on New Year’s Day in the high-caliber scientific journal Nature. The results held even when the algorithm was tested internationally, a hurdle that few AI tools have been capable of overcoming given how inconsistent datasets be across borders.
“It’s a very nice contribution,” Dr. Eric Topol, director of the Scripps Translational Science Institute, told STAT. “It’s still underperforming from where we want it to be, but it’s paving the way.”
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Interpreting the signs of breast cancer can be equal parts art and science: Only an experienced radiologist can look at a series of blurry, black-and-white mammograms and distinguish cloudy, nefarious masses from clusters of healthy tissues.
Still, even an expertly trained eye can falter. Clinicians fail to find about 20% of all breast cancers, according to the American Cancer Society. Meanwhile, about half of all American women getting yearly mammograms are, over the course of a decade, wrongly told they might have cancer when they are in fact disease-free.
At the same time, burnout and regional shortages are putting increased pressure on radiologists across the globe.
AI may never replace human experts, at least in a field as nuanced and complex as breast cancer. Instead, such tools could be used as a kind of second opinion — something radiologists turn to as a means of guiding their initial interpretation of a breast scan.
“I don’t expect an AI algorithm would ever replace a radiologist or other physician…not as long as there is still a need for creative thought,” Dr. Bonnie Joe, chief of breast imaging in radiology and biomedical imaging at the University of California, San Francisco, told STAT.
The new AI tool was developed by researchers at Google Health and its British subsidiary, DeepMind. In addition to the Northwestern University experts, they collaborated with radiology researchers at two British institutions: Cancer Research U.K. Imperial Centre and Royal Surrey County Hospital.
To train their algorithm, the researchers fed the tool clinical data that included breast cancer scans from about 91,000 women in the U.S. and the U.K. All of the scans had been scraped of personal identifiers like names and birth dates so as to render them anonymous. Breast cancer diagnoses had been confirmed by biopsy.
The researchers then tested the algorithm on new sets of de-identified breast scans from several thousand women in both countries.
Compared with standard clinical practice in the U.S., where women are screened as frequently as every year, the algorithm resulted in nearly 6% fewer false alarms and 9% fewer false reports of an all-clear. In the U.K., where women are screened only every three years, the algorithm resulted in 1% fewer false alarms and 3% fewer false reports of an all-clear.
The algorithm was still superior to standard clinical practice when, in a separate portion of the study, the researchers trained the tool on U.K. breast scans and then tested it on a sample of U.S. breast scans.
Finally, the algorithm also won out when it was pitted directly against six human radiologists, who were tasked with interpreting 500 cases from the U.S. test set.
“We believe this is just the beginning,” Daniel Tse, one of the Google researchers who co-authored the study, told STAT.
Importantly, just as there were cases in the study where the AI spotted breast cancer that the human experts missed, there were also cases where the human experts pinpointed disease that would have gone undetected by the AI.
The researchers aren’t sure precisely why this happened, but they weren’t surprised by it either.
“There are things that these models and technology are really good at, and there are things that radiologists, who spend their whole lives doing this, are really good at,” Tse said.
The study results appeared to suggest that the AI had a particular edge when it came to invasive cancers, which can be hard for the human eye to spot. No specific pattern was documented among the cases where only the human experts spotted disease.
Neither the Google researchers nor the outside experts believe that AI tools like these are ready to be used in isolation. But that doesn’t mean there’s not a role for them. “What patient is going to trust an algorithm and not have any opinion from an expert radiologist or doctor?” Topol said. “I like to think there’s a blend of the two where you get the best possible answer.”
Republished with permission from STAT. This article originally appeared on January 1, 2019
