Google Health researchers published a paper evaluating the use of AI for breast cancer prediction in mammogram images, finding that it performed better than radiologists and claimed that it "paves the way" for clinical trials to improve breast cancer screening accuracy.
In the "International evaluation of an AI system for breast cancer screening," Google Health and DeepMind researchers describe an AI system for breast cancer prediction and shared results on data from female patients in the U.S. and the U.K. The Google AI breast cancer screening tool outperformed human experts in breast cancer prediction in almost all cases, according to the paper.
While the work has been applauded by some experts, it has also stirred debate. Vinay Prasad, M.D., associate professor of medicine at Oregon Health and Science University, said on Twitter, "The Google AI mammogram paper is FLAWED," which resulted in hundreds of responses, as healthcare stakeholders chimed in.
The Google AI mammogram paper is FLAWED.
Want to learn why?
AND why cancer screening is the LAST thing you should pick FIRST to work on with AI?
The "flaw" is not whether the AI breast cancer screening tool works. Instead, Prasad's major criticism is why the algorithm was built in the first place. Prasad believes the goal for breast cancer researchers should not be to diagnose more masses but to better and less invasively determine the insidiousness of masses such as whether they're benign or if the cancer has already spread. Those are tricky questions even for radiologists to answer.
But others argue that the Google AI system for breast cancer screening demonstrates an important development for AI in radiology.
Criticism of Google Health research
Google researchers applied the AI tool to breast cancer prediction, noting that interpretation of mammogram images is a big challenge in radiology today.
Screenings are affected by high rates of false positives, which indicate a condition is present when it isn't, and false negatives, which indicate a condition is not present when it is. AI could be "uniquely poised" to help with the issue, according to the paper.
The researchers used breast cancer screening mammograms from women in the U.S. and U.K. to test the AI system. The AI breast screening tool outperformed radiologists in breast cancer prediction. It reduced the number of false positives by 5.7% and false negatives by 9.4% in U.S. data in biopsy-confirmed breast cancer. In the U.K. data, the tool reduced false positives by 1.2% and false negatives by 2.7%.
Vinay Prasad
The research is still in a preliminary stage and questions about how and whether the tool should be used in the clinical setting remain. Regardless, Prasad argues that Google's AI breast cancer screening tool does not move the needle in providing better outcomes for patients.
Making a diagnosis from cancer screening is the worst possible use of AI because cancer screening is something that is already a very tricky thing.
Vinay Prasad, M.D.Associate professor of medicine, Oregon Health and Science University
"Making a diagnosis from cancer screening is the worst possible use of AI because cancer screening is something that is already a very tricky thing," he said.
Diagnosing breast cancer involves differentiating between four main diagnosis categories: non-cancerous, non-harmful cancer, curable cancer and spread-already cancer. That's something even radiologists can't do from a screening yet, he said.
While he believes AI will eventually be valuable for cancer screening, Prasad thinks researchers should start with simpler problems first. Tackling an area like breast cancer screening with AI is like climbing Mt. Everest before doing practice runs on smaller mountains, he said.
But it's early days for the AI breast cancer screening tool, something both Prasad and Google Health researchers acknowledge. Google Health researchers noted that clinical studies will be necessary to "understand the full extent to which this technology can benefit patient care."
For his part, Prasad is sounding a warning bell in the hopes of stopping history from repeating itself. Roughly twenty years ago, new technology for breast cancer screening was embraced with "irrational exuberance" that has now resulted in high rates of false positives and over diagnoses, he said. Prasad believes federal regulators are eager to get new products to market, which could set a low bar for products like an AI breast cancer screening tool that isn't ready.
A step forward
Google is not alone in its efforts to build an AI breast cancer screening tool. Last year, New York University researchers published "Deep neural networks improve radiologists' performance in breast cancer screening," stating that its home-grown AI system performed as well as radiologists.
Siddharth Shah
AI tools like these may not address every nuance in breast cancer screening but could be important developments for AI in radiology, according to Siddharth Shah, transformational health program manager at consultancy Frost & Sullivan.
Google Health researchers used international data from the U.S. and U.K., creating a more robust dataset compared to other studies, which Shah described as a big step forward for AI in radiology.
Shah agreed that some concerns about the nuances to cancer screening are valid, but he said for an AI breast cancer screening tool to be adopted by the industry, it will have to prove its value, efficiency savings, and early detection capabilities first.
"AI holds potential, no doubt," he said. "But the commercial aspects all need to be judged and therefore caution must be exercised before going full steam ahead with it. Thankfully, the healthcare industry is one that is driven by physicians and doctors, and scientists and researchers who demand evidence for everything."
Vinay Prasad
Siddharth Shah
