Google AI chatbot performs well in primary care settings

A new report published in The Lancet paints a promising picture of Google's AI chatbot, showing that the technology can provide clinically accurate information to patients in the primary care setting.

The report assessed 98 patients who had both an AI interaction and a primary care appointment between April and November of 2025. According to researchers from Beth Israel Deaconess Medical Center and Google, the data also showed that patients are receptive to the tools.

Healthcare professionals view these types of chatbots, which use large language models to answer patients' questions and help them parse complex medical information, as key solutions to the industry's growing patient engagement and provider shortage challenges. The technology helps connect patients to health information that would otherwise be inaccessible or difficult to understand.

However, primary care providers need to do their due diligence before adopting AI chatbots, too, according to Adam Rodman, director of AI Programs at the Carl J. Shapiro Institute for Research and Education at BIDMC.

"Before we begin evaluating them as workload interventions, AI tools must first be shown to operate safely and acceptably in real care workflows," Rodman said in a press release.

"With this research, we set out to assess whether this kind of technology could improve both patient and physician experience."

The researchers partnered with experts from Google to assess the tech titan's Articulate Medical Intelligence Explorer, or AMIE. More than 100 patients were given access to AMIE after scheduling an urgent primary care appointment with a BIDMC physician. AMIE asked about symptoms, gathered medical histories, offered potential diagnoses for patients and created a summary for clinician review.

Notably, the technology performed well, providing accurate and safe information, according to a team of clinician reviewers who were tasked with monitoring AMIE interactions and intervening when patient safety was at risk.

While, none of the 98 analyzed interactions required a safety stop intervention,  the researchers did flag one hallucination, and they provided clinical clarification for five cases.

"These baseline characteristics provide a foundation for all future prospective trials aimed at discovering when human-on-the-loop workflows may be appropriate," Peter Brodeur, a clinical fellow in cardiovascular medicine at BIDMC and the study's co-first author, said in the press release.

Patients themselves viewed AMIE favorably, with participants' general attitudes around AI improving after interactions. Patients gave high ratings specifically to the AI's ability to listen, explain information and make them feel at ease. However, they did express concern about the confidentiality of the system and the chatbot's honesty or trustworthiness.

According to Rodman, a human-in-the-loop approach and strong patient-provider relationships could put patients more at ease.

"Research suggests that patient trust in medical AI systems flows from relationships with their human providers," said Rodman. "Future research will need to explore which interaction characteristics can build patient trust and how AI interactions can enhance the patient-physician relationship."

The study also got at a key question about patient-facing AI chatbots: can these technologies ultimately add depth to the patient-provider relationship?

According to the data, it could.

For 44 of the patient cases analyzed, primary care providers were able to review an AI-generated transcript of the chatbot encounter before meeting with the patient. This might have given clinicians insights into patients' concerns, questions or priorities before the visit. For instance, one clinician flagged an event in which a patient's anxiety seemed to have increased after the AI included lymphoma, a type of blood cancer, as a possible diagnosis.

In three-quarters of those cases, providers said the review helped them feel more prepared for the visit. In more than half (57%) of instances, clinicians said insights from the chatbot conversation influenced their clinical approach.

According to Marc L. Cohen, clinical chief of primary care in the division of general medicine at BIDMC, these findings underscore AI's potential as a tool to build out stronger relationships between patients and providers.

"We believe the future involves continued involvement of human clinicians in patient care, but see a future where the triad of patient, human clinician, and AI help to provide our patients with the very best evidence-based care possible," Cohen said in the press release.

Sara Heath is an executive editor at Xtelligent Healthcare Media, where she covers patient engagement, healthcare policy and health IT.