Patients want a human in the loop for AI healthcare tools

Dive Brief

• Healthcare consumers are receptive to more artificial intelligence being used to manage their care, as long as there's a human involved in the process, according to a new survey from the Pew Research Center.
• Patients said they're open to AI handling administrative tasks, like appointment scheduling and ambient scribing.
• But patients are less enthusiastic about AI diagnosing conditions and adjudicating claims, with respondents saying they aren’t comfortable with the technology even with a human in the loop.

Dive Insight

The survey of 3,488 U.S. adults showed that healthcare consumers primarily consider AI effective for low-level tasks, but don’t find it appropriate for clinical decision-making.

About half of respondents think AI could be useful for translating medical jargon for patients, but only if there is a human reviewing the output. Only a quarter think AI can do this without any human input. Similarly, 44% of respondents think AI can effectively orchestrate appointment scheduling if there's a human in the loop.

Ordering prescription refills, ambient scribing and analyzing medical scans are also popular use cases among patients, especially if the AI is monitored by a human.

These are already among the most popular AI use cases in medicine. Data shows more clinicians are adopting tools like ambient scribing, claiming the technology makes their administrative tasks more efficient. Meanwhile, more technology vendors are deploying AI in their appointment scheduling processes, and electronic health record companies are launching AI to help patients understand their medical notes.

But the promise of AI also has its limits, the survey showed.

Four in 10 consumers said AI should not be making diagnoses, even with a human reviewing the output. When considering their interactions with their own doctors, 60% of respondents said they'd be uncomfortable if their provider used AI for a diagnosis.

Notably, 46% of respondents said AI should not be making decisions about insurance claims, even if there is a human in the loop. This comes as the industry debates the use of AI in payer processes like prior authorizations.

In fact, healthcare consumers largely distrust payers to use AI in the best interest of the patient, with 58% saying they have little to no confidence in a healthcare payer's good faith. Patients are also leery of AI in the hands of the federal government, AI companies and their own state government.

Instead, healthcare consumers mostly trust their healthcare providers to use AI in a way that benefits the patient, with a quarter saying they have a great deal of confidence and 34% saying they have some confidence.

Still, consumers are unsure of AI's net impact on care.

About a third of respondents said they think AI will make medical care better, while the same proportion said it'll make care worse. Nearly half of respondents are pessimistic about AI's influence on the patient-provider relationship, the survey added, while 51% think AI will have a negative impact on patient privacy and data security.

Previously published on sister site Healthcare Dive.

Sara Heath is an executive editor at Xtelligent Healthcare Media, where she covers patient engagement, healthcare policy and health IT.