Kaiser's AI patient portal success hinges on quality checks

AI patient portals require quality assurance

AI for AI's sake isn't going to effectively move the needle on improving patient outcomes, Nguyen stressed. It's easy to fall in love with the bright shiny object that is AI, he added, but without thoughtful consideration of organizational workflows and data sources, the technology won't be effective.

"As a learning organization, we're continually trying to learn from this," Nguyen said of the Intelligent Navigator tool. "One of the key things that's important here is the people involved -- the physicians and the operational people that help to inform the KP Intelligent Navigator because it's constantly evolving to educate, to teach the algorithms so that we are able to get people to the right place."

Kaiser has a group of subject matter experts focused on examining how the Intelligent Navigator works from the time the patient logs into the portal to the receipt of care and everything in between. This enables a cycle of learning that should ensure the product creates a quality result most of the time.

In fact, it was those human-powered quality checks that made Kaiser keen on trying AI in the patient portal in the first place.

Despite AI's buzzword status in healthcare and beyond, Nguyen suggested that Kaiser was primarily comfortable testing out the system because it could guarantee it had been tested and controlled by a qualified team.

"It wasn't so much the technology, but it was the process of quality control that we had and that we continue to have using the right people -- that's what made us confident that this is something that we can continue to learn from and evolve while we keep people safe, which is the bottom line," he noted.

Next steps for AI patient navigation

Nguyen said KP's Intelligent Navigator should yield ROI in terms of better organizational workflows and a reallocation of work out of the call center and into patient self-service.

But to get there, he added that the healthcare organization will need to tie in more data sources to help NLP make even better decisions about patient care access.

"The next level as an organization that we're looking into is to not only try to understand what the patient is typing, but to tie it to their attributes -- their age, their gender, their sex, perhaps even their situation and then making informed levels of understanding what the patient is actually asking for," Nguyen said.

For example, a parent making a wellness visit for their child will get better navigation when KP's Intelligent Navigator can tell the child is a six-month-old, not a six-year-old.

But even amidst the organization's big plans for NLP advancements, Nguyen said they are careful not to fall prey to common pitfalls, like automation bias. "Just because the machine says something doesn't necessarily mean it's right," he noted.

"Organizations need to make sure that they have a quality improvement process to challenge things and to make sure that things are always heading in the right direction," he added. "Healthcare is constantly evolving."

For example, a patient telling Intelligent Navigator that they have a cough, fever and sore throat six years ago would indicate the flu. Today, it could also mean COVID, Nguyen said. Making sure these systems have the most updated information will ensure they can continue to yield good outcomes for patients.

"We're always thinking about how to best serve our members, not just have another interaction," Nguyen concluded. "These tools are long overdue to help us accelerate that mission and vision. But at the same time, we have to approach them with caution, and that really means getting people with boots on the ground involved in understanding."

Sara Heath has reported news related to patient engagement and health equity since 2015.