Speech recognition technology in healthcare tested at Boston hospital

Using voice technology in the clinical setting

Although the hospital used the Amazon Alexa platform for all three pilots, they used different Amazon devices in different settings. In the ICU, clinicians used the Amazon Echo Dot. 

In this pilot, the smart speakers allowed clinicians to verbally record who made up the care team for each patient, as well as ask for policy and guidelines. Because of the length of policies and guidelines, Nadar said the hospital opted to use a mobile app rather than having the Echo Dot read them out loud.

For example, Nadar said, "If you say, 'Pull up massive transfusion protocol,' it would send to your phone so you could read it as you're leaving."

A list of questions hanging on the wall of the ICU helped clinicians and nurses know what questions they could ask of the technology, such as "Who is the charge nurse right now?" and "Who is the respiratory therapist in bay five?" During the ICU pilot, about 70% of interactions resulted in "exactly what a clinician was looking for," Nadar said.

Devices used for the transplant organ verification process and surgical checklist pilots were usually Amazon Echo devices, but the devices often changed for those programs depending on what physicians wanted to use, Nadar said. More frequently, she said, the hospital began using Amazon Echo Show, which has a screen for displaying information.

Nadar said the Echo Show was a "pretty big game-changer in a lot of ways" because it eliminated the need to connect speakers to a computer screen or phone to pull up information.

Using voice technology as part of the transplant organ verification process benefited clinicians by allowing them to conduct an organ validation and checklist procedure hands-free. Without voice technology, clinicians had to conduct the validation and checklist procedure by typing information into a computer.

"As people talked and went through that checklist, it was being recorded by the Amazon skill," Nadar said.

The surgical checklist pilot was designed to assist surgical team members through a procedure-specific checklist, Nadar said.

While some skills required "imaginative solutions" due to Amazon devices not being HIPAA-compliant, Nadar said the hospital's research and development team has been developing skills on the Amazon platform for several years and tailored the skills to fit the healthcare setting while protecting patient privacy.

Learning from the voice technology pilot programs

The hospital's experience with the pilot programs, Nadar said, has demonstrated the importance of teaching clinicians and nurses how to use voice technology so they know what they can and cannot ask. Nadar said people don't have time for speech recognition technology in healthcare that doesn't work.

All these different teams approached us about doing this. The units were coming to us and said this would be a cool thing, could you build it.

Nadar's sentiments are echoed by Yaa Kumah-Crystal, M.D., an assistant professor of biomedical informatics at Vanderbilt University and lead researcher for Vanderbilt University Medical Center's EHR voice assistant, V-EVA.

"The thing about these technologies is it has to work really well the first time or else people will get frustrated and abandon the concept altogether," Kumah-Crystal said. "The principle we work with is it should be easier to ask a question and get an answer back versus searching for it."

Nadar said having reliable content is also important for voice technology to be successful. She explained that there is a "general unease" with voice assistants, but if clinicians know the information is coming from Boston Children's Hospital, they're more trusting of the information. Nadar added that the ICU pilot involved heavy integration with the hospital's internal database. 

"Showing individuals that you are being thoughtful in how you're developing these voice skills and you're taking into account their pain points, it goes a long way," Nadar said.

Applications, benefits of speech recognition technology in healthcare

A primary reason the pilot programs were successful, Nadar said, is that they were user-inspired. The ideas came from the clinicians themselves who said they wanted to use voice technology because of its hands-free nature, which helps reduce germ transmission and saves time in emergency situations by eliminating the need to remove their gloves and search for information on a computer.

"All these different teams approached us about doing this," Nadar said. "The units were coming to us and said this would be a cool thing, could you build it. It was a clinical team saying 'We saw an opportunity.'"

Another benefit to voice technology is affordability, Nadar said. Amazon's Alexa rings in around $100, while the Amazon Echo Dot costs about $50. Along with affordability, Nadar said there are flexible and easy-to-use developer platforms for creating a hands-free interface that enables immediate access to vital information in an engaging medium.

As the hospital expands and plans to build new clinical buildings in the next couple of years, Nadar said "We want everything to be as forward-thinking as possible," adding that "voice will have a very strong presence there."