AI testing, validation common but inconsistent across healthcare providers

As AI adoption accelerates across the U.S. healthcare industry, testing and validation are becoming increasingly vital to ensure the tools are safe and effective. However, a recent report revealed that health systems' validation approaches vary widely, and most lack a dedicated testing environment.

The report, compiled by the Center for Connected Medicine at UPMC and KLAS Research, is based on interviews with 27 U.S.-based healthcare leaders on AI deployment, testing and validation, data environments and AI strategy maturity, among other topics. The interviews were conducted in May and June 2026.

The vast majority of healthcare leaders (93%) said their organizations have deployed third-party AI solutions. Clinical documentation/ambient scribing tools were the most commonly deployed, with 52% saying their organizations had implemented these tools, followed by AI-driven revenue cycle, coding and billing (36%) solutions.

Testing and validation are common before AI deployment, according to the report.

Most of the organizations that deployed third-party AI tools (92%) reported evaluating the solutions. However, only 44% of leaders reported having a dedicated data platform or environment for testing them. Most of the leaders interviewed said their organization uses EHR-based or vendor-supplied analytics tools for testing, followed by cloud data warehouses or lakehouses.

The report further noted that organizations without dedicated AI testing environments may be unable to effectively validate the accuracy and real-world impact of their AI solutions.

Another challenge to AI validation is the fact that the very definition of 'validation' varies across health systems. For some, it means ensuring the efficacy of a workflow, while for others, it means assessing risks, like accuracy, bias, drift and clinical safety, over time. As such, health systems use a wide array of validation strategies, including formal vendor testing protocols, limited pilot programs and sandbox testing.

Additionally, health systems use different metrics to assess the value of an AI tool, with most citing efficiency and productivity, followed by return on investment and clinical outcomes.

"The health care industry has moved remarkably quickly from discussing the potential of AI to actively deploying solutions across the enterprise," said Rob Bart, M.D., chief medical information officer at UPMC, in the press release. "What's emerging from this research is a clear recognition that implementation is only the first step. Health systems are now focused on building the governance structures, testing capabilities and organizational strategies necessary to ensure AI delivers meaningful and measurable value."

But most healthcare leaders (63%) stated that their approach to AI is developing, with their organizations working to establish various aspects of their AI strategy, such as how to prioritize use cases, choose partners and decide whether to buy or build tools.

Only one leader described their organization's AI strategy as advanced, that is, a strategy with a scaled and measurable impact.

The primary challenges to executing an AI strategy included resource, budget, time or talent constraints (11 of 27 leaders), governance, security, compliance or quality assurance challenges (10) and change management, adoption and education barriers (eight).

Still, despite these challenges, the report authors emphasized that a "stronger data infrastructure, clearer governance, repeatable testing models, and practical approaches for communicating with patients and clinicians" will be critical for organizations to build more mature AI strategies.

Anuja Vaidya has covered the healthcare industry since 2012. She currently covers healthcare IT and innovation, including artificial intelligence, digital healthcare, EHRs and interoperability.