Q&A: Preparing for health AI oversight amid regulatory uncertainty

AI has transformed healthcare at an unprecedented pace. The past few years have been marked by rapid tool adoption, the industry's embrace of AI agents and early data showing notable returns on investment following the implementation of AI-powered workflows. From streamlining clinical notes to automating the revenue cycle, healthcare appears to be committed to using AI to reduce administrative friction and increase efficiency.

However, regulations on AI use are not moving as quickly. As the sector continues to innovate, organizations must also be mindful of complying with existing regulations and preparing for new ones.

In an interview with Healthtech Analytics, Jeff Wurzburg, a healthcare partner at the law firm Norton Rose Fulbright, discussed the current AI regulatory landscape and recommended that healthcare organizations prepare now for increased oversight.

This interview has been edited for clarity and brevity.

HEALTHTECH ANALYTICS: There's no question that healthcare is embracing AI. I'd love to hear your thoughts on the rise of autonomous and agentic AI in healthcare and some of the potential regulatory risks associated with it.

JEFF WURZBURG: I think in terms of capabilities, we're at a moment where it seems endless. When you think of healthcare spending in the United States, a significant portion of that goes into the administrative side in terms of reimbursement, authorizations and the paperwork side of the house. And so there appears to be endless optimism and opportunity to automate some of that work and, hopefully, over time, do so in a manner that will reduce costs for both providers and payers and be more responsive and more accurate. And so, presuming that can happen, that's a win for our system.

And we don't often get good news about our health system, certainly in terms of reducing costs. That said, at the outset, there is certainly a lot of investment required from both the provider and payer sides. Everybody's investing in those tools to see what works right now. And there's going to be some growing pains and challenges. And I think the challenge for regulators and enforcers is going to be doing that in a way that ensures beneficiaries are protected.

AI in healthcare is moving so fast. But regulations are much slower to come into effect. Could you summarize for our readers what the current enforcement and regulatory landscape looks like, and outline some of the major considerations that health systems should take into account before adopting these tools?

I think in one word: uncertainty. We are still very much in the infancy, and by its very design, government and regulators are always going to be one step behind innovation, which raises its own challenges.

I think we're crossing from AI that advises to AI that now takes actions, and the law hasn't yet caught up to that. And at the rapid pace of change, there's a natural challenge for lawmakers to be able to do so in a manner that both protects citizens and encourages innovation because there could be real benefits to society and to healthcare from that innovation.

But we have to be careful about the allocation of responsibility and ensure that there is still some humanity behind this, and that real eyes are laid on the determinations made by artificial intelligence programs. I think right now, the risk from AI is really the uncontrolled nature in which it's being deployed in some of these segments.

Are there some specific privacy and security concerns that come to mind for you at the intersection of AI and regulation?

Absolutely. I think privacy and security concerns in the healthcare space are paramount. Those employing AI tools still must follow all the HIPAA requirements and considerations.

At the same time, we're using a law from 1996 within this very, very new world that we're in. And so, it is definitely a concern. It's something that all healthcare systems, providers and boards need to be thinking about.

You mentioned that enforcement frameworks are still in their infancy. How do you see them evolving in the near-term?

At the moment, it's very hard to see the direct line to regulation. It's coming, but we aren't really seeing traditional enforcers using AI facts to pursue actions just yet. I think it's a matter of time. Especially as we see AI being used in billing audits, utilization disputes and payer-provider disputes, I think we will start to see more enforcement and direct consequences.

But I think it's important to remember that while AI is now being utilized in healthcare, the regulations and the laws underneath any sort of enforcement action haven't changed. It's still the same requirements that exist and utilizers of AI need to be ensuring that they are implementing programs within that framework. The False Claims Act will still be the False Claims Act.

Where I think AI will come into play there as well -- was there human oversight? Was there reckless disregard in the use of an AI program that led to the submission of false claims? Did you knowingly use a system that really hadn't been tested in terms of the accuracy of the claims that were being submitted for reimbursement under a federal healthcare program? In terms of thinking about enforcement, that's where governance is critical to ensure that guardrails are in place internally to ensure that programs are creating outputs with a correct factual underpinning.

Governance is clearly crucial to ensuring responsible AI use. Do you feel like healthcare is in a good place in its overall governance maturity, or are there specific areas that have room to grow?

Healthcare is particularly unique because health systems are used to having to pivot and encountering new uses and technology as part of the regular course. And when you look at the many pivots that health systems have had to do over the last several years, between COVID, inflation, and now, significant downward reimbursement pressure, health systems are good at adopting and employing appropriate governance and pivots to accommodate whatever they've encountered.

So, from that standpoint, I think health systems are well-suited to adapt to technological changes driven by improvements in artificial intelligence and its use across their systems.

Healthcare, because it's such a highly regulated environment, is used to working in a system that requires constant examination of its practices within the complex regulatory framework.

AI is a very dynamic tool and seeing how that is employed and regulated moving forward is a pretty exciting place to be, but it's not without risk for providers.

Jill Hughes has covered health tech news since 2021. Her coverage areas include cybersecurity, HIPAA compliance, interoperability, AI and EHRs.