RSAC panel warns AI poses unintended security consequences

SAN FRANCISCO -- While a panel of experts at RSA Conference 2023 touted generative AI for a host of security uses, including incident response, they also warned the rapid adoption of the technology will present unintended consequences, particularly around the spread of disinformation.

Ram Shankar Siva Kumar, data scientist in Azure Security at Microsoft, moderated a panel titled "Security as Part of Responsible AI: At Home or At Odds" on Tuesday during the conference. Panelists Vijay Bolina, CISO at Google DeepMind; Rumman Chowdhury, founder of Bias Buccaneers; and Daniel Rohrer, vice president of software security at Nvidia joined him. The discussion addressed if and how security can keep pace with the whirlwind of large language model (LLM) use that OpenAI's ChatGPT sparked beginning in November.

The panelists emphasized how the rapid adoption of AI has even affected the discussion points of their panel because so much has changed in the space over the last six months. However, one of the consistent main challenges with responsible AI use was possible unintended consequences.

Chowdhury defined unintended consequences as well-meaning people who accidentally implement bad elements as opposed to malicious attackers. The difference between the two matters, she said, because it affects the approach to solving the problem.

"In one case, you're looking for people who create bots or spread disinformation intentionally. And then there are people who spread it unintentionally because they believe it. Both need to be resolved," Chowdhury said. "People can make deepfakes that are malicious, but if no one shares them, it doesn't have a big impact."

She attributed the heightened use of generative AI to enterprises' needs for critical thinking and fast analysis, which the technology does address. On the other hand, the panelists emphasized their concerns, such as the potential for job loss in particular fields, inherent bias and even "hallucinations."

Hallucinations occur when an LLM provides responses that are inaccurate responses or not based in facts. This can lead to the spread of mass disinformation, Chowdhury warned.

"At Twitter, we worried a lot about election disinformation. AI may take that and amp it up. We have a politically contentious situation in a world in which it is very simple to make and spread disinformation at scale," she said.

Ryan Kovar, distinguished security strategist at Splunk, attended the panel Tuesday and said the hallucination aspect rings true. When Kovar asked ChatGPT to produce a summary on himself, he received a mix of work published by him as well as a colleague. The AI conflated his and his colleague's work all the way through.

"Another problem is AI, it doesn't lie, but it infers. You have to be more specific and get it down to what you're actually asking," Kovar said. "Still, the only people who are going to lose are the ones who don't adapt to AI."

To curb these problems, Rohrer said there needs to be a focus on building tools to manage those unintended consequences, and problems need to be addressed with a systems-based approach. He believes there's currently an overemphasis on the model itself rather than the embedded system.

Since the onset of increased AI use, Rohrer has dealt with the legal and ethics teams more than ever.

"What I learned in those conversations is that a lot of the things we do and the way we think in security applies very well -- looking at risks versus harms. It's the same things we want to do here," Rohrer said.

Similarly, Chowdhury referred to generative AI as the merging of ethics and security. It's similar, she said, but not the same.