AI coding tools run riot over SBOM controls as EU CRA looms

The inadequacy of SBOMs in the age of AI

A Linux Foundation World of Open Source survey of more than 600 global respondents found that 94% of organizations are using or piloting GenAI coding tools, and 48% said the tools have increased their use of open source software, according to a report issued during this week's Summit.

An agent could grab some old open source code with CVEs attached, copy it into an application, and then carry these vulnerabilities along, without the original component ever appearing in the SBOM.

Meanwhile, the number of vulnerabilities that any developer can potentially introduce into production software by relying on AI-generated code has grown exponentially, according to Torsten Volk, an analyst at Omdia, a division of Informa TechTarget.

AI coding tools make it easy to experiment with – and ultimately put into production -- a much wider range of code libraries and artifacts than developers could previously manage, Volk said.

Developers no longer have to set up and configure libraries, and even more importantly, they don't have to weed through API documentation to get new libraries to work, he said.

"However, an agent could grab some old open source code with CVEs attached, copy it into an application, and then carry these vulnerabilities along, without the original component ever appearing in the SBOM," Volk said.

SBOMs ostensibly list every component in a software product, and will be required under the EU CRA as of December 2027, but are woefully inadequate against the deluge of AI-generated code, especially for open source packages and libraries, Volk said.

"As AI agents are able to insert all kinds of code into our apps, it is crucial not to rely on a static analysis that depends on the assumption that these agents always listen to us when we say, 'You must disclose every piece of code you re-use from elsewhere,'" Volk said. "To solve this challenge, we need to continuously scan all new code within the context of the existing codebase."

The inadequacy of SBOMs for open source includes the hundreds of projects governed by the Cloud Native Computing Foundation (CNCF), said Mario Fahlandt, a customer delivery architect at Kubermatic, a CNCF Technical Oversight Committee member, and a maintainer of the Kubermatic CNCF SBOM project.

"We don't know what's inside of our environments. All those dependencies are a mystery to everyone, the maintainers included," Fahlandt said during a Summit presentation. "We have fewer maintainers for more work because we get more PRs [pull requests] that are AI-generated. If we don't know which packages are inside of our software, we don't know if there's a security issue, if we have a CVE, if there's a supply chain attack [happening]."

AI risks extend beyond agents hunting vulnerabilities

"Slopsquatting" is one example of an AI software supply chain security risk unrelated to agents tracking down vulnerabilities in existing legitimate code. While the term is relatively new, the practice has been around for at least a couple of years: the bad-actor agent or human asks AI agents such as Claude to list libraries, which then guess at library names for libraries that do not yet exist. The attacker publishes those names on npm or PyPI, for example, with exploitable code. Another developer, in good faith, installs the code thinking it is the right library (after all, Claude provides the name and link). 

The developer remains unaware that the attacker has access to their program and can execute various exploits, such as key theft, data theft or other actions. The SBOM ultimately fails because Claude invented the library's name, and the library exists under that name with code no one wants to install.

AI-assisted and copilot-generated code for basic BASH commands or tasks also poses a major risk. Even seemingly innocuous tasks, such as setting up a kernel on a Kubernetes cluster in cloud infrastructure or configuring YAML files with vibe coding instead of relying on documentation and strictly following best practices, can expose ports, secrets, and other sensitive information, leaving backdoors open. Human error that has exposed software to attacks has existed for years; to say that AI assistance and co-pilots compound this risk would be a euphemism.

The dynamic is a major issue for maintaining the Linux kernel, according to comments from Linus Torvalds, creator of Linux and Git, during a Summit keynote presentation on AI challenges, noting, among other things, a high number of kernel patches now being released.

For example, the 7.2 Linux kernel release in August was among the busiest update cycles in the kernel's history, adding 600,000 lines of code contributed by a record 2,652 developers. There were 1,111 commits in Linux 7.2 with an Assisted-by tag, compared with 31 in Linux 7.0. Those tags mark disclosed AI use on patches that landed as commits, but the real share of AI-assisted commits could be higher.

Some of those patches could stem from AI vulnerability noise, Torvalds said.

"We are getting a lot more patches…that are sometimes very important and for very real security issues, and sometimes things that nobody has touched in 20 years, because people don't care," Torvalds said. "But an AI bot, you ask, is there a memory leak here? AI will say, 'I don't care that nobody uses this driver anymore. I'll give you all the error reports I can find.'"

While the increasing number of patches is improving the Linux kernel codebase and AI is useful for supporting patch monitoring, the deluge of patches is "stressing maintainers out," Torvalds said. "That's part of why having the AI for review is important, to ease that burden."

AI automation, microVM isolation as potential fixes

The bulk of the discussions at the Open Source Summit were about how to make AI tools, both for code generation and review, "less stressful and more successful and more useful for everybody," as Torvalds put it.

During his presentation, Kubermatic's Fahlandt described a GitHub Action that runs every Sunday and spawns 2,500 workers to scan CNCF project repositories. The workers generate SBOMs based on release tags and store them in an Amazon S3 bucket, which currently contains 16,000 SBOMs.

"Every Sunday, the cron job is running, and I'm pinging someone from GitHub to fix the problems, and it gets better every Sunday," Fahlandt said.

Advances in container workload isolation are another potential countermeasure against the pressure from AI on supply chain security, as they can prevent attackers from exploiting a vulnerability to reach the broader network via the container host, according to a presentation by reps from container and network security vendor Edera at the Open Source Summit.

Open source Kata containers are one approach to solving the isolation problem, but the cost of running them at scale can be high, according to Kavitha Daula, vice president of engineering at Edera, during the presentation. Shared MicroVMs can introduce guest-kernel CVEs, host- and guest-kernel drift, a second debugging surface, GPU passthrough and other risks, she said.

Edera markets a competing isolation mechanism that wraps AI agents and container workloads in zones, or hardware-isolated microVMs, in Kubernetes.

"Every single agent – and we really mean every one – has its own microVM and private Linux kernel," Daula said in an interview. "They also run on the Kubernetes clusters you already own." 

Bruce Gain is founder and principal analyst at ReveCom. His byline has appeared in Wired, PC World, CIO, Technology Review, Popular Science, and EETimes.