putilov_denis - stock.adobe.com

How AI leaders can build an AI feature inventory

Traditional software inventories miss many embedded AI features. Learn what an AI feature inventory should include and how it improves visibility, governance and planning.

If shadow AI is defined as AI tools employees use without company approval, it presents only one challenge to creating a comprehensive AI feature inventory. Much of the hidden AI is embedded in enterprise software that the organization has already approved for employee use. The greatest risk is "false confidence" in the software the organization has already vetted and trusts, said Richa Taldar, staff product manager of AI Agents and MarTech at Walmart. "For any business serving millions of customers, an unnoticed capability can quickly become a trust, privacy or reputational issue," she said.

Unfortunately, traditional software inventories are not much help because they typically cannot identify all the embedded AI features across enterprise applications. Even when they detect embedded AI features, they are unlikely to capture important details such as AI capabilities, usage and data access rights.

It is hard to overstate the myriad issues resulting from the widespread use of embedded AI across all software types. AI features are now nearly ubiquitous across CRM platforms, productivity suites, security tools, databases, collaboration software, development environments and numerous other enterprise applications. The challenge is compounded by how they arrive: New AI features regularly appear through routine software updates that typically do not require a separate procurement process or IT review.

The risks inherent throughout this AI avalanche tend to be "far greater than many organizations realize,” according to Bihag Karnani, a senior product manager at Google, where he leads AI and machine learning-powered product development across platforms, including personalization systems inside Google's AI Overviews and AI Mode.

"As soon as AI becomes invisible within your product stack, you lose control over the governing of AI, auditing of AI and the ability to simply 'turn off' AI if or when there's a problem," Karnani said.

Building an AI feature inventory is the first step toward regaining visibility, managing risk, controlling costs and ensuring AI is deployed intentionally rather than by accident.

What belongs in an AI feature inventory?

Some prefer a comprehensive record to keep AI information current and AI feature management on track. For example, Taldar recommends that the record identify the platform and vendor, the business problem being addressed, the affected users and workflows, the data involved, the accountable business and technical owners, and any required human review. "It should also include cost, contract and renewal information, usage levels, known risks and measurable outcomes," she said.

Key components of an AI feature inventory

  • Application, vendor and product version. The application, vendor and version where the AI capability resides.
  • AI capabilities. All AI capabilities and features included in the product.
  • Feature status. Whether each AI capability is enabled or disabled.
  • Authorized users. The users, roles or groups permitted to use each capability.
  • Data sources accessed. The data, repositories and systems the AI can access.
  • Permitted actions. The actions the AI can perform, such as generating content, executing workflows or updating records.
  • Licensing and consumption model. How costs are incurred, such as per-seat, per-token, API usage or bundled.
  • Business owner. The individual or departments accountable for the AI capability.
  • Usage metrics. Adoption, utilization and other relevant usage data.
  • Governance requirements. Applicable security, compliance, privacy, risk management and policy requirements.
  • Contract and budget cycle. Renewal dates, license review periods and budget cycles.

Others think an inventory with fewer data points is more practical and more likely to be regularly updated. Allan Dabre, who leads Technology Compliance and AI Risk at PwC, is in this camp. He advocates keeping to five essentials with each tied to a governance function: identification and ownership; purpose and deployment status; data involved; risk tier and regulatory relevance; and oversight and lifecycle dates. "More becomes too heavy to maintain, less stops being useful," Dabre said. "I'd rather organizations start with these five fields and keep them current than build an elaborate 15-field template nobody maintains past month two," he added.

In practice, both highly detailed and basic AI inventories can become outdated without clear ownership and accountability for keeping them up to date. Consider including the following data points in an AI feature inventory.

When cataloging AI capabilities across the organization, log each feature, and not just each app, against the following fields:

  • Application and vendor. The enterprise application that contains the AI feature and the company that develops, owns or hosts it, including whether it's first-party, embedded in a larger platform or a bolt-on third-party add-on.
  • AI capabilities available. What the AI feature does, such as drafting, summarization, transcription, code generation, image generation or natural-language data analysis.
  • Licensing and consumption model. How the feature is licensed and billed (e.g., per seat, per token, API usage or bundled into an existing subscription), plus any usage caps or throttling limits. Also include contract renewal dates, license review periods and budget cycles.
  • Enabled vs. disabled features. Which sub-capabilities are switched on versus deliberately turned off, such as for web-grounded search, third-party plugins and auto-send.
  • User access and permissions. Who has access, including the teams, their roles, seat counts and other user access information. Add how that access is provisioned and scoped, ideally inheriting from existing file or system permissions rather than being granted separately.
  • Data sources accessed. What systems and data the feature can access and whether that is limited to data the user is already authorized to view. Also note the data generated by the feature, how it is stored and whether the vendor uses it to train AI models.
  • Actions the AI can perform. What the feature does autonomously versus what requires human approval, such as sending, deleting or writing data back to another system, versus just drafting or suggesting.
  • Feature ID. This unique identifier is used for cross-referencing in audits or tickets.
  • Risk tier and review status. The feature’s risk classification and the date it was last reviewed or updated to support ongoing governance rather than a one-time snapshot.

Who should own the inventory?

Only one team should be assigned accountability, but it cannot be the only team responsible for contributing information. For example, Taldar said that the central inventory owner in large enterprises can be in AI governance, enterprise architecture, technology portfolio management or a similar function that already has visibility across internal systems and vendor contracts.

But other teams should be assigned responsibility for contributing information to the inventory according to their primary functions and capabilities.

As soon as AI becomes invisible within your product stack, you lose control over the governing of AI, auditing of AI and the ability to simply 'turn off' AI if or when there's a problem.
Bihag KarnaniSenior product manager, Google

"No single group can independently manage this process. Management responsibility should exist at both an organizational document level and per entry by the responsible party," said Karnani.

So, what teams or parties should be responsible for contributing to the living AI feature inventory?

"Product and engineering teams can explain how a capability works, while business teams can explain its purpose and impact. Procurement, security, privacy, legal, compliance and finance contribute to the contract, risk and cost perspectives," said Taldar.

Smaller organizations might find project management to be a good practical owner, suggested Taldar.

"Regardless of company size, representatives for the people affected by the capability should participate in its evaluation. Technical approval alone does not mean that an AI experience is useful or appropriate for the end user," Taldar said.

How to get started

Start by choosing a dynamic, rather than static, form to house the organization's inventory, making it easier to update. The organization could also automate portions of the update process, but governance should not be fully automated. Human owners must still determine materiality, validate business use and make decisions about controls, according to Jessica Block, chief AI officer at RGP, a global professional services firm. 

"However, AI can reduce the manual burden of discovery, identify discrepancies and route likely changes or exceptions to the appropriate system or business owner," Block said.

Ultimately, an AI feature inventory should not be a one-and-done list of vendors and models on a spreadsheet, according to Kevin Schawinski, CEO and co-founder of Modulos AG, a Zurich‑based AI governance company.

"Can you keep all of this in a spreadsheet? Maybe, but it will be fiction within a quarter. Vendors flip feature flags every month, and your spreadsheet doesn't know that," Schawinski said.

Maintaining the inventory is itself a strong AI use case, according to Block. AI can be used to review vendor release notes, product documentation, licensing changes, contracts, application telemetry, configuration data and internal support activity to identify newly available or newly used AI capabilities, she said.

"AI can also help reconcile information across application inventories, software asset-management systems, vendor records, data catalogs and governance repositories," Block added.

As for which AI feature details to track, Taldar recommends maintaining separate records for current AI features and planned AI investments. "I would keep planned investments in a separate but connected roadmap. The inventory provides a view of what the organization has today, while the roadmap shows what internal teams and existing vendors expect to deliver next," she said.

Schawinski recommends keeping a separate inventory record for AI agents, too, since they affect the organization in ways other AI features cannot.

"AI agents need their own treatment because a copilot embedded in your office suite and an autonomous agent are very different animals, even when they come from the same vendor," Schawinski said. "For agents, you want to record what permissions they hold, what tools they can call and how much they're allowed to do on their own."

Building and maintaining an AI feature inventory offers multiple benefits. It eliminates surprises about which AI features an organization owns, uses and pays for but does not use, enabling better decisions.

"The feature inventory gives you a place to apply risk-based guardrails instead of one blanket policy and makes it easy to know where AI touches regulated data," said Dr. Ryan Ries, chief AI and data scientist at Mission Cloud, CDW's AWS-focused cloud and AI practice.

But there are advantages beyond compliance and risk mitigation. 

"Organizations are trying to get to predictable costs and forecast, but that has mostly failed due to unknown token usage, which is one of the primary billing mechanisms. At renewal, usage data can also serve as leverage. For example, your procurement team could negotiate down features nobody used," said Ries.

The organizations best equipped to manage AI at scale will not necessarily be those with the most advanced governance frameworks. Instead, they will be the ones that clearly understand the AI capabilities they have, know how those capabilities are being used and can make informed decisions as enterprise software continues to evolve.

Pam Baker is a freelance journalist and the author of books including ChatGPT for Dummies and Generative AI for Dummies. Baker is also an instructor on AI topics for LinkedIn Learning and a member of the National Press Club, the Society of Professional Journalists and the Internet Press Guild.

Dig Deeper on ERP administration and management