The dollars and sense of implementing AI
Calculating ROI of an AI project to prove business value requires a complicated mix of costs, including data prep, infrastructure, integration, staffing, training and power needs.
More businesses are deploying AI in one or more use cases, yet most struggle to track and quantify the return on their AI investments.
ROI is simple to calculate, right? Divide the net profit from an investment by the amount invested. As with any other technology-driven transformation, investment costs for an AI project come in the form of capital expenses and operating expenses. The business value derived from AI deployments depends on new dollars as AI drives higher revenue and saved dollars as AI decreases operating costs or reduces the risk of additional expenses. New dollars and saved dollars can be a one time or recurring sources of revenue.
While the ROI formula is simple, gathering all the data necessary to do the calculation is not. Businesses must know not only understand the Capex and Opex needed for an AI deployment, but also the business value and how to measure it.
When planning to deploy AI, businesses have to consider several key questions early on, including the following:
- Will the application run in-house or be delivered via a third party?
- Will the AI model running in-house also be trained and retrained in-house?
- Will an externally developed model be trained and retrained in-house?
- Will the data the model operates on be held in-house or stored in a provider cloud?
- Who and what will have access to the model?
- What will the model have access to?
Decisions based on these questions will determine which Opex and Capex items to prioritize.
Planning for the AI investment lifecycle
The investment cycle should start with a specific AI deployment followed by strategy development and project planning that includes the following steps:
- Determine which use cases to pursue.
- Plan the development and deployment of the AI tool.
- Set milestones for the deployment, including go/no-go decision points, to monitor progress.
- Define metrics that will quantify ROI.
For businesses that are hosting in-house or in IaaS clouds, planning and provisioning infrastructure is the next phase in the cycle followed by preparing data for AI model development, training and deployment. The model is then placed in production to generate actual value, which can be weighed against factors such as training for AI end users, process redesign costs and ongoing change management.
An AI model trained on company data will need periodic retraining to address behavioral problems, model drift or the need for a data refresh. Long-term, the model might have to be redeveloped to incorporate new architectural approaches.
Calculating AI capital expenses
Capex calculations include the physical or virtual infrastructure required for all parts of the AI project a company hosts, whether in its own data centers, colocation facilities or IaaS clouds. In addition to GPUs and other AI-targeted compute, infrastructure might include processor interconnects, additional storage and upgraded data center networking. An in-house deployment could also require capital upgrades to the cooling plant, power feeds, generators and uninterruptible power supply systems.
All these infrastructure costs must be factored in, even if they're prorated across multiple AI projects within the organization. Any data sets or intellectual property purchased outright must also be included in the Capex calculation.
Calculating AI operating expenses
Infrastructure costs and other Capex for AI vary a lot by the project's scale and very little by region or industry. Dell servers or ChatGPT API tokens, for example, cost the same in New York City and Idaho Falls, Idaho. By contrast, much of the Opex for AI projects are devoted to staffing costs and vary by region, industry and company size.
Time = money
Planning an AI project can take a few hours to several weeks and typically involves mid-tier IT staff, such as project managers; top-tier IT staff, such as AI, cloud, data center and network architects; and business line analysts and planners.
Infrastructure, APIs and intellectual property
Some Opex costs are associated with running and using capital resources, including the following:
- Network services consumed for the AI model.
- Rack space if hosting in colocation centers.
- Power for data centers if hosting in-house.
- IaaS capacity -- compute, storage, security and networking -- if the AI model is running in-house but hosting in the cloud. Data egress costs need to be closely watched because they can become a significant unexpected expense if the data flows from an IaaS environment aren't carefully planned.
Cloud AI services track the cost of API tokens, plus any associated data, network and security services, in addition to the costs of integration services and APIs used to connect AI tools to other platforms and systems. Licensing costs for items like commercial base models and data sets must also be factored in.
Model development and training
Opex includes the following staffing roles for various core activities:
- Data managers cleanse data and prepare it for labeling and subsequent use in training. They're typically on staff and considered medium salary employees.
- Data scientists handle model design, development, data labeling, training and retraining. They command high salaries whether they're on staff or contractors.
- AI architects and programmers can be on staff or contractors and typically command high salaries.
- Project managers oversee the development and production lifecycle and are usually on staff and considered medium salary employees.
- Integration developers connect AI tools to everything else and typically command medium salaries whether they're on staff or contractors.
As with any production system, multiple levels of change management are devoted to the data, the model and the application integrations, which can take a significant amount of time. Change management costs are highest in the first year of production deployment and should decrease after that yet remain as an ongoing cost.
Finally, consider any costs associated with retraining staff on using a new AI system. Some managers assume retraining isn't necessary because a properly built tool is user-friendly and intelligent. Testing a small group can help determine the staff's ability to adapt to the new AI system and whether retraining is needed. Retraining might actually cost less than the amount of productivity lost due to a lack of training.
Many happy returns?
When weighing an AI model's ROI, businesses typically compare the changes in revenue and costs after deployment.
One consideration is whether the AI system is driving revenue growth by producing new customers and increasing purchases from existing customers. Properly defined metrics and instrumentation should help determine the project's success or failure. For example, a business could measure customer spending on purchase add-ons before and after the rollout of AI-driven recommendations.
Another consideration is whether the AI system is reducing the cost of generating revenue. Metrics and instrumentation can pinpoint where the model is driving higher productivity. For example, a business can monitor whether a salesperson sells the same number of widgets in a quarter with reduced overhead expenses after AI agents replace human support staff.
When calculating an AI project's ROI, keep in mind that the most reliable result depends on the inclusion and accuracy of every Capex or Opex component.
John Burke is CTO and a research analyst at Nemertes Research. Burke joined Nemertes in 2005 with nearly two decades of technology experience. He has worked at all levels of IT, including as an end-user support specialist, programmer, system administrator, database specialist, network administrator, network architect and systems architect.
