Essential KPI considerations for IT operations managers

Connect IT initiatives to measurable business outcomes

One of the best ways to begin identifying which KPIs to use is to talk with the organization's C-level executives and ask what KPIs they rely on to evaluate the organization's overall health. After all, someone probably has already put significant effort into determining which KPIs most accurately represent the organization's overall well-being. Although not all these KPIs will be relevant to IT operations managers, at least some likely will be.

For example, suppose your organization operates an e-commerce website and is about to implement a new AI-based tool that recommends products to customers based on their past purchase history. That feature should have a quantifiable effect on business outcomes.

Support metrics that reveal what's really driving change

The most obvious way to determine the new feature's effectiveness in delivering the desired business outcome is to track revenue related to e-commerce. However, even if the revenue does increase, you can't know with 100% certainty that the increase stems from the new feature that you have implemented.

That being the case, consider what other metrics you might be able to track to prove or disprove the idea that the new feature added to your online store was responsible for increasing revenue.

Because the feature we're using in this example is designed to upsell products to customers based on their past purchases, you might begin by examining the number of unique customers who visit your site within a day, week or some other time frame.

If the customer count stays about the same as it has always been but revenue increases, then it means that existing customers are spending more -- which is the desired effect.

If, on the other hand, your revenue is increasing but so too are the number of customers visiting your online store, then the revenue increase can most likely be attributed to all the new customers as opposed to the new feature that you have implemented.

Track customer behavior to prove feature effectiveness

Another way to track the effectiveness of the store's new feature is to monitor the number of items customers are purchasing. Because the new feature is designed to target existing customers, you should be able to review all past transactions and determine how many items each customer has purchased in each transaction. You can then determine if there is an increase following the implementation of your new feature.

Although you could track the average number of items purchased as a generalized metric, you might get better results by comparing averages on a customer-by-customer basis. If a particular customer typically makes a purchase once a month and normally buys three items at a time, but now they are purchasing six items, then that customer's item count has increased significantly.

By tracking item count on a customer-by-customer basis, you can then figure out the average increase or decrease in item count across all your customers. Such a metric might, for example, indicate that on average, customers purchased 1.3 additional items following the implementation of the new feature.

Measure operational performance within the IT department

Although business outcomes are among the most important metrics you can track, there are other metrics that indicate how well the IT department is functioning. For example, you could track the first call resolution rate for your help desk.

Other help desk-related metrics might include the mean time to resolve, mean time between failures -- how often users experience problems -- tickets per day and user satisfaction ratings. By tracking these metrics, you can gauge whether the help desk is performing better or worse over time.

Evaluate the true cost of running mission-critical workloads

Another way you can use KPIs to assess IT effectiveness is to track various metrics associated with the systems that host mission-critical workloads. To illustrate how this works, let's revisit the previous example of an organization that operates an e-commerce site.

Most likely, the organization has multiple revenue streams; therefore, the income from the e-commerce site represents only a portion of the organization's total revenue. If that's the case, then the organization will most likely want to know if the e-commerce site is generating a profit or incurring a loss, as it's entirely possible for a site's operational costs to exceed the revenue it generates. If the site is making money, how profitable is it?

If the site is cloud-based, determining profitability is relatively easy. You can simply compare the cloud costs to the site's revenue.

If the site is operated on-premises, then you must determine the infrastructure costs (hardware and software) and operational costs (power, cooling, etc.). Depending on the accounting methods your organization uses, you might need to depreciate the hardware costs over several years. Eventually, however, you should be able to determine the average annual cost associated with hosting the workload. This will indicate whether the site is profitable. You can also use these same metrics to determine the per-minute cost associated with an outage.

Monitor IT efficiency through cost and maintenance trends

From an IT perspective, there are various metrics that you can track as a way of determining whether workload costs are increasing or decreasing. Some of these metrics might include staff hours (for maintenance and repairs), outages, replacement parts (for on-premises workloads) and costs associated with upgrades. You can use these metrics to gauge both workload profitability and IT efficiency.

In any organization, there are countless KPIs that could potentially be used. The key to selecting the right KPIs is to ensure that the chosen KPIs are relevant, can be measured accurately and provide actionable insights. It's also important to make sure that KPIs can be tracked in a timely manner. Otherwise, important decisions might be delayed.

Brien Posey is a former 22-time Microsoft MVP and a commercial astronaut candidate. In his more than 30 years in IT, he has served as a lead network engineer for the U.S. Department of Defense and a network administrator for some of the largest insurance companies in America.