How to overcome Kubernetes cost management struggles

The complexity of Kubernetes cost optimization

Kubernetes cost management can be highly challenging. This is largely due to the sheer complexity of Kubernetes, which requires organizations to adopt unique cost-optimization tactics that extend beyond classical FinOps strategies. Key complications in Kubernetes cost optimization include the following:

Highly dynamic environments

Most Kubernetes clusters are dynamic because workloads are constantly moving around. Part of what makes Kubernetes so powerful is its ability to migrate applications from one server to another based on factors such as changes in resource utilization patterns.

However, Kubernetes's dynamic nature also makes it hard to consistently track costs. To track Kubernetes spending on a per-workload basis, you need to know which servers host each workload and how many resources each workload consumes. Since this varies as workloads migrate between servers and as their resource consumption scales up and down, collecting the data necessary to monitor spending in real time is tough.

Varying Kubernetes deployment models

There are multiple ways to deploy Kubernetes. The three main options include:

• Deploying Kubernetes on your own infrastructure using an on-prem or private cloud deployment model.
• Running Kubernetes in a public cloud by setting up a Kubernetes cluster on cloud-based VMs that you manage yourself.
• Using a fully managed Kubernetes service -- like Amazon EKS or Azure AKS -- which provides both the host infrastructure and Kubernetes software through a single, integrated offering.

Each of these deployment architectures comes with different cost variables. For example, with an on-prem deployment, a primary cost consideration is the capital investment required to purchase the hardware that hosts Kubernetes. In contrast, with a cloud-based deployment, a business would typically pay monthly for the infrastructure it uses every month to operate its cluster.

Due to the variability in pricing models, cost tracking and management strategies that work for one type of Kubernetes deployment may not work for another.

The sharing of resources across projects and teams

A common FinOps goal is the ability to align spending with different teams, users or projects. This provides much more granular insight into costs. With Kubernetes, however, granular cost tracking can be tough because multiple teams or projects frequently share a single cluster. Per-workload or per-container cost monitoring is possible, but it requires the ability to track costs as workloads move across different servers within a cluster and as they scale up and down. This adds significant complexity to the cost management process.

Lack of native cost monitoring features

Kubernetes doesn't provide any built-in cost-tracking or management functionality. It can report data, such as how much CPU or memory a workload is using. It's possible to extrapolate from this data to estimate the workload's cost, but Kubernetes doesn't do that translation natively.

The gap between FinOps and Kubernetes

Kubernetes has been an outlier within businesses' FinOps strategies. In theory, FinOps can help organizations optimize the value of virtually any type of IT platform or workload. But in practice, it has typically been easier to manage and optimize spending for other types of resources, such as cloud-based VMs and databases.

The problem isn't just the inherently complex nature of Kubernetes cost management. It's also that FinOps tooling for Kubernetes is more limited than FinOps offerings that manage other types of platforms or workloads. For example, while the major public clouds offer built-in cost management tools -- such as AWS Cost Explorer -- that track spending for most types of cloud services, they usually report only very basic data about Kubernetes.

Common examples of Kubernetes FinOps failures

To contextualize how organizations can inadvertently waste money on Kubernetes, consider the following common examples of cost-inefficient deployments.

• Too many nodes. A Kubernetes cluster that includes more nodes than are necessary to support its workloads. The organization has to pay for each node, even though some would likely sit idle or be under-utilized in this scenario.
• Excess reservations. A set of containers that have more CPU or memory reserved for them than they need. The business pays for more resources than its workloads use.
• Incorrect instance type. A cluster that uses standard, pay-as-you-go cloud server instances instead of taking advantage of more cost-effective reserved instances.
• Replica issues. A configuration where admins deploy more application replicas than necessary to achieve a workload's reliability requirements. Replicas are a Kubernetes feature that runs multiple copies of the same app to ensure availability but can drain resources.
• Unused persistent volumes. The existence of persistent volumes (PVs) that are no longer actively being used. PVs are a way of storing data persistently in Kubernetes. Typically, admins should delete them when they're no longer in use to avoid wasting storage resources.

Best practices for optimizing Kubernetes spending

The good news for businesses aiming to maximize the ROI of Kubernetes is that doing so is possible, with help from practices and tools designed for the purpose. Here's a look at actionable best practices for maximizing the ROI of Kubernetes.

Reduce server costs

One of the simplest and most impactful steps a business can take to cut its Kubernetes infrastructure costs is to use lower-cost servers. In a cloud-based Kubernetes cluster, reserved instances offer discounted pricing in exchange for a commitment to long-term use. Since Kubernetes clusters are typically operational for an indefinite period, it makes sense to use reserved instances.

In some cases, businesses could even use spot instances, which provide very steep discounts but can shut down without warning. While you wouldn't want to use a spot instance for a workload that requires high availability, it can help reduce Kubernetes costs for workloads that can be interrupted, such as application testing or AI model training.

Reducing infrastructure costs in an on-prem cluster can be more challenging because businesses can't simply "swap and replace" virtual servers, as they would in the cloud. Still, strategies like extending the life of physical servers or increasing the density of VMs on a physical server can help.

Assess resource quotas and limits

As mentioned above, resource quotas assign a specific amount of resources to Kubernetes workloads. Limits restrict the total resources that workloads can use.

Quotas and limits are useful features for efficiently dividing up Kubernetes resources and preventing situations where one workload "hogs" resources from others. However, poorly configured quotas and limits can waste resources and money.

Businesses need to revisit resource quotas and limits at least periodically. In the best-case scenario, businesses will continuously compare the quotas and limits for Kubernetes workloads with observability data showing what those workloads are actually consuming. An imbalance -- such as a workload's quota that is significantly higher than its actual use -- requires attention.

Delete orphaned resources

A staple best practice in FinOps is to delete orphaned resources. It's especially important in Kubernetes, since the platform doesn't automatically report or delete resources such as orphaned persistent volumes. Businesses need to audit resources to determine which ones have become orphaned. Admins can do this manually using kubectl -- the Kubernetes command-line administration tool. Also, Kubernetes observability and cost management tools can help automate the practice by identifying orphaned resources automatically.

Separate teams and projects by namespace

Namespaces are a Kubernetes feature that creates virtual clusters. A business can more easily apply different types of resource management and cost-tracking policies with separate namespaces for each team or project that uses Kubernetes.

For instance, a namespace that hosts experimental workloads could be configured to use less expensive servers, while more costly, higher-performing nodes are reserved for mission-critical workloads in a different namespace.

Enable autoscaling

Autoscaling capabilities automatically modify resource configurations in Kubernetes based on demand changes. There are three main types:

1. Horizontal pod autoscaling. Increases or decreases workload replicas based on workload demand.
2. Vertical pod autoscaling. Adjusts workload resource allocations based on workload demand.
3. Cluster autoscaling. Adds or removes nodes from a cluster. This type is typically only available in cloud-based Kubernetes environments, since it requires the ability to provision new servers automatically.

In most Kubernetes distributions, autoscaling is not turned on by default, but admins can enable it to help strike a healthy balance between resource allocations and usage.

Deploy Kubernetes cost management tools

Although Kubernetes itself offers no built-in cost management features, add-on tools are available that provide these capabilities. The main offering in this vein is Kubecost, a free and open source tool. Kubecost automatically tracks spending metrics and generates alerts when they exceed the thresholds admins set. The tool can also provide recommendations for optimizing Kubernetes spending.

Commercial Kubernetes cost management capabilities are also available in some FinOps tools -- although, unlike Kubecost, most of these don't focus specifically on Kubernetes cost optimization.

Chris Tozzi is a freelance writer, research adviser, and professor of IT and society who has previously worked as a journalist and Linux systems administrator.