One of the best things about Kubernetes is its built-in scalability. Because Kubernetes does all the heavy lifting for you, deploying instances is a breeze.
On the flip side, a virtual Kubernetes cluster is still located in a physical data center, which means it shares the risks affecting that data center, however modest. The risks to a Kubernetes cluster come in many forms, including physical dangers -- such as earthquakes, tornadoes, flooding and theft -- as well as data loss, ransomware, and internal IT or hardware failures.
Even big cloud service providers (CSPs) are not immune from this reality. For this reason, CSPs have designed multiple redundant physical data centers within a single cloud region with locally and geographically redundant storage.
Benefits of multi-cluster Kubernetes
Similarly, using multiple clusters has several advantages when it comes to Kubernetes management. Losing one cluster when your applications are appropriately spread across several other clusters is much better than losing a single cluster with everything on it.
Multi-cluster Kubernetes environments have many advantages, even more so when geographically redundant -- if they are configured correctly and managed appropriately. The benefits of running multiple Kubernetes clusters include the following:
- Ability to run different cluster software versions. Upgrading a cluster isn't risk free. Separating a properly architected application across multiple clusters and regions means that if an error occurs in one cluster, the other appropriately configured clusters can take the load. Sometimes a cluster must remain on a specific software version, but there's no reason not to have that version on one cluster and other versions on different clusters.
- Ability to separate environments. It is considered best practice to have separate development and production clusters to reduce the risk of being served beta or non-production code versions. In addition, some environments are subject to data sovereignty rules requiring they be geographically located in a certain data center or region.
- Better end user experience. No one likes a slow internet browsing experience. Splitting workloads using a load balancer such as Azure Traffic Manager can improve performance and reduce latency by providing users with a geographically close environment.
Kubernetes clusters vs. namespaces
Kubernetes namespaces are a way to logically group applications or parts of an application in the same cluster without having to consider multi-cluster Kubernetes configurations. A namespace is not a separate cluster and does not improve redundancy. Namespaces are merely a way to logically segment applications when needed.
Challenges of multi-cluster Kubernetes
All this resilience doesn't come free. Most modern cloud providers, such as Azure, have a reference design for multi-cluster setups. Because of the highly ephemeral nature of containers, it makes sense to design for failure as much as possible at the outset.
When properly configured with appropriate traffic routing, providing resources from several different geographic areas enables requests to be served quicker because the cluster is geographically closer than it otherwise might have been. Obviously, this requires some upfront investment to configure the appropriate load balancers and routing.
Isolating data can also be necessary for legal reasons. The legal considerations are much simpler if all your data is located and served within a country or region that's required to follow well-defined data sovereignty and management rules, such as the GDPR. However, the details depend upon your customer base and their locations in the world. This is a key consideration, and mistakes can be financially ruinous. In short, there is more to consider than availability and technology.
From a technical standpoint, managing multiple Kubernetes clusters is more complex than managing a single cluster. Multiple clusters require multiple cluster configuration files. Centralized management avoids duplication of effort and reduces management overhead and scope for errors.
Tools for Kubernetes multi-cluster management
There are many options for automation and desired state tools, and the decision often comes down to the level of control needed and the environmental complexity. These factors need to be appropriately considered before adopting a tool.
Infrastructure as code
Kubernetes and infrastructure as code (IaC) go hand in hand. IaC helps ensure deployment consistency and reduce human error. Terraform is the go-to tool when creating an IaC environment because it can destroy and create infrastructure on the fly.
All these different deployment templates require version control to be useful. When deployment templates are properly managed, reverting to a previous configuration becomes as simple as using Git's git revert command and rerunning the process.
Managed Kubernetes services
While it is possible to create your own clusters on virtual or physical machines, there are many reasons to use the Kubernetes services that most major CSPs provide, such as Google Kubernetes Engine or Amazon Elastic Kubernetes Service.
With these managed Kubernetes services, the vendor does all the heavy lifting and maintenance. When implemented correctly, this can make multi-cluster management substantially cheaper.
Finally, service meshes provide tools to manage more complex infrastructures. Open Service Mesh is a popular example, but most service meshes have a broad range of capabilities:
- Service discovery.
- Traffic management.
- The ability to monitor the whole environment rather than individual service components.
Service mesh tools also help enforce security constraints, such as encryption between all services using Mutual Transport Layer Security. Service meshes also help ensure conformity of the running environment at scale.