While load tests are one of the most basic routines that software teams perform, there's a level of difficulty associated with load testing for microservices. Distributed applications experience volatile changes in terms of scale and resource consumption, which makes apps vulnerable to performance issues and monitoring continuously difficult. A microservices-specific load testing strategy can help minimize this risk, but testing teams will likely face some challenges along the way when they need to make a practical plan.
There are no shortcuts to perform microservices load testing. Application managers can't just test individual, high-risk application components because these components' shared relationships typically create performance and stability issues.
The appropriate way to address this is to account for load testing during the planning stage of a microservices-based application and continuously perform load tests throughout the application lifecycle. By following the principles below, development and testing teams can create a comprehensive load testing strategy for microservices-based applications.
1. Start with APM
Microservices load testing shouldn't start with tests on simulated data loads. Such QA should start with a focus on application performance monitoring (APM).
Highly componentized applications, such as microservices-based applications, are notoriously difficult to analyze. Even with comprehensive load testing, it can be difficult to determine overall performance or to pinpoint the cause of any performance issues. An APM strategy ensures teams know what to observe during a load test and also enables them to correlate test results with production behavior.
2. Focus on observability
Observability is key for effective load testing during microservices design. IT teams typically establish performance and scalability parameters for microservices. If the data necessary to monitor these parameters isn't available, it's time to look at new types of APM tools. Otherwise, developers will have to add automated monitoring processes to the code.
Kubernetes provides strong monitoring tools that will not likely require developers to manually add monitoring routines. In addition, a service mesh -- such as Istio or Linkerd -- will provide a full spectrum of load testing parameters and observability.
3. Maintain a holistic view
Teams shouldn't rely on partial or unit testing for microservices load tests. Microservices are easy to scale and redeploy, and the amount of logic in a single service is rarely the cause of performance issues. What really matters is the application's overall workflow and scalability characteristics.
While it's okay to perform unit testing and section testing of an application's logic, the results won't serve as a guide to performance under load. Feed work into the application's normal workflow channels and assess responses from there.
If overall performance under load -- measured in the turnaround time for processing -- isn't satisfactory, then you can start to drill down and determine the individual components causing the problem.
For instance, you can look at orchestration data garnered from the service mesh to track workflows and deployment points. This tracking should reveal which components are scaling under load and how those scaling processes affect performance. Grafana and JMeter are a couple other tool examples that can help provide this type of centralized view.
4. Analyze scaling patterns
When testing microservices, it's important to evaluate the time it takes to connect workloads, load balance across instances, spin up new instances and scale back when workload volume decreases. Visibility into these operational patterns, as well as the application's overall behavior under certain scaling conditions, is critical.
To perform load testing on a microservices-based application using simulated scenarios, don't simply ramp up workloads and measure the effects of the added stress in isolation. Instead, introduce the same kind of sporadic demand and fluctuating workflow that would occur in a production environment.
5. Test across hosting domains
In hybrid and multi-cloud environments, microservices traffic might move across various functional boundaries and hosting domains. This movement can create congestion and network latency, leading to longer deployment times for application components in another domain.
While not all applications will span multiple hosting domains, things like cloud-bursting or failover between domains will inevitably lead to crossover. As such, it will be very difficult to measure load test results without centralized monitoring across all hosting domains the microservices might use.
6. Find an appropriate work distribution framework
The final microservices load testing principle is to select the framework that carefully guides workflow and load distribution. Examples of work distribution frameworks include things such as API gateways and service mesh implementations since they both play a central role in scaling and load balancing procedures.
Before teams select a specific framework, they need to determine whether they're adding unnecessary complexity to the movement of work between the microservices that make up an application.
For instance, service mesh is extremely useful when it comes to dealing with large, highly distributed environments in a way that may prove too much for a simple API broker. However, service mesh implementations often come equipped with a sophisticated array of features, such as distributed monitoring and tracing capabilities. If the environment is simple and relatively contained, it may be better to go with a simple API broker.