5 key software testability characteristics

Untestable architectures

These are examples of architectures with low software testability that are common in today's development organizations:

• The organization tests messages moving between IoT devices and the network, but without tracing individual service calls from the applications.
• Data center backup processes take a week to test, during which there is no environment available for regular testing.
• The test environment is such a scaled-down version of production that it cannot realistically simulate real workloads.
• Software is only testable through the user interface, and there are no tracing systems or logs to identify what went wrong during a failed test.
• Testing tools can recreate problem-causing bugs and errors, but operations and development teams point fingers at each other about where the problem lies.

Often, software teams find that, despite these testing shortcomings, applications still work well enough. The problem is that this status quo creates an architecture that is not very testable and these issues become exponentially problematic in a move to microservices. But testing problems need not be inevitable.

Elements of a testable architecture

Comprehensive software testability should identify dynamic elements and events of a system, as well as system boundaries and interfaces. Know what was done to each piece of a system, who made the changes and when the changes were made. Set up an environment and tooling to recreate the system that had the problem, reproduce that problem quickly, monitor the issue and then fix and retest as necessary. Finally, look at each subsystem in isolation to locate specific failures in the architecture.

Here are the five key elements that fortify an architecture's testability:

1. Logging. Effective system logs tell you what happened and when. For high software testability, set up logs so that you can list every web service call and see who made it, when they made it and in which order it was processed. These logs should also be searchable and provide explanations for why an end user saw an error, even if that error doesn't appear in subsequent requests.
2. Scriptable message passing. It's important to know what messages have transferred back and forth, but it's equally important to be able to replay the script of those message transfers on demand. Logging systems should relay capture traffic scripts in a way that enables developers to simulate messaging instances across all systems.
3. Scriptable components. While most teams can script commands made to the database, they might not be able to execute business logic on a web server through the code available to them. Create a test web server in the cloud to gain this ability. Modern software deployment technologies, such as Docker containers, make it possible to create test servers that are copies of the server in production.
4. Swappable components. Mocks, stubs and fakes can all be powerful methods of service virtualization to isolate a part of a system and test it. For instance, while a programmer could simply test a database connection through strings of connection code, a higher-testability version of this process would use a database connection object that is created at runtime. If a programmer can create an abstract base class that's passed along at runtime, then they can use that object to create a mock, stub or fake for testing. This is part of a process known as dependency injection.
5. Scriptable infrastructure. The coup-de-grace of testability is to create the entire infrastructure as code. For a complex system, that means a configuration file supplies what elements of the system the tester needs (such as a web server, database or file system), what version of the code to create, and what data to load into the database. With a simple, one-line command and a few seconds, a tester can create a realistic environment to test in.

The most broken system is the one with the least testability. Find a point where you can add the most testability for the least price and get to work. There will be a million reasons why this seems impossible. But you can either fix it or learn to live with the burden of an opaque architecture.