model-based testing
Model-based testing (MBT) is an approach to software testing that requires developers to create a second, lightweight implementation of a software build called a model. Typically, a model consists of business logic and is just a few lines of code. Another program, called the driver, sends the same information to the software under test (SUT) and the model and then compares the input results to make sure they are the same. Any result that is different is a failure that needs to be examined.
Any component of an application that can be simulated (by the model), driven (by the driver program) and compared (by the results) is a candidate for model-based testing. The simplest model is an algorithm that takes inputs and creates a single output. If the application does one thing well, perhaps interacting with a database, a model, drivers, and some sample input could be all that is required to test the application. In other cases, a model is part of the solution.
Model-based testing requires a higher order of thinking than traditional, linear, "click-click type-inspect" GUI-based test automation. Testers who write MBT tend to be either strong enough technically to be production programmers, or to partner in some way with production programmers to get the test creates.
Model-based testing can be particularly good at finding memory leaks and potential conflicts that will cause the software to crash because automated tests can submit random input and run for extended periods of time. In practice, batch applications that take a single input (like a file) and produce a single output (like populating a database) are the easiest to get started with MBT. Other use cases include applications that are transaction-oriented, lack a clear user interface, or make it is possible to get "below" the UI at the API level. Some models may also include a state transition diagram, a finite state machine, a dependency graph or decision table -- typically captured and expressed in code.
Advantages and disadvantages of model-based testing
Advantages to MBT:
- Once the model is created, model-based tests can cover an incredibly large variety of scenarios with relatively little effort.
- Random execution of the model can uncover problems that would not be revealed up front.
- The MBT process can find design and specification errors quickly.
- Changes to the model will increase coverage and some MBT tools can automatically update test suites.
Disadvantages to MBT:
- MBT has a steep learning curve -- for developers integrating testing knowledge, and for testers learning how modeling relates to testing.
- If there is a large random test, MBT requires a great deal of infrastructure (many machines), a great deal of time (measured in hours to days for high coverage) or both.
- MBT tools can be time-consuming and difficult to implement, but actual test creation and execution can be faster, which leads to faster test processes.
