What is system testing?
System testing, also referred to as system-level testing or system integration testing, is the process in which a quality assurance (QA) team evaluates how the various components of an application interact together in the full, integrated system or application.
System testing verifies that an application performs tasks as designed. It's a type of black box testing that focuses on the functionality of an application rather than the inner workings of a system, which white box testing is concerned with.
System testing, for example, might check that every kind of user input produces the intended output across the application. System testing is the third level of testing in the software development process. It's typically performed before acceptance testing and after integration testing.
Importance of system testing
System testing is imperative for interconnected systems because any system or software defect can cause extreme complications for the consumers. For example, in 2014, Nissan had to recall over 1 million cars due to a defect in the airbag sensor detectors.
With system testing, a QA team determines whether a test case corresponds to each of an application's most crucial requirements and user stories.
The following are the main benefits of system testing:
- Improved product quality. A comprehensive system testing process ultimately boosts the product quality. Since an integrated system is tested through multiple test sets in a product development cycle, it provides a glimpse into whether a product can successfully work across different platforms and environments.
- Error reduction. Some errors are bound to happen during the development of complex systems. System testing verifies a system's code and functionality against its requirements, so errors that aren't detected during integration and unit testing can be exposed during system testing.
- Cost savings. It can be more time-consuming to fix a system defect that's detected later in the project lifecycle. Conducting timely and continuous system testing not only reduces unexpected costs and project delays, but also provides project managers with better budget control.
- Security. Well-tested products are reliable. They ensure that the tested system doesn't contain potential vulnerabilities that can put end users and system data at risk of potential threats.
- Customer satisfaction. System testing offers visibility into the stability of a product at every stage of development. This builds customer confidence and improves the overall user experience.
- Easier code modification. System testing can identify code problems during software development. Fixing older code that has gone into the production environment is much harder than modifying it while it's still in development.
- Software performance. Performance-based system tests can help understand changes in a system's performance and behavior, such as memory consumption, central processing unit utilization and latency. These tests raise red flags if system performance degrades significantly, enabling developers to take proactive action.
Types of system testing
With system testing, a QA team gauges if an application meets all of its technical, business and functional requirements. To accomplish this, the QA team might utilize various types of software testing techniques that determine the overall test coverage for an application and help catch critical defects that hamper an application's core functionalities before release.
The following are the common types of system testing techniques:
- Performance testing. Performance testing measures the speed, average load time, stability, reliability and peak response times of the system under various conditions. It's typically coupled with stress testing and may include both hardware and software testing tools.
- Usability testing. These are tests to evaluate if a system is easy to use and functional for the end user. Metrics, including user error rates, task success rates, the time it takes a user to complete a task and user satisfaction, are used during testing.
- Load testing. This is testing to determine how a system or software performs under a real-life extreme load and test scenarios. Metrics, such as throughput, number of users and latency, are measured through this testing.
- Regression testing. Also known as sanity testing, it ensures that all changes introduced into an application or code during system testing, recent code changes or updates haven't caused any new bugs or issues. Regression testing is responsible for the functionality of the existing features of a system or software.
- Migration testing. This is conducted to ensure smooth migration of legacy systems to new systems without disruptions, data loss or downtimes.
- Scalability testing. This measures an application's or system's capability to scale up or down when trying to meet the changing user requirements.
- Functionality testing. This is conducted to validate a system's functionality against its functional and business requirements.
- Recovery testing. This is a type of nonfunctional testing done to ensure that a system is capable of recovering from certain system errors, crashes and failures.
Phases of system testing
System testing examines every component of an application to make sure that they work as a complete and unified whole. A QA team typically conducts system testing after it checks individual modules with functional or user story testing and then each component through integration testing.
If a software build achieves the desired results in system testing, it gets a final check via acceptance testing before it goes to production, where users consume the software. An app dev team logs all defects and establishes what kinds and numbers of defects are tolerable.
Typically, system testing goes through the following stages:
- Test environment. In this initial stage, a test server is set up for creating a testing environment, which enables a tester to run a set of predefined test cases and test scripts.
- Test case. This stage generates the test case for the testing process.
- Test data. At this stage, the data to be tested is generated.
- Test case execution. Once the test case and test data are generated, test cases are executed.
- Reporting of defects. This is the stage where defects in the system are identified.
- Regression testing. This is done to see if any problems were introduced into the development process by the previous stages.
- Defect logging. All the identified defects are fixed at this stage.
- Retest. A test is repeated if it's unsuccessful.
System testing tools
Various commercial and Open Source tools help QA teams perform and review system testing results. These tools can create, manage and automate tests or test cases, and they might also offer features beyond system testing, such as requirements management capabilities.
Commercial system testing tools include Froglogic's Squish and Inflectra's SpiraTest, while open source tools include Robotium and SmartBear's SoapUI.
Not every software tester has access to all of the available testing resources. For example, a tester working for a large enterprise likely has access to expensive automated testing that might not be available to smaller organizations.
Each software development team follows its own preferred method of application testing. Select a software testing model by exploring these five factors first.