Advanced Testing of Systems-of-Systems, Volume 2

10Test Tools and Test Automation

Test automation is often perceived as “the” solution to facilitate and accelerate the execution of application tests, including for systems-of-systems. This vision must be modulated according to the objectives of each of the stakeholders on the system-of-systems.

We will define a test tool as a product to help the execution of one or more test activities. So, we will talk about tools that will help in test design, static test tools and dynamic test tools.

Like any testing activity, automation must demonstrate added value to justify its implementation. This value can be seen in the short term or in the longer term.

10.1. Objectives of test automation

Test automation is often seen only as a way to run more dynamic tests in a shorter amount of time, or “automatically” find more defects, or even reduce the cost of testing. It is illusory and unrealistic.

Having good goals is critical if we hope to achieve those goals. It is surprising how many test automation investment decisions are made without clearly defined goals or with vague justifications. It is important to make a distinction between our goals for testing (e.g. find as many defects as possible, reduce risk) and the goals we can achieve with dynamic test automation.

Realistic goals for test automation include:

– ensure the early detection of defects, from the drafting of requirements and codes, for example, by implementing static code analysis tools and TDD- or ATDD-type techniques;