Methodology for Test
Negar Koochakzadeh and Vahid Garousi
Test redundancy detection reduces test maintenance costs and also ensures the
integrity of test suites. One of the most widely used approaches for this pur-
pose is based on coverage information. In a recent work, we have shown that
although this information can be useful in detecting redundant tests, it may
suer from large number of false-positive errors, that is, a test case being iden-
tied as redundant while it is really not. In this paper, we propose a semiauto-
mated methodology to derive a reduced test suite from a given test suite, while
keeping the fault detection eectiveness unchanged. To evaluate the methodol-
ogy, we apply the mutation analysis technique to measure the fault detection
eectiveness of the reduced test suite of a real Java project. e results conrm
that the proposed manual interactive inspection process leads to a reduced test
suite with the same fault detection ability as the original test suite.
164 Data Structure and Software Engineering: Challenges and Improvements
In today’s large-scale software systems, test (suite) maintenance is an inseparable
part of software maintenance. As a software system evolves, its test suites need to
be updated (maintained) to verify new or modied functionality of the software.
at may cause test code to erode [1, 2]; it may become complex and unmanage-
able  and increase the cost of test maintenance. Decayed parts of test suite that
cause test maintenance problems are referred to as test smells .
Redundancy (among test cases) is a discussed but a seldom-studied test smell.
A redundant test case is one, which if removed, will not aect the fault detection
eectiveness of the test suite. Another type of test redundancy discussed in the
literature (e.g., [5, 6]) is test code duplication. is type of redundancy is similar
to conventional source code duplication and is of syntactic nature. We refer to the
above two types of redundancy as semantic and syntactic test redundancy smells,
respectively. In this work, we focus on the semantic redundancy smell which is
known to be more challenging to detect in general than the syntactic one .
Redundant test cases can have serious consequences on test maintenance. By
modifying a software unit in the maintenance phase, testers need to investigate
the test suite to nd all relevant test cases which test that feature and update
them correctly with the unit. Finding all of the related test cases increases the
cost of maintenance. From the other hand, if test maintenance (updating) is not
conducted carefully, the integrity of the entire test suite will be under question.
For example, we can end up in a situation in which two test cases test the same
features of a unit, if one of them is updated correctly with the unit and not the
other one, one test may fail while the other may pass, making the test results am-
biguous and conicting.
e motivation for test redundancy detection is straightforward. By detecting
and dealing with redundant test case (e.g., carefully removing them), we reduce
test maintenance cost and the risk of loosing integrity in our test suite, while fault
detection capability of our test suite remains constant.
One of the most widely used approaches in the literature (e.g., [6–11]) for test
redundancy detection, also referred to as test minimization, is based on coverage
information. e rationale followed is that, if several test cases in a test suite ex-
ecute the same program elements, the test suite can then be reduced to a smaller
suite that guarantees equivalent test coverage ratio .
However, test redundancy detection based on coverage information does not
guarantee to keep fault detection capability of a given test suite. Evaluation results
from our previous work  showed that although coverage information can be
very useful in test redundancy detection, detecting redundancy only based on this