Data Structure and Software Engineering

A Tester-Assisted

Methodology for Test

Redundancy Detection

Negar Koochakzadeh and Vahid Garousi

ABSTRACT

Test redundancy detection reduces test maintenance costs and also ensures the

integrity of test suites. One of the most widely used approaches for this pur-

pose is based on coverage information. In a recent work, we have shown that

although this information can be useful in detecting redundant tests, it may

suer from large number of false-positive errors, that is, a test case being iden-

tied as redundant while it is really not. In this paper, we propose a semiauto-

mated methodology to derive a reduced test suite from a given test suite, while

keeping the fault detection eectiveness unchanged. To evaluate the methodol-

ogy, we apply the mutation analysis technique to measure the fault detection

eectiveness of the reduced test suite of a real Java project. e results conrm

that the proposed manual interactive inspection process leads to a reduced test

suite with the same fault detection ability as the original test suite.

164 Data Structure and Software Engineering: Challenges and Improvements

Introduction

In today’s large-scale software systems, test (suite) maintenance is an inseparable

part of software maintenance. As a software system evolves, its test suites need to

be updated (maintained) to verify new or modied functionality of the software.

at may cause test code to erode [1, 2]; it may become complex and unmanage-

able [3] and increase the cost of test maintenance. Decayed parts of test suite that

cause test maintenance problems are referred to as test smells [4].

Redundancy (among test cases) is a discussed but a seldom-studied test smell.

A redundant test case is one, which if removed, will not aect the fault detection

eectiveness of the test suite. Another type of test redundancy discussed in the

literature (e.g., [5, 6]) is test code duplication. is type of redundancy is similar

to conventional source code duplication and is of syntactic nature. We refer to the

above two types of redundancy as semantic and syntactic test redundancy smells,

respectively. In this work, we focus on the semantic redundancy smell which is

known to be more challenging to detect in general than the syntactic one [5].

Redundant test cases can have serious consequences on test maintenance. By

modifying a software unit in the maintenance phase, testers need to investigate

the test suite to nd all relevant test cases which test that feature and update

them correctly with the unit. Finding all of the related test cases increases the

cost of maintenance. From the other hand, if test maintenance (updating) is not

conducted carefully, the integrity of the entire test suite will be under question.

For example, we can end up in a situation in which two test cases test the same

features of a unit, if one of them is updated correctly with the unit and not the

other one, one test may fail while the other may pass, making the test results am-

biguous and conicting.

e motivation for test redundancy detection is straightforward. By detecting

and dealing with redundant test case (e.g., carefully removing them), we reduce

test maintenance cost and the risk of loosing integrity in our test suite, while fault

detection capability of our test suite remains constant.

One of the most widely used approaches in the literature (e.g., [6–11]) for test

redundancy detection, also referred to as test minimization, is based on coverage

information. e rationale followed is that, if several test cases in a test suite ex-

ecute the same program elements, the test suite can then be reduced to a smaller

suite that guarantees equivalent test coverage ratio [6].

However, test redundancy detection based on coverage information does not

guarantee to keep fault detection capability of a given test suite. Evaluation results

from our previous work [12] showed that although coverage information can be

very useful in test redundancy detection, detecting redundancy only based on this