Beautiful Testing

Equivalent Mutants

Whenever we find a mutation that is not detected by a test suite, we must add an appropriate test. Sometimes, though, there is no way to write such a test—because the mutation does not change the program behavior at all. Example 18-4 shows such an equivalent mutant.

Example 18-4. Mutating the result from +1 to +2 makes no difference

public int compareTo(Object other) {
  if (!(other instanceof BcelAdvice)) 
    return 0;
  BcelAdvice o = (BcelAdvice)other;

  if (kind.getPrecedence() != o.kind.getPrecedence()) {
    if (kind.getPrecedence() > o.kind.getPrecedence()) 
      return +1;   // Mutation: replace +1 with +2
    else
      return -1;
  }
  // More comparisons...
}  ...

In Java, a compareTo() method compares the method target with another object; it is supposed to return a positive, negative, or zero value depending on whether the target is greater, smaller, or equal to the other object. Every caller of compareTo() will simply check for the sign of the comparison result, including the test. By changing the return value from +1 to +2, we keep the semantics of compareTo()—and the program—unchanged.

(Of course, we could change the test to specifically test for a return value of +1, and thus successfully detect the mutant. But this will not improve our test suite in any way either.)

The problem with equivalent mutants is that they are not as rare as one may think; in an experiment with the Jaxen XPath engine, we found as many as 40% equivalent mutants in a sample of undetected mutants. None of these ...