Chapter 4. Writing Unit Tests
The previous chapters present a simple unit test framework and the fundamentals of xUnit. The unit test framework’s architecture is important to understand, but not something you have to think about often. Most of your time should be spent writing unit tests, implementing production code to make the tests pass, or refactoring. This chapter includes examples of common patterns used when writing unit tests, as well as related tips on unit test development.
The code examples in this chapter are unit tests of additional
virtual library functionality, including looking up books by author
and title, looking up multiple books by one author, and removing
books from the library. The Library and
Book code to implement the new features is given
at the end of the chapter.
Types of Asserts
The code examples shown
so
far use plain asserts
. These are the most generic type of
test assertion, which take a Boolean condition that must evaluate to
TRUE for the test to succeed. A plain assert, the
unit test for the
Library
method
removeBook( ), is shown in Example 4-1.
LibraryTest.java
public void testRemoveBook( ) {
library.removeBook( "Dune" );
Book book = library.getBook( "Dune" );
assertTrue( book == null );
}If the method removeBook( ) is stubbed out, the
test fails. The following test results report the
failure:
> java junit.textui.TestRunner LibraryTests
.....F.
Time: 0.06
There was 1 failure:
1) testRemoveBook(LibraryTest)junit.framework.AssertionFailedError
at LibraryTest.testRemoveBook(LibraryTest.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
FAILURES!!!
Tests run: 6, Failures: 1, Errors: 0Although the line of code where the failure occurred is shown, the output does not describe the specific cause of the failure. It often is helpful to add an informative message to the assertion. The xUnits generally have two versions of every assert method, one of which takes a message parameter describing the assert. Example 4-2 shows the test method using an assert with a message.
LibraryTest.java public void testRemoveBook( ) { library.removeBook( "Dune" ); Book book = library.getBook( "Dune" ); assertTrue( "book is not removed", book == null ); }
With the additional message, the rest results provide better information about the cause of the test failure:
1) testRemoveBook(LibraryTest)junit.framework.AssertionFailedError: book is not removed
Although all assert conditions ultimately must evaluate to a Boolean
result of TRUE or FALSE, it can
be tedious to constantly reduce every expression to this form. The
xUnits offer a variety of assert functions to help. Examples of
several of the assert methods from JUnit are as follows:
assertFalse( book == null ); assertFalse( "book is null", book == null ); assertNull( book ); assertNull( "book is not null", book ); assertNotNull( book ); assertNotNull( "book is null", book ); assertEquals( "Solaris", book.title ); assertEquals( "unexpected book title", "Solaris", book.title );
These assert methods all have variants that take a message parameter
to describe the failure, as shown above. The assertEquals()
method has variants that take different data types as arguments.
Defining Custom Asserts
The basic assert methods cover only a few common cases. It’s often useful to extend them to cover additional test conditions and data types. Custom assert methods save test coding effort and make the test code more readable.
So far, the Library tests check a
Book’s title attribute to verify
the expected Book object, as shown in Example 4-3 in the test method testGetBooks(
).
LibraryTest.java
public void testGetBooks( ) {
Book book = library.getBook( "Dune" );
assertTrue( book.getTitle( ).equals( "Dune" ) );
book = library.getBook( "Solaris" );
assertTrue( book.getTitle( ).equals( "Solaris" ) );
}To be really sure that the test Book is correct,
the tests should also check the
Book’s author, but this means
adding extra asserts to each test. It’s clearly
useful to have an assert method that compares an expected
Book to the actual Book,
checking all of the attributes. This new assert method is easy to
implement by building on the generic assertTrue()
method, as shown in Example 4-4.
BookTest.java
public class BookTest extends TestCase {
public static void assertEquals( Book expected, Book actual ) {
assertTrue(expected.getTitle( ).equals( actual.getTitle( ) )
&& expected.getAuthor( ).equals( actual.getAuthor( ) ));
}
}The assert method assertEquals()
takes expected and actual Book objects to compare.
It succeeds if the title and author attributes of the two
Books are equal. Example 4-5
shows how it is used.
LibraryTest.java public class LibraryTest extends TestCase { private Library library; private Book book1, book2; public void setUp( ) { library = new Library( ); book1 = new Book("Dune", "Frank Herbert"); book2 = new Book("Solaris", "Stanislaw Lem"); library.addBook( book1 ); library.addBook( book2 ); } public void testGetBooks( ) { Book book = library.getBook( "Dune" ); BookTest.assertEquals( book1, book ); book = library.getBook( "Solaris" ); BookTest.assertEquals( book2, book ); } }
The custom assert method makes the test clear and concise and
improves it by comparing all the Book attributes,
not just the title. While writing tests, watch for complex assert
conditions that are used repeatedly. They are good candidates for
replacement with custom assert methods.
Single Condition Tests
A useful rule of thumb is that a test method should only contain a single test assertion. The idea is that a test method should only test one behavior; if there is more than one assert condition, multiple things are being tested. When there is more than one condition to test, then a test fixture should be set up, and each condition placed in a separate test method.
The xUnits tend to enforce this rule when handling test assertion failures. A test method returns as soon as a failure occurs, skipping any additional code. Running the rest of the test is unnecessary, since the result (failure) is known.
Practically speaking, test methods containing several assertions are
not always a terrible thing. Tests may have conditions that can only
be combined into one expression with unnecessary complication of the
code. The testGetBooks( ) method in the previous
section verifies that the Library contains two
Books, which is most clearly expressed as two
separate asserts, although they could be combined into one compound
condition. A single behavior can have several side effects that you
should check with separate assertions. So, it’s not
a problem when a test method contains several
asserts, as long as the test method
is only testing a single behavior.
However, a test method with many asserts is a clear indicator that a single test is doing too much. Example 4-6 shows a test method with this problem.
LibraryTest.java
public void testLookupBooksByAuthor( ) {
// Add two books by same author
Book book3 = new Book( "Cosmos", "Carl Sagan" );
Book book4 = new Book( "Contact", "Carl Sagan" );
library.addBook( book3 );
library.addBook( book4 );
// Look up books by title and author
Book book = library.getBook( "Cosmos", "Carl Sagan" );
BookTest.assertEquals( book3, book );
book = library.getBook( "Contact", "Carl Sagan" );
BookTest.assertEquals( book4, book );
// Look up both books by author
Vector books = library.getBooks( "Carl Sagan" );
assertEquals( "two books not found", 2, books.size( ) );
book = (Book)books.elementAt(0);
BookTest.assertEquals( book3, book );
book = (Book)books.elementAt(1);
BookTest.assertEquals( book4, book );
}How is this test flawed? Let us count the ways. It tests two separate
behaviors: getting a Book by author and title and
getting multiple Books by the same author. Looking
up two books by two different methods means there are several results
to test; thus, there are many asserts—five in all. Although it
is sensible to check the results of all the operations, there are
redundant tests, such as the two tests of the getBook() method. To get the test to pass, numerous changes must be
made immediately to both Book and
Library. The complexity of the changes increases
the chance that a coding mistake will be made. When one assert in the
sequence fails, the rest will be skipped, leaving it uncertain
whether those asserts would succeed. So, if the
Book lookup by title and author fails, it has to
be fixed before the test that gets multiple Books
is run. In other words, the tests are coupled so that failure of one
may affect the success of the others.
When the number of asserts in a test method is excessive, change it into a test fixture with multiple test methods, each testing one behavior. In Example 4-7, refactoring the test method makes it apparent that the two lookup methods are distinct behaviors and should be tested separately.
LibraryTest.java
public void setUp( ) {
book3 = new Book( "Cosmos", "Carl Sagan" );
book4 = new Book( "Contact", "Carl Sagan" );
library.addBook( book3 );
library.addBook( book4 );
}
public void testGetBookByTitleAndAuthor( ) {
Book book = library.getBook( "Cosmos", "Carl Sagan" );
BookTest.assertEquals( book3, book );
}
public void testGetBooksByAuthor( ) {
Vector books = library.getBooks( "Carl Sagan" );
assertEquals( "two books not found", 2, books.size( ) );
Book book = (Book)books.elementAt(0);
BookTest.assertEquals( book3, book );
book = (Book)books.elementAt(1);
BookTest.assertEquals( book4, book );
}
Example 4-7 shows LibraryTest
with the two separate test methods, one for each behavior. The code
to add the two test Books is placed in the
setUp( )
method. The tests are isolated and the
code is simplified.
Testing Expected Errors
It is important to test the error-handling behavior of production code in addition to its normal behavior. Such tests generate an error and assert that the error is handled as expected. In other words, an expected error produces a unit test success.
The canonical example of a unit test that checks expected error handling is one that tests whether an expected exception is thrown, as shown in Example 4-8.
LibraryTest.java
public void testRemoveNonexistentBook( ) {
try {
library.removeBook( "Nonexistent" );
fail( "Expected exception not thrown" );
} catch (Exception e) {}
}The expected error behavior is that an exception is thrown when the
removeBook( ) method is called for a nonexistent
Book. If the exception is thrown, the unit test
succeeds. If it is not thrown, fail()
is
called. The fail( ) method is another useful
variation on the basic assert method. It is equivalent to
assertTrue(false), but it reads better.
Since the removeBook( ) method now throws an
exception, the
testRemoveBook( ) unit test should be updated, as
shown in Example 4-9.
LibraryTest.java
public void testRemoveBook( ) {
try {
library.removeBook( "Dune" );
} catch (Exception e) {
fail( e.getMessage( ) );
}
Book book = library.getBook( "Dune" );
assertNull( "book is not removed", book );
}This example uses fail( ) to cause the test to
fail when an unexpected exception is thrown. The
exception’s message attribute is used as the assert
message.
The same general pattern is followed to test expected error behavior
that is not represented by an exception: the test fails if the error
is not seen and succeeds if it is. Example 4-10 shows
a unit test that attempts to get a nonexistent
Book from the Library and
asserts that the expected null Book is returned.
(Not) Testing Get/Set Methods
Every
behavior should be covered
by a unit test, but every method doesn’t need its
own unit test. Many developers don’t test get and
set
methods, because a method that does nothing but get or set an
attribute value is so simple that it is considered immune to failure.
Tests of such methods are correspondingly trivial, as shown in the
test of getTitle( ) in Example 4-11.
BookTest.java public void testGetTitle( ) { Book book = new Book( "Solaris", "Stanislaw Lem" ); assertEquals( "Solaris", book.getTitle( ) ); }
If a get or set method produces any side effects or otherwise has nontrivial functionality, it should be tested. For example, with lazy initialization, a get method may compute an attribute value before returning it — behavior that deserves a unit test.
Testing Protected Behavior
A topic of much discussion within the unit testing community is how to test protected or private methods. Since access to such methods is restricted, writing unit tests for them is not straightforward.
Some developers deal with this quandary by simply ignoring protected or private methods and testing only the public interfaces. It’s argued that most of an object’s behavior is reflected in its public methods. The behavior of the protected methods can be inferred by the exposed behavior.
There are some drawbacks to this approach. If there are private methods that contain complex functionality, they will not be tested directly. There is a tendency to make everything public so that it is testable. Some behaviors that should be private might be exposed.
It is possible to access and test protected and private methods,
depending on the specifics of how a language defines and enforces
object access permissions. In
C++, making the test class a
friend of the production class allows it to access
protected interfaces:
class Library {
#ifdef TEST
friend class LibraryTest;
#endif
}This introduces a reference to the test code into the production
code, which is not good. Preprocessor directives such as
#ifdef TEST can omit such references when the
production code is built.
In Java, a simple technique that allows test classes to access protected and private methods is to declare the methods as package scope and place the test classes in the same package as the production classes. The next section, “Test Code Organization,” shows how to arrange Java code this way.
For Java developers who are not satisfied with the direct approach,
the Java Reflection API is a tricky way
to overcome access protection. The
JUnit extension
“JUnit-addons” includes a class
named PrivateAccessor that uses this approach to
access protected or private attributes and methods.
The truly hardcore can follow the examples given here to write their
own code that subverts access protection. In Example 4-12, the values of all of
Book’s fields are read,
regardless of protection. This approach is an ugly hack.
Don’t read this code just after a meal.
BookTest.java
import java.lang.reflect.*;
public void testGetFields( ) {
Book book = new Book( "test", "test" );
Field fields[] = book.getClass( ).getDeclaredFields( );
for ( int i = 0; i < fields.length; i++ ) {
fields[i].setAccessible( true );
try {
String value = (String)fields[i].get( book );
assertEquals( "test", value );
} catch (Exception e) {
fail( e.getMessage( ) );
}
}
}A Book with title and author
“test” is created. The Reflection
API method getDeclaredFields()
returns an array of all of the
Book’s fields, and the call to
setAccessible( )
allows access to a field. The Reflection API
method get( )
is used to obtain each
field’s value. The test asserts that the value of
the field is test.
Similarly, in Example 4-13, all of
Book’s get methods are called,
ignoring access protection (although the get methods actually are
public).
BookTest.java
public void testInvokeMethods( ) {
Book book = new Book( "test", "test" );
Method[] methods = book.getClass( ).getDeclaredMethods( );
for ( int i = 0; i < methods.length; i++ ) {
if ( methods[i].getName( ).startsWith("get") ) {
methods[i].setAccessible( true );
try {
String value = (String)methods[i].invoke( book, null );
assertEquals( "test", value );
} catch (Exception e) {
fail( e.getMessage( ) );
}
}
}
}Paralleling the previous example, the Reflection API method
getDeclaredMethods()
returns all of the
Book’s methods, and the call to
setAccessible( )
subverts the method’s access
protection. The test checks the method name and calls only those that
have names starting with get to avoid calling
Book’s constructor. The
Reflection API method invoke()
is used to call the methods. Both get
methods should return the value test, so this
condition is asserted.
Hacks aside, the recommended approach is to design objects so that their important behaviors are public and test those behaviors. Structure the code so that the tests have access to the protected behaviors as well, so that they can be accessed if necessary.
Test Code Organization
As a project grows in size, organizing the files containing production and test code becomes an issue. Although keeping the test and production code in the same directory is the simplest solution, it is better to have a clean separation between the two categories of code. This strategy helps avoid build complications that occur when a directory contains some code that should be linked into the production application, and some that should not. Including the test code in the delivered application is undesirable because it unnecessarily increases the size of the delivery, and also because the tests may expose behavior or design details that the developer meant to keep “under the hood.”
Organizing the code is a language-specific concern. In Java, the directory path to a source file parallels its package membership. The need to test protected interfaces means that unit tests should belong to the same package as the production classes they test, so they must have the same directory path. This can be done by creating separate but parallel hierarchies for the production and test code.
Figure 4-1
shows how the source code for the final
version of the virtual library application is organized. There are
three Java packages, com.utf.library,
com.utf.library.gui, and
com.utf.library.xml.
The production and test code are located in separate directories,
src and test, which are
located under the project’s top level
DEVROOT directory. For example, the production
class Library resides in the directory
src/com/utf/library, and the test class
LibraryTest is in
test/com/utf/library. The test
classes’ package names parallel the production
classes’ package names, so the test classes can
access and test protected behavior of the production code. Since the
code is in separate directory trees, it is simple to build and run
only the production or test code as desired.
For many other languages, an effective way to organize the code is to place all test code in a subdirectory named test within each production code directory. This arrangement keeps the test code separate, but makes linking it to the production code simple.
Mock Objects
Applications often use interfaces to external objects such as databases, web servers, network services, or hardware devices. Sometimes you must write and test code to interface with objects before they are actually available. Even when the external object is available in the development environment, using it in testing may involve lots of time-consuming, fragile set-up effort, such as loading test data, running services, or placing hardware in a known state. Mock objects are a way of dealing elegantly with this type of situation.
A mock object is a simulation of a real object. Mocks implement the interface of the real object and behave identically with it, to the extent necessary for testing. Mocks also validate that the code that uses them does so correctly. To pass the mock’s validation, other objects must call the correct methods, with the expected parameters, in the expected order. A test object that simply stands in for a real object without providing such verification is not a mock; it is a stub.
Databases are commonly mocked objects. Code that interfaces to a database clearly is important to test. To be tested realistically, the code must be able to perform database operations such as opening and closing connections, reading and writing data, and performing transactions. However, running a live database in the development environment can be a pain. Tests often require that the database is in a specific state or that it contains a specific set of test data. If multiple developers run tests simultaneously, their database operations may interfere.
Mocking the database makes having an actual database unnecessary for testing. The mock has the same interface as the actual database object and the same behavior from the perspective of the client software, but it doesn’t need to actually contain anything but a minimal implementation and possibly some test data. Once the database mock is created, it becomes much simpler to write tests that assume that the database is in various states. Testing becomes faster and easier without the overhead of interfacing with an actual database engine.
To illustrate this, let’s create a mock object
representing a database connection object. An interface called
DBConnection
represents a database connection, as
shown in Example 4-14.
DBConnection.java
public interface DBConnection {
void connect( );
void close( );
Book selectBook( String title, String author );
}The class LibraryDB retrieves
Books from a database using
DBConnection. It is shown in Example 4-15.
LibraryDB.java public class LibraryDB { private DBConnection connection; public LibraryDB( DBConnection c ) { connection = c; } Book getBook( String title, String author ) { connection.connect( ); Book book = connection.selectBook( title, author ); connection.close( ); return book; } }
We would like to build a unit test for LibraryDB,
but we don’t have an actual database yet. So,
we’ll mock DBConnection as shown
in Example 4-16.
MockDBConnection.java public classMockDBConnectionimplementsDBConnection{ private boolean connected = false; private boolean closed = false; public void connect( ) { connected = true; } public void close( ) { closed = true; } public Book selectBook( String title, String author ) { return null; } public boolean validate( ) { return connected && closed; } }
MockDBConnection
implements the public interface
of DBConnection, so it can be used in the
interface’s place.
MockDBConnection uses the attributes
connected and closed to record
that the connect( ) and close() methods have been called. The validate(
) method verifies the connection’s state
by checking these flags. So, the expectation set by the mock is that
code using DBConnection must call both
connect( ) and close( ).
The test class
LibraryDBTest
is shown in Example 4-17.
LibraryDBTest.java
import junit.framework.*;
import java.util.*;
public class LibraryDBTest extends TestCase {
public void testGetBook( ) {
MockDBConnection mock = new MockDBConnection( );
LibraryDB db = new LibraryDB( mock );
Book book = db.getBook( "Cosmos", "Carl Sagan" );
assertTrue( mock.validate( ) );
}
}The test method testGetBook( ) creates an instance
of MockDBConnection, uses it to construct a
LibraryDB, and then calls the
LibraryDB method getBook( ).
The success of the test depends on the result of the
mock’s validate( ) function. If
the mock is in the expected state, its validation succeeds and the
test passes. The mock object verifies the expected sequence of calls
to the database connection and validates that
LibraryDB is interacting with it correctly. It
also allows LibraryDB and
DBConnection to be tested without an actual
database.
More sophisticated mock objects go beyond simply setting flags for each method called by recording the arguments provided for method calls, the order of calls, and other details of the method’s state. In this way, mock objects can perform sophisticated validation of interobject interactions.
Mock objects are a deep topic, covered by numerous web sites, books, and online groups. Also, a variety of tools are available to support mock object development for various domains and languages, including EasyMock, jMock, and MockRunner.
AbstractTest
Just like regular classes, abstract classes and interfaces should have their own unit tests. Designing such tests is not straightforward, because these object types cannot be directly instantiated. We’d also like to ensure that every descendant of an abstract class passes the parent object’s tests. The AbstractTest pattern is the answer.
An AbstractTest is itself abstract, like the tested object. It contains an abstract factory method, which produces an instance of the object to test. It also contains the test methods for the abstract class. They resemble ordinary unit test methods, but test instances of the abstract class created by the factory method.
To test a concrete class that is descended from the abstract class, the unit test is subclassed from the AbstractTest. Its factory method returns an instance of the concrete class. When the concrete unit test is run, the AbstractTest is run as well. So, the AbstractTest tests every concrete implementation of the abstract class.
Let’s create an AbstractTest for the interface
DBConnection
. We’ll add the method
isOpen( )
to it, as shown in Example 4-18.
DBConnection.java
public interface DBConnection {
void connect( );
void close( );
boolean isOpen( );
Book selectBook( String title, String author );
}The AbstractTest should test the behavior of the interface to make
sure that any concrete implementation of it is correct. Tests of the
isOpen( ) method should verify that it returns
TRUE after connect(
)
is called, and
FALSE after close()
is called. The AbstractTest
class
AbstractDBConnectionTestCase
, shown in Example 4-19, provides these tests.
AbstractDBConnectionTestCase.java import junit.framework.*; public abstract class AbstractDBConnectionTestCase extends TestCase { public abstract DBConnection getConnection( ); public void testIsOpen( ) { DBConnection connection = getConnection( ); connection.connect( ); assertTrue( connection.isOpen( ) ); } public void testClose( ) { DBConnection connection = getConnection( ); connection.connect( ); connection.close( ); assertTrue( !connection.isOpen( ) ); } }
The AbstractTest specifies a factory method, getConnection()
.
Concrete tests that descend from it will implement the factory
method, allowing the test methods testIsOpen( )
and testClose( ) to test an instance of the
concrete class. Notice how these methods use getConnection() to get the DBConnection to test.
AbstractTests have names ending in “TestCase,” which is different from other test classes. A separate naming convention for AbstractTest classes makes them easily recognizable. Some unit test tools assume that any class named ending with “Test” are test classes that should be instantiated and run, and the different naming convention avoids confusion.
To see the AbstractTest run, we need to define a concrete class
descended from DBConnection and a corresponding
concrete unit test descended from
AbstractDBConnectionTestCase. The concrete class
JDBCConnection
is shown in Example 4-20.
JDBCConnection.java public class JDBCConnection implements DBConnection { private String connectString; private boolean open; public JDBCConnection( String connect ) { connectString = connect; open = false; } public void connect( ) { open = true; } public void close( ) { open = false; } public boolean isOpen( ) { return open; } public String getConnectString( ) { return connectString; } public Book selectBook( String title, String author ) { return null; } }
JDBCConnection is an initial version of an
interface to a JDBC database engine. It differs from the base
DBConnection by its member
connectString, which contains the URL of a JDBC
database connection.
The unit test
JDBCConnectionTest
tests JDBCConnection.
It is derived from the AbstractTest. It is shown in Example 4-21.
JDBCConnectionTest.java public classJDBCConnectionTestextendsAbstractDBConnectionTestCase{ public DBConnection getConnection( ) { return new JDBCConnection( "jdbc:odbc:testdb" ); } public voidtestConnectString( ){ JDBCConnection connection = (JDBCConnection)getConnection( ); String connStr = connection.getConnectString( ); assertTrue( connStr.equals("jdbc:odbc:testdb") ); } }
JDBCConnectionTest implements the factory method
getConnection( )
and one test method,
testConnectString( ). When the test is
instantiated and run, the two test methods in the parent AbstractTest
also will be run to test instances of
JDBCConnection. This way, the AbstractTest
verifies that the concrete subclass passes the tests of the parent
interface.
Performance Tests
Like the mock object, unit testing for performance is its own significant topic. Software performance often is neglected at the unit testing level, and is only taken into consideration during functional testing. However, performance-oriented unit tests are powerful tools, especially for applications that require specific performance goals be met. It’s been reported that Apple’s Safari browser was developed in an environment that automatically ran performance tests on any code that was checked in. The code was rejected if it did not meet or exceed the speed standards of previous versions. Thus, the unit tests ensured that the code’s performance is continuously improving.
When a piece of code has a performance problem, it is very useful to first write a test that reveals the problem. This performance test not only lets you know when the code has achieved the desired performance, but also acts as a “canary in the coal mine” that indicates if the performance degrades again.
Tools intended specifically for performance-oriented unit testing are
available, such as JUnitPerf. However, it is not difficult to develop
performance tests within any unit test framework. This section gives
an example of a unit test that tests the speed of retrieving a
Book from a Library.
The initial question when writing a performance test is this: what is the performance criterion that the test must meet to pass? Usually, this is expressed in terms of the amount of time that a certain action may take. If the action takes too long, the criterion has not been met, and the test fails.
The Library class developed so far has a very
poorly performing algorithm to get a Book. It
serially reads through the collection of Books,
doing string comparisons on each one until the desired
Book is found. This awful lookup stratagem is
ideal for demonstrating a performance test that fails initially, but
succeeds after a little refactoring. Example 4-22
shows the unit test class
LibraryPerfTest
.
LibraryPerfTest.java import junit.framework.*; import java.util.*; public class LibraryPerfTest extends TestCase { private Library library; public void setUp( ) { library = new Library( ); for ( int i=0; i < 100000; i++ ) { String title = "book" + i; String author = "author" + i; library.addBook(new Book( title, author )); } } public void testGetBookPerf( ) { double maxTime = 100; // milliseconds long startTime = System.currentTimeMillis( ); Book book = library.getBook( "book99999" ); long endTime = System.currentTimeMillis( ); long time = endTime-startTime; assertTrue( time < maxTime ); assertEquals( "book99999", book.getTitle( ) ); } }
LibraryPerfTest is implemented as a
test fixture since it is likely that
more performance tests will be implemented. The setUp()
method adds 100,000
Books to the Library. The test
method testGetBookPerf( ) tests the amount of time
it takes to look up a Book. It uses the method
currentTimeMillis(
)
to get the system time before and after the
getBook( ) operation, calculates the elapsed time,
and compares it to a performance criterion of 100 milliseconds (0.1
second). As a sanity check, it also asserts that the expected
Book was found.
With the
Vector-based
implementation of Library, the unit test fails:
> java -classpath ".;junit.jar" junit.textui.TestRunner LibraryPerfTest
.F
Time: 0.562
There was 1 failure:
1) testGetBookPerf(LibraryPerfTest)junit.framework.AssertionFailedError
at LibraryPerfTest.testGetBookPerf(LibraryPerfTest.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
FAILURES!!!
Tests run: 1, Failures: 1, Errors: 0
Library can be refactored to use a
Hashtable
to store
Books. (The refactored Library
code is given in the next section, “New Library and
Book Code.”) With this change, lookups by title are
efficient, and the test passes:
> java -classpath ".;junit.jar" junit.textui.TestRunner LibraryPerfTest . Time: 0.734 OK (1 test)
The total test time has increased. This is because addBook() takes longer to add a Book with the
Hashtable implementation.
The hardcoded time value of 100 milliseconds used in this example can produce different results when the test is run on faster or slower platforms. Even when run on the same platform, varying machine loads and process priorities mean that a performance test can succeed or fail on subsequent runs without any code changes. Accounting for such variations can present a challenge when designing performance tests. There are a number of techniques to deal with these problems. Consistently running performance tests on the same platform is helpful. Test timing can be based on the time required to run a reference operation rather than on a hardcoded time value, allowing for system performance variations. Timing multiple repetitions of an operation reduces the effect of transient glitches. Finally, performance tests can use order-of-magnitude timing ranges rather than exact minimum timings, so that code meeting general performance goals will pass.
New Library and Book Code
Example 4-23 gives the code for the
version of Book
referenced in this chapter.
Book.java
public class Book {
private String title = "";
private String author = "";
Book(String title, String author) {
this.title = title;
this.author = author;
}
public String getTitle( ) { return title; }
public String getAuthor( ) { return author; }
}The code for the final version of
Library
is given in Example 4-24. It uses a Hashtable to
store the collection of Books.
Library.java
import java.util.*;
public class Library {
private Hashtable books;
Library( ) {
books = new Hashtable( );
}
public void addBook( Book book ) {
books.put( book.getTitle( ), book );
}
public Book getBook( String title ) {
return (Book)books.get( title );
}
public Book getBook( String title, String author ) {
return (Book)books.get( title );
}
public Vector getBooks( String author ) {
Vector auth_books = new Vector( );
for ( Enumeration e = books.elements( ); e.hasMoreElements( ); ) {
Book book = (Book)e.nextElement( );
if ( book.getAuthor( ).equals(author) )
auth_books.add( book );
}
return auth_books;
}
public void removeBook( String title ) throws Exception {
if ( books.remove( title ) == null )
throw new Exception("Book not found");
}
public int getNumBooks( ) {
return books.size( );
}
public void empty( ) {
books.clear( );
}
}Get Unit Test Frameworks now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
