By Bill Hamilton
Book Price: $9.95 USD
£6.95 GBP
PDF Price: $7.99

Cover

Table of Contents

Chapter 1: NUnit Pocket Reference

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

I was inspired to write NUnit Pocket Reference because I was frustrated with the lack of documentation for NUnit—an excellent, indispensable tool for unit testing software. My aim was to write a book that my colleagues and I would find useful—a short, concise reference for installing, using, and extending NUnit.

Unit testing software started with xUnit—a testing framework developed by Kent Beck and Erich Gamma to test SmallTalk code. Test frameworks now exist for most programming languages—for example, JUnit for Java, CppUnit for C++, and NUnit for .NET.

NUnit is a port from JUnit. NUnit is similar to JUnit but is for the .NET Framework development. Unlike JUnit, NUnit is language independent, and can be used to test C#, VB.NET, and J# programs as well as most other programs developed using the .NET Framework. You can write a test in a different language from the code being tested and use NUnit to test projects that use multiple languages.

NUnit Version 2.2 supports Microsoft .NET Framework Versions 1.0 and 1.1, and Mono—an open source version of the Microsoft .NET development platform that runs on Unix, Windows, and Mac OS X. This book has been written with the .NET Framework version in mind; however, most of the information applies equally to the Mono version. For details about Mono, see http://www.mono-project.com/.

The examples in this book are written in C#, but NUnit can be used with any .NET language—I just happen to prefer C#. Regardless of your programming language preference, you should find the short examples easy to understand and adapt.

Italic is used for filenames, URLs, and to introduce new terms.

Constant width is used for code and to indicate keywords, parameters, attributes, elements, arguments, methods, constructors, switches, and other code items within text

Constant width bold is used to highlight code.

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you're reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O'Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product's documentation does require permission.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Introduction

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

I was inspired to write NUnit Pocket Reference because I was frustrated with the lack of documentation for NUnit—an excellent, indispensable tool for unit testing software. My aim was to write a book that my colleagues and I would find useful—a short, concise reference for installing, using, and extending NUnit.

Unit testing software started with xUnit—a testing framework developed by Kent Beck and Erich Gamma to test SmallTalk code. Test frameworks now exist for most programming languages—for example, JUnit for Java, CppUnit for C++, and NUnit for .NET.

NUnit is a port from JUnit. NUnit is similar to JUnit but is for the .NET Framework development. Unlike JUnit, NUnit is language independent, and can be used to test C#, VB.NET, and J# programs as well as most other programs developed using the .NET Framework. You can write a test in a different language from the code being tested and use NUnit to test projects that use multiple languages.

NUnit Version 2.2 supports Microsoft .NET Framework Versions 1.0 and 1.1, and Mono—an open source version of the Microsoft .NET development platform that runs on Unix, Windows, and Mac OS X. This book has been written with the .NET Framework version in mind; however, most of the information applies equally to the Mono version. For details about Mono, see http://www.mono-project.com/.

The examples in this book are written in C#, but NUnit can be used with any .NET language—I just happen to prefer C#. Regardless of your programming language preference, you should find the short examples easy to understand and adapt.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Conventions Used in This Book

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Italic is used for filenames, URLs, and to introduce new terms.

Constant width is used for code and to indicate keywords, parameters, attributes, elements, arguments, methods, constructors, switches, and other code items within text

Constant width bold is used to highlight code.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Using Code Examples

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you're reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O'Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product's documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: "NUnit Pocket Reference by Bill Hamilton. Copyright 2004 O'Reilly Media, Inc., 0-596-00739-6."

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Acknowledgments

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Thanks to everyone who has helped to develop NUnit and to the rest of the NUnit community for creating a great open source framework that makes it much easier to produce quality software.

I'd like to thank my editor, Brian Jepson, who is not only diligent, but a pleasure to work with, and my technical reviewer, Brad Merrill, for his helpful feedback.

Finally, I'd like to thank Molly, my family, and the usual suspects for their support and humor.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Unit Testing

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section provides an overview of software testing, describes unit testing and its place in the testing cycle, provides an overview of test driven development, and recommends best practices for unit testing.

Software testing measures the performance of software against both functional and nonfunctional acceptance criteria. Unit testing ensures that software does what it commits or is expected to do.

Successful unit testing starts with acceptance criteria that you identify during the requirements analysis; the acceptance criteria and the requirements are specified together. Acceptance criteria must be measurable—a statement such as "the software must perform well" is not enough. You must identify each combination of inputs and the expected output, and the final state for each based on the starting state.

Testing is approached from two different but complementary focuses:

White box

Tests the unit, component, or system by looking at what happens within the code. Tests are determined based on how functionality is actually implemented. The flow of both control and data within the code is tested. In addition to verifying inputs and outputs, you can examine all code paths, and see exactly how the code processes the inputs to generate the outputs, how errors are raised and propagated, and the internal state of the code after it has been executed.

Black box

Examines the behavior of a unit, component, or system from a functional perspective—this is why such tests are sometimes referred to as functional tests. These tests evaluate whether the system performs successfully from an external perspective. In a black box test, you supply inputs to a piece of software—method, component, etc.—and verify that the outputs are what you expected without looking at the inner functioning of the software. You cannot directly inspect how the inputs are processed, how the outputs are generated, how resources are handled, or how errors are handled or propagated. You cannot look within the code, so you cannot ensure that you are testing all branches of the code.

A successful testing strategy, including unit testing, uses both black box and white box testing

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What Is NUnit?

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

NUnit is an easy-to-learn, open source framework built using Visual Studio .NET 2003 and the .NET Framework 1.1. NUnit automates unit tests for all .NET languages.

NUnit facilitates unit testing by providing a framework to support test development and test runners to run the tests and assess their results. NUnit includes test runner executables with both GUI and command-line interfaces. The command-line interface is particularly useful for automated build and test environments.

NUnit tests are easily repeatable by anyone with access to the code and test assemblies. This allows you to test the impact of your changes on the work of the entire development team in a distributed development environment. Verifying your code against the entire project maintains the integrity of the overall build.

The NUnit framework provides a collection of base classes and attributes to support unit testing. Unit testing code is organized into tests, test fixtures, and test suites, as described in "Setting Up NUnit."

.NET reflection lets you dynamically examine and modify code at runtime. Reflection queries metadata in the portable executable (PE) file (i.e., a DLL or EXE) using services in the System.Reflection namespace. The Common Language Runtime (CLR) allows you to add descriptive declarations called attributes to annotate programming elements, such as methods, fields, properties, and types. These attributes are saved with the metadata. Attributes are used both to describe code and affect application behavior at runtime. The .NET Framework supplies built-in attributes, and you can also define your own.

NUnit 2.1 or later uses attributes to identify tests and to specify testing metadata within the test code. NUnit scans the attributes to identify tests, and then uses reflection to locate and execute all of the tests in an assembly.

NUnit loads a copy of the code being tested into its own application domain—a unit of isolation that prevents an application from interfering with other applications running within the same or different process. This allows NUnit to check for changes to assemblies you are testing and automatically reload them as necessary without restarting to ensure you test the latest assembly. NUnit will not automatically recompile code that has changed, however.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Setting Up NUnit

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section describes how to get, install, and configure NUnit, and explains what is included in the installation.

NUnit 2.2, the fourth major release of NUnit, runs on Windows 98, ME, NT, 2000, 2003, and XP (as well as Linux, Unix, and Mac OS X). If you are running Windows 98, ME, NT, or 2000, you might need to download the Windows Installer from the Microsoft Download Center http://www.microsoft.com/downloads. This will allow you to install the NUnit .msi file.

Select Windows Installer from the Products/Technology dropdown menu, and pick the installer for your operating system.

NUnit supports all .NET programming languages and works across all .NET languages. For example, you can write unit tests in VB.NET to test C# code.

NUnit supports testing assemblies written in any .NET language. Support is limited to C#, VB.NET, J#, and C++ if you intend to open Visual Studio .NET projects and solutions directly rather than open the compiled assemblies.

The official NUnit site is http://www.nunit.org. At this site, you will find download instructions, documentation, and contact information for the developers.

You can download the most recent version of NUnit from the Files section of the SourceForge.net project page at http://sourceforge.net/projects/nunit/. The site has discussion groups about NUnit, as well as add-ins for NUnit that are discussed later in "Extending NUnit." Discussion groups are located at http://sourceforge.net/forum/?group_id=10749.

The NUnit download is either a Windows Installer package or a zip file for Linux, Unix, and Mac OS X versions of Mono. On Windows, double-click on the file to begin the installation. The installer copies the files to a directory structure and installs shortcuts in the Start menu. On Linux, Unix, and Mac OS X, extract the zip file and follow the instructions in the doc subdirectory.

On Windows, NUnit installs files in the directory %SystemDrive%\Program Files\NUnit x.y, where x.y is the version number, as in c:\Program Files\NUnit 2.2. The installer also creates three subdirectories:

bin

Contains the class libraries, test assemblies used to build NUnit, and both the GUI and console-based test runners and configuration files

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Unit Testing with NUnit

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section discusses the different elements of the NUnit framework and how to use them to create unit tests and test structures. shows the relationship between the NUnit elements.

In the simplest case, NUnit requires only test cases and one or more test fixtures. Each test must contain at least one assertion. A description of these elements follows.

Figure : NUnit test elements

Test cases

A test is the lowest building block of unit testing and tests a single piece of software functionality. Programmatically, a test corresponds to a method in the unit test code.

You identify a test by decorating a method with the [Test] attribute. For example:

[Test]
public void Test1()
{
    // test case implementation
}

A test method must be public (so that the test runner can locate it using reflection), returns void, and take no arguments.

A unit test performs one or more assertions that determine whether the functionality being tested works properly. An assertion simply tests an actual post-condition against the expected post-condition required for the test to pass. Assertions are described later in "Assertions."

The [Test] attribute has an optional argument named Description that defines the description that appears in the test properties dialog in the test runner GUI. For example:

[Test (Description = "MyTest")]

For backward compatibility with earlier versions of NUnit, a method is automatically identified by NUnit as a test if the first four characters of the method name are "test"; this identification is not case sensitive.

Test fixtures

A test fixture is used to group and run multiple tests that test a logical collection of functionality. Programmatically, a test fixture corresponds to a class that in turn contains unit tests as methods of the class.

A test fixture class must have either a public default constructor or no constructor, which implicitly creates a public default constructor. Identify a class as a test fixture by decorating it with the [TestFixture] attribute. For example:

[TestFixture]
public class Class1Test
{
    // test fixture implementation
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Test Runner GUI

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

NUnit provides two test runner interfaces—GUI and console. Both interfaces provide similar functionality. The test runner GUI is most useful for testing code while you are developing. The test runner console is useful in automated testing environments.

The test runner GUI can be launched using either the shortcut that is created when NUnit is installed or from the command line, as discussed later in this section.

This section describes the test runner GUI. Examples use the nunit.tests.dll assembly that the NUnit developers use to test NUnit; this assembly installs with NUnit into the bin subdirectory.

The following sections describe the different parts of the test runner GUI.

Test tree view

The test tree view displays on the Test tab at the left side of the test runner GUI. It displays the hierarchy of test suites, test fixtures, and tests in the assembly.

The success or failure of each test is shown in the tree view—a green dot indicates success, a red dot indicates failure, and a yellow dot indicates that a test was not run. The tree view also allows you to selectively run single tests or groups of tests.

shows the test runner GUI.

Figure : A test runner GUI without checkboxes

You can display checkboxes for each node in the tree view, allowing you to select tests to run. Select or deselect the menu item View → CheckBoxes.

shows the test runner GUI with checkboxes.

Select multiple checkboxes and click the Run button to run the tests for the checked nodes. All child tests of a checked parent are run.

Figure : Test runner GUI with checkboxes

The Clear button deselects all checkboxes. The Check Failed button selects only the tests that failed in the last run.

You can right-click on any node in the tree view and select Properties from the context menu to find out what the node represents and information about the node. shows the properties for a test case.

Figure : The Test Case Properties dialog's Test tab

The Test tab displays the following information:

Full Name

Fully qualified name of the test.

Description

Description of the test as specified by the optional

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Test Runner Console

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The test runner console is text-based and not interactive. It is most useful in automated testing environments where you use an automated build tool, such as NAnt, to run NUnit tests automatically as part of the build cycle.

The test runner console has a number of options in addition to those available in the test runner GUI. The console interface outputs test results to an XML file. By default, the file is named TestResult.xml and is placed in the working directory. This XML file is overwritten without warning if it already exists. The schema and contents of the file are described earlier in "Results in XML format."

To use the test runner console, add the directory for nunit-console.exe (the default directory is c:\Program Files\NUnit V2.2\bin) to the environment path by selecting Control Panel→System, selecting the Advanced tab, and clicking the Environment Variables button. Append the directory containing nunit-console.exe to either the user or system path. Separate the path from any other entries with a semicolon.

Test an assembly by specifying its name on the command line:

nunit-console.exe NUnitPocketReference.dll

The output includes:

• Output that you have directed to the console in the tests. This output is the same as that sent to the Console.Out window in the test runner GUI.
• Statistics about the tests run including the number of successes, failures, tests not run, and the time it took to run the tests.
• A list of tests that failed.
• A list of tests that were ignored.

shows a test runner console run.

Figure : A test runner console

There are several other ways that you can specify the tests to run—for example, you can specify the .NET project name. NUnit loads the assembly specified by the .NET project definition:

nunit-console.exe NUnitPocketReference.csproj

You can also specify a .NET solution file with the same result:

nunit-console.exe NUnitPocketReference.sln

Finally, you can specify an NUnit project file:

nunit-console.exe NUnitPocketReference.nunit

Run a test in multiple assemblies by adding the names of all assemblies to the command line:

nunit-console.exe MyAssembly1.dll MyAssembly2.dll

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Debugging with NUnit

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The Visual Studio .NET debugger can attach to a program running in an external process. Use this feature to debug NUnit tests as follows:

1. Open the Visual Studio .NET project that you are testing with the Visual Studio .NET IDE.
2. Run NUnit and load the Visual Studio .NET project.
3. From the Visual Studio .NET menu, select Tools→Debug Processes… to launch the Processes dialog as shown in .
   

   Figure : A Processes dialog
4. Select nunit-gui.exe from the list of processes and press the Attach… button to launch the Attach to Process dialog as shown in .
5. Check Common Language Runtime in the Attach to Process dialog and click the OK button. The nunit-gui.exe process appears in the Debugged Process window of the Processes dialog as shown in . Press the Close button to exit the Processes dialog (see ).
6. Run the tests in the NUnit GUI. You will now be able to debug the test assembly in Visual Studio .NET.
   

   Figure : An Attach to Process dialog
   

   Figure : A Debugged Processes dialog
7. Once you have finished debugging, use one of the following techniques to detach debugging:
   1. Select Debug→Stop Debugging from the Visual Studio .NET IDE menu.
   2. Close the NUnit GUI.
   3. Select Debug→Processes from the Visual Studio .NET IDE menu. Select the nunit-gui.exe process from the Debug Processes window and click the Detach button.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Testing with Mock Objects

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Mock objects simulate real-world objects that are external to your application. During development and testing, mock objects let you isolate your application from objects in the outside world with which your application would normally collaborate. Eliminating reliance on this external code leaves only the code that you need to test.

Mock objects:

• Offer better control when unit testing because you have complete control of the mock object.
• Improve test performance dramatically by replacing slow external dependencies.
• Simplify testing by letting you easily simulate states that are difficult or time-consuming to reproduce in the outside world.
• Raise any type of exception on demand, allowing you to test these conditions in a single test cycle.

The downside of mock objects is the effort required to identify and program their behavior to accurately simulate the behavior of the real-world object. Fortunately, as the use and popularity of mock objects increases, libraries of reusable mock objects are becoming available that might meet your requirements.

To use mock objects for testing, you must design an interface for the real-world object and implement that interface for both test and production code. Using an interface to represent the object allows you to use the mock object in debug code and the real object in production code.

NUnit 2.2 adds a lightweight mock object facility that allows you to:

• Dynamically create a mock implementation of any interface or marshal by reference class.
• Set the order in which methods are called and specify which methods are not to be called.
• Specify return values for specific or arbitrary sets of arguments.
• Verify that expected actions occurred.

The NUnit mock objects were developed primarily to support the testing of NUnit and are not meant to replace one of the full-featured mock object frameworks available for .NET such as NMock. For more information about mock objects and a list of available mock object frameworks, visit http://mockobjects.com.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Compatibility

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This section describes how to run tests created in older versions of NUnit using NUnit 2.2 and highlights the most significant changes made in the last two releases of NUnit.

To run NUnit 2.1 tests with NUnit 2.2, recompile the tests using the NUnit 2.2 nunit.framework assembly. You might also need to add a reference to the nunit.core assembly if you are using core types.

There are two options for running NUnit 2.0 tests with NUnit 2.2. The first option is to upgrade the 2.0 tests and recompile the test assemblies using the new nunit.framework assembly.

The NUnit 2.2 documentation states that the Assertion class is deprecated and marked obsolete. Although not necessary, you might want to convert all assertions to use the new Assert class to ensure compatibility with future versions of NUnit.

The second option allows you to run NUnit 2.0 tests with NUnit 2.2 without changing the NUnit 2.0 tests. Create the appropriate binding redirect in the test runner configuration file (nunit-gui.exe.config or nunit-console.exe.config). The nunit20under21.config file in the src subdirectory of the NUnit 2.2 installation directory has the required redirect in the bindingRedirect element and can be used as is or as a template.

You cannot run NUnit 1.x tests using NUnit 2.2—the only option is to upgrade the Version 1.x tests. The following points outline the changes you must make to the NUnit 1.x test code:

• In the test source code, remove the constructor with the string argument that inherits from the TestCase class. The class must have a default constructor or no constructor (which implicitly generates the default constructor).
• In the test source code, update the reference to the NUnit 1.x framework with the NUnit 2.2 framework nunit.framework.dll assembly.
• The Suite property is no longer used. Replace any occurrences with the [Suite] attribute or use namespaces to automatically create test suites. If you use the [Suite] attribute, you must add the NUnit core types in the nunit.core.dll assembly.
• An AssertionException exception is now thrown when an assertion fails instead of an AssertionFailedError

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Extending NUnit

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

A number of external tools are available to enhance NUnit or to overcome some of its limitations. Some of the most useful are discussed in the following sections.

NUnit 2.2 is not fully integrated with Visual Studio .NET—it does not use the Visual Studio extensibility model. You must set up a custom tool entry to launch NUnit from Visual Studio. To do this, select Tools→External Tools from the Visual Studio menu. Click the Add button and complete the dialog as follows:

In this entry…	Type
Title:	NUnit
Command:	C:\Program Files\NUnit 2.2\bin\nunit-gui.exe
Arguments:	$(TargetPath)
Initial Directory:	$(TargetDir

shows the completed dialog.

NUnit will appear under the Tools menu and you can select it to launch NUnit.

The NUnit AddIn is an open source project that simplifies using the NUnit framework from the Visual Studio .NET IDE by integrating NUnit functionality into the IDE.

Figure : An External Tools dialog for NUnit

Download the latest version of the NUnit AddIn from http://sourceforge.net/projects/nunitaddin. Once you have downloaded the NUnit AddIn Windows Installer File (.msi file), double click on it to install—the default installation options work well. The installation does not create any desktop icons or a program folder in the startup menu. The installer does not install any documentation or a README file and no documentation is available separately for download from SourceForge.

To run NUnit tests using the Nunit AddIn, open your test project, right-click on the solution or a project in the Solution Explorer window, and select Run Tests from the pop-up menu as shown in . The add-in will rebuild the project if necessary.

Figure : A Solution Explorer window with NUnit Addin

Test results are displayed in the Output window of the Visual Studio .NET IDE as shown in .

Figure : NUnit Addin test results

If a test fails, double-click on the line that failed in the Output window and the cursor will be positioned on the assertion that failed in the class.

The NUnit Addin does not provide all of the functionality of the NUnit GUI—for example, it is not possible to run a single test. However, it does provide a quick way to run tests without leaving the IDE; this is especially convenient for running tests repeatedly during test driven development.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Return to NUnit Pocket Reference