Ruby Best Practices

Testing Fundamentals

A few good habits go a long way when it comes to TDD. We’ll now take a look at some key techniques that help make writing solid and maintainable tests much easier.

Well-Focused Examples

A common beginner habit in testing is to create a single example that covers all of the edge cases for a given method. An example of this might be something along these lines:

class VolumeTest < Test::Unit::TestCase
  must "compute volume based on length, width, and height" do
    # defaults to l=w=h=1
    assert_equal 1, volume

    #when given 1 arg, set l=x, set w,h = 1
    x = 6
    assert_equal x, volume(x)

    # when given 2 args, set l=x, w=y and h=1
    y = 2
    assert_equal x*y, volume(x,y)

    # when given 3 args, set l=x, w=y and h=z
    z = 7
    assert_equal x*y*z, volume(x,y,z)

    # when given a hash, use :length, :width, :height
    assert_equal x*y*z, volume(length: x, width: y, height: z)
  end
end

Though it is relatively easy to type things out this way, there are some limitations that are worth noting. One of the most obvious issues with this approach is that it isn’t very organized. Compare the previous example to the next, and you’ll see how much easier it is to read things when they are cleanly separated out:

class VolumeTest < Test::Unit::TestCase

  must "return 1 by default if no arguments are given" do
    # defaults to l=w=h=1
    assert_equal 1, volume
  end

  must "set l=x, set w,h = 1 when given 1 numeric argument" do
    x = 6
    assert_equal x, volume(x)
  end

  must "set l=x, w=y, and h=1 when given 2 arguments" do
    x, y = 6, 2
    assert_equal x*y, volume(x,y)
  end

  must "set l=x, w=y, and h=z when given 3 arguments" do
    x,y,z = 6, 2, 7
    assert_equal x*y*z, volume(x,y,z)
  end

  must "use :length, :width, and :height when given a hash argument" do
    x,y,z = 6, 2, 7
    assert_equal x*y*z, volume(length: x, width: y, height: z)
  end

end

However, the improved clarity is actually one of the lesser reasons why this code is better. In the former example, your failure report will include only the first assertion that was violated; the code that follows it will not even be executed. When you get the report back, you’ll get a message that shows you the numeric expected/actual values, but it will be titled something like, “a volume function should compute volume based on length width and height,” which is not very instructive for determining which case caused the problem.

In the latter approach, every single example will run, testing all of the cases simultaneously. This means that if a change you make to your code affects three out of the four cases, your tests will report back three out of four cases rather than just the first failed assertion in the example. They’ll have more useful names, too, each uniquely pointing back to the individual must() call that failed.

Although the code shown here is unlikely to have side effects, there is an additional benefit to splitting up examples: each one runs in its own clean-slate environment. This means you can use setup and teardown methods to manage pre- and postprocessing, but the code will run largely independent of your other examples. The benefit here is that you’ll avoid the problem of accidentally depending on some side effect or state that is left hanging around as a result of another method call. Because of this, your tests will be more isolated and less likely to run into false positives or strange errors.

Testing Exceptions

Code is not merely specified by the way it acts under favorable conditions. Although it’d be great if we could assume conservative input and liberal output constraints, this just doesn’t seem to be practical in most cases. This means that our code will often need to raise appropriate exceptions when it isn’t able to handle the request it has been given, or if it detects misuse that deserves further attention. Luckily, Test::Unit makes it easy for us to specify both when code should raise a certain error, and when we expect it to run without error. We’ll take a look at a trivial little lockbox object that provides rudimentary access control to get a feel for how this looks. See if you can understand the tests just by reading through them:

class LockBoxTest < Test::Unit::TestCase

  def setup
    @lock_box = LockBox.new( password: "secret",
                              content: "My Secret Message" )
  end

  must "raise an error when an invalid password is used" do
    assert_raises(LockBox::InvalidPassword) do
      @lock_box.unlock("kitten")
    end
  end

  must "Not raise error when a valid password is used" do
    assert_nothing_raised do
      @lock_box.unlock("secret")
    end
  end

  must "prevent access to content by default" do
    assert_raises(LockBox::UnauthorizedAccess) do
      @lock_box.content
    end
  end

  must "allow access to content when box is properly unlocked" do
    assert_nothing_raised do
      @lock_box.unlock("secret")
      @lock_box.content
    end
  end

end

As you can see, these tests read pretty clearly. Testing your exceptions is as easy as using the assert_raises() and assert_nothing_raised() methods with the relevant error class names. We can take a quick look at the implementation of LockBox to see what the code that satisfies these tests looks like:

class LockBox

  UnauthorizedAccess = Class.new(StandardError)
  InvalidPassword    = Class.new(StandardError)

  def initialize(options)
    @locked   = true
    @password = options[:password]
    @content  = options[:content]
  end

  def unlock(pass)
    @password == pass ? @locked = false : raise(InvalidPassword)
  end

  def content
    @locked ? raise(UnauthorizedAccess) : @content
  end
end

Nothing too fancy is going on here—just a few conditional arguments and a pair of custom exceptions.^[4] But if we failed to test the cases that generated the exceptions, we wouldn’t have full test coverage. Generally speaking, any time your methods might intentionally raise an error, you’ll want to set up test cases that cover both the condition where this error is raised as well as the case where it is not. This will help make sure that your error can actually be raised, while ensuring that it isn’t raised unconditionally. Testing this way will help you catch trivial mistakes up front, which is always a good thing.

Run the Whole Suite at Once

Though the examples we have worked with so far might fit well in a single file, you’ll eventually want to split up your tests across several files. However, that doesn’t mean that you should run them only in isolation!

A key feature of automated testing is that it gives you a comprehensive sense of how your software is running as a system, not just on a component-by-component basis. To keep aware of any problems that might occur during refactoring or wiring in new features, it is beneficial to run your entire suite of examples on every change. Luckily, using Ruby’s standard project automation tool, this is trivial. Here is a sample Rakefile that uses some of the most common conventions:

require "rake/testtask"

task :default => [:test]

Rake::TestTask.new do |test|
  test.libs << "test"
  test.test_files = Dir[ "test/test_*.rb" ]
  test.verbose = true
end

This code makes it so rake test will run every Ruby file in the test/ folder of your project that starts with test_ and ends with the .rb extension. A typical directory layout that works with this sort of command looks like this:

test/
  test_foo.rb
  test_bar.rb

You can tweak which files get run by changing the glob pattern passed to Dir. These work pretty much the same as they do on the command line, so you can just put one together that suits your file layout.

Now, if you’ve got some expensive resources you’re writing tests against, such as file I/O, database interaction, or some network operation, you may be a bit nervous about the idea of running all your tests on every change you make. This may be due to performance concerns or due to the fact that you simply can’t afford to do frequent live tests of your external resources. However, in most cases, this problem can be worked around, and actually leads to better tests.

The solution I’m alluding to is mock objects, and how they can be used to avoid dependencies on external resources. We’ll go over several advanced concepts in the following section, but mocks are as good a place to start as any, so we’ll work with them first. Before we do that though, let’s review some of the key guidelines that outline testing fundamentals:

• Keep your test cases atomic. If you are testing a function with multiple interfaces, write multiple examples. Also, write an example for each edge case you want to test.

• Don’t just check function input and output, also use assert_raises() and assert_nothing_raised() to test that exceptions are being thrown under the right conditions, and not unexpectedly.

• Use a rake task to automate running your test suite, and run all of your examples on every change to ensure that integration issues are caught as soon as they are introduced. Running tests individually may save time by catching problems early, but before moving from feature to feature, it is crucial to run the whole suite.

---

^[4] The syntax used for creating errors here is just a shortcut for class MyCustomError < StandardError; end.