Beautiful Testing

Comparison

There are many solutions out there for testing web applications in the browser: Watir, Selenium, Twill, and WebDriver, to name a few. Although they all have strengths and weaknesses, Windmill clearly has some very useful features, and appears to have the least barriers to entry for new test writers, regardless of the platform or browser they need to test on.

Windmill boasts a recorder, integrated Firebug Lite, integrated inspector, and support for cross-domain and SSL testing. However, unlike Watir, Windmill operates within the content security model, which means that it can’t access some parts of the browser, such as the confirmation dialog or file uploads. But Windmill has come up with a solution or workaround for many of these challenges.

Selenium boasts the GRID project, which allows for distributed tests running across many machines. Windmill has yet to come up with a comparable solution, but with test runtimes being such an important factor with large sets of tests, it’s being factored into the roadmap.

Other projects such Twill can provide much faster test runtimes, and in many ways more humanly readable test syntax. However, these frameworks lack the ability to test applications that heavily use JavaScript, limiting the scope of their utility. If you find yourself testing an application that doesn’t use any JavaScript and you don’t worry about cross-browser bugs, you will probably find the test running experience much more enjoyable in a tool such as Twill.