Testing Flex RIAs

The sooner you start testing your application, the shorter the development cycle will be. It seems obvious, but many IT teams haven’t adopted agile testing methodologies, which costs them dearly. ActionScript supports dynamic types, which means that its compiler won’t be as helpful in identifying errors as it is in Java. To put it simply, Flex applications have to be tested more thoroughly.

To switch to an agile test-driven development, start with accepting the notion of embedding testing into your development process rather than scheduling testing after the development cycle is complete. The basic types of testing are:

  • Unit

  • Integration

  • Functional

  • Load

The sections that follow examine the differences between these testing strategies, as well as point out tools that will help you to automate the process.

Unit and Integration Testing

Unit testing is performed by a developer and is targeted at small pieces of code to ensure, for example, that if you call a function with particular arguments, it will return the expected result.

Test-driven development principles suggest that you write test code even before you write the application code. For example, if you are about to start programming a class with some business logic, ask yourself, “How can I ensure that this function works fine?” After you know the answer, write a test ActionScript class that calls this function to assert that the business logic gives the expected result. Only after the test is written, start programming the business logic. Say you are in a business of shipping goods. Create a Shipment class that implements business logic and a ShipmentTest class to test this logic. You may write a test that will assert that the shipping address is not null if the order quantity is greater than zero.

In addition to business logic, Flex RIAs should be tested for proper rendering of UI components, changing view states, dispatching, and handling events. Integration testing is a process in which a developer combines several unit tests to ensure that they work properly with each other. Both unit and integration tests have to be written by application developers.

Several tools can help you write unit and integration tests.

FlexUnit4

FlexUnit4 is a unit testing framework for Flex and ActionScript 3.0 applications and libraries. With FlexUnit4 and Flash Builder, you can generate individual unit tests and combine them into test suites. Flash Builder 4 allows automatic creation of test cases (see New TestCase Class in the menus). Just enter the name of the class to test, and Flash Builder will generate a test application and a test case class in a separate package.

For each method of your class, say calculateMonthlyPayment(), Flash Builder will generate a test method, for example testCalculateMonthlyPayment(). You just need to implement it:

public function testCalculateMonthlyPayment(){
   //A $200K mortgage at 7% for 30 years should have
   // a monthly payment of $1199.10
   Assert.assertEquals(
    MortgageCalculator.calculateMonthlyPayment (200000, 7,30 ),1199.1 );
}

After the test case class is ready, ask Flash Builder to generate the test suite for you (see New Test Suite Class). To execute your test suite, right-click on the project in Flash Builder and select Execute FlexUnit Tests.

Unit testing of visual components is not as straightforward as unit testing of business logic in ActionScript classes. The Flex framework makes lots of internal function calls to properly display your component on the Flash Player’s stage. And if you need to get a hold of a particular UI component to ensure that it’s properly created, laid out, and populated, use the Application.application object in your tests.

FlexMonkey

A free tool from Gorilla Logic, FlexMonkey is a unit testing framework for Flex applications that also automates testing of Flex UI functionality. FlexMonkey can record and play back UI interactions. For example, Figure 4-4 illustrates the command list that results from the user entering the name of the manager and selecting a date.

Recording command list in FlexMonkey

Figure 4-4. Recording command list in FlexMonkey

FlexMonkey not only creates a command list, but also generates ActionScript testing scripts for FlexUnit (Figure 4-5) that you can easily include within a continuous integration process.

Test-generated matching command list

Figure 4-5. Test-generated matching command list

Technically, if the test scripts generated by FlexMonkey would allow a programming language simpler than ActionScript, you could consider it both a unit and functional testing framework. In the small IT shops where developers have to perform all kinds of testing, you may use FlexMonkey in this double mode. Even in larger organizations it may be beneficial if a developer runs these prefunctional tests to minimize the number of errors reported by the QA team. For more information on FlexMonkey, see http://www.gorillalogic.com/flexmonkey.

Visual Flex Unit

An open source framework for testing the visual appearance of components, Visual Flex Unit also introduces visual assertions, which assert that a component’s appearance is identical to a stored baseline image file. Developers can instantiate and initialize UI components, define view states and styles, and test that these components look the same as presaved images of the same. For output, you’ll get a report on how many pixels differ. You can run tests in Ant mode and send notifications about the test results. At the time of this writing, Visual Flex Unit is still in alpha version, but you can find more information at http://code.google.com/p/visualflexunit/

Functional Testing

Functional testing (a.k.a. black-box, QA, or acceptance testing) is aimed at finding out whether the application properly implements business logic. For example, if the user clicks on a row in the customer data grid, the program should display a form view with specific details about the selected customer. In functional testing business users should define what has to be tested, unlike unit or integration testing where tests are created by software developers.

Functional tests can be performed manually, in which a real person clicks through each and every view of the RIA, confirming that it operates properly or reporting discrepancies with the functional specifications. A better approach, however, is to engage specialized software that allows you to prerecord the sequence of clicks (similar to what FlexMonkey does) and replay these scripts whenever the application has been modified to verify that the functionality has not been broken by the last code changes.

Writing scripts for testing may sound like an annoying process, but this up-front investment can save you a lot of grief and long overtime hours during the project life cycle. Larger organizations have dedicated Quality Assurance teams who write these tests. In smaller IT shops, Flex developers write these tests, but this is a less efficient approach, as developers may not have the correct vision of the entire business workflow of the application and their tests won’t cover the whole functionality of the system.

Automated test scripts should be integrated with the build process of your application and run continuously. There are several commercial (and expensive) offerings for automation of functional testing:

QuickTest Professional (QTP) by HP (formerly Mercury)

During the recording phase, QTP creates a script in the VBScript language in which each line represents an action of the user. The checkpoints included in the script are used for comparison of the current value with expected values of the specified properties of application objects. Flex 3 Professional includes the libraries (.swc) required for automated testing with QTP, and your Flex application has to be compiled with these libraries. In addition, the QA testers need to have a commercial license for the QTP itself. The process of installing QTP for testing Flex applications is described at http://tinyurl.com/5wyqgb.

Rational Functional Tester by IBM

Rational Functional Tester supports functional and regression testing of Flex applications. You can see the demo and download a trial version of this product at http://www-01.ibm.com/software/awdtools/tester/functional/index.html.

RIATest by RIATest

RIATest (Figure 4-6) is a commercial testing tool for QA teams working with Flex applications. It includes Action Recorder (an RIAScript language similar to ActionScript), a script debugger, and synchronization capabilities.

Because of the event-driven nature of Flex, UI testing tools need to be smart enough to understand that some events take time to execute and your tests can run only after a certain period of time. RIATest allows you to not only rely on this tool to make such synchronization decisions, but also to specify various wait conditions manually. For example, if a click on the button requires an asynchronous remote call to populate a data grid, RIATest offers you the script command waitfor, which won’t perform the data verification until the data grid is populated. The Action Recorder creates human-readable scripts. To download a demo, go to http://riatest.com.

RIATest: Visual creation of verification code

Figure 4-6. RIATest: Visual creation of verification code

Load Testing

While rearchitecting an old-fashioned HTML-based application with RIA, you should not forget that besides looking good, the new application should be at least as scalable as the one you are replacing. Ideally, it should be more scalable than the old one if faster data communication protocols such as AMF and Real Time Messaging Protocol (RTMP) are being used. How many concurrent users can work with your application without bringing your server to its knees? Even if the server is capable of serving a thousand users, will performance suffer? If yes, how bad is it going to be?

It all comes down to two factors: availability and response time. These requirements for your application should be well defined in the service level agreement (SLA), which should clearly state what’s acceptable from the user’s perspective. For example, the SLA can include a clause stating that the initial download of your application shouldn’t take longer than 30 seconds for users with a slow connection (500 kbps). The SLA can state that the query to display a list of customers shouldn’t run for more than five seconds, and the application should be operational 99.9 percent of the time.

To avoid surprises after going live with your new mission-critical RIA, don’t forget to include in your project plan a set of heavy stress tests, and do this well in advance before it goes live. Luckily, you don’t need to hire 1,000 interns to find out whether your application will meet the SLA requirements. The automated load (a.k.a. stress or performance testing software) allows you to emulate required number of users, set up the throttling to emulate a slower connection, and configure the ramp-up speed. For example, you can simulate a situation where the number of users logged on to your system grows at the speed of 50 users every 10 seconds. Stress testing software also allows you to prerecord the action of the business users, and then you can run these scripts emulating a heavy load.

Good stress-testing software allows simulating the load close to the real-world usage patterns. You should be able to create and run mixed scripts simulating a situation in which some users are logging on to your application while others are retrieving the data and performing data modifications. Each of the following tools understands AMF protocol and can be used for stress testing of Flex applications:

NeoLoad by Neotys

NeoLoad is a commercial stress-testing tool. It offers analysis of web applications using performance monitors without the need to do manual scripting. You start with recording and configuring a test scenario, then you run the tests creating multiple virtual users, and finally, you monitor client operational system load and web and application server components. As you’ll learn in Chapter 6, we at Farata Systems have been using a scalable stress-test solution based on BlazeDS installed under a Jetty server. For more information on NeoLoad, go to http://neotys.com.

WebLOAD 8.3 by RadView Software

A commercial stress-testing software, WebLOAD 8.3 offers similar functionality to NeoLoad. It includes analysis and reporting, and a workflow wizard that helps with building scripts. It also supports AJAX. WebLOAD also allows you to enter SLA requirements right into the tests. To learn more, visit http://www.radview.com.

SilkPerformer and SilkTest by Borland

The commercial Borland test suite includes Borland SilkPerformer, stress-testing software for optimizing performance of business applications, and the functional testing tool Borland SilkTest, among other tools.

SilkPerformer allows you to create thousands of users with its visual scenario modeling tools. It supports Flex clients and the AMF 3 protocol.

SilkTest automates the functional testing process, and supports regression, cross-platform, and localization testing. For more details, see http://www.borland.com/us/products/index.html.

Data Services Stress Testing Framework by Adobe (open source)

An open source load-testing tool, Data Services Stress Testing Framework helps developers with stress testing of LiveCycle Data Services ES. This is a tool for putting load on the server and is not meant for stress testing an individual Flex/LCDS application running in the Flash Player. This framework is not compatible with BlazeDS. To download it or learn more, visit http://labs.adobe.com/wiki/index.php/Flex_Stress_Testing_Framework. For testing BlazeDS, consider using JMeter as described at the JTeam blog.

Code Coverage

Even if you are using testing tools, can you be sure that you have tested each and every scenario that may arise in your application?

Code coverage describes the degree to which your code has been tested. It’s also known as white-box testing, which is an attempt to analyze the code and test each possible path your application may go through. In large projects with hundreds of if statements, it’s often difficult to cover each and every branch of execution, and automated tools will help you with this.

An open source project, Flexcover is a code coverage tool for Flex and AIR applications. This project provides code coverage instrumentation, data collection, and reporting tools. It incorporates a modified version of the ActionScript 3 (AS3) compiler, which inserts extra function calls in the code within the .swf or .swc output file. At runtime, these function calls send information on the application’s code coverage to a separate tool. The modified compiler also emits a separate coverage metadata file that describes all the possible packages, classes, functions, code blocks, and lines in the code, as well as the names of the associated source code files. For more information, go to http://code.google.com/p/flexcover/.

Note

The document “Flex SDK coding conventions and best practices” lays out the coding standards for writing open source components in ActionScript 3, but you can use it as a guideline for writing code in your business application, too. This document is available at the following URL: http://tinyurl.com/3xphtd.

FlexPMD is a tool that helps to improve code quality by auditing any AS3/Flex source directories and detecting common bad practices, such as unused code (functions, variables, constants, etc.), inefficient code (misuse of dynamic filters, heavy constructors, etc.), overly long code (classes, methods, etc.), incorrect use of the Flex component life cycle (commitProperties, etc.), and more.

The code coverage tools will ensure that you’ve tested all application code, and the coding conventions document will help you in adhering to commonly accepted practices, but yet another question to be answered is, “How should you split the code of a large application into a smaller and more manageable modules?” This becomes the subject of the brief discussion that comes next.

Get Agile Enterprise Application Development with Flex now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.