Black Box Web Testing with HttpUnit

HttpUnit Main Classes

The main HttpUnit classes are:

• WebConversation, which emulates a web browser talking to a single site. It keeps track of the session context on the client side by holding cookies.
• WebRequest, which represents an HTTP Request sent to the server.
• WebResponse, which represents an HTTP Response returned by the server.
• WebForm, which represents an HTML form.

A typical HttpUnit interaction consists of creating a WebConversation instance, submitting a WebRequest, and navigating and validating the result returned by the WebResponse.

The Dice Job-Seeker HttpUnit Black Box Test

Step 1

The first thing to do is open a WebConversation with Dice:

URL serverUrl                = new URL(url);

WebConversation conversation = new WebConversation();

WebRequest request           = new GetMethodWebRequest(serverUrl,"");

WebResponse response         = conversation.getResponse(request);

assertNotNull("response of url: " + url, response)

Notice that the implementation of this HttpUnit test case uses the JUnit framework. The JUnit framework provides assertions, test runners, and Ant integration. Don't be misled by the name JUnit; our Black Box tests are not unit tests, we just use some of JUnit's capabilities.

Step 2

So far, the test uses very little knowledge about the structure of the web site. This is about to change. Here is the HttpUnit code used to click on the Find a Job image button:

WebImage image =

    response.getImageWithSource("images/topnav_findajob_off.gif");

assertNotNull("find a job impage",image);



WebLink findJobLink = image.getLink();

assertNotNull("find a job link",findJobLink);



System.out.println("found job seeker link");

response = findJobLink.click();

assertNotNull("job seeker page",response);



System.out.println("got job seeker page");

The first line in the code above searches for the image using the call response.getImageWithSource("images/topnav_findajob_off.gif");

The first question that comes to mind is, "How does a test coder find the name of the image?" The answer lies in HTML. To discover the image name, the test coder must use the web browser's View Source functionality. (Remember to View Source within the boundaries of the frame holding the HTML control you seek.)

Using the link associated with the image, the script opens the Job Seekers page. Like many Dice pages, this page uses HTML frames. The code section below selects the ENGINE_WIN frame:

response = response.getSubframeContents("ENGINE_WIN");

assertNotNull("got the engine frame",response);

The ENGINE_WIN frame uses HTML forms to submit user input to the Dice job search engine. The first necessary step is finding the input form within the page. This can be done by either the form name or form number within the containing page:

WebForm findJobsForm = response.getForms()[0];

assertNotNull("find jobs form",findJobsForm);

Step 3

The next step is populating the form with the job search criteria. The names of the form parameters are found again by viewing the page source.

findJobsForm.setParameter("FREE_TEXT",openSearchText);

findJobsForm.setParameter("DAYSBACK",daysBack);

findJobsForm.setParameter("NUM_PER_PAGE",jobsPerPage);

Step 4

After the form is populated, all that is left is calling the submit() method.

response = findJobsForm.submit();

assertNotNull("got search result frame",response);

Step 5

The job opening HTML links resulting from the job search form submission are sent to the same frame that originated the form submission. After receiving these links, the test script needs to filter out links not related to job openings:

WebLink resultLinks[] = response.getLinks();

assertNotNull("search links", resultLinks);

System.out.println("got search links back");



// count only the job opening links

int openingCounter = 0;

for(int i=0; i < resultLinks.length; i++)

{

    String url =  resultLinks[i].getURLString();

    if(url.indexOf("/jobsearch/servlet/JobSearch?op") != -1)

    {

        openingCounter++;

        System.out.println("found job opening: " + resultLinks[i].asText());

    }

}

Step 6

If the number of job openings found is greater than 0, the user can click on the job openings link:

if(openingCounter > 0)

{

    response = resultLinks[0].click();

    assertNotNull("job opening page",response); 

}

Congratulations! You just finished a complete Black Box test of a commercial web site by emulating an entire use case. Assuming the Dice.com site doesn't change soon, running the test will produce output similar to the following:

[java] .found job seeker link

[java] got job seeker page

[java] got engine frame

[java] got the find jobs search form

[java] got search result

[java] got search links back

[java] found job opening: Embedded Linux Guru

[java] found job opening: Visionary Software Design Manager

[java] found job opening: Senior Software Performance Engineer

[java] found job opening: Senior Java - Websphere Developer

[java] found job opening: Configuration and Deployment

Sadly, you might notice that the number of job openings for Java professionals is smaller than one would hope. (It's not a great time to jump ship.)

Please be considerate about running tests against someone else's site.

Integrating HttpUnit Tests Into a Build Script

A J2EE build script generally executes in the following order:

Figure 2. Software development lifecycle

The build tool of choice in many J2EE projects is Ant. Ant's feature list is out of the scope of this article, but of interest to testers are its features for reporting the success or failure of build tasks via email, and for executing the JUnit script. An Ant build script file is bundled with this article.

Ant Launch-and-Forget Build Script

The Ant mail logger enables SMTP email reporting of Ant build target successes and failures. Configuring the Ant mail logger requires several simple components:

• A property file holding the mailer information.
• An entry in an Ant build file to import the property file.
• Several .jars of Java classes enabling SMTP communication.

Invoke Ant using the command:

$ ant -logger org.apache.tools.ant.listener.MailLogger

Here is sample build.properties file used to set the Ant mail logger configuration:

MailLogger.mailhost=smtphost@domain.com

MailLogger.from=BuildAdmin@domain.com

MailLogger.failure.notify=true

MailLogger.success.notify=true

MailLogger.failure.to=FailureRecipient@domain.com

MailLogger.success.to=SuccessRecipient@domain.com

MailLogger.failure.subject=Ant Build Failure Subject

MailLogger.success.subject=Ant build Success Subject

Here is a sample entry in an Ant XML build file used to import the mailer properties:

<property file="build.properties"/>

Running the Ant XML build file bundled with this project will send an email message to the designated recipient.

Beware that reporting build failures with the Ant mail logger can be vicious. I've seen corporations whose build script notifies the entire galaxy whenever a build or test failure occurs. Use this feature with caution. (See an Anti-Pattern named: Email is Dangerous.)

HttpUnit Black Box Test Usage

Let's conclude with a real world example where Black Box tests saved the day. We had a three-tier web application. Our hosting app server was clustered and monitored. A user reported that the application was not responding. The developer assigned to the case successfully used the application.

Further exploration revealed that one node of the cluster contained an invalid configuration for the JDBC connection pool. Users assigned to that node faced an unresponsive application.

The lesson we learned was that application server monitoring can't guarantee application health. Application health must be checked via a Black Box test executed periodically by a daemon thread.

Black Box tests can also serve well as stress tests. HttpUnit scripts are very valuable for tests of concurrent access to a web site. The tests are a good representation of client behavior.

Usage Advice

• Attempt to mimic the user web-browser runtime behavior as closely as possible. Black box test scripts should not create environment entries and data not present in the runtime environment of the emulated user.
• Just do the necessary minimum. If you find yourself writing an artificial intelligence engine to dynamically test HTML field names, you have gone a tad too far. Black box tests must be simple and concrete.

Conclusion

Black Box tests are the most valuable type of tests available in the project team arsenal. HttpUnit Black Box tests can serve for use case completeness validation, application availability validation, stress testing, and regression tests.

Software packages like HttpUnit enable developers to automate Black Box tests quickly, while covering the entire functionality offered by today's web sites. Integration between Ant build scripts and HttpUnit is a best practice for protecting developers from hastily rolling unfinished changes to production.

Giora Katz-Lichtenstein is a J2EE professional consulting to the financial industry in New York City.

Return to ONJava.com.