Black Box Web Testing with HttpUnit

White or Black?

Automated software tests are crucial for IT projects. They enable continuous modifications to an existing code base without the fear of damaging existing functionality. They are executed at will and don't carry the costs and inconsistencies associated with manual tests.

There are two fundamental approaches for automated software tests:

• White Box, where an application is tested from the inside using its internal application programmatic interface (API).
• Black Box, where an application is tested using its outward-facing interface (usually a graphical user interface).

I consider White Box tests as a nice-to-have feature, while Black Box tests are a necessity. White Box tests suffer from the following shortcomings:

• Creating a false sense of stability and quality. An application can often pass all unit tests and still appear unavailable to users due to a non-functioning component (a database, connection pool, etc.). This is a critical problem, since it makes the IT application team look incompetent.

• Inability to validate use-case completeness because they use a different interface to the application. White Box tests focus on pleasing the wrong crowd -- developers. The right crowd to please in the real world is application users, because they fund IT projects. It is more important to understand how a user views an application than how a developer does. Developers and users use different interfaces to interact with an application; developers use a programmatic API, while users use a GUI. Application users care mostly about the inputs and outputs on their GUI screen. The only technique for testing these is interacting with the application via the same interface used by the users.

Sometimes White Box tests are sufficient. In general, I am no great supporter of White Box tests. White Box unit tests are sufficient to test programmatic interfaces (API). For example, a unit test is sufficient to test a square root function.

Black Box tests are useful in evaluating the overall health of the application and its ability to provide value to users. Most Black Box tests emulate the sequence of interactions between a user to an application in its exact order. Failure of a Black Box test indicates that a user received insufficient value from the application.

The focus of this article is Black Box tests of web browsers, perhaps the most common user interface in today's IT environment. The article demonstrates a simple yet effective way to perform a Black Box test of a web site using an open source package called HttpUnit.

Figure 1. Where White Box and Black Box tests interface to an application

Emulating User Activity via Black Box Tests

Black Box tests emulate user interaction with an application. Keeping the tests closely aligned with the manner in which users interact with the application is the key for quality tests.

Historically, writing Black Box tests was difficult. Applications used different GUI technologies and communication protocols; thus, the potential for tool reuse across applications was low. Commercial testing-tool packages relied on capturing mouse clicks and keystrokes, which proved brittle to change. The combination of wide usage of HTTP/HTML standards and an emerging number of open source projects makes Black Box testing an easier task. Emulating user activity on web sites can be done in several manners:

• Automating Internet Explorer via the SHDocVw COM interface.
• Using a commercial testing tool.
• Using an open source Java package like HttpUnit or jWebUnit.

HttpUnit is the focus of this article for several reasons:

• HttpUnit covers the useful spectrum of HTTP/HTML standards functionality, including forms, frames, JavaScript, SSL, cookies, headers. HttpUnit can be used to test a web site written in any programming language.
• HttpUnit is written in Java, so the test cases can use the entire spectrum of Java functionality.
• HttpUnit executes within a JVM as a standard Java library and can be integrated easily with Ant scripts and daemon test agents.
• HttpUnit is an open source project.

Swing and AWT applications are not suitable for HttpUnit tests. Those can be tested with sibling tools like jfcUnit.

Installing HttpUnit

Running HttpUnit will require the following CLASSPATH entries:

• httpunit.jar
• js.jar
• junit.jar
• nekohtml.jar
• xercesImpl.jar
• xmlParserAPIs.jar

A Word of Warning About Client-side JavaScript

HttpUnit supports client-side JavaScript. However, JavaScript is always an enigma. Sometimes it is appropriate to turn off support for client-side JavaScript by either removing js.jar from the CLASSPATH or calling the HttpUnit method HttpUnitOptions.setExceptionsThrownOnScriptError( false );.

Writing an HttpUnit Test Case

Site Under Test

The demonstration of HttpUnit's capabilities requires a web page accessible to the public. Ironically, I selected Dice.com. In this day and age it is likely that Java developers will know this resource. (If you detect an underlying pain in my writing, you can probably guess that I do know Dice.)

We shall write a Black Box test using HttpUnit to test a use case on the Dice web site.

To be clear: the Dice web site is not written in Java. HttpUnit Black Box tests are compatible with any HTML/HTTP-based web site. Furthermore, the test is performed against a code base completely hidden from us. This is not to be taken lightly in scenarios of delegation and design by contract, where the tester and coder are not the same. The set is ready now for testing a real-world use case:

The Dice Job-Seeker Use Case:

The following are the steps of the use case tested:

1. Navigate to www.dice.com.
2. Click on the Find a Job button.
3. Set the form data.
      • Set the Full Text Search.
      • Set the Days back to search.
      • Set the Jobs per Page.
4. Click on the Search button.
5. View the Result in the next page.
6. Click on a Job Opening link.
7. View a Job Opening.

If all of the above work, the test case is considered successful.