SPMark™Java06 is the third mobile device benchmark from Futuremark. The first was SPMark04 which provides a standard benchmark suite for Symbian OS devices. We looked at this product in the features "How Fast Is Your Smartphone?" (http://www.symbianone.com/content/view/778/) last September. The other is 3DMarkMobile06 that tests 3D performance for future mobile platforms using OpenGL ES 1.0 and 1.1. 3DMarkMobile 06 is targeted mainly for semiconductor and handset companies' product development use and is available to Futuremark's BDP members as a source code product.

SPMarkJava06 is available to the public, developers and hardware vendors, and, not the least to IT media. It is integrated with Futuremark's ORB (Online Results Browser) service, which allows user to upload and compare benchmark results.

SPMarkJava06 runs a series of tests covering 2D and 3D gaming, fundimental 3D graphics performance including fillrate and polygon throughput, and other tests that stress the handset in ways that consumers use phones when running Java applications: video playback and PNG image coding and decoding, and a JVM test.

The 2D test uses a simple racing game with a single car. The test is designed to assess the performance of the JVM in rendering of 2D bitmaps with heavy use of javax.microedition.lcdui.game package class. The image below show the test running on a Nokia 9500.

In the Advanced and Professional versions of SPMark Java06 the game is also available to play, although there will not be too much worry in the games publishing houses, as it is fairly basic.

There are two different 3D gaming tests featuring a simulated car racing game. Both versions test JSR-184 (Mobile 3D Graphics API for J2ME, also known as M3G). The versions differ in the number of polygons, objects and features used in each game scene. The scenes played also differ. It was not possible to capture this test on any of our machines, so the image below is from Futuremark.

The Fillrate test measures the 3D fill rate using two 64 by 64 textures on a single triangle, seen below on a S60 device.

The polygon throughput measure the rate at which polygons can be drawn on the screen using a set of 19 by 19 cubes, with each wall created using 2 triangles for a total of 4332 triangles. Again the image is from Futuremark.

The video playback test simply determines whether the device can play h.263 compressed video. On a Nokia 9500 it shows a small section of video, as shown below.

The PNG Image tests simply encode and decode portions of an on screen image (the track bitmap from the 2D test) using PNG (Portable Network Graphic) encoding, seen here from a S60 device.

The JVM tests consist of a battery of 32bit and 64bit integer and floating point, trigonometry and square root mathematics operations along with method calls, object creation and array copying. The final test is a battery test, this test simply runs the main set of tests until the battery runs out. The following results were obtained from our test runs. Note that only the Nokia 6630 was reset so the only running application on the device was SPMarkJava06. The device only gets a SPMarkJava06 score if all the tests run successfully.

The results show a couple of interesting thing. The Nokia 9500, with its significantly larger display was not significantly slower than either the Nokia 6630 or Sony Ericsson P910i, with their much smaller screens. Also, despite its superior performance in almost all tests, the Nokia 6630 was the slowest device at the PNG encoding and decoding tests.

For any user of SPMark Java06 simply seeing a set of tests is merely a curiosity without something to compare it with. Even a hardware manufacturer, who may use the benchmark to judge how well improvement in their hardware performance are going, will also want to know how its hardware performs compared to others.

To this end Futuremark have extended their ORB (Online Results Browser) to include results from SPMarkJava06. After a set of benchmark tests have been run, one of the results options is "Submit" this sends the results over the device's internet connection to ORB. Once the send is complete the devices browser is launched to allow the user to create an account and assign the results to that account or log onto to view the results and compare them with those uploaded by other users.

SPMarkJava06 comes in three versions, Basic (free), Advanced and Professional. The Basic version lacks the playable game, high definition 3D, Video processing and battery tests. The Advanced version, which is for private use, costs $3, adds the playable game, the ability to select the test run and complete system information display. The Professional version, for companies, costs $20,000, the jump in price includes high detail 3D game, video and battery tests as well as the right to publish the results of tests outside of Futuremark's ORB.

SPMarkJava06 is a welcome addition to the Futuremark family of benchmarks. It has something to offer everyone, from the curious end user who wants to know how well their device performs to silicon and device manufacturers looking to measure the performance of their products. Continued development of mobile products by Futuremark underlies the fact that mobile devices are moving from the sidelines, where any level of performance thrilled the owner to the mainstream where the quest for optimum performance will be a key factor in attracting the next generation of mobile device owners.

To find out more about SPMarkJava06 visit Futuremark's web site: www.futuremark.com.