An Introduction to Streaming Transformations for XML

XSLT's popularity has grown over the past three years, both aiding and riding on the adoption of XML. In comparison to API-level programming with Document Object Model (DOM), XSLT provides a loosely-typed, declarative environment tailored for tree-oriented transformation of XML documents, which has achieved wide adoption as a general-purpose XML manipulation tool despite the proscription:

DOM versus SAX

"Which is better, DOM or SAX?" is a common question for newcomers on XML-related discussion lists. (And, one could legimately also ask "DOM or JDOM or DOM4J or XOM?" and "SAX or XNI"?)

DOM provides an overall view of an XML document through tree traversal and manipulation. DOM is heavyweight, however, in that it typically imposes a memory footprint of around five times the size of the underlying XML text for simple documents. DOM also imposes a significant time overhead for creating the necessary objects.

SAX provides a sequential view of an XML document through a stream of events. SAX-based programs typically maintain some amount of state information that encapsulates already-received events, but SAX processing requires a negligible amount of memory (typically only the representation of the current event and the buffer for the parser).

SAX is the event-oriented sibling of the DOM API. (See the sidebar for a short discussion of DOM versus SAX.) STX provides a streaming analog for XSLT by adopting some of the now familiar concepts from XSLT (e.g., matching based on templates and an XPath 1.0-like expression language) but using SAX as the underlying interface to the XML document. In line with the proscription about XSLT, STX is neither a general purpose XML transformation language, nor is it an attempt to improve, extend, or replace XSLT.

Like SAX, STX is a completely free, grassroots effort by the XML community, initiated by Petr Cimprich; the specification and a mailing list are hosted on SourceForge. The current version of the STX specification contains a list of other contributors. There are currently two STX processor implementations:

An Illustrative Example

This section illustrates some of the relative strengths of STX through an example. The scenario is one of a nursery that stocks a selection of trees and flowers and receives orders in a simple XML format, illustrated below. (The full text of the example order document is available in XML.) Every order consists of the name, an optional quantity which defaults to 1, and a price per unit quantity.



<?xml version="1.0"?>

<order-list>

  <order>

    <name>Achillea Millefolium</name>

    <quantity>50</quantity>

    <price>2.93</price>

  </order>

  ...

  <order>

    <name>Camelia Japonica</name>

    <quantity>20</quantity>

    <price>39.90</price>

  </order>

  <order>

    <name>Ginkgo Biloba</name>

    <price>19.90</price>

  </order>

  ...

</order-list>

Example 1. An order

The initial goal is to create a stylesheet that computes the total price (the sum over all items of the the quantity multiplied by the unit price). This goal could be accomplished with XSLT, using recursive invocation of named templates with parameter passing.



  1 <?xml version="1.0"?>                                      

    <stx:transform version="1.0"                               

      xmlns:stx="http://stx.sourceforge.net/2002/ns"

      xmlns="http://www.w3.org/1999/xhtml">

  5 

      <stx:variable name="sum" select="0" />                   

      

      <stx:template match="order-list">                        

        <html>                                                 

 10       <body>

            <table>

              <tr>

                <th colspan="5">Order Summary</th>

              </tr>

 15           <tr>

                <td>Line</td>

                <td>Item</td>

                <td>Quantity</td>

                <td>Unit Price</td>

 20             <td>Item Total</td>

              </tr>

              <stx:process-children />

              <tr>

                <td colspan="4">TOTAL:</td>

 25             <td><b><stx:value-of select="$sum" /></b></td> 

              </tr>

            </table>

          </body>

        </html>

 30   </stx:template>

    

      <stx:variable name="name" />

      <stx:variable name="quantity" />

      <stx:variable name="price" />

 35   

      <stx:template match="order">

        <stx:assign name="quantity" select="1" />

        <stx:process-children />

        <stx:assign name="sum"

 40       select="$sum + $quantity * $price" />                

        <tr>

          <td><stx:value-of select="position()" /></td>

          <td><b><stx:value-of select="$name" /></b></td>

          <td><stx:value-of select="$quantity" /></td>

 45       <td><stx:value-of select="$price" /></td>

          <td><stx:value-of

            select="$quantity * $price" /></td>

        </tr>

      </stx:template>

 50 

      <stx:template match="name">

        <stx:assign name="name" select="." />

      </stx:template>

    

 55   <stx:template match="quantity">

        <stx:assign name="quantity" select="." />

      </stx:template>

    

      <stx:template match="price">

 60     <stx:assign name="price" select="." />

      </stx:template>

    

    </stx:transform>

Example 2. STX stylesheet to compute the order total

Here is XML source for the stylesheet; and here is a pretty-printed HTML rendition.

The output from the transformation is, as expected, an XHTML document containing the order formatted into a table and the total cost computed.

Running the Example with Joost

To execute the transformation with Joost under JDK 1.4 (the SAX parser, a version of Crimson, bundled with JDK 1.4 is less than perfect; Ã†lfred2 or Xerces-J2 both make better choices for general use XML parsers), download binary distributions of Joost and log4j and then run:

The process should complete almost instanteously. On a reasonable PC, Joost will process around 1Mb per second through the example stylesheet. This was estimated by timing the processing of a 250,000 line-item order.

Running under JDK 1.3 requires adding a JAXP-compliant SAX parser to the classpath.

Running the stylesheet with STX::XML

To execute the transformation with STX::XML under Perl, install the STX::XML module and the prerequisite Perl modules XML::SAX and XML::NamespaceSupport from CPAN. (The modules XML::SAX::Expat and XML::SAX::Writer are also recommended but not required.) Then execute the stxcmd.pl script (located in the STX::XML installation directory) as follows:

Similarities with XSLT

A quick glance over the example stylesheet should remind the reader of an XSLT stylesheet:

Furthering the resemblance, the STX namespace (http://stx.sourceforge.net/2002/ns) shares many elements (by local name) with the XSLT 1.0 namespace (http://www.w3.org/1999/XSL/Transform), and these elements provide similar functions to their XSLT 1.0 counterparts:

The transform element encloses the stylesheet. (STX does not support the "literal result element as stylesheet" syntax described in section 2.3 of the XSLT 1.0 specification.) (3)

STX provides variables, both global and local. (STX and XSLT 1.0 treat variables differently, however.) (4)

STX uses declarative, rule-based matching to drive the flow of stylesheet execution. (STX matching is different from XSLT 1.0 in some important ways; read on.) (6)

STX has a value-of element to place the value of an expression on the output. (7)

STX uses a loosely-typed expression syntax derived from [XPath2.0], so the basic syntax is similar to the version of XPath ([XPath1.0]) used in XSLT 1.0.

STX also has choose, if, param, and with-param that behave similarly to their XSLT 1.0 counterparts.

Differences Between STX and XSLT 1.0

Nodes and Events

The most important difference between XSLT and STX is the difference between XPath 1.0 nodes and SAX events. The fundamental, atomic unit of operation in XPath 1.0 is a node that exists in the context of its containing document and inter-node relationships (parents, siblings, children, and so on). In contrast, the fundamental unit of operation in SAX is an event that exists without any additional context.

Programming at the level of SAX events is non-trivial and typically involves tracking the context of an event, directing the event to an appropriate handler based on its context, and performing an operation. To this end, an STX processor adds a thin layer of context to a stream of SAX events to create a notion of node robust enough to support a useful subset of XPath expressions -- called "STXPath" -- without requiring the creation of a DOM or similar tree-like data structure. The available context for a node in STXPath includes:

stx:process-* versus xsl:apply-templates

STX does not include an analog of the apply-templates instruction from XSLT, since an STX processor receives SAX events in document order as opposed to being able to look for nodes in an object model. STX provides several different process-* instructions to control the handling of subsequent SAX events (with an optional group="..." specification; see below for more about groups).

One way to think about the process-* instructions is as valves that let a certain set of events pass through part of the stylesheet, returning control to the current template once those events have been consumed.

Variables in STX

In contrast to side-effect-free XSLT, variables in STX are mutable and can be freely reassigned at any point just like variables in programming languages like Java, C, Python, and Perl. The example stylesheet uses this to maintain the sum during the execution of the stylesheet and to re-populate the name, quantity, and price variables for each successive order.

Unique and Distinguishing Features of STX

The example STX stylesheet illustrates some of the similarities and differences between STX and XSLT 1.0, but STX has a host of additional features.

Conclusion

We have demonstrated that STX is an approachable means for transforming streaming XML with two usable implementations and an active, community-driven development effort.