Here is a list of XML benchmarking tools that might be interesting to use. It was originally collected for my course on parallel XML processing.

There are tons of small performance test scripts out there. I tried to reduce the collection to full-fledged benchmark applications that focus on comparative results for different XML processors. Another investigation is that the two famous XSLT benchmarks (XSLTMark and XSLBench) somehow disappeared. Maybe somebody still has a link for download.

Drop a comment if you have other suggestions.

Parser / XSLT benchmarking

• XMLBench
  • Parser / XSLT benchmark kit for all major C and Java parsers
  • Single benchmark implementations: Non-validating DOM / SAX parsing, DOM tree creation, Schema validation, XSL transformation, XML Security encryption
  • Measures initialization time, execution time, and memory consumption for different input file sizes (uses XMark generator)
  • Mainly a prepared set of test applications for a huge set of XML processors
  • Last update in 2004
• XPB4J
  • Java XML parser / XSLT benchmark
  • Framework for comparative measurements (e.g. DOM vs. SAX parsing)
  • Can create random XML data, or work with given data set
  • Code included to gather XML file statistics with different methods (SAX, DOM (JAXP, JDOM, DOM4J), XPP, XSLT stylesheet calling Java functions)
  • Measures processing time and memory usage
• XMLTest
  • Simulation of multithreaded server processing XML documents in parallel (Java / C# version)
  • Contains multiple processing steps: Parsing, data access, modification, serialization
  • Measures throughput of processed XML documents
  • Support for different parser API’s (DOM, SAX, JAXB, StAX)
  • Successor for Sun’s version of XMLMark (Microsoft’s version of XMLMark is still available)
• SPECjvm2008
  • Overall benchmark for JVM implementations
  • Contains a sub-benchmark for XML transformation and validation (javax.xml.transform, javax.xml.parsers)

Test data generation

• XMark
  • Data generator for valid XML documents (DTD available)
  • Certain probability distributions
  • Only one factor to tweak, influences the resulting document size
  • Simple C program, available in source code
  • Intended for XQuery performance measurement, several example queries
  • Last update in 2003
• ToXgene
  • Synthetic data generator for XML
  • Takes XML schema annotated with probability values as input

XQuery benchmarking

• XMach-1
  • XQuery benchmark for XML-based 2-tier server systems (database + application server)
  • Measurement of XQuery throughput and response time (query and update operations)
  • Java implementation for server-side servlet and multiple parallel clients
• The Michigan Benchmark
  • Micro benchmark for XML databases, with single data set
  • Carefully chosen set of atomic query elements, implemented in XPath / XQuery
  • Intended to allow the benchmarking of processing steps in XML query evaluation
• XOO7
  • XQuery benchmark containing different queries for XML databases
  • Data sets with various sizes
  • Data generator, DTD and list of XQuery statements freely available
• XBench
  • Exhaustive description of test data for XML databases
  • Covers all possible aspects of XQuery
  • According database generator implementation available
• TPoX
  • XML database benchmark based on financial transaction scenario
  • Data generation toolset, produces large collection of small files with referential consistency
  • Pre-defined XQuery workload; includes transactional insert, update, and delete operations
  • Developed by IBM DB2 performance team