Regular Expression Benchmark Utility

Regular Expression Benchmark Utility suite is an application written 100% in Java. It was implemented to test FIRE execution speed and compatibility with other Regular Expressions Engines in Java. The architecture of the regular expressions benchmarking utility is illustrated in the following figure:

Downloads

You can download the distribution from here. Current version is 0.7. See ChangeLog

• fire-benchmark.zip (~250Kb, includes binaries for all supported engines)

Supported engines

The benchmark suite has modular design in order to easily add new regular expression engines for testing. For the time being it supports the following libraries (module name, description):

SUN1.4 - Java Sun Regular Expression library (java.util.regex) 1.4.x, http://java.sun.com/
AUTOMATON - Automaton Regular Expression library (dk.brics.automaton), http://www.brics.dk/~amoeller/automaton/
JAKARTA-REGEXP - Jakarta Regular Expression library, http://jakarta.apache.org/regexp/index.html
JAKARTA-ORO - Jakarta ORO PERL5 Compatible Regular Expression Library, http://jakarta.apache.org/oro/index.html
GNU-REGEXP - GNU Regular Expressions Library, http://www.cacas.org/java/gnu/regexp/

This distribution provides the binaries for these libraries in the directory lib.

Tests

The regular expression library supports three types of tests.

• Iteration (Speed) - Execute N times, full-length match for a specific set of REGEX and DATA. Returns the execution time in msecs.
• Compatibility Benchmark (Compatibility) - Performs a full-length match for a specific set of REGEX and DATA. Returns true or false.
• Compile Iteration (Speed) - Performs for N iterations, compile speed test for a specific set of REGEX and DATA. Returns execution time in msecs.
  

In the future we will be add more tests.

Execution

Usage:
java org.benchmark.main.BenchmarkMain <output-method> <xml-file> [output-file]
Or
java.org.benchmark.main.BenchmarkMain XSLT <xml-file> <xslt-file> [output-file] for custom XSLT tranformation.

Available Output methods:
INFO - Prints out only information about the requested file (does not perform benchmark), example output for xml benchmark input file
HTML - HTML formatted document, example output for xml benchmark input file
XSLT - Transform with custom XSLT
TEXT - RAW Text output, example output for xml benchmark input file
XML - RAW XML, example output for xml benchmark input file
LATEX - LaTeX output, example output (postscript compressed gz) for xml benchmark input file

example:
java -jar fire-benchmark HTML test/benchmark.xml output.html

Plugin Development

In order to support more libraries, a modular framework was developed. For each regular expression engine a specific module has to be implemented and declared into the module library collection. The development procedure is simple.

1. Create a java module that implements the LibraryPlugin interface. Example for SUN JDK1.4 Regular Expression engine
2. Register the new plugin in a plugin.properties file (must be in the same directory with fire-benchmark.jar). Example
3. Execute the benchmark utility with the new module included in the XML input file and the compiled classes available in the CLASSPATH.

Back

Last Update: $Date: 2007-10-26 15:16:41 +0300 (Παρ, 26 Οκτ 2007) $