Berlin SPARQL Benchmark (BSBM)

The SPARQL Query Language for RDF and the SPARQL Protocol for RDF are implemented by a growing number of storage systems and are used within enterprise and open web settings. As SPARQL is taken up by the community there is a growing need for benchmarks to compare the performance of storage systems that expose SPARQL endpoints via the SPARQL protocol. Such systems include native RDF stores, Named Graph stores, systems that map relational databases into RDF, and SPARQL wrappers around other kinds of data sources.

The Berlin SPARQL Benchmark (BSBM) defines a suite of benchmarks for comparing the performance of these systems across architectures. The benchmark is built around an e-commerce use case in which a set of products is offered by different vendors and consumers have posted reviews about products. The benchmark query mix illustrates the search and navigation pattern of a consumer looking for a product.

• Benchmark Specification V2.0
• Benchmark Results for Virtuoso, Sesame, Jena, and D2R Server
• Data Generator and Test Driver

News

• 04/16/2009: Additional data generator for producing Linked Data style named graph stets released by Olaf Hartig from Humboldt University.
• 03/24/2009: Updated benchmark results from running BSBM against the current versions of four RDF stores, two RDB-to-RDF wrappers and two SQL databases released.
• 09/17/2008: Results from running BSBM Version 2 against three RDF stores, two RDB-to-RDF wrappers and two SQL databases released.
• 09/12/2008: Version 2 of the BSBM benchmark specification, data generator and test driver released. The new version includes 2 additional benchmark queries, fine-tuned data generation rules, and a relational representation of the dataset as well as a SQL version of the query mix. Multi-threaded version of the test-driver in order to simulate multiple clients simultaneously working against a server.
• 07/30/2008: Results from running BSBM Version 1 with 100 million triple datasets on Virtuoso and D2R Server added.
• 07/23/2008: New benchmark results for D2R Server added.
• 07/19/2008: Initial benchmark results for Virtuoso, Sesame and SDB published.

Please send comments and feedback about the benchmark to Chris Bizer and Andreas Schultz.

Publications

• Christian Bizer, Andreas Schultz: The Berlin SPARQL Benchmark . To appear in: International Journal On Semantic Web and Information Systems - Special Issue on Scalability and Performance of Semantic Web Systems, 2009
• Christian Bizer, Andreas Schultz: Benchmarking the Performance of Storage Systems that expose SPARQL Endpoints. In: Proceedings of the 4th International Workshop on Scalable Semantic Web knowledge Base Systems (SSWS2008).