Introduction

This page presents a performance evaluation for LDIF using life science use cases.

The following data sources have been used to evaluate the performance of LDIF:

• Allen Mouse Brain Atlas is a growing collection of online public resources integrating extensive gene expression and neuroanatomical data.
• KEGG GENES, a collection of gene catalogs for all complete genomes generated from publicly available resources
• KEGG Pathway, a collection of pathway maps representing knowledge on the molecular interaction and reaction networks
• PharmGKB, which provides data on gene information, disease and drug pathways, and SNP variants
• Uniprot, a dataset containing protein sequence, genes and function

Benchmark Machine

We used a machine with the following specification for the benchmark experiments:

Hardware:

• Processors: Intel i7 950, 3.07GHz (quadcore)
• Memory: 24GB
• Hard Disks: 2 × 1.8TB (7,200 rpm) SATA2

Software:

• Operating System: Ubuntu 11.04 64-bit, Kernel: 2.6.38-10
• Java version: 1.6.0_22

Test Procedure

We applied the following test procedure to each test:

1. Clear Operating System caches

    echo 2 > /proc/sys/vm/drop_caches 

2. Run test

    java -server -Xmx20G -jar ldif-0.1-single-machine.jar 

Use case A

In this use case, we evaluated the performance of LDIF applied to integrate data originating from the five life science sources described above and translate those into a common target vocabulary.

Datasets

In order to measure the performance at different scale we used several samples of the input data which differ in the amount of resources (1, 10 and 100 thousand) for each data source.
For this use case we extracted only data of human (Homo sapiens, hsa) and mouse (Mus musculus, mmu) species.

Details about the benchmark datasets are summarized in the following table:

Mappings

We defined R2R mappings for translating vocabularies used by the source datasets into the target vocabulary.

Link Specifications

The following Silk identity resolution heuristics are used to find genes and other expressions that are described in multiple datasets.

• ABA-to-Wiki.silk.xml,
• KEGG-GENES-to-Wiki.silk.xml,
• KEGG-Pathway-to-Wiki.silk.xml,
• PharmGKB-to-Wiki.silk.xml,
• Uniprot-to-Wiki.silk.xml

Results

The following table summarizes the LDIF run times for the different dataset sizes. The overall run time is split according to the different processing steps of the integration process.

Use case B

In this use case, we evaluated the performance of LDIF integrating two larger life science datasets and translate those into a common target vocabulary.

Datasets

For this use case we used only KEGG GENES and UniProt datasets. There is a huge difference in dataset size between the two datasets. Converted to N-Triples the complete KEGG GENES dump is about 28GB in size whereas the UniProt dataset contains over 400GB worth of data. Because of this size mismatch we extracted only data of five different species from UniProt: bos taurus (cattle), canis familiaris (dog), danio rerio (zebrafish), drosophila melanogaster (fruitfly) and sus scrofa (pig).

For the test, we generated subsets of both data sources amounting together to 25 million, 50 million and 100 million RDF triples.

Details about the benchmark datasets are summarized in the following table. It provides statistics about the data integration process for each dataset. The original number of input triples decreases in the process as LDIF discards input triples which are irrelevant for the defined mappings, and therefore can not be translated into the target vocabulary. The number decreases again after the actual translation, as the input data uses more verbose vocabularies and as multiple triples from the input data are thus combined into single triples in the target vocabulary. The size of the final dataset is the number of quads after the mapping phase plus any provided provenance data.

Mappings

We defined R2R mappings for translating genes, diseases and pathways from KEGG GENES and genes from UniProt into a proprietary target vocabulary. Some more sophisticated mappings from the use case translate complex structural
patterns and perform value transformations (e.g. extracting an integer value from a URI. The prevalent value transformations are extracting strings with a regular expression and modifying the target data types.

• mapping a KEGG Gene 'gene' into a target vocabulary 'Gene'

   mp:Gene
     a r2r:ClassMapping;
     r2r:prefixDefinitions
          "category: <http://mywiki/resource/category/> .
           property: <http://mywiki/resource/property/> .
           pathway:  <http://wiking.vulcan.com/neurobase/kegg_pathway/resource/vocab/> .
           genes:    <http://wiking.vulcan.com/neurobase/kegg_genes/resource/vocab/> .
           xsd:      <http://www.w3.org/2001/XMLSchema#> .";
     r2r:sourcePattern "?SUBJ a genes:gene";
     r2r:targetPattern "?SUBJ a category:Gene";
     .
  

• mapping a KEGG Gene 'externalLink' property into a target vocabulary 'UniprotId' property

   mp:GeneLinkUniProt
     a r2r:PropertyMapping;
     r2r:mappingRef mp:Gene;
     r2r:sourcePattern  "?SUBJ genes:externalLink ?x";
     r2r:transformation "?id = regexToList('UniProt:(.+)', ?x)";
     r2r:targetPattern  "?SUBJ property:UniprotId ?'id'^^xsd:string";
     . 
  

Link Specifications

We defined the following Link Specification for the identity resolution phase:

<Silk>
	<Prefixes>
		<Prefix id="rdf" namespace="http://www.w3.org/1999/02/22-rdf-syntax-ns#" />
		<Prefix id="rdfs" namespace="http://www.w3.org/2000/01/rdf-schema#" />
		<Prefix id="owl" namespace="http://www.w3.org/2002/07/owl#" />
		<Prefix id="property" namespace="http://mywiki/resource/property/" />
		<Prefix id="category" namespace="http://mywiki/resource/category/" />
	</Prefixes>

	<DataSources>
		<DataSource id="Source" type="sparqlEndpoint" >
			<Param name="endpointURI" value="http://localhost:2020/sparql/read" />
		</DataSource>
		<DataSource id="Target" type="sparqlEndpoint" >
			<Param name="endpointURI" value="http://localhost:2020/sparql/read" />
		</DataSource>
	</DataSources>

	<Interlinks>
		<Interlink id="link">
			<LinkType>owl:sameAs</LinkType>

			<SourceDataset dataSource="Source" var="a">
				<RestrictTo>
					{ ?a rdf:type category:Gene }
					UNION { ?a rdf:type category:Disease }
					UNION { ?a rdf:type category:Pathway }
				</RestrictTo>
			</SourceDataset>

			<TargetDataset dataSource="Target" var="b">
				<RestrictTo>
					{ ?b rdf:type category:Gene }
					UNION { ?b rdf:type category:Disease }
					UNION { ?b rdf:type category:Pathway }
				</RestrictTo>
			</TargetDataset>

			<LinkageRule>
			<Aggregate type="max">
					<Compare metric="equality">
						<Input path="?a/property:UniprotId" />
						<Input path="?b/property:UniprotId" />
					</Compare>
					<Compare metric="equality">
						<Input path="?a/property:EntrezGeneId" />
						<Input path="?b/property:EntrezGeneId" />
					</Compare>
					<Compare metric="equality">
						<Input path="?a/property:MgiMarkerAccessionId" />
						<Input path="?b/property:MgiMarkerAccessionId" />
					</Compare>
					<Compare metric="equality">
						<Input path="?a/property:KeggDiseaseId" />
						<Input path="?b/property:KeggDiseaseId" />
					</Compare>
					<Compare metric="equality">
						<Input path="?a/property:KeggPathwayId" />
						<Input path="?b/property:KeggPathwayId" />
					</Compare>
			</Aggregate>
			</LinkageRule>

			<Filter threshold="1.0" />

		</Interlink>

	</Interlinks>
</Silk>

Results

The following table summarizes the LDIF run times for the different dataset sizes. The overall run time is split according to the different processing steps of the integration process.