Current Behavior
We are currently investigating the RDF serialization of Asset Administration Shell (AAS).
During performance testing we've compared RDF4J (MemoryStore und LmdbStore) with Virtuoso and QLever.
We were suprised that LmdbStore performed much worse than MemoryStore for two of the three queries.
Therefore, I've created an isolated benchmark for further investigation:
https://github.com/kenwenzel/rdf4j/tree/aas-benchmarks
UPDATE: I've found out that the original queries did not returned any results and therefore more or less just measured the query optimization overhead, which is larger for LmdbStore than for MemoryStore.
The results on develop are as follows:
Benchmark (driveUnitCount) (productionLineCount) (sensorCount) (store) Mode Cnt Score Error Units
AASQueriesBenchmark.query1PropertyProjection 100 100 100 lmdb avgt 3 0.288 ± 0.108 ms/op
AASQueriesBenchmark.query1PropertyProjection 100 100 100 memory avgt 3 0.104 ± 0.187 ms/op
AASQueriesBenchmark.query1PropertyProjection 100 100 100 native avgt 3 0.126 ± 0.007 ms/op
AASQueriesBenchmark.query2ThresholdCount 100 100 100 lmdb avgt 3 171.104 ± 74.601 ms/op
AASQueriesBenchmark.query2ThresholdCount 100 100 100 memory avgt 3 51.067 ± 17.995 ms/op
AASQueriesBenchmark.query2ThresholdCount 100 100 100 native avgt 3 333.817 ± 309.181 ms/op
AASQueriesBenchmark.query3LineAggregates 100 100 100 lmdb avgt 3 417.807 ± 703.935 ms/op
AASQueriesBenchmark.query3LineAggregates 100 100 100 memory avgt 3 77.809 ± 19.704 ms/op
AASQueriesBenchmark.query3LineAggregates 100 100 100 native avgt 3 459.011 ± 269.580 ms/op
Query 1:
PREFIX aas: <https://admin-shell.io/aas/3/>
SELECT ?propertyName ?propertyValue ?valueType
WHERE {
<urn:aas:DriveUnitAAS:DU-101> a aas:AssetAdministrationShell ;
aas:submodel/aas:submodelElement/(aas:value)* ?prop .
?prop a aas:Property ;
aas:idShort ?propertyName ;
aas:value ?propertyValue .
OPTIONAL { ?prop aas:valueType ?valueType }
}
Query 2:
PREFIX aas: <https://admin-shell.io/aas/3/>
SELECT (COUNT(*) AS ?c)
WHERE {
?aas a aas:AssetAdministrationShell ;
aas:assetInformation/aas:specificAssetId [ aas:value "DriveUnit" ] ;
aas:submodel/aas:submodelElement/(aas:value)* ?p1 .
?p1 aas:idShort "ratedPower" ;
aas:value ?v1 .
FILTER (?v1 > 15.0)
}
Query 3:
PREFIX aas: <https://admin-shell.io/aas/3/>
PREFIX aas-ext: <https://admin-shell.io/aas/3/extended/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?lineAAS (COUNT(DISTINCT ?compAAS) AS ?numDrives) (AVG(xsd:double(?val)) AS ?avgPower)
WHERE {
?lineAAS a aas:AssetAdministrationShell ;
aas:submodel / aas:submodelElement ?rel .
?rel a aas:RelationshipElement ;
aas:second / aas-ext:resolvesTo ?compAAS .
?compAAS aas:submodel / aas:submodelElement / (aas:value)* ?prop .
?prop aas:idShort "ratedPower" ;
aas:value ?val .
}
GROUP BY ?lineAAS
Expected Behavior
I've expected that the performance is in the same order of magnitude when comparing the 3 stores.
Steps To Reproduce
- checkout https://github.com/kenwenzel/rdf4j/tree/aas-benchmarks
- run
AASQueriesBenchmark
Version
current develop branch
Are you interested in contributing a solution yourself?
Perhaps?
Anything else?
It's probably due to cardinality estimation.
Current Behavior
We are currently investigating the RDF serialization of Asset Administration Shell (AAS).
During performance testing we've compared RDF4J (MemoryStore und LmdbStore) with Virtuoso and QLever.
We were suprised that LmdbStore performed much worse than MemoryStore for two of the three queries.
Therefore, I've created an isolated benchmark for further investigation:
https://github.com/kenwenzel/rdf4j/tree/aas-benchmarks
UPDATE: I've found out that the original queries did not returned any results and therefore more or less just measured the query optimization overhead, which is larger for LmdbStore than for MemoryStore.
The results on develop are as follows:
Query 1:
Query 2:
Query 3:
Expected Behavior
I've expected that the performance is in the same order of magnitude when comparing the 3 stores.
Steps To Reproduce
AASQueriesBenchmarkVersion
current develop branch
Are you interested in contributing a solution yourself?
Perhaps?
Anything else?
It's probably due to cardinality estimation.