Skip to content

Sketch based optimizer 4 new unified planners and optimizers2#5796

Open
hmottestad wants to merge 65 commits intodevelopfrom
sketch-based-optimizer-4-new-unified-planners-and-optimizers2
Open

Sketch based optimizer 4 new unified planners and optimizers2#5796
hmottestad wants to merge 65 commits intodevelopfrom
sketch-based-optimizer-4-new-unified-planners-and-optimizers2

Conversation

@hmottestad
Copy link
Copy Markdown
Contributor

GitHub issue resolved: #

Briefly describe the changes proposed in this PR:


PR Author Checklist (see the contributor guidelines for more details):

  • my pull request is self-contained
  • I've added tests for the changes I made
  • I've applied code formatting (you can use mvn process-resources to format from the command line)
  • I've squashed my commits where necessary
  • every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change

…nified-planners-and-optimizers2

# Conflicts:
#	.gitignore
#	core/queryalgebra/evaluation/src/main/java/org/eclipse/rdf4j/query/algebra/evaluation/ArrayBindingSet.java
#	core/sail/lmdb/src/main/java/org/eclipse/rdf4j/sail/lmdb/LmdbSailStore.java
#	core/sail/lmdb/src/main/java/org/eclipse/rdf4j/sail/lmdb/LmdbStore.java
#	core/sail/lmdb/src/main/java/org/eclipse/rdf4j/sail/lmdb/TripleStore.java
HIGHLY_CONNECTED    +56.6%
PHARMA              +54.6%
MEDICAL_RECORDS      -2.0%
ENGINEERING         -26.7%
TRAIN               -54.6%
LIBRARY             -55.0%
ELECTRICAL_GRID     -57.9%
@hmottestad
Copy link
Copy Markdown
Contributor Author

Worst query is about 8x slower than on develop. Best query is 1817x faster :)

latest-vs-develop-theme-query-compare

@hmottestad
Copy link
Copy Markdown
Contributor Author

Seems that the overhead is around 0.2 ms, so any query that is faster than that on develop will be slower on this branch.

@kenwenzel
Copy link
Copy Markdown
Member

The results look a lot more balanced in comparison to the develop branch.
Especially the very slow queries disappeared completely.
I wonder how much additional memory (RAM and harddisk) is required.

@hmottestad
Copy link
Copy Markdown
Contributor Author

This now uses mmap, so you don't need as much memory as before. It'll just page out. But you do need memory for the actual sketch objects and maps and lists, so I think it still requires a more than it should.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants