bug (?) : FILTER NOT EXISTS size limit

[YvanDesPatates]

Hello everyone, I am working for opensilex. We are working on a Java API that use RDF4J as database. I'm writing here in order to know if my issue is an issue, if I should open a GitHub issue, or if the only issue is the way I'm using SPARQL queries 😆

Description

I recently changed the way we delete things. Doing this, I ended up with a smaller request but a huge FILTER NOT EXISTS field that should avoid deleting some triples (because some of our data models have an IgnoreUpdateIfNull annotation on some fields).

I know the issues do not come from the query size. I had the issue on a query with 362 504 chars while I am able to perform simple CREATE queries with more than a million chars.

RDF4J version

I am using this RDF4J docker image : eclipse/rdf4j-workbench:amd64-3.4.3

Error

RDF4J server response :

Transaction handling error: java.util.concurrent.ExecutionException: java.lang.StackOverflowError

Query

This is a very short example of the FILTER NOT EXISTS field to understand the huge query above.

{ FILTER NOT EXISTS
    { GRAPH <http://phenome.inrae.fr/m3p/set/germplasm>
      { ?uriToDelete  ?p  ?o
        FILTER (
          ( ?uriToDelete = "http://falseuri" && ?p = "http://www.w3.org/2000/01/rdf-schema#label" )
          || ( ?uriToDelete = "http://falseuri" && ?p = "http://www.opensilex.org/vocabulary/oeso#hasId" )
        )
      }
    }
 }

The entire query:

full_request_formatted.txt

I'm sorry for this awful file with a lot of horizontal scrolling.

Expectation

As I told you at the beginning of these messages, I want to know if you consider it as an issue or if you think that this type of query is too much for RDF4J and so I should change the way I handle my needs. I already have some ideas to prevent this problem (batching, using many smaller filters instead of one big one...), but I was thinking that others should probably have the same issue one day.

Thank you for reading !
__
Yvan

[hmottestad]

I believe this was fixed fairly recently. Is this still reproducible with 5.2.1?

A workaround would be to increase your stack size: https://www.baeldung.com/jvm-configure-stack-sizes

[YvanDesPatates]

I confirm this is still reproducible with the 5.2.1-tomcat docker version. I have just tested this morning.

Thank you for the workaround. I will try this. I just hope this will be ok for the devops team to increase JVM stack size. Thank you again.

[YvanDesPatates]

I finally managed to work with the workaround. Here is my new RDF4J section of my Docker Compose file. I set 2Mo instead of 1Mo stack size and it is enough for now.

    rdf4j:
        container_name: opensilex-rdf4j
        image: eclipse/rdf4j-workbench:5.2.1-tomcat
        networks:
            - backend
        ports:
            - "8667:8080"
        restart: always
        volumes:
            - rdf4j_data:/var/rdf4j
        command: [ "/bin/sh", "-c", "export JAVA_OPTS='-Dorg.eclipse.rdf4j.appdata.basedir=/var/rdf4j/ -Xss2m'; exec catalina.sh run" ]

[JervenBolleman]

Hi @YvanDesPatates besides the issue with the stack size. Perhaps changing the first FILTER IN into an VALUES clause might give better performance. I suspect that the second filter can then be much smaller as an expression.

[YvanDesPatates]

The point is that we have decided to use Jena ARQ for query building. However, Jena ARQ only handles the VALUES clause in a SELECT query. I really don't understand why, and it's starting to cause problems with complex queries and optimisation. Otherwise, I could also use VALUES to filter by a list of predicate bindings to values of URIs.

[afs]

However, Jena ARQ only handles the VALUES clause in a SELECT query.

It should work in an update. Please file a report with a minimal :-), complete example.

(Note: the handling of named graphs is different in Jena.)

[YvanDesPatates]

I have to admit that we are not at the latest version of Jena. We are 4.9 and we haven't taken time yet to upgrade to 5.x due to breaking changes.

here is the minimal example :

        Var s = Converters.makeVar("s");
        Node[] uris = new Node[] {
                NodeFactory.createURI("my:uri1"),
                NodeFactory.createURI("my:uri2")
        };

        WhereBuilder where = new WhereBuilder();
        where.addWhereValueVar(s, uris);
        where.addWhere("?s", "?p", "?o");

        UpdateBuilder update = new UpdateBuilder();
        update.addDelete("?s", "?p", "?o");
        update.addWhere(where);

        System.out.println(where);
        System.out.println("----------------------------------------");
        System.out.println(update.buildRequest());

expected output :

WHERE
  { ?s  ?p  ?o
    VALUES ?s { <my:uri1> <my:uri2> }
  }

----------------------------------------
DELETE {
  ?s ?p ?o .
}
WHERE
  { ?s  ?p  ?o }

As you see the VALUES clause magically disappear

[JervenBolleman]

@YvanDesPatates @afs maybe it is best that Jena bugs are discussed at https://github.com/apache/jena/issues

[YvanDesPatates]

Good idea. For the Jena topic see here @afs : apache/jena#3656

About the initial RDF4J topic. The stack overflow error is still reproducible on 5.2.1 as I mentioned before. Do I have to create an issue ? Is it something the team wants to work on ?

I confirm that increasing JVM stack size works.