docker compose for one read-write and many read-only instances

[ticapix]

Problem description

Hello,

I'm using RDF4J as a database backend for a web+api application written in python.
RDF4J is deployed using docker compose.

I'm using the SAIL API to create the repositories and then the SPARQL endpoint to DELETE/INSERT and ASK/SELECT/CONSTRUCT.

Most requests to the DB are ReadOnly and I'm wondering how I could have one RW instance and several RO instances, using the same filesystem.

I'm trying with this docker compose

services:
  rdf4j-readwrite:
    image: eclipse/rdf4j-workbench:5.2.0-tomcat
    volumes:
      - rdf4j_data:/var/rdf4j:rw

  rdf4j-readonly:
    image: eclipse/rdf4j-workbench:5.2.0-tomcat
    volumes:
      - rdf4j_data:/var/rdf4j:ro
      - rdf4j_logs_readonly:/var/rdf4j/server/logs
    deploy:
      replicas: 3

volumes:
  rdf4j_data:
  rdf4j_logs_readonly:

I'm getting this error on the RO instance:

javax.servlet.ServletException: org.eclipse.rdf4j.repository.RepositoryLockedException: SAIL is already locked by: 1@5c8cc1fa0d60 in /var/rdf4j/workbench/queries
	org.eclipse.rdf4j.workbench.commands.QueryServlet.init(QueryServlet.java:127)
	org.eclipse.rdf4j.workbench.proxy.ProxyRepositoryServlet.createServlet(ProxyRepositoryServlet.java:127)
	org.eclipse.rdf4j.workbench.proxy.ProxyRepositoryServlet.init(ProxyRepositoryServlet.java:55)
	org.eclipse.rdf4j.workbench.proxy.WorkbenchServlet.service(WorkbenchServlet.java:230)
	org.eclipse.rdf4j.workbench.proxy.WorkbenchServlet.handleRequest(WorkbenchServlet.java:136)
	org.eclipse.rdf4j.workbench.proxy.WorkbenchServlet.service(WorkbenchServlet.java:111)
	org.eclipse.rdf4j.workbench.proxy.WorkbenchGateway.service(WorkbenchGateway.java:119)
	org.eclipse.rdf4j.workbench.base.AbstractServlet.service(AbstractServlet.java:129)
	org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:115)
	org.eclipse.rdf4j.workbench.proxy.CacheFilter.doFilter(CacheFilter.java:64)
	org.eclipse.rdf4j.workbench.proxy.CookieCacheControlFilter.doFilter(CookieCacheControlFilter.java:56)

Is this scenario is at all supported by RDF4J ?

Best regards,
Pierre

Preferred solution

A tutorial for one RW and several RO instances on the same host, sharing the filesystem where data is stored.

Are you interested in contributing a solution yourself?

I don't have much JAVA experience, but I can test deployment variation.

Alternatives you've considered

No response

Anything else?

No response

[hmottestad]

No, none of our backends are designed to share storage. They all assume that the storage is dedicated to the instance.

What is your reason for wanting to run multiple instances of RDF4J on the same host on the same db files? Are you running into performance issues with RDF4J when you only run a single instance?

[ticapix]

No, none of our backends are designed to share storage. They all assume that the storage is dedicated to the instance.

Ok, thank you for the info.

What is your reason for wanting to run multiple instances of RDF4J on the same host on the same db files? Are you running into performance issues with RDF4J when you only run a single instance?

No performance issue so far. We have a small db (<300k tuples) and a low number of concurrent SPARQL requests.
Even under load with ab -n 50000 -c 1000 .. it works fine.

At this point in time, it's more an exercise for later and to better understand how to operate a RDF4J DB in production.
(I'm used to the 1x RW many RO with postgresql)

1. Should I deploy it on a VM and backup/restore the VM ? (that's heavy)
2. Should I deploy it with docker and backup/restore the docker volume ? (that's more subtle but what is exact minimal mounting point /var/rdf4j, /var/rdf4j/server/repositories, ...)
3. Other

[ticapix]

Thank you @hmottestad

To keep a clean backlog, the initial question been answered, I'm closing the ticket.