Skip to content

Full RocksDB compaction causes OOM kill — memory usage undocumented #215

@0xCMars

Description

@0xCMars

The periodic full compaction on the history RocksDB database consumes significantly more memory than normal operation, often leading to OOM kills on machines that otherwise have plenty of RAM for steady-state indexing. The memory requirements for full compaction are not documented anywhere.
Environment

electrs version: built from source
OS: Ubuntu 24, systemd managed
RAM: 16 GB
Mode: --lightmode with --jsonrpc-import
RocksDB settings: --db-write-buffer-size-mb 64, --db-block-cache-mb 128, --db-parallelism 1

Relevant logs

INFO starting full compaction on RocksDB { path: "/home/ubuntu/.electrs/db/mainnet/newindex/history" }
electrs.service: A process of this unit has been killed by the OOM killer.
electrs.service: Failed with result 'oom-kill'.
electrs.service: Consumed 1h 22min 3.542s CPU time, 15G memory peak.

I tried to resolve this by Setting systemd MemoryHigh=10G and MemoryMax=12G to throttle rather than OOM-kill. But it seem like it will never end the compaction process:

May 07 06:16:14 btc-node electrs[234837]: INFO Salt rotated
May 06 09:24:26 btc-node electrs[234837]: INFO starting full compaction on RocksDB { path: "/home/ubuntu/.electrs/db/mainnet/newindex/history" }
May 06 06:16:33 btc-node electrs[234837]: INFO processing blocks 948088/948125 (100.0%)
May 06 06:16:29 btc-node electrs[234837]: INFO fetching blocks 948088/948125 (100.0%)

Questions

  1. How much memory does full compaction actually require?
  2. Would it be possible to document minimum memory requirements for full compaction, especially for mainnet?

THX!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions