Skip to content

feat(sei-db): expose state-store pebbledb block cache size as a config option#3623

Open
kylee9612 wants to merge 4 commits into
sei-protocol:mainfrom
kylee9612:feat/ss-pebble-cache-size-config
Open

feat(sei-db): expose state-store pebbledb block cache size as a config option#3623
kylee9612 wants to merge 4 commits into
sei-protocol:mainfrom
kylee9612:feat/ss-pebble-cache-size-config

Conversation

@kylee9612

Copy link
Copy Markdown

Problem

Historical EVM queries (debug_traceBlockByNumber,
debug_traceTransaction) are slow under heavy trace load.

Root cause

These reads bypass the in-memory memIAVL (SC) layer and are
served directly from the state-store (SS) pebbledb backend, whose block
cache is hardcoded to 32 MiB in sei-db/db_engine/pebbledb/mvcc/db.go
(pebble.NewCache(1024 * 1024 * 32)). For trace-heavy workloads this is too
small, so reads become disk-bound and there is no way to tune it without
recompiling.

Fix

Expose the cache size as ss-cache-size-bytes in the
[state-store] config. Default stays at 32 MiB (the previous hardcoded
value); values <= 0 fall back to the default, so existing configs are
unaffected. Type is int64 to match pebble.NewCache(int64).

@cursor

cursor Bot commented Jun 23, 2026

Copy link
Copy Markdown

PR Summary

Low Risk
Operational tuning only; default cache size matches the previous hardcoded value and does not change on-disk format or query semantics.

Overview
Makes the state-store Pebble block cache tunable via config instead of a fixed 32 MiB in OpenDB.

Adds CacheSizeBytes on StateStoreConfig (default 32 MiB, <= 0 uses the default) and documents it in the generated TOML as ss-cache-size-bytes. mvcc.OpenDB now passes that value into pebble.NewCache so operators can raise cache size for trace-heavy historical reads without rebuilding.

Also renames the config import in mvcc/db.go to seidbconfig (no behavior change).

Reviewed by Cursor Bugbot for commit 434de74. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread sei-db/config/ss_config.go Outdated
Comment thread sei-db/config/ss_config.go Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ee4960b. Configure here.

Comment thread sei-db/config/toml.go
@yzang2019

Copy link
Copy Markdown
Contributor

Thanks for the contribution, do you have any evidence or any data to show that increasing block cache size would help making debug_trace query faster? If so, how much faster? Would appreciate to link some test result here. (We've internally tried that and not seeing much improvement)

We have other ways of improving debug_trace, for example you can switch to RocksDB backend, which gives you 2x perf improvement

@kylee9612

Copy link
Copy Markdown
Author

Thanks for the contribution, do you have any evidence or any data to show that increasing block cache size would help making debug_trace query faster? If so, how much faster? Would appreciate to link some test result here. (We've internally tried that and not seeing much improvement)

We have other ways of improving debug_trace, for example you can switch to RocksDB backend, which gives you 2x perf improvement

Thanks for the feedback. I re-ran a controlled comparison and do see a consistent improvement, so sharing the numbers.

Setup: two nodes, identical hardware(AWS i7i.4xlarge) , same chain height, tracing the same block (debug_traceBlockByNumber with callTracer). Only difference is the SS pebbledb block cache size. Each value is the average of 11 calls.

SS block cache avg latency
32 MiB (current default) ~1769 ms
4 GiB (ss-cache-size-bytes) ~403 ms

Roughly a 4x speedup, with cache size as the only variable. The gap shows up on dense / trace-heavy blocks where the working set exceeds 32 MiB.

We're not considering the RocksDB backend at the moment, and in our testing simply increasing the cache size was the most effective option. The default stays at 32 MiB (<= 0 falls back), so existing deployments are unaffected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants