From 466fd1b061ec3203a89ebe06a9bbd56a2199ab87 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Tue, 16 Jun 2026 09:24:53 -0700 Subject: [PATCH 1/5] feat: update docs even more --- docs/client-drivers.md | 26 ++- docs/configuration/index.md | 23 ++- docs/configuration/pgdog.toml/admin.md | 13 +- docs/configuration/pgdog.toml/control.md | 30 ++- docs/configuration/pgdog.toml/databases.md | 42 ++-- docs/configuration/pgdog.toml/memory.md | 19 +- docs/configuration/pgdog.toml/mirroring.md | 23 ++- docs/configuration/pgdog.toml/network.md | 25 ++- docs/configuration/pgdog.toml/plugins.md | 35 ++-- docs/configuration/pgdog.toml/rewrite.md | 23 ++- .../pgdog.toml/sharded_schemas.md | 50 +++-- .../pgdog.toml/sharded_tables.md | 155 ++++++++++----- docs/configuration/users.toml/users.md | 33 ++-- docs/enterprise_edition/plans.md | 7 +- docs/enterprise_edition/schema.md | 13 +- docs/features/authentication.md | 39 ++-- docs/features/connection-recovery.md | 39 ++-- docs/features/load-balancer/healthchecks.md | 78 +++++--- docs/features/load-balancer/index.md | 98 +++++++--- docs/features/load-balancer/manual-routing.md | 13 +- .../load-balancer/replication-failover.md | 74 ++++--- docs/features/multi-tenancy.md | 127 ++++++++---- docs/features/plugins/index.md | 28 ++- docs/features/prepared-statements.md | 39 ++-- docs/features/pub_sub.md | 13 +- docs/features/sharding/2pc.md | 26 ++- .../sharding/cross-shard-queries/index.md | 13 +- .../sharding/cross-shard-queries/insert.md | 34 +++- .../sharding/cross-shard-queries/update.md | 17 +- docs/features/sharding/dry-run.md | 13 +- docs/features/sharding/explain.md | 13 +- docs/features/sharding/omnishards.md | 141 +++++++++----- docs/features/sharding/query-routing.md | 43 ++-- docs/features/sharding/resharding/cutover.md | 58 ++++-- docs/features/sharding/resharding/move.md | 98 +++++++--- docs/features/sharding/resharding/schema.md | 22 ++- .../sharding/schema_management/cache.md | 17 +- docs/features/sharding/sharding-functions.md | 183 ++++++++++++------ docs/features/sharding/unique-ids.md | 26 ++- docs/features/tls.md | 42 ++-- docs/features/transaction-mode.md | 96 ++++++--- docs/installation.md | 37 ++-- docs/migrating-to-pgdog/from-pgbouncer.md | 60 +++++- 43 files changed, 1395 insertions(+), 609 deletions(-) diff --git a/docs/client-drivers.md b/docs/client-drivers.md index 6a328bca..6ca59429 100644 --- a/docs/client-drivers.md +++ b/docs/client-drivers.md @@ -50,10 +50,15 @@ Connection conn = DriverManager.getConnection(url, "user", "password"); This doesn't cause any query routing issues; however, PgDog can't effectively cache the query syntax tree and has to parse queries _every time_ they are executed. This is computationally expensive. Consider switching to `psycopg` (version 3) or enabling our Rust-native query parser: -```toml -[general] -query_parser_engine = "pg_query_raw" -``` +=== "pgdog.toml" + ```toml + [general] + query_parser_engine = "pg_query_raw" + ``` +=== "Helm chart" + ```yaml + queryParserEngine: pg_query_raw + ``` We benchmarked this to be 5 times faster than normal `pg_query` parsing, which should help. @@ -61,7 +66,12 @@ We benchmarked this to be 5 times faster than normal `pg_query` parsing, which s Prisma doesn't correctly use the `IN` clause with arrays, causing it to generate a very large number of unique prepared statements. This is not a big problem, but if left unchecked, can cause heavy memory usage in PgDog. Consider setting a lower prepared statements [cache limit](features/prepared-statements.md#cache-limit): -```toml -[general] -prepared_statements_limit = 1_000 -``` +=== "pgdog.toml" + ```toml + [general] + prepared_statements_limit = 1_000 + ``` +=== "Helm chart" + ```yaml + preparedStatementsLimit: 1_000 + ``` diff --git a/docs/configuration/index.md b/docs/configuration/index.md index a59ce844..9364626b 100644 --- a/docs/configuration/index.md +++ b/docs/configuration/index.md @@ -24,6 +24,16 @@ PgDog uses the [TOML](https://toml.io/en/) configuration language for its two co password = "pgdog" database = "pgdog" ``` +=== "Helm chart" + ```yaml + databases: + - name: pgdog + host: 127.0.0.1 + users: + - name: pgdog + password: pgdog + database: pgdog + ``` By default, PgDog looks for both configuration files in the current working directory (`$PWD`). Alternatively, you can pass the `--config=` and `--users=` arguments on startup. @@ -47,10 +57,15 @@ Hot reload can be triggered by sending `SIGHUP` to the `pgdog` process or by con To make things simpler, all units of time are in milliseconds. For example, if you want to set the pool checkout timeout to 5 seconds, convert it to 5000ms instead: -```toml -[general] -checkout_timeout = 5_000 -``` +=== "pgdog.toml" + ```toml + [general] + checkout_timeout = 5_000 + ``` +=== "Helm chart" + ```yaml + checkoutTimeout: 5_000 + ``` !!! note "TOML syntax" Since PgDog uses TOML, both `5000` and `5_000` are valid numbers. Configuration will fail to load if non-integer values are used, e.g. "5s" or "53.5". diff --git a/docs/configuration/pgdog.toml/admin.md b/docs/configuration/pgdog.toml/admin.md index 97b4785b..64b033b6 100644 --- a/docs/configuration/pgdog.toml/admin.md +++ b/docs/configuration/pgdog.toml/admin.md @@ -7,10 +7,15 @@ icon: material/security Admin database settings control access to the [admin](../../administration/index.md) database which contains real time statistics about internal operations of PgDog. For example: -```toml -[admin] -password = "hunter2" -``` +=== "pgdog.toml" + ```toml + [admin] + password = "hunter2" + ``` +=== "Helm chart" + ```yaml + adminPassword: "hunter2" + ``` ### `name` diff --git a/docs/configuration/pgdog.toml/control.md b/docs/configuration/pgdog.toml/control.md index 2429e97c..ca4d0fbd 100644 --- a/docs/configuration/pgdog.toml/control.md +++ b/docs/configuration/pgdog.toml/control.md @@ -9,15 +9,27 @@ Control settings configure PgDog's connection to the PgDog control plane. !!! note "Enterprise edition" This feature is available in [Enterprise Edition](../../enterprise_edition/index.md) only. -```toml -[control] -endpoint = "http://localhost:8080" -token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" -metrics_interval = 1000 -stats_interval = 5000 -active_queries_interval = 5000 -request_timeout = 1000 -``` +=== "pgdog.toml" + ```toml + [control] + endpoint = "http://localhost:8080" + token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + metrics_interval = 1000 + stats_interval = 5000 + active_queries_interval = 5000 + request_timeout = 1000 + ``` +=== "Helm chart" + ```yaml + control: + enabled: true + endpoint: "http://localhost:8080" + token: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + metricsInterval: 1000 + statsInterval: 5000 + activeQueriesInterval: 5000 + requestTimeout: 1000 + ``` ### `endpoint` diff --git a/docs/configuration/pgdog.toml/databases.md b/docs/configuration/pgdog.toml/databases.md index 37f8e2e2..30f7ad8b 100644 --- a/docs/configuration/pgdog.toml/databases.md +++ b/docs/configuration/pgdog.toml/databases.md @@ -7,20 +7,34 @@ Database settings configure which databases PgDog is managing. This is a TOML li For each database instance, add a `[[databases]]` entry to `pgdog.toml`. For example: -```toml -[[databases]] -name = "prod" -host = "10.0.0.1" -port = 5432 -shard = 0 - -[[databases]] -name = "prod" -host = "10.0.0.2" -port = 5432 -role = "replica" -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "prod" + host = "10.0.0.1" + port = 5432 + shard = 0 + + [[databases]] + name = "prod" + host = "10.0.0.2" + port = 5432 + role = "replica" + shard = 0 + ``` +=== "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + port: 5432 + shard: 0 + - name: prod + host: 10.0.0.2 + port: 5432 + role: replica + shard: 0 + ``` ### `name` diff --git a/docs/configuration/pgdog.toml/memory.md b/docs/configuration/pgdog.toml/memory.md index 0d36a3e8..74b61e9e 100644 --- a/docs/configuration/pgdog.toml/memory.md +++ b/docs/configuration/pgdog.toml/memory.md @@ -6,12 +6,19 @@ icon: material/memory Memory settings control buffer sizes used by PgDog for network I/O and task execution. -```toml -[memory] -net_buffer = 4096 -message_buffer = 4096 -stack_size = 2097152 -``` +=== "pgdog.toml" + ```toml + [memory] + net_buffer = 4096 + message_buffer = 4096 + stack_size = 2097152 + ``` +=== "Helm chart" + ```yaml + memoryNetBuffer: 4096 + memoryMessageBuffer: 4096 + memoryStackSize: 2097152 + ``` ### `net_buffer` diff --git a/docs/configuration/pgdog.toml/mirroring.md b/docs/configuration/pgdog.toml/mirroring.md index 813fd346..d38ef2a1 100644 --- a/docs/configuration/pgdog.toml/mirroring.md +++ b/docs/configuration/pgdog.toml/mirroring.md @@ -8,13 +8,22 @@ icon: material/mirror-rectangle For example: -```toml -[[mirroring]] -source_db = "source" -destination_db = "dest" -queue_length = 500 -exposure = 0.1 -``` +=== "pgdog.toml" + ```toml + [[mirroring]] + source_db = "source" + destination_db = "dest" + queue_length = 500 + exposure = 0.1 + ``` +=== "Helm chart" + ```yaml + mirrors: + - sourceDb: source + destinationDb: dest + queueLength: 500 + exposure: 0.1 + ``` ### `source_db` diff --git a/docs/configuration/pgdog.toml/network.md b/docs/configuration/pgdog.toml/network.md index 6dbf7cba..8accf0ab 100644 --- a/docs/configuration/pgdog.toml/network.md +++ b/docs/configuration/pgdog.toml/network.md @@ -6,14 +6,23 @@ icon: material/network PgDog speaks the Postgres protocol which, underneath, uses TCP. Optimal TCP settings are necessary to quickly recover from database incidents. For example: -```toml -[tcp] -keepalive = true -time = 60_000 -interval = 60_000 -retries = 3 -user_timeout = 5_000 -``` +=== "pgdog.toml" + ```toml + [tcp] + keepalive = true + time = 60_000 + interval = 60_000 + retries = 3 + user_timeout = 5_000 + ``` +=== "Helm chart" + ```yaml + tcpKeepalive: true + tcpTime: 60_000 + tcpInterval: 60_000 + tcpRetries: 3 + tcpUserTimeout: 5_000 + ``` To be consistent with the rest of PgDog documentation, units of time are in milliseconds. However, many TCP implementations only support seconds. Consider using round units, e.g., `1_000` milliseconds = 1 second. diff --git a/docs/configuration/pgdog.toml/plugins.md b/docs/configuration/pgdog.toml/plugins.md index 62a162b1..acb525da 100644 --- a/docs/configuration/pgdog.toml/plugins.md +++ b/docs/configuration/pgdog.toml/plugins.md @@ -8,13 +8,20 @@ icon: material/power-plug Plugins are a TOML list, so for each plugin you want to enable, add a `[[plugins]]` entry to `pgdog.toml`. For example: -```toml -[[plugins]] -name = "bob_router" - -[[plugins]] -name = "alice_router" -``` +=== "pgdog.toml" + ```toml + [[plugins]] + name = "bob_router" + + [[plugins]] + name = "alice_router" + ``` +=== "Helm chart" + ```yaml + plugins: + - name: bob_router + - name: alice_router + ``` !!! note Plugins can only be configured at PgDog startup. They cannot be changed after @@ -27,9 +34,15 @@ name is `router`, PgDog will look for `librouter.so` on Linux, `librouter.dll` o Additionally, you can pass the relative or absolute path to the shared library itself: -```toml -[[plugins]] -name = "/opt/plugins/librouter.so" -``` +=== "pgdog.toml" + ```toml + [[plugins]] + name = "/opt/plugins/librouter.so" + ``` +=== "Helm chart" + ```yaml + plugins: + - name: /opt/plugins/librouter.so + ``` Make sure the user running PgDog has read & execute permissions on the library. diff --git a/docs/configuration/pgdog.toml/rewrite.md b/docs/configuration/pgdog.toml/rewrite.md index db3396cd..d68748e4 100644 --- a/docs/configuration/pgdog.toml/rewrite.md +++ b/docs/configuration/pgdog.toml/rewrite.md @@ -6,13 +6,22 @@ icon: material/alpha-r-box-outline The `rewrite` section controls PgDog's automatic SQL rewrites for sharded databases. It affects sharding key updates and multi-tuple inserts. Either one can be toggled separately: -```toml -[rewrite] -enabled = false -shard_key = "error" -split_inserts = "error" -primary_key = "ignore" -``` +=== "pgdog.toml" + ```toml + [rewrite] + enabled = false + shard_key = "error" + split_inserts = "error" + primary_key = "ignore" + ``` +=== "Helm chart" + ```yaml + rewrite: + enabled: false + shardKey: "error" + splitInserts: "error" + primaryKey: "ignore" + ``` | Setting | Description | Default | | --- | --- | --- | diff --git a/docs/configuration/pgdog.toml/sharded_schemas.md b/docs/configuration/pgdog.toml/sharded_schemas.md index 3a84b3d0..cfe83cf9 100644 --- a/docs/configuration/pgdog.toml/sharded_schemas.md +++ b/docs/configuration/pgdog.toml/sharded_schemas.md @@ -10,17 +10,28 @@ icon: material/set-split Each schema needs to have a shard mapping in the config, for example: -```toml -[[sharded_schemas]] -database = "prod" -name = "customer_a" -shard = 0 - -[[sharded_schemas]] -database = "prod" -name = "customer_b" -shard = 1 -``` +=== "pgdog.toml" + ```toml + [[sharded_schemas]] + database = "prod" + name = "customer_a" + shard = 0 + + [[sharded_schemas]] + database = "prod" + name = "customer_b" + shard = 1 + ``` +=== "Helm chart" + ```yaml + shardedSchemas: + - database: prod + name: customer_a + shard: 0 + - database: prod + name: customer_b + shard: 1 + ``` All queries that fully qualify the table names will be routed correctly, for example: @@ -35,11 +46,18 @@ You can add multiple entries per database. Mappings are matched by schema name f For queries that don't specify a schema or for which a mapping doesn't exist, the default behavior is to send it to all shards. If this is not desirable, add an entry without a `name` to choose a default shard: -```toml -[[sharded_schemas]] -database = "prod" -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_schemas]] + database = "prod" + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedSchemas: + - database: prod + shard: 0 + ``` PgDog now sends any unmapped schema to shard zero, including plain references (`SELECT * FROM pg_stat_activity`) and schemas created after the mapping file was generated. diff --git a/docs/configuration/pgdog.toml/sharded_tables.md b/docs/configuration/pgdog.toml/sharded_tables.md index 35a86f2a..e40d9d9f 100644 --- a/docs/configuration/pgdog.toml/sharded_tables.md +++ b/docs/configuration/pgdog.toml/sharded_tables.md @@ -12,13 +12,22 @@ To detect and route queries with sharding keys, PgDog expects the sharded column The following configuration will match queries referring to this exact table and column exclusively: -```toml -[[sharded_tables]] -database = "prod" -name = "users" -column = "id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + name = "users" + column = "id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + name: users + column: id + dataType: bigint + ``` The table `users` is sharded on the column `id`, which has the data type `BIGINT`. Queries that reference that column will be automatically routed to one or more of the shards: @@ -32,12 +41,20 @@ WHERE users.id = $1 The following configuration will match queries referring to this column, irrespective of table name: -```toml -[[sharded_tables]] -database = "prod" -column = "user_id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + dataType: bigint + ``` In this example, the table name is omitted so all tables that have the `user_id` column (data type `BIGINT`) will be routed automatically to the right shard(s): @@ -109,17 +126,29 @@ By default, all tables unless otherwise configured as sharded, are considered om Sticky routing disables round robin for omnisharded tables and sends the queries touching those tables to the same shard, guaranteeing consistent results for the duration of a client's connection: -```toml -[[omnisharded_tables]] -database = "prod" -sticky = true -tables = [ - "settings", - "cities", - "terms_of_service", - "ip_blocks", -] -``` +=== "pgdog.toml" + ```toml + [[omnisharded_tables]] + database = "prod" + sticky = true + tables = [ + "settings", + "cities", + "terms_of_service", + "ip_blocks", + ] + ``` +=== "Helm chart" + ```yaml + omnishardedTables: + - database: prod + sticky: true + tables: + - settings + - cities + - terms_of_service + - ip_blocks + ``` All queries referencing only these tables will be sent to one of the shards, using the round robin algorithm. If the query contains a sharding key, it will be used instead and omnisharded tables will be ignored by the query router. @@ -184,14 +213,24 @@ The target shard number for matched queries. Lists are defined as a list of values and a corresponding shard number. Just like sharded tables, the mapping is database and column (and optionally, table and schema) specific: -```toml -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "list" -values = [1, 2, 3, 4, 5] -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "list" + values = [1, 2, 3, 4, 5] + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: tenant_id + kind: list + values: [1, 2, 3, 4, 5] + shard: 0 + ``` All queries that match the values defined in the mapping will be routed to that specific shard, for example: @@ -204,15 +243,26 @@ WHERE tenant_id = 4 AND user_id = 1235 Ranges are defined with a starting value (included) and the end value (excluded), just like `PARTITION BY RANGE` in Postgres: -```toml -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "range" -start = 1 -end = 100 -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "range" + start = 1 + end = 100 + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: tenant_id + kind: range + start: 1 + end: 100 + shard: 0 + ``` All sharding key values matching the range will be routed to the specified shard: @@ -225,13 +275,22 @@ WHERE tenant_id IN (1, 2, 5, 10, 56) The `default` kind specifies a fallback shard for values that don't match any list mapping: -```toml -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "default" -shard = 2 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "default" + shard = 2 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: tenant_id + kind: default + shard: 2 + ``` Any sharding key value that doesn't match an explicit list mapping will be routed to the default shard. diff --git a/docs/configuration/users.toml/users.md b/docs/configuration/users.toml/users.md index 905ac8d8..6ba258d3 100644 --- a/docs/configuration/users.toml/users.md +++ b/docs/configuration/users.toml/users.md @@ -6,17 +6,28 @@ icon: material/account-key This configuration controls which users are allowed to connect to PgDog. This is a TOML list so for each user, add a `[[users]]` section to `users.toml`. For example: -```toml -[[users]] -name = "alice" -database = "prod" -password = "hunter2" - -[[users]] -name = "bob" -database = "prod" -password = "opensesame" -``` +=== "users.toml" + ```toml + [[users]] + name = "alice" + database = "prod" + password = "hunter2" + + [[users]] + name = "bob" + database = "prod" + password = "opensesame" + ``` +=== "Helm chart" + ```yaml + users: + - name: alice + database: prod + password: hunter2 + - name: bob + database: prod + password: opensesame + ``` ### `name` diff --git a/docs/enterprise_edition/plans.md b/docs/enterprise_edition/plans.md index 41e388a4..346b7229 100644 --- a/docs/enterprise_edition/plans.md +++ b/docs/enterprise_edition/plans.md @@ -22,7 +22,7 @@ Query plan collection is disabled by default and can be enabled in `pgdog.toml`: ```yaml queryStats: enabled: true - queryPlansThreshold: 100 + queryPlanThreshold: 100 ``` The plans are stored in memory and deduplicated by _normalized_ SQL of the query. This means that queries with different parameters will be tied to the same query plan, making sure PgDog doesn't plan the same query multiple times. @@ -53,10 +53,9 @@ The query plan cache is configurable: ```yaml queryStats: enabled: true - queryPlansThreshold: 100 - queryPlansSampleRate: 0.5 + queryPlanThreshold: 100 queryPlansCache: 1000 - queryPlansMaxAge: 15000 + queryPlanMaxAge: 15000 ``` ### Admin database diff --git a/docs/enterprise_edition/schema.md b/docs/enterprise_edition/schema.md index f97e2d56..50565a3a 100644 --- a/docs/enterprise_edition/schema.md +++ b/docs/enterprise_edition/schema.md @@ -22,9 +22,14 @@ This process takes place in the background, so the schema snapshot across nodes Schema reloading on DDL is enabled by default. This is configurable in [`pgdog.toml`](../configuration/pgdog.toml/general.md#reload_schema_on_ddl): -```toml -[general] -reload_schema_on_ddl = true -``` +=== "pgdog.toml" + ```toml + [general] + reload_schema_on_ddl = true + ``` +=== "Helm chart" + ```yaml + reloadSchemaOnDdl: true + ``` In the open source edition, this reloads the schema on the same node only, while in the Enterprise edition, this triggers a schema reload on all PgDog nodes part of the same deployment. diff --git a/docs/features/authentication.md b/docs/features/authentication.md index eb1d8891..b20320ad 100644 --- a/docs/features/authentication.md +++ b/docs/features/authentication.md @@ -25,10 +25,15 @@ PgDog implements a subset of authentication methods supported by Postgres. We're By default, client connections will use `scram-sha-256` for password encryption during the authentication handshake. This method is secure and recommended in production. PgDog does support using others, and you can change it with configuration: -```toml -[general] -auth_type = "scram" -``` +=== "pgdog.toml" + ```toml + [general] + auth_type = "scram" + ``` +=== "Helm chart" + ```yaml + authType: scram + ``` Available options currently are: @@ -100,10 +105,15 @@ Passthrough authentication is a feature where instead of storing passwords in `u Passthrough authentication is disabled by default and can be enabled with configuration: -```toml -[general] -passthrough_auth = "enabled" -``` +=== "pgdog.toml" + ```toml + [general] + passthrough_auth = "enabled" + ``` +=== "Helm chart" + ```yaml + passthroughAuth: enabled + ``` This will require clients to send passwords in plain text. PgDog will create a connection pool for the database/user pair and the provided password. The database must exist in [`pgdog.toml`](../configuration/pgdog.toml/databases.md). @@ -115,10 +125,15 @@ Sending plain text passwords over unencrypted connections isn't ideal, even if P If you don't want to set up TLS (it has some impact on latency), you can override this behavior and send passwords via plain text (unsecured) connection: -```toml -[general] -passthrough_auth = "enabled_plain" -``` +=== "pgdog.toml" + ```toml + [general] + passthrough_auth = "enabled_plain" + ``` +=== "Helm chart" + ```yaml + passthroughAuth: enabled_plain + ``` ## Password security diff --git a/docs/features/connection-recovery.md b/docs/features/connection-recovery.md index df235bf5..1127fe40 100644 --- a/docs/features/connection-recovery.md +++ b/docs/features/connection-recovery.md @@ -71,10 +71,15 @@ Just like [abandoned transactions](#abandoned-transactions), this protects Postg Connection recovery is an optional feature, enabled by default. You can change how it behaves through configuration: -```toml -[general] -connection_recovery = "recover" -``` +=== "pgdog.toml" + ```toml + [general] + connection_recovery = "recover" + ``` +=== "Helm chart" + ```yaml + connectionRecovery: recover + ``` | Configuration value | Description | |-|-| @@ -84,10 +89,15 @@ connection_recovery = "recover" To make sure abandoned server connections don't block normal operations, PgDog supports a configurable timeout on the recovery operation. If connection recovery doesn't complete in time, the connection will be closed: -```toml -[general] -rollback_timeout = 5_000 -``` +=== "pgdog.toml" + ```toml + [general] + rollback_timeout = 5_000 + ``` +=== "Helm chart" + ```yaml + rollbackTimeout: 5_000 + ``` ## Client connections @@ -95,10 +105,15 @@ Just like server connections, PgDog can maintain client connections (application While enabled by default, some applications don't behave well when their queries return errors instead of results. Therefore, this feature is configurable and can be disabled: -```toml -[general] -client_connection_recovery = "drop" -``` +=== "pgdog.toml" + ```toml + [general] + client_connection_recovery = "drop" + ``` +=== "Helm chart" + ```yaml + clientConnectionRecovery: drop + ``` | Configuration value | Description | |-|-| diff --git a/docs/features/load-balancer/healthchecks.md b/docs/features/load-balancer/healthchecks.md index 3e1c999d..12f5a26d 100644 --- a/docs/features/load-balancer/healthchecks.md +++ b/docs/features/load-balancer/healthchecks.md @@ -39,10 +39,15 @@ If the health check query finishes successfully, the connection is marked health While the health check is cheap, running it on every single transaction is unnecessary and would cause undesirable latency. For this reason, the connection health check is performed once per configurable interval, controlled by the `healthcheck_interval` setting: -```toml -[general] -healthcheck_interval = 30_000 # Run a health check every 30 seconds -``` +=== "pgdog.toml" + ```toml + [general] + healthcheck_interval = 30_000 # Run a health check every 30 seconds + ``` +=== "Helm chart" + ```yaml + healthcheckInterval: 30_000 # Run a health check every 30 seconds + ``` The **default** value is **30 seconds** (`30_000` milliseconds). @@ -52,10 +57,15 @@ If your databases are relatively idle, connection health checks don't provide en The load balancer runs health check queries independently and asynchronously in the background. The frequency of background health checks is controlled by the `idle_healthcheck_interval` setting: -```toml -[general] -idle_healthcheck_interval = 30_000 # Run a health check every 30 seconds -``` +=== "pgdog.toml" + ```toml + [general] + idle_healthcheck_interval = 30_000 # Run a health check every 30 seconds + ``` +=== "Helm chart" + ```yaml + idleHealthcheckInterval: 30_000 # Run a health check every 30 seconds + ``` The **default** value for this setting is **30 seconds** (`30_000` milliseconds). @@ -63,10 +73,15 @@ The **default** value for this setting is **30 seconds** (`30_000` milliseconds) When PgDog is first started, it's possible that the database or the network is not yet ready to handle requests. To make sure there are no false positives caused by a slow start, database health checks are started after a configurable delay, controlled by the `idle_healthcheck_delay` setting: -```toml -[general] -idle_healthcheck_delay = 5_000 # 5 seconds -``` +=== "pgdog.toml" + ```toml + [general] + idle_healthcheck_delay = 5_000 # 5 seconds + ``` +=== "Helm chart" + ```yaml + idleHealthcheckDelay: 5_000 # 5 seconds + ``` The **default** value for this setting is **5 seconds** (`5_000` milliseconds). @@ -81,10 +96,15 @@ Databases are automatically put back into the load balancer after a period of ti The amount of time the database is banned from serving traffic is controlled with the `ban_timeout` setting: -```toml -[general] -ban_timeout = 300_000 # 5 minutes -``` +=== "pgdog.toml" + ```toml + [general] + ban_timeout = 300_000 # 5 minutes + ``` +=== "Helm chart" + ```yaml + banTimeout: 300_000 # 5 minutes + ``` The **default** value is **5 minutes** (`300_000` milliseconds). @@ -105,10 +125,15 @@ By default, the load balancer gives the database a limited amount of time to ans This behavior is configurable with the `healthcheck_timeout` setting: -```toml -[general] -healthcheck_timeout = 5_000 # 5 seconds in ms -``` +=== "pgdog.toml" + ```toml + [general] + healthcheck_timeout = 5_000 # 5 seconds in ms + ``` +=== "Helm chart" + ```yaml + healthcheckTimeout: 5_000 # 5 seconds in ms + ``` The default value is `5_000` (5 seconds). @@ -131,10 +156,15 @@ Since PgDog itself is a TCP application, no additional configuration is required If your load balancer supports sending HTTP health checks to a configurable port (like AWS NLBs, for example), you can configure PgDog to run an HTTP server to respond to them: -```toml -[general] -healthcheck_port = 8080 -``` +=== "pgdog.toml" + ```toml + [general] + healthcheck_port = 8080 + ``` +=== "Helm chart" + ```yaml + healthcheckPort: 8080 + ``` This is configurable on startup only and will spin up an HTTP server on `http://0.0.0.0:8080` (or whatever port you set). diff --git a/docs/features/load-balancer/index.md b/docs/features/load-balancer/index.md index 20c4afb8..78c6f6c9 100644 --- a/docs/features/load-balancer/index.md +++ b/docs/features/load-balancer/index.md @@ -46,10 +46,15 @@ This algorithm makes no assumptions about the capacity of each database or the c Round robin is used **by default**, so no config changes are required. You can still set it explicitly in [pgdog.toml](../../configuration/pgdog.toml/general.md), like so: -```toml -[general] -load_balancing_strategy = "round_robin" -``` +=== "pgdog.toml" + ```toml + [general] + load_balancing_strategy = "round_robin" + ``` +=== "Helm chart" + ```yaml + loadBalancingStrategy: round_robin + ``` ### Random @@ -59,10 +64,15 @@ This algorithm is often effective when queries have unpredictable runtime. By ra ##### Configuration -```toml -[general] -load_balancing_strategy = "random" -``` +=== "pgdog.toml" + ```toml + [general] + load_balancing_strategy = "random" + ``` +=== "Helm chart" + ```yaml + loadBalancingStrategy: random + ``` ### Least active connections @@ -72,10 +82,15 @@ This algorithm is useful when you want to "bin pack" the replica cluster. It ass ##### Configuration -```toml -[general] -load_balancing_strategy = "least_active_connections" -``` +=== "pgdog.toml" + ```toml + [general] + load_balancing_strategy = "least_active_connections" + ``` +=== "Helm chart" + ```yaml + loadBalancingStrategy: least_active_connections + ``` ## Single endpoint @@ -111,17 +126,28 @@ The load balancer recursively checks CTEs and, if any of them contains a query t The load balancer is **enabled by default** when more than one database with the same `name` property is configured in [pgdog.toml](../../configuration/pgdog.toml/databases.md), for example: -```toml -[[databases]] -name = "prod" -role = "primary" -host = "10.0.0.1" - -[[databases]] -name = "prod" -role = "replica" -host = "10.0.0.2" -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "prod" + role = "primary" + host = "10.0.0.1" + + [[databases]] + name = "prod" + role = "replica" + host = "10.0.0.2" + ``` +=== "Helm chart" + ```yaml + databases: + - name: prod + role: primary + host: 10.0.0.1 + - name: prod + role: replica + host: 10.0.0.2 + ``` ## Primary reads @@ -129,19 +155,29 @@ By default, if replica databases are configured, the primary is treated as one o This behavior is configurable in [pgdog.toml](../../configuration/pgdog.toml/general.md#read_write_split). You can isolate your primary from read queries and allow it to only serve writes: -```toml -[general] -read_write_split = "exclude_primary" -``` +=== "pgdog.toml" + ```toml + [general] + read_write_split = "exclude_primary" + ``` +=== "Helm chart" + ```yaml + readWriteSplit: exclude_primary + ``` #### Failover for reads In case one of your replicas fails, you can configure the primary to serve read queries temporarily while you (or your cloud vendor) bring the replica back up. This is configurable, like so: -```toml -[general] -read_write_split = "include_primary_if_replica_banned" -``` +=== "pgdog.toml" + ```toml + [general] + read_write_split = "include_primary_if_replica_banned" + ``` +=== "Helm chart" + ```yaml + readWriteSplit: include_primary_if_replica_banned + ``` ## Learn more diff --git a/docs/features/load-balancer/manual-routing.md b/docs/features/load-balancer/manual-routing.md index c1d1cf17..b8781004 100644 --- a/docs/features/load-balancer/manual-routing.md +++ b/docs/features/load-balancer/manual-routing.md @@ -159,10 +159,15 @@ In certain situations, the overhead of parsing queries may be too high, e.g., wh If you've configured the desired database role (and/or shard) for each of your application connections, you can disable the query parser in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#query_parser): -```toml -[general] -query_parser = "off" -``` +=== "pgdog.toml" + ```toml + [general] + query_parser = "off" + ``` +=== "Helm chart" + ```yaml + queryParser: "off" + ``` Once it's disabled, PgDog will rely solely on the `pgdog.role` and `pgdog.shard` parameters to make its routing decisions. diff --git a/docs/features/load-balancer/replication-failover.md b/docs/features/load-balancer/replication-failover.md index d96dc89b..3f3afd0d 100644 --- a/docs/features/load-balancer/replication-failover.md +++ b/docs/features/load-balancer/replication-failover.md @@ -39,14 +39,23 @@ This formula assumes that when the replica's LSN is behind the primary, the prim By default, PgDog will not query databases for their replication status. To enable this feature, configure it in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#replication): -```toml -[general] -# Start running the LSN check immediately. -lsn_check_delay = 0 - -# Run LSN check every second. -lsn_check_interval = 1_000 -``` +=== "pgdog.toml" + ```toml + [general] + # Start running the LSN check immediately. + lsn_check_delay = 0 + + # Run LSN check every second. + lsn_check_interval = 1_000 + ``` +=== "Helm chart" + ```yaml + # Start running the LSN check immediately. + lsnCheckDelay: 0 + + # Run LSN check every second. + lsnCheckInterval: 1_000 + ``` | Setting | Description | |-|-| @@ -66,11 +75,17 @@ If a replica has fallen far behind the primary, it may start serving stale data PgDog supports this with configurable banning thresholds: -```toml -[general] -ban_replica_lag = 60_000 # 1 minute -ban_replica_lag_bytes = 25_000_000 # 25 MiB -``` +=== "pgdog.toml" + ```toml + [general] + ban_replica_lag = 60_000 # 1 minute + ban_replica_lag_bytes = 25_000_000 # 25 MiB + ``` +=== "Helm chart" + ```yaml + banReplicaLag: 60_000 # 1 minute + banReplicaLagBytes: 25_000_000 # 25 MiB + ``` | Setting | Description | |-|-| @@ -100,17 +115,28 @@ If the `pg_is_in_recovery()` function returns `false`, PgDog will assume that th Failover is disabled by default. To enable it, change all configured databases in [`pgdog.toml`](../../configuration/pgdog.toml/databases.md) to use the `"auto"` role, for example: -```toml -[[databases]] -name = "prod" -host = "10.0.0.1" -role = "auto" - -[[databases]] -name = "prod" -host = "10.0.0.2" -role = "auto" -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "prod" + host = "10.0.0.1" + role = "auto" + + [[databases]] + name = "prod" + host = "10.0.0.2" + role = "auto" + ``` +=== "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + role: auto + - name: prod + host: 10.0.0.2 + role: auto + ``` On startup, PgDog will connect to each database, find out if they are in recovery, and automatically reload its configuration with the determined roles. diff --git a/docs/features/multi-tenancy.md b/docs/features/multi-tenancy.md index bcf41693..6a275a95 100644 --- a/docs/features/multi-tenancy.md +++ b/docs/features/multi-tenancy.md @@ -22,17 +22,28 @@ Physical multitenancy is a form of [sharding](sharding/index.md). To make it wor #### Example -```toml -[[databases]] -name = "prod" -host = "10.0.0.1" -shard = 0 - -[[databases]] -name = "prod" -host = "10.0.0.2" -shard = 1 -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "prod" + host = "10.0.0.1" + shard = 0 + + [[databases]] + name = "prod" + host = "10.0.0.2" + shard = 1 + ``` +=== "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + shard: 0 + - name: prod + host: 10.0.0.2 + shard: 1 + ``` ### Routing queries @@ -43,12 +54,20 @@ To route queries to the right tenant (aka shard), you need to configure the shar The first one is configurable using [sharded tables](../configuration/pgdog.toml/sharded_tables.md), for example: -```toml -[[sharded_tables]] -database = "prod" -column = "tenant_id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "tenant_id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: tenant_id + dataType: bigint + ``` This tells PgDog that the column `tenant_id` in _all_ tables will have the data type `BIGINT` and should be used for routing queries between shards. See [supported data types](sharding/sharding-functions.md#supported-data-types) for a list of data types you can use for sharding. @@ -58,14 +77,24 @@ The mapping is configurable separately for each tenant ID. Here, you have two op List-based is the most natural mapping for multitenant systems where your tenants are uniquely identified by a value. For example, using configuration, you can tell PgDog to route tenants 1, 5, and 1,000 to shard 0: -```toml -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "list" -values = [1, 5, 1_000] -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "list" + values = [1, 5, 1_000] + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: tenant_id + kind: list + values: [1, 5, 1_000] + shard: 0 + ``` You can specify as many mappings as you need and the list of values can contain as many tenants as you need. PgDog is using an efficient algorithm for mapping tenants to shards and supports millions of tenants as part of the same deployment. @@ -73,22 +102,38 @@ You can specify as many mappings as you need and the list of values can contain Range-based mapping is identical to `PARTITION BY RANGE` in Postgres and can be used creatively to separate tenants based on a range of values. For example, you can map tenants with IDs between 1 and 100 to shard 0 and everyone else to shard 1 like so: -```toml -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "range" -start = 1 -end = 100 -shard = 0 - -[[sharded_mappings]] -database = "prod" -column = "tenant_id" -kind = "range" -start = 100 -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "range" + start = 1 + end = 100 + shard = 0 + + [[sharded_mappings]] + database = "prod" + column = "tenant_id" + kind = "range" + start = 100 + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: tenant_id + kind: range + start: 1 + end: 100 + shard: 0 + - database: prod + column: tenant_id + kind: range + start: 100 + shard: 0 + ``` Much like Postgres partitions, the start of the range is included in the range while the end is excluded. diff --git a/docs/features/plugins/index.md b/docs/features/plugins/index.md index 4e75b489..af5da925 100644 --- a/docs/features/plugins/index.md +++ b/docs/features/plugins/index.md @@ -159,17 +159,29 @@ Plugins need to be compiled and placed into a folder on your machine where PgDog You then need to specify which plugins you'd like PgDog to load at runtime: -```toml -[[plugins]] -name = "my_plugin" -``` +=== "pgdog.toml" + ```toml + [[plugins]] + name = "my_plugin" + ``` +=== "Helm chart" + ```yaml + plugins: + - name: my_plugin + ``` This can be the name of the library (without the `lib` prefix or the `.so`/`.dylib` extension) or a relative/absolute path to the shared library, for example: -```toml -[[plugins]] -name = "/usr/lib/libmy_plugin.so" -``` +=== "pgdog.toml" + ```toml + [[plugins]] + name = "/usr/lib/libmy_plugin.so" + ``` +=== "Helm chart" + ```yaml + plugins: + - name: /usr/lib/libmy_plugin.so + ``` ## Examples diff --git a/docs/features/prepared-statements.md b/docs/features/prepared-statements.md index a164d203..ea14946f 100644 --- a/docs/features/prepared-statements.md +++ b/docs/features/prepared-statements.md @@ -30,10 +30,15 @@ at a global level, so clients can't evict a prepared statement potentially used Prepared statements are stored in memory. On machines constrained by RAM, it's reasonable to introduce a limit on how many statements should be stored. This limit is controlled through configuration: -```toml -[general] -prepared_statements_limit = 500 -``` +=== "pgdog.toml" + ```toml + [general] + prepared_statements_limit = 500 + ``` +=== "Helm chart" + ```yaml + preparedStatementsLimit: 500 + ``` This limit is strictly enforced on server connections: if a prepared statement needs to be sent to a server connection and it would exceed this capacity, the _least recently used_ statement will be closed to allow for more space on the connection. @@ -77,10 +82,15 @@ global name. Since this requires PgDog to parse _each_ incoming query, and that' You can enable it in [`pgdog.toml`](../configuration/pgdog.toml/general.md#prepared_statements): -```toml -[general] -prepared_statements = "full" -``` +=== "pgdog.toml" + ```toml + [general] + prepared_statements = "full" + ``` +=== "Helm chart" + ```yaml + preparedStatements: full + ``` Statements prepared using this method can be executed normally with `Bind` and `Execute` messages. Result data types can be inspected with `Describe`, just like a regular prepared statement. @@ -95,7 +105,12 @@ By default, unnamed (or anonymous) prepared statements are not cached and are se To make those drivers work, consider caching and rewriting unnamed prepared statements, like so: -```toml -[general] -prepared_statements = "extended_anonymous" -``` +=== "pgdog.toml" + ```toml + [general] + prepared_statements = "extended_anonymous" + ``` +=== "Helm chart" + ```yaml + preparedStatements: extended_anonymous + ``` diff --git a/docs/features/pub_sub.md b/docs/features/pub_sub.md index a17d0238..d7e962e9 100644 --- a/docs/features/pub_sub.md +++ b/docs/features/pub_sub.md @@ -15,10 +15,15 @@ PgDog supports this in [transaction mode](transaction-mode.md), removing this li You can enable pub/sub support by configuring the asynchronous message channel size in [`pgdog.toml`](../configuration/pgdog.toml/general.md): -```toml -[general] -pub_sub_channel_size = 4096 -``` +=== "pgdog.toml" + ```toml + [general] + pub_sub_channel_size = 4096 + ``` +=== "Helm chart" + ```yaml + pubSubChannelSize: 4096 + ``` Clients can then use Postgres pub/sub like normal. PgDog will intercept all commands and process them internally. How each command is handled is described below. diff --git a/docs/features/sharding/2pc.md b/docs/features/sharding/2pc.md index 4e5a1543..b19aa78b 100644 --- a/docs/features/sharding/2pc.md +++ b/docs/features/sharding/2pc.md @@ -32,10 +32,15 @@ Alternatively, if you're running on managed Postgres (e.g., AWS RDS), this param Once prepared transactions are enabled in Postgres, two-phase commit can be enabled in [`pgdog.toml`](../../configuration/pgdog.toml/general.md): -```toml -[general] -two_phase_commit = true -``` +=== "pgdog.toml" + ```toml + [general] + two_phase_commit = true + ``` +=== "Helm chart" + ```yaml + twoPhaseCommit: true + ``` ### Error handling @@ -89,10 +94,15 @@ While it's often desirable to ensure cross-shard writes are atomic, rewriting si If your writes are idempotent and can be safely retried, or your application doesn't have consistency requirements, you don't need to use this. Otherwise, you can enable it in [`pgdog.toml`](../../configuration/pgdog.toml/general.md): -```toml -[general] -two_phase_commit_auto = true -``` +=== "pgdog.toml" + ```toml + [general] + two_phase_commit_auto = true + ``` +=== "Helm chart" + ```yaml + twoPhaseCommitAuto: true + ``` ## Reads diff --git a/docs/features/sharding/cross-shard-queries/index.md b/docs/features/sharding/cross-shard-queries/index.md index 00112736..e3fa5b4b 100644 --- a/docs/features/sharding/cross-shard-queries/index.md +++ b/docs/features/sharding/cross-shard-queries/index.md @@ -57,10 +57,15 @@ Since PgDog needs to process rows before sending them to the client, we implemen If you don't want PgDog to route cross-shard queries, e.g., because you have a [multitenant](../../multi-tenancy.md) system with no interdependencies, cross-shard queries can be disabled with a configuration setting: -```toml -[general] -cross_shard_disabled = true -``` +=== "pgdog.toml" + ```toml + [general] + cross_shard_disabled = true + ``` +=== "Helm chart" + ```yaml + crossShardDisabled: true + ``` When this setting is enabled and a query doesn't have a sharding key, instead of executing the query, PgDog will return an error and abort the transaction. diff --git a/docs/features/sharding/cross-shard-queries/insert.md b/docs/features/sharding/cross-shard-queries/insert.md index 120cf295..a93fc793 100644 --- a/docs/features/sharding/cross-shard-queries/insert.md +++ b/docs/features/sharding/cross-shard-queries/insert.md @@ -120,11 +120,18 @@ INSERT INTO "users" ("id", "email", "created_at") VALUES (pgdog.unique_id(), $1, This works for regular queries and prepared statements alike. This feature is **disabled** by default and can be enabled in [`pgdog.toml`](../../../configuration/pgdog.toml/rewrite.md): -```toml -[rewrite] -enabled = true -primary_key = "rewrite" -``` +=== "pgdog.toml" + ```toml + [rewrite] + enabled = true + primary_key = "rewrite" + ``` +=== "Helm chart" + ```yaml + rewrite: + enabled: true + primaryKey: rewrite + ``` ### Composite primary keys @@ -162,11 +169,18 @@ This works for all queries, including prepared statements. PgDog will rewrite al Since this feature has additional overhead by using multiple shards for each query, it is **disabled** by default and can be enabled in [`pgdog.toml`](../../../configuration/pgdog.toml/rewrite.md): -```toml -[rewrite] -enabled = true -split_inserts = "rewrite" -``` +=== "pgdog.toml" + ```toml + [rewrite] + enabled = true + split_inserts = "rewrite" + ``` +=== "Helm chart" + ```yaml + rewrite: + enabled: true + splitInserts: rewrite + ``` ### Transaction required diff --git a/docs/features/sharding/cross-shard-queries/update.md b/docs/features/sharding/cross-shard-queries/update.md index 2f10cc21..b99a0a94 100644 --- a/docs/features/sharding/cross-shard-queries/update.md +++ b/docs/features/sharding/cross-shard-queries/update.md @@ -73,11 +73,18 @@ If the application doesn't start a transaction, PgDog will return an error and a This feature is **disabled** by default and can be enabled with configuration in [`pgdog.toml`](../../../configuration/pgdog.toml/rewrite.md): -```toml -[rewrite] -enabled = true -shard_key = "rewrite" -``` +=== "pgdog.toml" + ```toml + [rewrite] + enabled = true + shard_key = "rewrite" + ``` +=== "Helm chart" + ```yaml + rewrite: + enabled: true + shardKey: rewrite + ``` ### Updating multiple rows diff --git a/docs/features/sharding/dry-run.md b/docs/features/sharding/dry-run.md index 8faaa16e..a3a30e3c 100644 --- a/docs/features/sharding/dry-run.md +++ b/docs/features/sharding/dry-run.md @@ -9,10 +9,15 @@ In dry run mode, PgDog will parse every single query and record the routing deci You can enable dry run in the config: -```toml -[general] -dry_run = true -``` +=== "pgdog.toml" + ```toml + [general] + dry_run = true + ``` +=== "Helm chart" + ```yaml + dryRun: true + ``` PgDog supports hot reload of its configuration files, so you can toggle this setting without restarting the connection pooler. When enabled, PgDog enables its internal query parser (powered by `pg_query`) and attempts to "shard" every single query it receives. diff --git a/docs/features/sharding/explain.md b/docs/features/sharding/explain.md index f73325a4..19e61801 100644 --- a/docs/features/sharding/explain.md +++ b/docs/features/sharding/explain.md @@ -12,10 +12,15 @@ PgDog can annotate the query plan returned by Postgres with its own routing deci To avoid breaking existing tooling which relies on the standard `EXPLAIN` format, PgDog's `EXPLAIN` is disabled by default and can be turned on with configuration: -```toml -[general] -expanded_explain = true -``` +=== "pgdog.toml" + ```toml + [general] + expanded_explain = true + ``` +=== "Helm chart" + ```yaml + expandedExplain: true + ``` When configured, all `EXPLAIN` queries will start returning two query plans: one for Postgres and one for PgDog. For example: diff --git a/docs/features/sharding/omnishards.md b/docs/features/sharding/omnishards.md index 6af375d4..ac47547a 100644 --- a/docs/features/sharding/omnishards.md +++ b/docs/features/sharding/omnishards.md @@ -11,11 +11,18 @@ Other names for these tables include **mirrored tables** and **replicated tables Unless otherwise specified as a [sharded table](../../configuration/pgdog.toml/sharded_tables.md), all tables are omnisharded by default. This makes configuration simpler, and doesn't require explicitly enumerating all tables in `pgdog.toml`. For example: -```toml -[[sharded_tables]] -database = "prod" -column = "user_id" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + ``` This will configure all tables that have the `user_id` as sharded and all others as omnisharded. @@ -63,54 +70,98 @@ For example, system catalogs (e.g. `pg_database`, `pg_class`, etc.) could have d For example: -```toml -[[omnisharded_tables]] -database = "prod" -sticky = true -tables = [ - "pg_class", - "pg_database" -] -``` +=== "pgdog.toml" + ```toml + [[omnisharded_tables]] + database = "prod" + sticky = true + tables = [ + "pg_class", + "pg_database" + ] + ``` +=== "Helm chart" + ```yaml + omnishardedTables: + - database: prod + sticky: true + tables: + - pg_class + - pg_database + ``` You can enable sticky routing for all omnisharded tables in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#omnisharded_sticky): -```toml -[general] -omnisharded_sticky = true -``` +=== "pgdog.toml" + ```toml + [general] + omnisharded_sticky = true + ``` +=== "Helm chart" + ```yaml + omnishardedSticky: true + ``` The following system catalogs are using sticky routing by default: -```toml -[[omnisharded_tables]] -database = "prod" -sticky = true -tables = [ - "pg_class", - "pg_attribute", - "pg_attrdef", - "pg_index", - "pg_constraint", - "pg_namespace", - "pg_database", - "pg_tablespace", - "pg_type", - "pg_proc", - "pg_operator", - "pg_cast", - "pg_enum", - "pg_range", - "pg_authid", - "pg_am", -] -``` +=== "pgdog.toml" + ```toml + [[omnisharded_tables]] + database = "prod" + sticky = true + tables = [ + "pg_class", + "pg_attribute", + "pg_attrdef", + "pg_index", + "pg_constraint", + "pg_namespace", + "pg_database", + "pg_tablespace", + "pg_type", + "pg_proc", + "pg_operator", + "pg_cast", + "pg_enum", + "pg_range", + "pg_authid", + "pg_am", + ] + ``` +=== "Helm chart" + ```yaml + omnishardedTables: + - database: prod + sticky: true + tables: + - pg_class + - pg_attribute + - pg_attrdef + - pg_index + - pg_constraint + - pg_namespace + - pg_database + - pg_tablespace + - pg_type + - pg_proc + - pg_operator + - pg_cast + - pg_enum + - pg_range + - pg_authid + - pg_am + ``` This is configurable with the `system_catalogs` setting in [`pgdog.toml`](../../configuration/pgdog.toml/general.md#system_catalogs): -```toml -[general] -system_catalogs = "omnisharded_sticky" -``` +=== "pgdog.toml" + ```toml + [general] + system_catalogs = "omnisharded_sticky" + ``` +=== "Helm chart" + ```yaml + systemCatalogs: omnisharded_sticky + ``` If enabled (it is by default), commands like `\d`, `\d+` and others sent from `psql` will return correct results. diff --git a/docs/features/sharding/query-routing.md b/docs/features/sharding/query-routing.md index 692c1402..f32b0958 100644 --- a/docs/features/sharding/query-routing.md +++ b/docs/features/sharding/query-routing.md @@ -126,25 +126,42 @@ Adding the sharding key column is often best, because it makes writing queries a If most or all of your tables have the sharding key and the column name is the same, you can add it to [pgdog.toml](../../configuration/pgdog.toml/sharded_tables.md) without specifying a table name, for example: -```toml -[[sharded_tables]] -database = "prod" -column = "user_id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + dataType: bigint + ``` This will match all queries referring to all tables with the `user_id` column and route them to a shard accordingly. For the table storing the actual data referred to by the foreign keys, you can make another entry in the config, this time with the table name explicitly stated: -```toml -[[sharded_tables]] -database = "prod" -name = "users" -column = "id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + name = "users" + column = "id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + name: users + column: id + dataType: bigint + ``` The latter will match queries referring to the `users.id` column only. Together with the `user_id` entry, all tables that contain the sharding key will be supported by the query router for direct-to-shard queries. diff --git a/docs/features/sharding/resharding/cutover.md b/docs/features/sharding/resharding/cutover.md index cdf74993..8988273d 100644 --- a/docs/features/sharding/resharding/cutover.md +++ b/docs/features/sharding/resharding/cutover.md @@ -50,10 +50,15 @@ To suspend traffic, PgDog turns on [maintenance mode](../../../administration/ma The replication lag threshold at which PgDog will pause traffic automatically is configurable in [`pgdog.toml`](../../../configuration/pgdog.toml/general.md#cutover_traffic_stop_threshold): -```toml -[general] -cutover_traffic_stop_threshold = 1_000_000 # 1 MiB -``` +=== "pgdog.toml" + ```toml + [general] + cutover_traffic_stop_threshold = 1_000_000 # 1 MiB + ``` +=== "Helm chart" + ```yaml + cutoverTrafficStopThreshold: 1_000_000 # 1 MiB + ``` By default, it's set to 1MB, which is low enough that when traffic is paused, the two databases will synchronize very quickly. @@ -69,10 +74,15 @@ For this to work, the original database must remain in the [configuration files] The configuration swap happens in memory, but PgDog has the ability to write the new configuration files to disk as well. This is disabled by default, but can be enabled with a setting: -```toml -[general] -cutover_save_config = true -``` +=== "pgdog.toml" + ```toml + [general] + cutover_save_config = true + ``` +=== "Helm chart" + ```yaml + cutoverSaveConfig: true + ``` When enabled, PgDog will backup both configuration files, `pgdog.toml` as `pgdog.bak.toml` and `users.toml` as `users.bak.toml`, and save its in-memory configuration to `pgdog.toml` and `users.toml` respectively, so the new cutover configuration persists in case of an error. @@ -83,11 +93,17 @@ When enabled, PgDog will backup both configuration files, `pgdog.toml` as `pgdog Before swapping the configuration, PgDog waits for the two databases to be completely identical. These thresholds are configurable as follows: -```toml -[general] -cutover_replication_lag_threshold = 0 # 0 bytes -cutover_last_transaction_delay = 1_000 # 1 second -``` +=== "pgdog.toml" + ```toml + [general] + cutover_replication_lag_threshold = 0 # 0 bytes + cutover_last_transaction_delay = 1_000 # 1 second + ``` +=== "Helm chart" + ```yaml + cutoverReplicationLagThreshold: 0 # 0 bytes + cutoverLastTransactionDelay: 1_000 # 1 second + ``` Due to vacuum activity and transactions affecting other tables not in the publication, the replication lag between the two databases may never reach zero. For this reason, PgDog provides two triggers for the configuration swap: @@ -100,11 +116,17 @@ The latter is computed from messages received via the replication stream and is If these thresholds are not hit within a reasonable amount of time, PgDog will abort the cutover and resume traffic on the source database. This behavior is configurable: -```toml -[general] -cutover_timeout = 30_000 # 30 seconds -cutover_timeout_action = "abort" # or "cutover" -``` +=== "pgdog.toml" + ```toml + [general] + cutover_timeout = 30_000 # 30 seconds + cutover_timeout_action = "abort" # or "cutover" + ``` +=== "Helm chart" + ```yaml + cutoverTimeout: 30_000 # 30 seconds + cutoverTimeoutAction: abort # or "cutover" + ``` If `cutover_timeout_action` is set to `"cutover"` instead, PgDog will flip the traffic to the destination database. This is an acceptable course of action in environments where data integrity is not paramount or the operator is absolutely certain that both databases are identical. diff --git a/docs/features/sharding/resharding/move.md b/docs/features/sharding/resharding/move.md index 364a644b..4bc4e9d2 100644 --- a/docs/features/sharding/resharding/move.md +++ b/docs/features/sharding/resharding/move.md @@ -103,17 +103,28 @@ If you are running PostgreSQL 16 or later and have configured replicas on the so To set this up, make sure to add your read replicas to [`pgdog.toml`](../../../configuration/pgdog.toml/databases.md), for example: -```toml -[[databases]] -name = "source" -host = "10.0.0.1" -role = "replica" - -[[databases]] -name = "source" -host = "10.0.0.2" -role = "replica" -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "source" + host = "10.0.0.1" + role = "replica" + + [[databases]] + name = "source" + host = "10.0.0.2" + role = "replica" + ``` +=== "Helm chart" + ```yaml + databases: + - name: source + host: 10.0.0.1 + role: replica + - name: source + host: 10.0.0.2 + role: replica + ``` PgDog will distribute the table copy load evenly between all replicas in the configuration. The more replicas are available for resharding, the faster it will complete. @@ -124,13 +135,22 @@ PgDog will distribute the table copy load evenly between all replicas in the con To make sure dedicated replicas are not used for read queries in production, you can configure PgDog to use them for resharding only: - ```toml - [[databases]] - name = "prod" - host = "10.0.0.1" - role = "replica" - resharding_only = true - ``` + === "pgdog.toml" + ```toml + [[databases]] + name = "prod" + host = "10.0.0.1" + role = "replica" + resharding_only = true + ``` + === "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + role: replica + reshardingOnly: true + ``` #### Binary `COPY` @@ -150,10 +170,15 @@ If your primary keys are using the `INTEGER` data type (like in older Rails vers If this is the case, the binary `COPY` will not work, and you need to use text copy protocol instead. This can be configured in [`pgdog.toml`](../../../configuration/pgdog.toml/general.md#resharding_copy_format): -```toml -[general] -resharding_copy_format = "text" -``` +=== "pgdog.toml" + ```toml + [general] + resharding_copy_format = "text" + ``` +=== "Helm chart" + ```yaml + reshardingCopyFormat: text + ``` #### Monitoring progress @@ -228,11 +253,17 @@ If a shard goes down mid-copy, PgDog retries that table with exponential backoff To change the defaults, configure [`resharding_copy_retry_max_attempts`](../../../configuration/pgdog.toml/general.md#resharding_copy_retry_max_attempts) and [`resharding_copy_retry_min_delay`](../../../configuration/pgdog.toml/general.md#resharding_copy_retry_min_delay) in `pgdog.toml`: -```toml -[general] -resharding_copy_retry_max_attempts = 5 # per-table retry attempts -resharding_copy_retry_min_delay = 1000 # base backoff in ms; doubles each attempt, max 32× -``` +=== "pgdog.toml" + ```toml + [general] + resharding_copy_retry_max_attempts = 5 # per-table retry attempts + resharding_copy_retry_min_delay = 1000 # base backoff in ms; doubles each attempt, max 32× + ``` +=== "Helm chart" + ```yaml + reshardingCopyRetryMaxAttempts: 5 # per-table retry attempts + reshardingCopyRetryMinDelay: 1000 # base backoff in ms; doubles each attempt, max 32× + ``` ### Transient errors during replication streaming @@ -266,10 +297,15 @@ Run the `TRUNCATE` on the destination (you can copy the command above, but make Different major Postgres versions can produce incompatible binary `COPY` data. PgDog surfaces this as a `BinaryFormatMismatch` error. Switch to text: -```toml -[general] -resharding_copy_format = "text" -``` +=== "pgdog.toml" + ```toml + [general] + resharding_copy_format = "text" + ``` +=== "Helm chart" + ```yaml + reshardingCopyFormat: text + ``` See [Integer primary keys](#integer-primary-keys) for the other common reason to use text format. diff --git a/docs/features/sharding/resharding/schema.md b/docs/features/sharding/resharding/schema.md index fb17da6e..b53320b6 100644 --- a/docs/features/sharding/resharding/schema.md +++ b/docs/features/sharding/resharding/schema.md @@ -157,13 +157,21 @@ Since creating indexes on large tables can take some time, PgDog provides an adm Schema sync creates tables, indexes, and other entities on the destination database. To make sure that this is done with a user with sufficient privileges (e.g., `CREATE` and `REPLICATION` permissions on the database), make sure to add such a user to [`users.toml`](../../../configuration/users.toml/users.md) and mark it as the schema administrator: -```toml -[[users]] -name = "migrator" -database = "prod" -password = "hunter2" -schema_admin = true -``` +=== "users.toml" + ```toml + [[users]] + name = "migrator" + database = "prod" + password = "hunter2" + schema_admin = true + ``` +=== "Helm chart" + ```yaml + users: + - name: migrator + database: prod + password: hunter2 + ``` PgDog will use this user to connect to the source and destination databases, so make sure to specify one for both databases in the configuration. diff --git a/docs/features/sharding/schema_management/cache.md b/docs/features/sharding/schema_management/cache.md index 204947f2..5b76b62d 100644 --- a/docs/features/sharding/schema_management/cache.md +++ b/docs/features/sharding/schema_management/cache.md @@ -29,11 +29,18 @@ INSERT INTO users VALUES ($1, $2); In addition to detecting sharding keys, PgDog uses the schema cache to determine if a table is sharded or omnisharded. If a sharded table is configured using a column name only in [`pgdog.toml`](../../../configuration/pgdog.toml/sharded_tables.md), inspecting its schema is the only way to find if it has the sharding key, for example: -```toml -[[sharded_tables]] -database = "prod" -column = "user_id" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + ``` When a query comes in that doesn't specify this column, PgDog needs to decide if the table is sharded or omnisharded: diff --git a/docs/features/sharding/sharding-functions.md b/docs/features/sharding/sharding-functions.md index 115f99e5..2999ab87 100644 --- a/docs/features/sharding/sharding-functions.md +++ b/docs/features/sharding/sharding-functions.md @@ -37,12 +37,20 @@ The hash function evenly distributes data between all shards. It ingests bytes a The hash function is used by default when configuring sharded tables in [`pgdog.toml`](../../configuration/pgdog.toml/sharded_tables.md): -```toml -[[sharded_tables]] -database = "prod" -column = "user_id" -data_type = "bigint" -``` +=== "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + data_type = "bigint" + ``` +=== "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + dataType: bigint + ``` All queries referencing the `user_id` column will be automatically sent to the matching shard(s) and data in those tables will be split between all data nodes evenly. See below for a list of [supported](#supported-data-types) data types. Each can be specified as follows: @@ -50,30 +58,54 @@ All queries referencing the `user_id` column will be automatically sent to the m !!! note "Integer types" Different integer types are treated the same by the query router. If you're using `BIGINT`, `INTEGER` or `SMALLINT` as your sharding key, you can specify `bigint` in the configuration: - ```toml - [[sharded_tables]] - database = "prod" - column = "user_id" - data_type = "bigint" - ``` + === "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "user_id" + data_type = "bigint" + ``` + === "Helm chart" + ```yaml + shardedTables: + - database: prod + column: user_id + dataType: bigint + ``` === "Text" !!! note "Text types" `VARCHAR`, `VARCHAR(n)`, and `TEXT` use the same encoding and are treated the same by the query router. For either one, you can specify `varchar` in the configuration: - ```toml - [[sharded_tables]] - database = "prod" - column = "serial_number" - data_type = "varchar" - ``` + === "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "serial_number" + data_type = "varchar" + ``` + === "Helm chart" + ```yaml + shardedTables: + - database: prod + column: serial_number + dataType: varchar + ``` === "UUID" !!! note "UUID types" Only UUIDv4 is currently supported for sharding in the query router. - ```toml - [[sharded_tables]] - database = "prod" - column = "unique_id" - data_type = "uuid" - ``` + === "pgdog.toml" + ```toml + [[sharded_tables]] + database = "prod" + column = "unique_id" + data_type = "uuid" + ``` + === "Helm chart" + ```yaml + shardedTables: + - database: prod + column: unique_id + dataType: uuid + ``` ## List @@ -81,14 +113,24 @@ The list sharding function distributes data between shards according to a value To enable this sharding function on a table or column, you need to specify additional value <-> shard mappings in [`pgdog.toml`](../../configuration/pgdog.toml/sharded_tables.md), for example: -```toml -[[sharded_mappings]] -database = "prod" -column = "user_id" -kind = "list" -values = [1, 2, 3] -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "user_id" + kind = "list" + values = [1, 2, 3] + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: user_id + kind: list + values: [1, 2, 3] + shard: 0 + ``` This example will route all queries with `user_id` equal to one, two, or three to shard zero. Unlike [hash](#hash) sharding, a value <-> shard mapping is required for _all_ values of the sharding key. If a value is used that doesn't have a mapping, the query will be sent to [all shards](cross-shard-queries/index.md). @@ -99,15 +141,26 @@ This example will route all queries with `user_id` equal to one, two, or three t Sharding by range is similar to [list](#list) sharding, except instead of specifying the values explicitly, you can specify a bounding range. All values that are included in the range will be sent to the specified shard, for example: -```toml -[[sharded_mappings]] -database = "prod" -column = "user_id" -kind = "range" -start = 1 -end = 100 -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_mappings]] + database = "prod" + column = "user_id" + kind = "range" + start = 1 + end = 100 + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedMappings: + - database: prod + column: user_id + kind: range + start: 1 + end: 100 + shard: 0 + ``` This will route queries that refer to the `user_id` column, with values between 1 and 100 (exclusively), to shard zero. For open-ended ranges, you can specify either the `start` or the `end` value. The start value is included in the range, while the end value is excluded. @@ -138,17 +191,28 @@ When enabled, PgDog will route queries that fully qualify tables based on their Schemas are mapped to their shards in [pgdog.toml](../../configuration/pgdog.toml/sharded_schemas.md), for example: -```toml -[[sharded_schemas]] -database = "prod" -name = "customer_a" -shard = 0 +=== "pgdog.toml" + ```toml + [[sharded_schemas]] + database = "prod" + name = "customer_a" + shard = 0 -[[sharded_schemas]] -database = "prod" -name = "customer_b" -shard = 1 -``` + [[sharded_schemas]] + database = "prod" + name = "customer_b" + shard = 1 + ``` +=== "Helm chart" + ```yaml + shardedSchemas: + - database: prod + name: customer_a + shard: 0 + - database: prod + name: customer_b + shard: 1 + ``` Queries that include the schema name in the tables they are referring to can be routed to the right shard. For example: @@ -181,11 +245,18 @@ If a schema isn't mapped to a shard number, PgDog will fallback to using other c To avoid this behavior and send all other queries to a particular shard, you can add a default schema mapping: -```toml -[[sharded_schemas]] -database = "prod" -shard = 0 -``` +=== "pgdog.toml" + ```toml + [[sharded_schemas]] + database = "prod" + shard = 0 + ``` +=== "Helm chart" + ```yaml + shardedSchemas: + - database: prod + shard: 0 + ``` This will send all queries that don't specify a schema or use a schema without a mapping to shard zero. diff --git a/docs/features/sharding/unique-ids.md b/docs/features/sharding/unique-ids.md index f88bdb40..554de902 100644 --- a/docs/features/sharding/unique-ids.md +++ b/docs/features/sharding/unique-ids.md @@ -84,10 +84,15 @@ When configured correctly, you can get each node's identifier by querying the [a If you're migrating data from an existing database, you can ensure that all IDs generated by PgDog start at a minimum value. This is configurable in [`pgdog.toml`](../../configuration/pgdog.toml/general.md), like so: -```toml -[general] -unique_id_min = 5_000_000 -``` +=== "pgdog.toml" + ```toml + [general] + unique_id_min = 5_000_000 + ``` +=== "Helm chart" + ```yaml + uniqueIdMin: 5_000_000 + ``` When set, all generated IDs are guaranteed to be larger than this value. This feature however is normally not needed, since IDs generated by this function are very large. @@ -118,10 +123,15 @@ The unique ID algorithm generates very large 64-bit integers. This is because th For this reason, we added a more "compact", 53-bit unique ID generator function. It can be used by enabling it in [`pgdog.toml`](../../configuration/pgdog.toml/general.md): -```toml -[general] -unique_id_function = "compact" -``` +=== "pgdog.toml" + ```toml + [general] + unique_id_function = "compact" + ``` +=== "Helm chart" + ```yaml + uniqueIdFunction: compact + ``` !!! warning "Switching to the compact generator" If you're currently using the `"standard"` unique ID generator (default), be careful switching because the IDs it will generate will be considerably smaller, breaking the monotonic guarantee diff --git a/docs/features/tls.md b/docs/features/tls.md index 3fac8e98..871b3152 100644 --- a/docs/features/tls.md +++ b/docs/features/tls.md @@ -13,11 +13,17 @@ To enable encryption for client connections, you need to provide (or generate) a Add the following settings to your `pgdog.toml`: -```toml -[general] -tls_certificate = "/path/to/certificate.pem" -tls_private_key = "/path/to/private_key.pem" -``` +=== "pgdog.toml" + ```toml + [general] + tls_certificate = "/path/to/certificate.pem" + tls_private_key = "/path/to/private_key.pem" + ``` +=== "Helm chart" + ```yaml + tlsCertificate: /path/to/certificate.pem + tlsPrivateKey: /path/to/private_key.pem + ``` Both the certificate and private key need to be formatted using the [PEM](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail) format. The private key should not have a passphrase. @@ -46,10 +52,15 @@ The default value for most PostgreSQL connection drivers is typically `prefer`, By default, PgDog will attempt to use TLS when connecting to PostgreSQL. This is configurable via a setting: -```toml -[general] -tls_verify = "prefer" -``` +=== "pgdog.toml" + ```toml + [general] + tls_verify = "prefer" + ``` +=== "Helm chart" + ```yaml + tlsVerify: prefer + ``` This setting accepts almost identical values to the `sslmode` parameter used by clients: @@ -62,10 +73,15 @@ This setting accepts almost identical values to the `sslmode` parameter used by If you use `verify_ca` or `verify_full` and your certificate is not signed by a well known CA, you can configure PgDog to validate it using your own certificate chain: -```toml -[general] -tls_server_ca_certificate = "/path/to/ca/certificate.pem" -``` +=== "pgdog.toml" + ```toml + [general] + tls_server_ca_certificate = "/path/to/ca/certificate.pem" + ``` +=== "Helm chart" + ```yaml + tlsServerCaCertificate: /path/to/ca/certificate.pem + ``` ## Performance diff --git a/docs/features/transaction-mode.md b/docs/features/transaction-mode.md index 25544875..13d7c939 100644 --- a/docs/features/transaction-mode.md +++ b/docs/features/transaction-mode.md @@ -22,25 +22,40 @@ In practice, this allows thousands of client connections to re-use just one Post Transaction mode is **enabled** by default. This is controllable via configuration, at the global, user and database levels: -=== "pgdog.toml (global)" - ```toml +=== "Global" + ```toml title="pgdog.toml" [general] pooler_mode = "transaction" ``` -=== "pgdog.toml (database)" - ```toml + ```yaml title="Helm chart" + poolerMode: transaction + ``` +=== "Database" + ```toml title="pgdog.toml" [[databases]] name = "prod" host = "127.0.0.1" pooler_mode = "transaction" ``` -=== "users.toml" - ```toml + ```yaml title="Helm chart" + databases: + - name: prod + host: 127.0.0.1 + poolerMode: transaction + ``` +=== "User" + ```toml title="users.toml" [[users]] name = "alice" database = "prod" pooler_mode = "transaction" ``` + ```yaml title="Helm chart" + users: + - name: alice + database: prod + poolerMode: transaction + ``` ## Session state @@ -51,17 +66,10 @@ To avoid session-level state leaking between clients, PgDog tracks connection pa This is performed efficiently, and server parameters are updated only if they differ from the ones set on the client. !!! note "Parsing SET commands" - PgDog uses `pg_query` to parse SQL statements, which includes the `SET` command. For each command sent by the client, PgDog will decode the setting - and save it in the client connection's internal state. - - This feature is enabled only if **at least** one of the following conditions is met: - - 1. The database has a primary and replica(s) - 2. The database has more than one shard - 3. [`prepared_statements`](../configuration/pgdog.toml/general.md#prepared_statements) is set to `"full"` - 4. [`query_parser`](../configuration/pgdog.toml/general.md#query_parser_enabled) is set to `"on"` - - This is to avoid unnecessary overhead of using `pg_query` (however small), when we don't absolutely have to. + PgDog automatically detects `SET` commands and uses the `pg_query` SQL parser to extract the GUC/session variable. This feature is **enabled** by default. + + For deployments that don't normally need the parser (i.e. unsharded, read-only or no replicas), PgDog can selectively enable its parser for `SET` commands only. This is very fast + and shouldn't have a noticeable impact on pooler performance. ### Connection parameters @@ -95,17 +103,36 @@ SELECT pg_advisory_lock(1234); In transaction mode, server connections are re-used between clients, so additional care needs to be taken to keep the server connection tied to the client that created the lock. +### Handling advisory locks + PgDog is able to detect advisory lock usage and will pin the server connection to the client connection until one of the following conditions is met: 1. The client releases the lock with `pg_advisory_unlock` 2. The client disconnects -!!! note "Performance" - If multiple clients use advisory locks and don't release them quickly, the effectiveness of transaction pooling will be reduced because server connections will not be effectively re-used between client transactions. +!!! note "Query parser" + This feature requires the query parser to be enabled, which happens if the deployment is sharded + or is using the read/write split feature of the [load balancer](load-balancer/index.md). + +If your PgDog deployment is unsharded and isn't using the [load balancer](load-balancer/index.md) for read/write separation, this feature is **disabled** by default. To enable it, turn on the query parser with the following setting: + +=== "pgdog.toml" + ```toml + [general] + query_parser = "session_state_and_locks" + ``` +=== "Helm chart" + ```yaml + queryParser: session_state_and_locks + ``` + +This will scan all incoming queries for `pg_advisory_*` functions and selectively enable the query parser to handle them correctly. + +### Performance -### Limitations +If multiple clients use advisory locks and don't release them quickly, the effectiveness of transaction pooling will be diminshed because server connections will not be quickly re-used between client transactions. -PgDog doesn't keep track of multiple advisory locks inside client connections. If a client acquires two different locks, for example, and only releases one, the server connection will still be returned back to the pool with the acquired lock. +Generally speaking, Postgres advisory locks should be used inside transactions, (i.e. using `pg_xact_advisory_lock()` function), or by the schema migrations tooling only (e.g. Alembic, Rails migrations, etc.). ## Statement mode @@ -113,24 +140,39 @@ Statement mode is a subset of transaction mode. In statement mode, clients are n To use statement mode, you can configure it globally or per user/database, for example: -=== "pgdog.toml (global)" - ```toml +=== "Global" + ```toml title="pgdog.toml" [general] pooler_mode = "statement" ``` -=== "pgdog.toml (database)" - ```toml + ```yaml title="Helm chart" + poolerMode: statement + ``` +=== "Database" + ```toml title="pgdog.toml" [[databases]] name = "prod" host = "127.0.0.1" pooler_mode = "statement" ``` -=== "users.toml" - ```toml + ```yaml title="Helm chart" + databases: + - name: prod + host: 127.0.0.1 + poolerMode: statement + ``` +=== "User" + ```toml title="users.toml" [[users]] name = "alice" database = "prod" pooler_mode = "statement" ``` + ```yaml title="Helm chart" + users: + - name: alice + database: prod + poolerMode: statement + ``` Statement mode is useful when you want to avoid holding server connections idle while the client executes long transactions, but it does remove an important feature of Postgres, so additional care needs to be taken on the client to handle concurrent database updates. diff --git a/docs/installation.md b/docs/installation.md index 60641858..f205b9c8 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -123,22 +123,37 @@ pgdog \ Most configuration options have sensible defaults. This makes single-database configuration pretty short: -```toml -[[databases]] -name = "postgres" -host = "127.0.0.1" -``` +=== "pgdog.toml" + ```toml + [[databases]] + name = "postgres" + host = "127.0.0.1" + ``` +=== "Helm chart" + ```yaml + databases: + - name: postgres + host: 127.0.0.1 + ``` #### [users.toml](configuration/users.toml/users.md) This config file contains a mapping between databases, users, and passwords. Unless you configured [passthrough authentication](features/authentication.md#passthrough-authentication), users not specified in this file won't be able to connect: -```toml -[[users]] -name = "alice" -database = "postgres" -password = "hunter2" -``` +=== "users.toml" + ```toml + [[users]] + name = "alice" + database = "postgres" + password = "hunter2" + ``` +=== "Helm chart" + ```yaml + users: + - name: alice + database: postgres + password: hunter2 + ``` !!! note "Configuring users" diff --git a/docs/migrating-to-pgdog/from-pgbouncer.md b/docs/migrating-to-pgdog/from-pgbouncer.md index 7d331fbc..647e5f2b 100644 --- a/docs/migrating-to-pgdog/from-pgbouncer.md +++ b/docs/migrating-to-pgdog/from-pgbouncer.md @@ -44,6 +44,17 @@ Databases are a list of PostgreSQL databases proxied by the connection pooler, w !!! note "Configuring users" Unlike PgBouncer, PgDog connection pool users are configured separately in [`users.toml`](../configuration/users.toml/users.md). +=== "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + port: 5432 + - name: staging + host: 10.0.0.1 + port: 5432 + ``` + #### Overrides Both PgBouncer and PgDog can override the user's password used to connect to PostgreSQL. @@ -62,6 +73,16 @@ Both PgBouncer and PgDog can override the user's password used to connect to Pos password = "hunter2" pooler_mode = "transaction" ``` +=== "Helm chart" + ```yaml + databases: + - name: prod + host: 10.0.0.1 + port: 5432 + user: postgres + password: hunter2 + poolerMode: transaction + ``` #### Database-level settings @@ -113,6 +134,12 @@ PgDog follows the same principles and has a [`[general]`](../configuration/pgdog default_pool_size = 100 pooler_mode = "transaction" ``` +=== "Helm chart" + ```yaml + port: 6432 + defaultPoolSize: 100 + poolerMode: transaction + ``` #### Supported settings @@ -301,16 +328,35 @@ The PgBouncer's user list is a list of username/password pairs. The users listed database = "staging" password = "oranges" ``` +=== "Helm chart" + ```yaml + users: + - name: postgres + database: prod + password: hunter2 + - name: apples + database: staging + password: oranges + ``` In addition to passwords, [`users.toml`](../configuration/users.toml/users.md) entries allow to configure user-specific overrides, for example: -```toml -[[users]] -name = "postgres" -database = "postgres" -password = "hunter2" -pooler_mode = "session" -``` +=== "users.toml" + ```toml + [[users]] + name = "postgres" + database = "postgres" + password = "hunter2" + pooler_mode = "session" + ``` +=== "Helm chart" + ```yaml + users: + - name: postgres + database: postgres + password: hunter2 + poolerMode: session + ``` ## Read more From f665accfb3764c349a2feb06cade2bb52c5eb8b0 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Tue, 16 Jun 2026 09:26:24 -0700 Subject: [PATCH 2/5] wrong --- docs/enterprise_edition/plans.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/enterprise_edition/plans.md b/docs/enterprise_edition/plans.md index 346b7229..844eca9b 100644 --- a/docs/enterprise_edition/plans.md +++ b/docs/enterprise_edition/plans.md @@ -22,7 +22,7 @@ Query plan collection is disabled by default and can be enabled in `pgdog.toml`: ```yaml queryStats: enabled: true - queryPlanThreshold: 100 + queryPlansThreshold: 100 ``` The plans are stored in memory and deduplicated by _normalized_ SQL of the query. This means that queries with different parameters will be tied to the same query plan, making sure PgDog doesn't plan the same query multiple times. @@ -53,7 +53,7 @@ The query plan cache is configurable: ```yaml queryStats: enabled: true - queryPlanThreshold: 100 + queryPlansThreshold: 100 queryPlansCache: 1000 queryPlanMaxAge: 15000 ``` From ed4bf7d28d38d1d4a0db35c2abf58db7096f9a97 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Tue, 16 Jun 2026 09:27:25 -0700 Subject: [PATCH 3/5] wrong --- docs/enterprise_edition/plans.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/enterprise_edition/plans.md b/docs/enterprise_edition/plans.md index 844eca9b..702657df 100644 --- a/docs/enterprise_edition/plans.md +++ b/docs/enterprise_edition/plans.md @@ -55,7 +55,7 @@ The query plan cache is configurable: enabled: true queryPlansThreshold: 100 queryPlansCache: 1000 - queryPlanMaxAge: 15000 + queryPlansMaxAge: 15000 ``` ### Admin database From 7914b311017e14acd680bbecffa341fd360671a2 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Tue, 16 Jun 2026 09:31:42 -0700 Subject: [PATCH 4/5] wrong --- docs/enterprise_edition/plans.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/enterprise_edition/plans.md b/docs/enterprise_edition/plans.md index 702657df..41e388a4 100644 --- a/docs/enterprise_edition/plans.md +++ b/docs/enterprise_edition/plans.md @@ -54,6 +54,7 @@ The query plan cache is configurable: queryStats: enabled: true queryPlansThreshold: 100 + queryPlansSampleRate: 0.5 queryPlansCache: 1000 queryPlansMaxAge: 15000 ``` From 787ae0d1a546464217f4e2b4979c9faf6be45955 Mon Sep 17 00:00:00 2001 From: Lev Kokotov Date: Tue, 16 Jun 2026 09:33:39 -0700 Subject: [PATCH 5/5] wrong --- docs/features/transaction-mode.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/features/transaction-mode.md b/docs/features/transaction-mode.md index 13d7c939..81d1e355 100644 --- a/docs/features/transaction-mode.md +++ b/docs/features/transaction-mode.md @@ -119,11 +119,11 @@ If your PgDog deployment is unsharded and isn't using the [load balancer](load-b === "pgdog.toml" ```toml [general] - query_parser = "session_state_and_locks" + query_parser = "session_control_and_locks" ``` === "Helm chart" ```yaml - queryParser: session_state_and_locks + queryParser: session_control_and_locks ``` This will scan all incoming queries for `pg_advisory_*` functions and selectively enable the query parser to handle them correctly.