diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index bd1e4227..f24b3ce9 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -8,30 +8,14 @@ jobs: verify-tomls: runs-on: blacksmith-4vcpu-ubuntu-2404 timeout-minutes: 30 + env: + PGDOG_IMAGE: ghcr.io/pgdogdev/pgdog-enterprise:main-ent steps: - name: Checkout code uses: actions/checkout@v4 - - name: Checkout pgdog-enterprise - uses: actions/checkout@v4 - with: - repository: pgdogdev/pgdog-enterprise - ref: main-ent - path: pgdog-source - token: ${{ secrets.PGDOG_ENTERPRISE_TOKEN }} - - uses: actions-rs/toolchain@v1 - with: - toolchain: stable - override: true - - name: Install CMake 3.31 - run: | - sudo apt update && sudo apt install mold -y - sudo apt remove cmake - sudo pip3 install cmake==3.31.6 - cmake --version - - name: Build - run: cargo build - working-directory: pgdog-source + - name: Pull PgDog Enterprise image + run: docker pull "$PGDOG_IMAGE" - name: Install test dependencies run: sudo pip3 install -r tests/requirements.txt - name: Run tests - run: python3 tests/test_code_blocks.py pgdog-source/target/debug/pgdog + run: python3 tests/test_code_blocks.py tests/pgdog-docker.sh diff --git a/docs/enterprise_edition/control_plane/.pages b/docs/enterprise_edition/control_plane/.pages new file mode 100644 index 00000000..1697fd8d --- /dev/null +++ b/docs/enterprise_edition/control_plane/.pages @@ -0,0 +1,5 @@ +nav: + - index.md + - installation.md + - features + - ... diff --git a/docs/enterprise_edition/control_plane/features/.pages b/docs/enterprise_edition/control_plane/features/.pages new file mode 100644 index 00000000..8ae5ab46 --- /dev/null +++ b/docs/enterprise_edition/control_plane/features/.pages @@ -0,0 +1,3 @@ +nav: + - index.md + - ... diff --git a/docs/enterprise_edition/control_plane/features/alerts.md b/docs/enterprise_edition/control_plane/features/alerts.md new file mode 100644 index 00000000..7e61e886 --- /dev/null +++ b/docs/enterprise_edition/control_plane/features/alerts.md @@ -0,0 +1,62 @@ +--- +icon: material/alert-circle-outline +--- + +# Alerts + +Since the control plane has access to real-time PgDog [metrics](../../../features/metrics.md), it can detect outliers and trigger automatic alerts. + +## How it works + +The control plane evaluates the metrics it receives from all PgDog instances on a continuous loop. When they exceed pre-configured thresholds, an incident +is created using one of the configured integrations. + +
+ Alerts +
+ +### Integrations + +The following incident management providers are supported: + +| Provider | Description | +|-|-| +| [incident.io](https://incident.io/) | The control plane will create an incident with the highest available priority. | + +### Configuration + +Alerts are **disabled** by default. To enable them, configure a provider's API key and set thresholds for each supported metric you would like to monitor: + +```yaml title="values.yaml" +control: + config: + alerts: + evaluation_window_secs: 300 + thresholds: + clients_waiting: 10 + cpu: 90.0 + memory: 2048 + server_connections: 100 + incident_io: + api_key: inc_live_xxx +``` + +#### Parameters + +!!! note "Disabled by default" + Thresholds have no defaults. If a threshold is not configured, its metric will not be monitored. + + +The following parameters are configurable via the [Helm chart](../installation.md): + +| Parameter | Description | Unit | +|-|-|-| +| `evaluation_window_secs` | The metrics evaluation window. Metrics are averaged over this period to produce an alert signal. | Seconds, e.g., `300` (5 minutes) | +| `clients_waiting` | Average number of clients waiting for a connection from a pool. | Clients, e.g., `10` | +| `cpu` | Average CPU utilization of each PgDog pod. | Percentage, e.g., `90.0` (90%) | +| `memory` | Average memory utilization of each PgDog pod. | MiB, e.g., `2048` (2048 MiB) | +| `server_connections` | Average number of open connections from PgDog to Postgres. | Connections, e.g., `100` | + +#### Evaluation window + +Longer evaluation windows reduce the chance of false positives in spiky workloads. Shorter windows will be evaluated quicker and will trigger faster notifications in case of an incident. diff --git a/docs/enterprise_edition/control_plane/features/autoscaling.md b/docs/enterprise_edition/control_plane/features/autoscaling.md new file mode 100644 index 00000000..85ac8906 --- /dev/null +++ b/docs/enterprise_edition/control_plane/features/autoscaling.md @@ -0,0 +1,137 @@ +--- +icon: material/arrow-expand-all +--- + +# Autoscaling + +Autoscaling automatically adjusts PgDog settings when the number of processes (e.g. pods in Kubernetes) in the same deployment changes. This simplifies configuration: you no longer need to perform manual calculations to resize connection pools. + +## Configuration + +Autoscaling is **disabled** by default. To enable it, add the following settings to the [control plane](../installation.md) Helm [chart](https://github.com/pgdogdev/helm-ee): + +```yaml title="values.yaml" +control: + config: + autoscaling: + pool_size: true +``` + +## How it works + +When a PgDog process connects to the [control plane](../../control_plane/index.md), the control plane provides it with the total number of processes that are part of the same deployment. PgDog then automatically adjusts its configuration by dividing all pool-related configuration values by that number, for example: + +=== "Configuration" + ```toml title="pgdog.toml" + [general] + default_pool_size = 200 + + [[databases]] + name = "prod" + host = "10.0.0.1" + min_pool_size = 50 + ``` + + ```toml title="users.toml" + [[users]] + name = "postgres" + database = "prod" + pool_size = 100 + ``` + +=== "2 processes" + ```toml title="pgdog.toml" + [general] + default_pool_size = 100 + + [[databases]] + name = "prod" + host = "10.0.0.1" + min_pool_size = 25 + ``` + + ```toml title="users.toml" + [[users]] + name = "postgres" + database = "prod" + pool_size = 50 + ``` + +=== "4 processes" + ```toml title="pgdog.toml" + [general] + default_pool_size = 50 + + [[databases]] + name = "prod" + host = "10.0.0.1" + min_pool_size = 12 + ``` + + ```toml title="users.toml" + [[users]] + name = "postgres" + database = "prod" + pool_size = 25 + ``` + +=== "8 processes" + ```toml title="pgdog.toml" + [general] + default_pool_size = 25 + + [[databases]] + name = "prod" + host = "10.0.0.1" + min_pool_size = 6 + ``` + + ```toml title="users.toml" + [[users]] + name = "postgres" + database = "prod" + pool_size = 12 + ``` + +### Supported settings + +The following configuration options are supported for pool size autoscaling: + +| Section | Configuration | +|-|-| +| [`[general]`](../../../configuration/pgdog.toml/general.md) | [`default_pool_size`](../../../configuration/pgdog.toml/general.md#default_pool_size) (alias: `max_pool_size`) | +| [`[general]`](../../../configuration/pgdog.toml/general.md) | [`min_pool_size`](../../../configuration/pgdog.toml/general.md#min_pool_size) | +| [`[[databases]]`](../../../configuration/pgdog.toml/databases.md) | [`pool_size`](../../../configuration/pgdog.toml/databases.md#pool_size) | +| [`[[databases]]`](../../../configuration/pgdog.toml/databases.md) | [`min_pool_size`](../../../configuration/pgdog.toml/databases.md#min_pool_size) | +| [`[[users]]`](../../../configuration/users.toml/users.md) | [`pool_size`](../../../configuration/users.toml/users.md#pool_size) | +| [`[[users]]`](../../../configuration/users.toml/users.md) | [`min_pool_size`](../../../configuration/users.toml/users.md#min_pool_size) | + +## Orchestrator integration + +Autoscaling actions are performed entirely using the internal PgDog <-> control plane protocol and, therefore, will work with all orchestrators, including Kubernetes, ECS, and manual deployments. + +### Kubernetes + +If using autoscaling and deploying PgDog with our [Helm chart](../../../installation.md), make sure to set the pool-related settings to reflect the _total_ number of connections. For example, if deploying 3 replicas and the total pool size across the 3 pods is 600 connections, set it accordingly in `values.yaml`: + +```yaml title="values.yaml" +defaultPoolSize: 600 +replicas: 3 + +users: + - name: "postgres" + database: "prod" + minPoolSize: 300 # 50% of the pool +``` + +When PgDog pods are started, they will connect to the control plane and automatically adjust the pool settings to reflect the total number of pods in the deployment: + +| Configuration | Value | Autoscaled value | +|-|-|-| +| `defaultPoolSize` | `600` | `200` | +| `minPoolSize` | `300` | `100` | + +!!! note "Automatic adjustment" + The configuration adjustment happens inside the PgDog process. The control plane does not + mutate the `ConfigMap` resource, so GitOps tools like ArgoCD will not detect this drift + and continue to operate normally. diff --git a/docs/enterprise_edition/control_plane/features/index.md b/docs/enterprise_edition/control_plane/features/index.md new file mode 100644 index 00000000..f33d33ed --- /dev/null +++ b/docs/enterprise_edition/control_plane/features/index.md @@ -0,0 +1,13 @@ +--- +icon: material/format-list-bulleted-type +--- + +# Features + +The control plane provides operational features for monitoring, configuring, and managing PgDog deployments. + +| Feature | Description | +|-|-| +| [Alerts](alerts.md) | Detect outliers in live PgDog metrics and send incidents to supported providers. | +| [Autoscaling](autoscaling.md) | Automatically resize connection pools when PgDog nodes are added or removed. | +| [Slack integration](slack.md) | Send deployment and configuration notifications to a Slack channel. | diff --git a/docs/enterprise_edition/control_plane/features/slack.md b/docs/enterprise_edition/control_plane/features/slack.md new file mode 100644 index 00000000..5e77da79 --- /dev/null +++ b/docs/enterprise_edition/control_plane/features/slack.md @@ -0,0 +1,36 @@ +--- +icon: material/slack +--- + +# Slack integration + +The control plane can send notifications to a Slack channel. Notifications include changes to managed PgDog deployments, like Helm chart updates, configuration reloading, and updates to database topology. + +## Configuration + +Slack integration is **disabled** by default and can be enabled with configuration: + +```yaml title="values.yaml" +control: + config: + slack: + bot_token: "xoxb-[...]" + channel: "C0123456789" +``` + +The following parameters can be configured via the [Helm chart](../installation.md): + +| Parameters | Description | +|-|-| +| `bot_token` | The Slack API bot token. | +| `channel` | The Slack channel ID where the Slack bot is allowed to post messages. | + +## Notifications + +If Slack integration is configured, the control plane will send the following notifications to a Slack channel: + +| Notification | Description | +|-|-| +| Helm install (start) | The control plane starts executing the `helm upgrade --install` command to modify/create a PgDog deployment. | +| Helm install (end) | The control plane is finished executing the `helm upgrade --install` command. If any errors occur, they are included in the notification. | +| Configuration reload | The control plane triggers a configuration reload on all PgDog instances in a deployment. | diff --git a/docs/enterprise_edition/control_plane/index.md b/docs/enterprise_edition/control_plane/index.md index 2ccc934d..3e22c144 100644 --- a/docs/enterprise_edition/control_plane/index.md +++ b/docs/enterprise_edition/control_plane/index.md @@ -73,3 +73,10 @@ PgDog transmits the following information to the control plane: | Queries | Queries that are currently executing through each PgDog node. | | Query plans | Output of `EXPLAIN` for slow and sampled queries, collected in the background. | | Configuration | Current PgDog settings and database schema. | + +## Read more + +{{ next_steps_links([ + ("Installation", "installation.md", "Deploy the control plane alongside your PgDog nodes and connect them together."), + ("Features", "features/index.md", "Explore control plane features like alerts, configuration management, instances, metrics, and Slack integration."), +]) }} diff --git a/docs/enterprise_edition/control_plane/installation.md b/docs/enterprise_edition/control_plane/installation.md index 5c31d9ed..51701643 100644 --- a/docs/enterprise_edition/control_plane/installation.md +++ b/docs/enterprise_edition/control_plane/installation.md @@ -15,7 +15,7 @@ helm install control pgdogdev-ee/pgdog-control The chart has a few external requirements, [documented below](#requirements). -## Guided install +## Dependencies While the chart creates and manages several resources, including an `Ingress`, some of them have external dependencies which cannot be created by Helm. @@ -31,9 +31,6 @@ The script requires that you have both the `awscli` and `kubectl` installed, whi !!! note "Read-only actions" The guided installation script is strictly **read-only** and will never make any modifications to your environment. - -## Requirements - Since the chart creates an `Ingress` resource for the web dashboard, an ingress controller is required to access the web dashboard. The chart supports four Ingress settings out of the box: | Ingress | Description | @@ -49,7 +46,7 @@ If deploying the dashboard with access to the Internet, make sure to configure a ## Ingress -Most of the settings that need to be provided are around the Ingress and OAuth authentication. The [guided install](#guided-install) will configure them automatically. However, if you're installing manually, they are documented below: +Most of the settings that need to be provided are around the Ingress and OAuth authentication. The [guided install](#dependencies) will configure them automatically. However, if you're installing manually, they are documented below: | Setting | Description | Example | |-|-|-| diff --git a/docs/images/ee/alerts.png b/docs/images/ee/alerts.png new file mode 100644 index 00000000..5339c487 Binary files /dev/null and b/docs/images/ee/alerts.png differ diff --git a/main.py b/main.py index 1e02e23b..e24a5b5f 100644 --- a/main.py +++ b/main.py @@ -10,7 +10,7 @@ # Latest released tag for the Enterprise Docker images. Update this in one # place; reference it in docs with {{ enterprise_tag }}. Can be overridden at # build time with the ENTERPRISE_TAG environment variable. -ENTERPRISE_TAG = os.environ.get("ENTERPRISE_TAG", "v2026-06-15") +ENTERPRISE_TAG = os.environ.get("ENTERPRISE_TAG", "v2026-06-17") def define_env(env): diff --git a/tests/pgdog-docker.sh b/tests/pgdog-docker.sh new file mode 100755 index 00000000..0cadfcd2 --- /dev/null +++ b/tests/pgdog-docker.sh @@ -0,0 +1,9 @@ +#!/bin/bash +set -eo pipefail + +PGDOG_IMAGE="${PGDOG_IMAGE:-ghcr.io/pgdogdev/pgdog-enterprise:main-ent}" + +docker run --rm \ + -v /tmp:/tmp \ + "$PGDOG_IMAGE" \ + /usr/local/bin/pgdog "$@"