Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
625 changes: 229 additions & 396 deletions README.md

Large diffs are not rendered by default.

127 changes: 127 additions & 0 deletions docs/pgx/benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# PGX benchmarks

The numbers behind the README's Benchmarks section, with every table and the isolation and query-speed results. The full report is [`linux-results.md`](linux-results.md) and the frozen v0.1 numbers are [`v0.1-baseline.md`](v0.1-baseline.md).

Measured on a 4 vCPU / 8 GB / $8.50-a-month VPS with ZFS block cloning. The
load generator ran on the same CPUs, so these are architecture numbers,
not capacity limits. Every figure has a script and a committed raw result
([`benchmarks/pgx/linux/README.md`](../../benchmarks/pgx/linux/README.md)).

## Density

| | PGX |
|---|---|
| 1000 idle databases in one runtime | **128–155 MB** in total |
| 1000 databases, each queried once | **213 MB** |
| Per idle database | ~70 KB untouched, ~128 KB once queried |
| 1000 databases with 100 active | 283 MB |
| Idle CPU with 1000 databases | 0.33–0.43 % of one core |

For comparison, one idle PostgreSQL 18 server uses 16.6 MB.

## Creating databases

| | p50 | p99 |
|---|---|---|
| Database from the warm pool | 6.2 ms | 9.4 ms |
| Cold database (copy-on-write clone of a 600-file template) | 74 ms | 102 ms |
| 500 simultaneous new-database requests | all served, 0 errors | drained in 12.6–13.8 s |

Times run from the client's connection to its first `SELECT 1` answer,
on the same host.

## Branch storage (ZFS copy-on-write)

| Template | Branch ready in | New physical storage |
|---|---|---|
| 1 GB | 135 ms | 2.3 MB |
| 2 GB | 198 ms | 3.7 MB |
| 5.2 GB | 417 ms | 7.0 MB |
| 10 GB | 674 ms | 13.6 MB |

A branch of a 10 GB database is ready in under a second and initially uses
hundreds of times less physical storage than a full copy.

## Running for a long time

| | Runtime memory |
|---|---|
| After 10,000 create → use → drop lifecycles | 108 MB |
| After 30,000 lifecycles | 111 MB (~0.2 KB retained per lifecycle) |

## Compatibility

| | Result |
|---|---|
| PostgreSQL regression suite (`src/test/regress`, PG 18.6 expected output) | 219 / 231 byte-exact; 12 differ only in EXPLAIN plan text; **0 result differences**. Measured under the PGX profile with parallel workers and `synchronous_commit` restored (`configs/pgx-regress-overlay.conf`), the two settings the suite asserts on; under the shipped ephemeral profile those two settings alone make `select_parallel` and `stats` differ. One suite, one platform: not a compatibility score. |
| Real Rails 8.1 app (migrations, fixtures, model tests, schema change, `db:schema:load`) | passes; within 0–5 % of PostgreSQL 18 per phase |
| Django 5.2, Prisma 6, SQLAlchemy 2, node-postgres, Rails-shaped SQL | pass |
| All framework runs | **36 / 36** |

Transactional behaviour follows PostgreSQL, including transactional DDL
and temporary tables:

```sql
BEGIN;
CREATE TEMP TABLE pgx_test (id integer PRIMARY KEY, value text);
INSERT INTO pgx_test VALUES (1, 'PGX works');
SAVEPOINT before_update;
UPDATE pgx_test SET value = 'changed' WHERE id = 1;
ROLLBACK TO SAVEPOINT before_update;
SELECT value FROM pgx_test WHERE id = 1; -- 'PGX works'
ROLLBACK;
SELECT to_regclass('pg_temp.pgx_test'); -- NULL: the table was rolled back
```

## Isolation

- **Memory limits.** Session, database and runtime limits refuse runaway
work with SQLSTATE 53200. The session stays usable, other databases see
0 errors, and the runtime does not crash.
- **Connection limits.** A neighbour running a 50-connection migration
mix pushes another database's p99 latency to 9.8 ms. Capped at 1 / 2 / 5
connections per database, it stays at 1.2 / 2.1 / 4.0 ms.
- **cgroups.** The kernel is the last line: under a cgroup memory limit,
the worst case is that one runtime is killed. The host survives.

## Query speed

PGX is not trying to beat Postgres on single queries.

- **Short statements:** about 0.05–0.15 ms slower than PostgreSQL 18; a
point SELECT takes 0.25 vs 0.13 ms.
- **Large single-threaded scans:** 1.5–2.2× slower.
- **Equal:** COPY and 4-client throughput.

Whole framework lifecycles take 0–15 % longer than on PostgreSQL 18.

PGX concentrates on database creation latency, density, memory
efficiency, copy-on-write storage, high churn and isolation between
tenants. For application lifecycle workloads it stays close enough to
PostgreSQL while making large numbers of isolated databases dramatically
cheaper to keep alive.

The full report:
[`docs/pgx/linux-results.md`](linux-results.md). The frozen v0.1
numbers: [`docs/pgx/v0.1-baseline.md`](v0.1-baseline.md).

## Reproducing the benchmarks

The harness is Python 3 standard library only and talks the wire protocol
directly. On a Linux host with a ZFS pool:

```bash
PGXBENCH_WORKDIR=/tank/pgx-bench benchmarks/pgx/linux/run-suite.sh scale churn pool cow regress
```

Results land in `benchmarks/pgx/results/<host>-<commit>/`.
[`benchmarks/pgx/linux/README.md`](../../benchmarks/pgx/linux/README.md) maps
every headline number in this document to its step, script and raw result.

The regression suite is PostgreSQL's own `src/test/regress`, vendored
unmodified at `crates/postgres-18.6-reference/src/test/regress`. It runs
with stock `pg_regress` (`benchmarks/pgx/regress-baseline.py`) under the
PGX profile plus `configs/pgx-regress-overlay.conf` (parallel workers and
`synchronous_commit` back on, because the suite asserts on them), and each
difference is classified as plan-only or semantic. The nightly run fails on
any semantic difference (`classify-regress-diffs.py --fail-on-semantic`).
Binary file added docs/pgx/img/arch-density.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pgx/img/arch-density.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pgx/img/architecture.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pgx/img/disposable.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pgx/img/disposable.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/pgx/img/star-history.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
77 changes: 77 additions & 0 deletions docs/pgx/local-dev/build-from-source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Building PGX from source

The prebuilt distribution (`npx pgxdb@latest` or `curl -fsSL https://pgx.dev/install | sh`, see [`install.md`](install.md)) is the normal way to run PGX. This page is for building the engine yourself, running a bare runtime without the `pgx` CLI, and minting databases from a sealed template by hand.

## Build

You need the Rust toolchain (`rust-toolchain.toml` pins Rust 1.96.0;
rustup fetches it), the RE2 regex library, and the PostgreSQL 18 client
tools for `initdb` and `psql`:

- Debian/Ubuntu: `sudo apt-get install -y build-essential pkg-config libre2-dev`,
plus `postgresql-18` and `postgresql-client-18` from the
[PGDG apt repo](https://www.postgresql.org/download/linux/ubuntu/)
(then `export PATH="/usr/lib/postgresql/18/bin:$PATH"`).
- macOS: `brew install re2 pkg-config postgresql@18`, then
`export PATH="$(brew --prefix postgresql@18)/bin:$PATH"`.

```bash
cargo build --release --locked --bin postgres
```

The server binary is `target/release/postgres`. Release builds refuse to
compile without RE2 on purpose: the fallback regex engine is much slower.

At run time the server loads two libraries with `dlopen` rather than
linking them: libxml2 for the `xml` type and SQL/XML functions, and ICU for
ICU collations. The PostgreSQL 18 packages above install both. A host
without libxml2 still starts, but XML functions raise `unsupported XML
feature` (SQLSTATE 0A000) with a hint naming the library, and
`pgrust_runtime_status()` reports `libraries.libxml2.available = false`.
Debian/Ubuntu package: `libxml2` (`libxml2-16` from Ubuntu 25.10); macOS
ships libxml2 with the system.

## Run a PGX runtime

```bash
# Create a cluster and apply the PGX profile.
initdb -D /tmp/pgx-data --no-locale --encoding UTF8 -U postgres
cat configs/pgx-ephemeral.conf >> /tmp/pgx-data/postgresql.conf

# The server reads timezone and other data files from a Postgres share dir.
export PGRUST_PGSHAREDIR=/usr/share/postgresql/18 # macOS: "$(brew --prefix postgresql@18)/share/postgresql"
export PGRUST_TZDIR=/usr/share/zoneinfo # macOS: "$PGRUST_PGSHAREDIR/timezone"

ulimit -s 65520
RUST_MIN_STACK=33554432 target/release/postgres \
-D /tmp/pgx-data -k /tmp -p 5432 \
-c listen_addresses= -c io_method=sync -c max_stack_depth=60000 \
-c pgrust.ephemeral_db_prefix=tdb_ \
-c pgrust.ephemeral_db_mint_roles=postgres
```

Put the data directory on a ZFS dataset with block cloning enabled to get
copy-on-write branches (the profile sets `file_copy_method = clone`). On a
filesystem without block cloning, remove that line from the profile and
databases are created by copying files.

## Make a template and mint databases

```bash
# Build a template once, then seal it.
psql -h /tmp -U postgres -c "CREATE DATABASE tpl_app"
psql -h /tmp -U postgres -d tpl_app -c "CREATE TABLE users (id bigserial PRIMARY KEY, email text UNIQUE)"
psql -h /tmp -U postgres -c "SELECT pgrust_seal_template('tpl_app')"

# Connecting to <prefix><template>__<token> creates that database.
psql -h /tmp -U postgres -d tdb_tpl_app__agent42 -c "INSERT INTO users (email) VALUES ('a@example.com')"
psql -h /tmp -U postgres -d tdb_tpl_app__agent43 -c "SELECT count(*) FROM users" # 0: isolated

# Runtime status as JSON.
psql -h /tmp -U postgres -Atc "SELECT pgrust_runtime_status()"
```

Minted databases are dropped `pgrust.ephemeral_db_grace` seconds after
their last session ends, unless they are pinned with
`pgrust_pin_database()`. All control functions are listed in
[`docs/pgx/pgrun-interface.md`](../pgrun-interface.md).
2 changes: 1 addition & 1 deletion docs/pgx/local-dev/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ that reaps processes.

## From source

See the repository README "Quickstart → Build": the Rust toolchain (pinned by
See [`build-from-source.md`](build-from-source.md): the Rust toolchain (pinned by
`rust-toolchain.toml`), RE2, and PostgreSQL 18 client tools, then
`cargo build --release --bin postgres --bin pgx` and `dist/assemble.sh`.

Expand Down
Loading