Tutorial 6 — growing up

The app code is identical at every rung. Climbing is an env var, a driver swap and someone else's infrastructure — never a rewrite. A small idea reaches a big output by changing where the rows and the messages live, not by changing what an entity, policy, action, mutator, query, job, route or task says.

As of 2026-08. The full ladder, with costs, seams and the places the invariant breaks, is docs/idea/17-scale-ladder.md; running it for real is docs/ops/. This page is the decision procedure, not a second copy of either.

Series: 1 — first app · 2 — first feature · 3 — auth and admin · 4 — jobs and realtime · 5 — deploy free · 6

Climb on a signal, never on a feeling

Skipping a rung to look serious buys a second product to maintain. A Kubernetes cluster running one app is that.

What is allowed to change

Rung 0 → 1: split the roles

Same platform, same image. One service per ROLE. This is the rung a free tier's sleeping instance forces whether or not traffic did — see tutorial 5.

scheduler leadership is an expiring lease row in x_scheduler_leader, never an advisory lock — that grant belongs to the session, not to the process: it outlives every transaction on the connection and is released only by an explicit unlock, the pool's reset on release, or the connection dying — so a node can neither renew it nor prove it still holds one — so a second instance is an idle standby rather than a duplicate. It has no /readyz at all: ROLE=scheduler binds the metrics port only, and nothing routes request traffic to it.

The shared cache tier

Add it when there is more than one web replica and a measured cross-replica miss. Not before.

// app.config.ts
cache: { tiers: ['request-memo', 'lru', 'redis'] },

Read order is request-memo → LRU → redis → origin, and a tier is never consulted for a request whose policy has not already passed. Adding one changes where a value is found, never how it is asked for.

Any Redis-protocol server, As of 2026-08. The tier-3 invalidation script used to DEL keys it never declared in KEYS, which Dragonfly and Redis Cluster reject. Fixed: the script touches only the buckets it was handed and returns the value keys for the client to drop one at a time, the buckets carry an {entity} hash tag, and invalidateTags issues one call per tag — so every key of a call co-slots. Redis, Valkey and Dragonfly all work. Clustered deployments are unmeasured, not unsupported: no test in the repo runs against a real cluster node.

Detail: Caching and invalidation.

Rung 1 → 2: one box, Compose

docker/docker-compose.prod.yml is the production topology — one service per role, one image, differing only by ROLE. migrate runs to completion and every serving role waits on service_completed_successfully.

bunx x deploy --image ghcr.io/you/myapp:1.2.3 --method compose

Start Postgres with wal_level=logical, max_replication_slots=8 and max_wal_senders=8 so the replicator role can preflight rather than fail on its first change.

The compose ceiling is one replica of each serving role, declared As of 2026-08. The framework's file and the scaffolded one both publish a host port on web and sync and both set replicas: 1 — one host port has exactly one binder. worker publishes no port and scales freely.

Two ways past it: delete their ports: lines and put a reverse proxy of your own on the compose network, which the service names resolve across every replica; or take the next rung, where the chart's per-role HPA does it. The framework ships neither proxy — one in docker-compose.prod.yml is a dependency every app inherits and a second answer to "how does traffic reach a role" beside the chart's Ingress.

Rung 2 → 3: Kubernetes

x new writes the chart, As of 2026-08-19 — docker/helm, 8 files — a Deployment for each of the four roles enabled by default (web, sync, worker, scheduler), replicator behind enabled: false, and migrate as a Job — so x deploy --method helm runs helm upgrade --install against it with nothing to copy in. On 3.0.0 and below x new writes none and the command exits X_NOT_IMPLEMENTED: copy docker/helm from the framework repo.

The alternative is the flat manifests in docs/ops/01-kubernetes.md. Carry one, never both — a repo with a chart and a manifest tree has two sources of truth for the same pod. Deleting docker/helm is how you choose the manifests: the command then fails with helm's own error, not the framework's.

Three things turn on at this rung, all by env, none by a code path:

Do not enable replicator until live queries are in use. The chart ships it enabled: false for that reason.

Per-role HPAs target rps, connections and queue_demand — CPU is a lagging proxy for all three serving roles. Every role serves /metrics on METRICS_PORT (default 9090) and http/realtime/jobs emit the series. The chart still cannot scrape them: no role declares a metrics container port and no scrape target ships, so an HPA sits at <unknown> until you add both plus a custom metrics adapter.

Rung 3 → 4: three separable moves

Observability is the part that is not optional here, and the framework's half is done As of 2026-11: there is no code to write. The production boot (apps/web/server.ts, every ROLE) reads the variables the OpenTelemetry SDKs read and wires all three signals — traces, metrics and logs — from them at boot:

OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318   # all three signals
OTEL_LOGS_EXPORTER=none                                   # optional: one signal off

Point it at your collector's HTTP receiver, :4318 — :4317 is gRPC and is refused with X_OTLP_PROTOCOL_UNSUPPORTED, permanently, because HTTP/2 plus protobuf is a dependency and a second wire format. A credential goes in OTEL_EXPORTER_OTLP_HEADERS, never in the URL (X_OTLP_ENDPOINT_INVALID). OTEL_SDK_DISABLED=true turns all three off. Metrics also stay scrapeable at METRICS_PORT if you prefer an agent. docs/ops/03-observability.md says what to scrape and what to alert on; anything else that wants the stream subscribes to it — subscribe('span.end', …), subscribe('log.record', …) (Observability).

The seams, so nothing surprises you

Every one keys on env or a boot-time constructor. One image resolves x dev, a Compose host and a cluster identically.

Rehearse the climb before you need it

Where the invariant genuinely breaks

Named rather than left to be discovered:

Each sits behind an interface that ships today and fails loudly, rather than pretending to work. The full list, with a workaround per row: Known gaps.

Read next

Related: Caching and invalidation · Realtime · Jobs and workflows · Troubleshooting