# 23.x → 24.0.0, entry by entry

**Two hundred and two entries** — `CHANGELOG.md`'s 24.0.0 entries in their order: grouped by package, lowest tier first. No legacy path, no
codemod, no compatibility shim — every break is a build error or an `X_*` error naming the rewrite.
`As of 2026-10` slices 01–10 have landed: `@ultimat3/schema` and `@ultimat3/core`; tier 1 —
`i18n`, `time`, `db`, `flags`; then `cache`, `seo`, `storage` and one `render` entry; then slice 04, complete — tier 2's `entity`, `policy` and `http`, with one `testing` entry; then slice 05, `auth`; then slice 06, complete — `query`, `mcp`, `admin`, then `action` and the `manifest` step; then slice 07, `jobs`; then slice 08, `realtime`; then slice 09, complete — `render` and `pwa`, with one `http` entry, then `ui`; then slice 10, complete — `http` again, `mcp`, `mail`, `notify` and `manifest`, then `ai`; then slice 11, complete — `testing`,
`scraping` and `admin`; then slice 12a — `cli`'s build, generator, test-selection and policy half, with
`seo`, `policy` and `testing`; then slice 12b — `cli`'s runtime and commands, with `http` and the
chart; then slice 13a, the repository's guards, with one `cli` entry; then slice 13b, CI and the
chart; then slice 14, one home per helper — `render`, `cli`, `schema` and `core`. **Realtime has three operator steps: entries
88–90, upgrade steps 32–34.** **A deployment with MFA-enrolled users has an
operator step: entry 58, upgrade steps 18–20.** **A gateway that declares `actor` or `org`
ceilings: entry 123, upgrade step 49.** **The `policy` and `drift` steps go red on an
existing app: entries 169–170, upgrade step 61.** **Every deploy runs `ROLE=migrate` before its
serving roles: entry 174, upgrade step 65.** **A chart upgrade needs its values read first:
entries 192–194, upgrade step 72.** A later slice appends
its rows below the last one and never renumbers; each row's `CHANGELOG.md` line names it as
`(#N)` after the dash.

### The upgrade, top to bottom

| # | Do | What you see until you do | Entries |
|---|---|---|---|
| 1 | pin every `@ultimat3/*` to the one new version, `bun install` | nothing yet — a mixed install is untested | — |
| 2 | `bun run typecheck`, then `x verify --only typecheck,unit` | `X_SCHEMA_DEFAULT_INVALID` or `X_CONFIG_INVALID` at the first import of the file that declares it | 4, 6 |
| 3 | read the deploy environment: `LOG_LEVEL`, `ULTIMATE_CURSOR_SECRET`, `OTEL_EXPORTER_OTLP_TRACES_HEADERS`, `OTEL_EXPORTER_OTLP_METRICS_HEADERS`, `OTEL_TRACES_SAMPLER` | a boot that exits on `X_INVARIANT` or `X_CURSOR_SECRET_DEV`; a collector that rejects one signal; every root trace sampled where a leftover ratio thinned them | 7, 13, 14, 16 |
| 4 | `x verify --only unit,contract,e2e` and fix the tests it fails | a date, URL, object or query number that validated and is now refused; a redacted field a test read | 1–3, 5, 8–11 |
| 5 | repoint error-monitor rules from `extra.<key>` to `extra.meta.<key>`; add an exact host rule for each internal address a wildcard used to admit | a saved search that matches nothing; a refused request to `127.0.0.1` | 12, 15 |
| 6 | `bun run typecheck` again for the tier-1 types: add `zone` to each `formatRelative` call, `wildcardTime` to a hand-built `CronExpression`, `case 'changed-primary-key':` to a `DriftKind` switch, `.expression` to a `CatalogColumn.generated` read | TS2741 / TS2339 at each site | 18, 20, 26, 28 |
| 7 | load the app once (`x verify --only unit`) and fix each cron and flag declaration it refuses | `X_CRON_INVALID`, `X_FLAG_EXPIRY_INVALID` at the first import of the declaring file | 19, 30 |
| 8 | `x db gen` where the database holds an object entry 29 lists; commit `packages/db/schema/` | a dump that no longer matches the one the gate regenerates | 29 |
| 9 | `x verify --only unit,contract,job,live` and fix the tests it fails; read every `catch` and every `Promise.all` inside a `withTransaction` body | `X_DB_TRANSACTION_ABORTED` where a call used to resolve; `X_DB_SIBLING_SCOPE_TIMEOUT` after a 30 s wait; `⟦name⟧` in a rendered string; a relative date counted in calendar days | 17, 20–25, 27, 31 |
| 10 | `bun run typecheck` for the storage types: add `policy` to each `promoteAttachment` call, `stat()` to a hand-written `StorageDriver`, a guard to each `lastModified` read, a base path to `canonicalRequest` / `signConstraints` | TS2741 / TS18048 / TS2554 at each site | 35–37, 41 |
| 11 | load the app once (`x verify --only unit`) and rename each cache tag and `revalidate.tags` entry it refuses | `X_ROUTE_MODE_INVALID` at registration; `X_CACHE_PURGE_FAILED` where a tagged response is built | 33, 43 |
| 12 | `x verify --only unit,contract,e2e` and fix the tests it fails; add the real media types to each upload policy; pass `{ formats: FORMAT_ORDER }` where your driver encodes AVIF | a header or `<picture>` snapshot that changed; `X_STORAGE_TOO_LARGE` from a `get()`; an upload refused for its sniffed type | 32, 34, 38, 40, 42 |
| 13 | after the deploy: list and delete the old-shape image variants; expect signed URLs minted before it to fail for 15 minutes; purge the CDN once if a collection bust must reach older edge copies | orphaned variant files; an outstanding signed URL that no longer verifies | 32, 39, 41 |
| 14 | `bun run typecheck` for tier 2: add `transactor()` to a hand-built or wrapping `Driver`, import `dbDrift` from `@ultimat3/db`, retype a `403` denial status as `DenialStatus` | TS2741 / TS2305 / TS2322 at each site | 44, 45, 48 |
| 15 | `x verify --only unit,contract,policy` and fix the tests it fails; read every `catch` around `assertAllowed` and every policy predicate's return | `X_UNAUTHENTICATED` and a 401 where a test expected `X_FORBIDDEN` and 403; a denial where a malformed decision allowed; `X_INVARIANT_VIOLATED` from a preload over 10,000 rows | 46–49 |
| 16 | set the http config the deploy needs: `trustClientCertHeader: true` where `ctx.peer` is read behind a proxy that strips the header, `healthDetailPeers` for an off-box health reader, `cors.origins` for a cross-origin browser form, `hostname` for an embedder | `ctx.peer` is `null`; a health body of three fields; `X_CSRF_BLOCKED` on an anonymous form post | 50, 51, 53, 56 |
| 17 | `x verify --only unit,contract,e2e` and fix the tests it fails: `await` every browser `close()`, send `content-type` with every body, write `private, max-age=N` where a handler meant a per-user lifetime | `X_BODY_INVALID`; a rewritten `cache-control`; 429 after repeated 401s; a leaked Chrome | 52, 54, 55, 57 |
| 18 | **before the deploy, where any user has MFA:** `x secrets init`, or set `ULTIMATE_SECRETS_KEY` in the deploy | `X_SEAL_KEY_MISSING` at `login()` for an enrolled user | 58 |
| 19 | `bun run typecheck` for auth: a custom `AuthLimiter` implements `reserve` / `refund`; a custom `AuthAdapter` implements the eight members; callers of `verifyApiKey` read `.record`; delete imports of `redeemRecoveryCode`, `mfaRequired`, `authNotImplemented`; enrolment writes through `saveTotpSecret`; set `APP_URL` | TS2741 / TS2305 / TS2339 at each site; `X_ENV_MISSING` from `oauthLogin` | 60, 61, 64, 65, 67 |
| 20 | **deploy, then `x auth seal-mfa --json` once** | `X_MFA_SECRET_UNSEALED` (500) at the second factor for every enrolled user | 58 |
| 21 | `x verify --only unit,contract,e2e`; move the second-factor client to `meta.challenge` and `completeMfa`; list each API key's scopes and pass `grantsOf` where users hold roles | a key that resolves with fewer scopes or 401; a client still reading `meta.userId`; `X_AUTH_WRITE_FAILED` where a test matched `X_DB_UNIQUE_VIOLATION`; `X_CDP_CALL_FAILED` from an `offline()` a page refused | 61–63, 66, 68 |
| 22 | once every replica runs this release: `x doctor --json`, then `psql "$DATABASE_URL" -c 'drop table if exists x_auth_failures'` | `X_FRAMEWORK_TABLE_ORPHANED` from `x doctor` | 59 |
| 23 | `bun run typecheck` for slice 06: delete `compareValues`, pass `kindsOf(shape.entity)` to `compareRows` / `matchesFilter` / `isAfterKey`, replace `adminPermissions` with `...ADMIN_PERMISSIONS` in `definePermissions([...])` | TS2305 / TS2554 at each site | 69, 72 |
| 24 | `x verify --only unit,contract,e2e`; drop `.limit()` from each read a client pages to the end; read the row, not `.rows`, from a `single: true` MCP tool | a listing that ends at its limit; `X_CURSOR_INVALID` once for a cursor minted before the deploy; an MCP answer of a different shape | 70, 71 |
| 25 | add `idempotent: true` to every `mutator({ … })`; on more than one replica also `configureIdempotency({ scope: 'shared' })`. A hand-built `postgresIdempotencyStore` adds `origin` and `reclaimAfterMs` | TS2741 at each site; `X_MUTATOR_NOT_IDEMPOTENT` at declaration from untyped code | 73, 77 |
| 26 | `x manifest`, commit `x.manifest.json` and `openapi.json` | the `manifest` step is `X_MANIFEST_STALE` | 74, 79 |
| 27 | `x db gen`, commit `packages/db/schema/` | the `drift` step is `X_SCHEMA_DUMP_DRIFT` for `x_idempotency` | 75 |
| 28 | `x verify --only unit,contract,job`; read each test that rolls back an idempotent action or asserts a cache bust inside `withTransaction` | a record left `in-flight` after a rollback; a bust that fires at commit; `X_IDEMPOTENCY_RESERVATION_LOST` (409) on a slow attempt | 76, 78 |
| 29 | `bun run typecheck` for jobs: a custom `EventBus` returns a promise from `purgeExpired()`, a custom `EventLookup` adds `now()`, a hand-built `Lease` adds `abandon()`, a hand-built `RetentionStores` adds `events`; a caller of `SQL_CLAIM` / `SQL_SCHEDULER_FIRE` binds the extra parameter | TS2741 / TS2322 at each site; a statement that fails to bind | 81, 85, 87 |
| 30 | read every job with `retry.attempts: 1`, every script that cancels a finished job, and every `runId` passed to `enqueue` | a job buried `dead` after its worker died; `X_JOB_NOT_CANCELLABLE`; `X_ID_INVALID` | 80, 82, 84, 86 |
| 31 | `x verify --only unit,job` and fix the tests it fails | `X_JOB_LEASE_LOST` from a step write after a lapsed lease; a dead letter where a test expected a re-run | 80, 83 |
| 32 | **before the deploy:** make `APP_URL` on the `sync` role the origin the pages are served on — a declared origin is the whole allow-list; add it to `.env.production` for an older compose file | every websocket upgrade is `403 X_SOCKET_ORIGIN_REFUSED` | 88 |
| 33 | **where a channel is declared with params:** `x db gen "replica identity full"`, then `x db migrate` | `replication.channel_identity_partial` in the replicator log; deletes on those channels not announced | 89 |
| 34 | **before the deploy:** check the replication role's password method and the connection's `sslmode`; set `?sslmode=verify-full`, move the role to scram-sha-256, or state `?sslmode=disable` | `X_REPLICATION_FAILED` when the replicator dials | 90 |
| 35 | roll `replicator` and `sync` together — they must run the same major; point an off-box health reader of the sync role at `healthDetailPeers` | rows that do not parse across a mixed fleet; a health body of three fields | 91, 93 |
| 36 | `bun run typecheck` for realtime: a custom `Transport` adds `onReconnect`, a custom `AdvisoryLock` adds `onLost` and `abandon`, a custom `ChangeFeed` adds `abandon`, an `UpgradeTarget` adds `requestIP`, `UpgradeDeps` adds `healthDetailPeers`; replace `.hasOwnProperty(` on decoded records with `Object.hasOwn(` | TS2741 at each site; a `TypeError` on a null-prototype object | 92, 96 |
| 37 | `x verify --only unit,live,e2e` and fix the tests it fails | `X_TOPIC_FORBIDDEN` where a channel kept delivering; `X_OFFLINE_QUEUE_ABANDONED` for a write queued across a sign-out | 94, 95 |
| 38 | `bun run typecheck` for render: add `case 'unparsable':` to an exhaustive `switch` over `NativeReason` | TS2322 / TS2366 at the switch | 99 |
| 39 | load the app once (`x verify --only unit`) and give each `isr` route whose only trigger is a zero-length `ttl` a positive `ttl` or `tags` | `X_ROUTE_MODE_INVALID` at registration, naming the file | 98 |
| 40 | `x build`, then `x verify --only unit,contract,e2e`; handle `path === ''` in each catch-all handler, read an escaped CSS-module class by its unescaped name, replace a stored `specificity` with a comparison, point each push `url` at this origin | a 200 where a test expected a 404 at a catch-all's bare prefix; a snapshot holding the old scoped class name; `styles['w-1']` undefined; a tap that opens the app root | 97, 100–102 |
| 41 | `bun run typecheck` for ui: add `current()` and `appDefault()` to a hand-built `ThemeEnv`; delete each import of `THEME_INLINE_SCRIPT` / `themeInlineScript*`, its hand-inlined `<script>` and its `sha256-…` in your CSP; widen a `Popover` trigger's `aria-controls` to `string \| undefined` | TS2741 / TS2305 / TS2322 at each site | 103, 105, 106 |
| 42 | `x build --target static`, then `x verify --only unit,e2e,budgets`; set `theme: { defaultMode: 'system' }` where the page must follow the OS; read each upload test that posts a file the control refuses | `X_BUDGET_UNMEASURED` on every budgeted route until the build; a "System" pick that stays dark; a refused file missing from the request | 104, 107 |
| 43 | `bun run typecheck` for http: a hand-written `RateLimitStore` adds `peek(key, bucket, nowMs)`, a hand-written `RateLimiter` adds `peek(key, bucketName)` | TS2741 at each site | 108 |
| 44 | **before the deploy:** list every probe, monitor, script or agent that sends a credential that can fail — to an `auth: 'required'` route or to the MCP endpoint — from an address real users or agents share; raise `rateLimit.defaultBucket` or `rateLimits.unauthenticated`, or move it | `429 X_RATE_LIMITED` / `429 X_MCP_RATE_LIMITED` for a right credential after a run of wrong ones | 109, 114 |
| 45 | `x manifest`, then `x db gen`; commit `x.manifest.json`, `openapi.json` and `packages/db/schema/`. Where `x manifest` refuses `X_MANIFEST_FACT_INVALID`, make the declaration at `meta.path` finite and run it again | `X_MANIFEST_STALE` from the `manifest` step; `X_MANIFEST_BREAKING` for `jobs.mail.send.input` from `contract-diff`; `X_SCHEMA_DUMP_DRIFT` for `x_notify_digests` from `drift` | 116–119 |
| 46 | `x verify --only unit,contract,job,e2e` and fix the tests it fails; read each MCP client that parses a 5xx `cause`, each `db.query` an agent runs, each hand-registered tool's `destructive`, each `manage_resource` list call, each `send()` inside `withTransaction` and each recipient a mail is sent to | a fixed cause where the server's was; `X_MCP_QUERY_REJECTED`; a writing tool listed as `query`; `X_INPUT_INVALID` for a list key; a mail sent only after the commit; `X_MAIL_ADDRESS_INVALID` or `X_VALIDATION_FAILED` at `send()` | 110–113, 115, 116 |
| 47 | before dropping a default from a NOT NULL column: bump the app's major in `package.json`, or keep the default | `X_MANIFEST_BREAKING` from `contract-diff` naming `…hasDefault` | 120 |
| 48 | `bun run typecheck` for ai: a hand-written `BudgetStore` adds an atomic `take(key, tokens, limit)`; a hand-written or wrapping `Gateway` implements `callLedger(keys)`; a hand-written `VectorStore` adds `prune(filter, keep)`; replace each `fnv1a` import with core's `fingerprint` | TS2741 / TS2554 / TS2305 at each site | 121, 122, 125, 129 |
| 49 | **before the deploy, where `createGateway({ budget })` declares `actor` or `org`:** size each ceiling for every `llm()`, `agent()` and hive member a caller makes — every anonymous caller shares one counter — and key each `gateway.scope()` with `budgetKeysFor(actor)` | `X_AI_BUDGET_EXCEEDED` where the call ran | 123 |
| 50 | load the app once (`x verify --only unit,eval`); give each `numericTolerance` a finite, non-negative bound; bind a tenant on each vector store read inside an org's request, or open the backfill store with `scope: UNSCOPED` | `X_INVARIANT` at the eval file's import; `X_VECTOR_UNSCOPED` (500) at the read | 128, 131 |
| 51 | `x verify --only unit,eval,e2e` and fix what it fails: re-index every store `HashEmbedder` filled; give each app code an agent tool throws for the model to read a 4xx `registerErrorStatus` row or a public cause; move a `source` passed to `chunk()` under another key; make a fixture's invented `finish_reason` `stop` | a ranking or vector snapshot that changed; a tool result reading `CODE: ` and a fixed sentence; `X_LLM_TRUNCATED` where an answer was repaired | 124, 126, 127, 130 |
| 52 | `bun run typecheck` for scraping: a hand-written launcher or test double's browser adds `target()` answering `createCDPSession()`, whose session has `send` and `on`; a `burnSession` call passes the version of the record the run used | TS2741 / TS2554 at each site | 137, 139 |
| 53 | load the app once (`x verify --only unit`) and write each `expect.maxDrop` as a fraction | `X_INVARIANT` at `scrape()`, naming the scrape | 140 |
| 54 | `x verify --only unit,e2e` and fix the tests it fails: read `app()` inside a `test` body, set up per mount the state a test carried between island mounts | `X_TEST_APP_NOT_BOOTED` where a test expected `ReferenceError`; a mount that no longer sees the last one's module state; an assertion on key order or a count of clock announcements that changed | 132–135 |
| 55 | `x verify --only unit,job` and fix the scraping runs it fails: move a proxy switch out of `localBrowser({ options: { args } })` into `proxy` or `egress`; add an exact `allowHosts` rule for each internal host a wildcard reached and for each host a `robots.txt` redirects to; re-record each fixture with no readable `recordedAt` | `X_SCRAPE_LAUNCH_ARGS_INVALID`; `X_SCRAPE_HOST_BLOCKED`; `X_NOT_IMPLEMENTED` from a jar-less browser; `X_SCRAPE_FIXTURE_STALE`; a redaction snapshot or a header that changed; robots rules that no longer apply | 136, 138, 141–145 |
| 56 | `bun run typecheck` for admin: a custom `AdminRepo#list` handles a `null` `keyset.value`; an `AdminForm` rendered by hand passes `version`; an exhaustive `switch` over `CrudResult` handles `'stale'` | TS2345 / TS18047 / TS2741 at each site; a switch that no longer compiles | 146, 148, 149 |
| 57 | load the app once (`x verify --only unit`) and rename a list scope declared as `*` | `X_ADMIN_FILTER_INVALID` where the resource is declared | 153 |
| 58 | `x verify --only unit,contract,e2e` and fix the tests it fails: post the rendered `_version` with each hand-posted edit form; read each policy rule over `input` for what an admin create or update writes; read each test that expects an action's or a batch's write to outlive a failed audit entry, or another org's audit rows; fix each link that sends a blank, hex, exponent or off-calendar filter | a 409 where an edit used to save; a denial on a create or update; a rolled-back write with a `failed` entry; an audit list without another org's rows; `X_ADMIN_FILTER_INVALID` | 147, 150–152, 154 |
| 59 | `bun run typecheck` for slice 12a: a hand-built `ProcessRegistrySnapshot` adds `permissionSites`; a one-argument `restorePermissions(names)` call passes the `permissionDeclarationSites()` captured with the names | TS2741 at each snapshot; TS2554 at each `restorePermissions` call | 156, 171 |
| 60 | **where a sitemap is split** (past `maxUrls`, 50,000 by default): `x build --target static`, upload the new `sitemaps/` folder, delete or redirect each old `sitemap-<n>.xml`, re-submit `/sitemap.xml`. A running web role answers SEO from a one-hour memo — restart it to publish a change sooner | a search console reporting the old parts as 404; a sitemap change that shows an hour late | 155, 166 |
| 61 | `x verify --only policy,drift`: add each permission it names to the app's own `definePermissions([...])`; `x db gen "record replica identity full"`, then `x db migrate` | `X_PERMISSION_BORROWED` naming the file; `X_DB_SCHEMA_UNMIGRATED` for `replica identity full` | 169, 170 |
| 62 | `x verify --only filesize,errors,i18n,unit,live,e2e` and fix what it fails | a guard or an `apps/*/server.ts` / `prerender.ts` over 500 lines or with an unrunnable `fix:`; a test under a slice named `build`, `examples` or `dummy` that never ran and now fails | 164, 165 |
| 63 | read each build and CI script: empty a custom `--out` once or build to `.x/static`; drop `--tag` / `--out` / `--no-preflight` where the target never reads it; fix each module `x verify --only manifest --json` lists; read `ok` and the findings of `x g --dry-run`; rename a generator name it refuses; expect more work from `--affected` after a change under `scripts/` or `guards/` | `X_BUILD_OUT_UNSAFE`; `X_BUILD_FAILED`; `X_CLI_BAD_FLAG`; `X_GENERATE_CONFLICT`; a CI job that runs every workspace | 157–163 |
| 64 | optional, to match a fresh scaffold: pin `solid-js` `1.9.15`; set `"typecheck": "tsc --noEmit -p ../../tsconfig.json"` in `apps/web/package.json` and `apps/admin/package.json` | an app typecheck that now reports what the gate already refused | 167, 168 |
| 65 | **before the deploy:** make `ROLE=migrate` run to completion before every serving role. The shipped chart's pre-install/pre-upgrade hook and `docker-compose.prod.yml` already do; a deploy of your own adds the step | every `web`, `sync`, `worker`, `scheduler` and `replicator` pod refuses to boot: `X_FRAMEWORK_SCHEMA_UNAPPLIED` | 174 |
| 66 | read the deploy environment and the monitors: `TRUSTED_PROXY_HOPS` decimal 1–64, `PORT` and `METRICS_PORT` decimal; repoint an alert on `X_PORT_INVALID` for hops to `X_TRUSTED_PROXY_HOPS_INVALID`, and a match on `x routes list --json` to `x routes --json` | a boot that exits on `X_PORT_INVALID` or `X_TRUSTED_PROXY_HOPS_INVALID`; an alert that never fires | 173, 175, 176 |
| 67 | roll the chart: `docker/helm` carries the replicator probe; a chart `x new` wrote copies the two edits from `docker/helm/templates/_helpers.tpl` and `service.yaml`; read each alert on a not-ready replicator | a replicator at 0/1 while its stream is down; a worker that stops claiming at SIGTERM | 178, 179 |
| 68 | `bun run typecheck`: a hand-built `DoctorProbe` adds `appUrl` | TS2741 at each site | 182 |
| 69 | `x verify --only unit,contract,e2e` and fix what it fails: read each handler that sets its own CSP, each link to an uploaded non-media file, each script or test of `/_x/db`, `x routes list`, `x verify merge` parts and MCP `tests.run` | a page under two policies; a download where a tab opened; `405`, `421` or `X_CSRF_BLOCKED` from `/_x`; `X_CLI_UNKNOWN_COMMAND`; `X_VERIFY_MERGE_INPUT`; `failed ≥ 1` | 172, 177, 183, 187, 188, 190 |
| 70 | read each script and CI job that calls `x`: `x help`, `x i18n add\|sync`, `x db seed`, `x db backfill`, `x doctor` behind a `PORT`, `x jobs` with an id, `x shot --out`, the `x dev` and `x mcp serve` port fixes | exit 1 where 0 was; a probe of `PORT`; `X_APP_URL_PORT_MISMATCH`; `X_JOB_UNKNOWN`; a screenshot in the cwd; `X_PORT_IN_USE` | 180–182, 184–186, 189 |
| 71 | where a repository of your own calls `checkErrorCodesThrown`: pass the codes it retired as the third argument | `X_ERROR_CODE_UNTHROWN` for a code whose row says "not thrown"; `X_ERROR_CODE_UNTHROWN_STALE` for a listed code that is constructed again | 191 |
| 72 | **before the chart upgrade:** read the values — a scraper outside the cluster in `networkPolicy.metricsFrom` (or `networkPolicy.enabled: false`); `tmp.sizeLimit` for a role that writes more than 512Mi; `existingSecret` or `roles.<role>.existingSecret` for every role. A chart `x new` wrote copies `templates/_volumes.tpl`, `templates/networkpolicy.yaml` and the two values blocks from a fresh scaffold | `helm template` failing on `tmp.sizeLimit` or on a role with no Secret; a scraper that reads nothing; a pod evicted for `/tmp` | 192–194 |
| 73 | read each test or tool that runs `x dev --port 0` or `startRoles({ port: 0 })` and dials the sync node at the web port + 1: read `syncUrl` from the boot's answer | a refused connection to web + 1 | 195 |
| 74 | `x verify` | green, or a finding whose `fix:` is the edit | — |

### Entry by entry

Tier 0 — `@ultimat3/schema` (1–5), `@ultimat3/core` (6–16). Tier 1 — `@ultimat3/i18n` (17),
`@ultimat3/time` (18–21), `@ultimat3/db` (22–29), `@ultimat3/flags` (30–31), `@ultimat3/cache`
(32–33), `@ultimat3/seo` (34), `@ultimat3/storage` (35–42). Tier 4 — `@ultimat3/render` (43).
Tier 2 — `@ultimat3/entity` (44–46), `@ultimat3/policy` (47–49), `@ultimat3/http` (50–56). Tier 5 —
`@ultimat3/testing` (57). Tier 2 — `@ultimat3/auth` (58–67). Tier 5 —
`@ultimat3/testing` again (68). Tier 3 — `@ultimat3/query` (69–70), with `@ultimat3/mcp` (71). Tier 5 —
`@ultimat3/admin` (72). Tier 3 — `@ultimat3/action` (73–78). Tier 5 — `@ultimat3/cli` (79).
Tier 3 — `@ultimat3/jobs` (80–86). Tier 5 — `@ultimat3/cli` again (87). Tier 3 —
`@ultimat3/realtime` (88–96). Tier 2 — `@ultimat3/http` again (97). Tier 4 — `@ultimat3/render`
(98–101), `@ultimat3/pwa` (102), `@ultimat3/ui` (103–107). Tier 2 — `@ultimat3/http` again
(108–109). Tier 4 — `@ultimat3/mcp` (110–114), `@ultimat3/mail` (115–116), `@ultimat3/notify`
(117), `@ultimat3/manifest` (118–120), `@ultimat3/ai` (121–131). Tier 5 — `@ultimat3/testing`
(132–135), `@ultimat3/scraping` (136–145), `@ultimat3/admin` (146–154). Tier 1 —
`@ultimat3/seo` (155). Tier 2 — `@ultimat3/policy` (156). Tier 5 — `@ultimat3/cli` (157–170),
`@ultimat3/testing` (171). Tier 2 — `@ultimat3/http` (172–173). Tier 5 — `@ultimat3/cli`
(174–190), and again (191). Deploy — the chart, with `@ultimat3/cli`'s scaffold (192–195).
Tier 4 — `@ultimat3/render` (196). Tier 5 — `@ultimat3/cli` (197). Tier 0 — `@ultimat3/schema`
(198–199), `@ultimat3/core` (200–202).

| # | Surface | Costs you an edit if |
|---|---|---|
| 1 | `t.date`, `isIsoDateTime`, `fromIso`, `timestamp()` columns, feed dates, `DateTime` | a fixture, seed, import or client sends a day its month does not have (`'2026-02-30'`, `'2026-04-31'`), month `00`/`13`, day `00`/`32`. Refused at validation; it used to roll over into the next month. Correct the date at its source |
| 2 | `t.url` | a value has a leading or trailing space or control character, or a tab, CR or LF anywhere. `value.trim()` before validating; an interior tab, CR or LF survives `.trim()` — strip or percent-encode it at the source. A stored row that already holds one fails the next time it is validated |
| 3 | `t.object`, `t.record`, `t.money` | you pass a `Map`, a `Date` or a class instance. Pass a plain object: `{ ...instance }`, `Object.fromEntries(map)`. Null-prototype objects are still accepted |
| 4 | `.default(v)` | `v` fails the schema it is declared on (`t.number.min(5).default(1)`). `X_SCHEMA_DEFAULT_INVALID` at the first import of the file; the cause quotes the rule. Edit the default, or relax the rule |
| 5 | HTTP query and form coercion | a client sends `0x10`, `0b11` or `0o17` for a number. It stays a string and fails as `expected a number`. Send decimal |
| 6 | `defineConfig` | `app.config.ts` or an overlay holds: an unknown `roles` entry, `jobs.backoff`, `database.driver` or `theme.defaultMode`; a non-boolean `database.ssl`, `realtime.enabled` or `ai.mcp.expose` (`'false'` from an env variable read as on — write `process.env.X === 'true'`); `auth.signInPath` with no leading `/` (`ai.mcp.path`, then also checked here, is deleted in 25.0.0); `cache.tiers: []`; an empty or non-string `jobs.queues` entry; one locale twice (`['EN', 'en']`); a section set to `null`. `X_CONFIG_INVALID` names each key |
| 7 | `LOG_LEVEL` | a deploy sets a value that is not `trace`, `debug`, `info`, `warn`, `error`, `fatal` or `silent`, lower-case — `DEBUG` and `verbose` included. The process exits at import (`X_INVARIANT`); it used to log at `info`. Unset and empty are unchanged |
| 8 | `retry()`, `retryDecision()` | `attempts` can be `NaN`, infinite, negative or a fraction, or `timeBudgetMs` `NaN` or infinite — typically `Number(process.env.X)` on an unset variable. `X_INVARIANT` before the first try. Parse and default the value before passing it. `attempts: 0` still runs once |
| 9 | `createFlightGate` | `maxConcurrent` or `maxQueued` is `NaN`, infinite, negative or a fraction (`X_INVARIANT` at construction), or `maxConcurrent` is `0` and you expected callers to wait: each is refused with `X_FLIGHT_GATE_OVERLOADED` |
| 10 | `withChildContext({ signal })` | the child's work was meant to survive the request. The child's signal now aborts when the parent's does. Move that work to a job |
| 11 | redaction: logs, audit rows, the error monitor | a test, a log query or an audit reader expects the value of a field named like a credential — `currentPassword`, `mfaSecret`, `resetToken`, `recoveryCode`, `webhookSecret`, `passwordHash`, `tokenHash`, `keyHash`. It reads `[redacted]`. A name ending in `token` (`NPM_TOKEN`, `confirmationToken`), qualified key material (`privateKey`, `AWS_ACCESS_KEY_ID`) and a value embedding a credential (`connectionString`, `databaseUrl`) read `[redacted]` too. Ask `isRedactedKey('<name>')` for any other name. `idempotencyToken`, `continuationToken`, `maxTokens`, `cacheKey` and `code` are unchanged |
| 12 | the Sentry envelope | a monitor rule, alert or saved search reads an error's `meta` key at `extra.<key>`. It is `extra.meta.<key>`. `scope.extra` keys stay at `extra.<key>`, except `fix`, `docs`, `stack`, `requestId` and `actorId`, which the framework's own values now win |
| 13 | `OTEL_EXPORTER_OTLP_TRACES_HEADERS`, `OTEL_EXPORTER_OTLP_METRICS_HEADERS` | a deploy sets either one and also `OTEL_EXPORTER_OTLP_HEADERS`. The per-signal variable was ignored; it now replaces the generic one for that signal. Put every header the signal needs in it, or unset it |
| 14 | `OTEL_TRACES_SAMPLER=parentbased_always_on` | `OTEL_TRACES_SAMPLER_ARG` is also set. The ratio is ignored and every root is sampled. For a ratio: `OTEL_TRACES_SAMPLER=parentbased_traceidratio` |
| 15 | `hostDecision`, every `allowHosts` list | a `'*'` or `'*.suffix'` rule was how a request reached a loopback, private, link-local or metadata address literal. Add the exact rule: `allowHosts: ['*', '127.0.0.1']`. Hostnames are unaffected, including one that resolves inward |
| 16 | `ULTIMATE_CURSOR_SECRET` | a compose file or chart sets it to the empty string. Outside local development the boot is `X_CURSOR_SECRET_DEV`: `x secrets set ULTIMATE_CURSOR_SECRET`. Cursors issued under the empty key stop verifying — clients restart from page one |
| 17 | `t(key)` with no vars | a caller relied on the template coming back raw: a key whose message holds `{name}` or `{{`. It renders `⟦name⟧` and `{`. Pass the vars, or read the template with `t.raw(key)` |
| 18 | interval crons, `CronExpression` | a task with `*` or `*/n` in its minute or hour field must not run twice in a fall-back hour — make the body idempotent for that hour, or give it a fixed time. A hand-built `CronExpression` adds `wildcardTime` (TS2741) |
| 19 | cron expressions | one spells a name loosely (`mond`, `thurs`, `sept`) or has an extra `-` or `/` part (`1-5-7`, `1/2/3`). `X_CRON_INVALID` where the task is declared. Write `mon` or `monday`; write one range, one step |
| 20 | `formatRelative` | always: add `zone` (TS2741). A test that asserted on "tomorrow" / "in N days" re-reads its expectation — the count is midnights crossed in that zone |
| 21 | `addDaysInZone`, `formatDuration`, `formatDurationIso`, `plainDateUtc`, `addPlainDays`, `plainDateIn` | a computed argument can be a fraction, `NaN`, infinite, or a date outside years 0000–9999. `X_SCHEDULE_INVALID` or `X_INVARIANT` at the call. Round or default the number before passing it |
| 22 | `withTransaction` | a body catches a statement error and carries on. `X_DB_TRANSACTION_ABORTED`; it used to resolve with nothing stored. `await withTransaction(() => fallible()).catch(fallback)` around the statement, or rethrow |
| 23 | `X_DB_COMMIT_UNKNOWN`, `onRollback` | you retry on `X_DB_UNAVAILABLE` from a commit, or an `onRollback` undo had to run when the socket died mid-`COMMIT`. Neither list runs now. In a `psql "$DATABASE_URL"` session, select a row the transaction wrote; re-run only when it is absent |
| 24 | a nested `withTransaction(fn, options)` | it passes `isolation`, `readOnly: true`, `deferrable: true` or another `client` (`X_INVARIANT`). Move the first three to the outermost call; run the other client's work after the outer scope returns |
| 25 | sibling nested `withTransaction` scopes | a nested body awaits another scope under the same parent (`X_DB_SIBLING_SCOPE_TIMEOUT` after 30 s — it is a cycle), code relied on two siblings interleaving under `Promise.all`, or the first sibling runs longer than 30 s. Await them in sequence: `await withTransaction(first); await withTransaction(second)`. For a long first sibling pass `{ siblingWaitMs }`; `0` removes the deadline |
| 26 | `DriftKind`, the drift check | you `switch` over `DriftKind` exhaustively: add `case 'changed-primary-key':`. A table re-keyed by hand now fails the check; the finding's `fix:` is one `psql "$DATABASE_URL" -c '…'` command to run against that database, then `x db migrate` re-checks |
| 27 | `introspect()` | you compare `ColumnDescription.dataType` to `'numeric'`, `'ARRAY'` or `'USER-DEFINED'`, or assume every `IndexDescription.columns` entry is a column name. Compare to `numeric(12,2)`, `text[]`, the enum's name; skip entries in parentheses |
| 28 | `CatalogColumn.generated` from `@ultimat3/db/schema-dump` | you read it as a string. Read `generated.expression`; `generated.storage` says `stored` or `virtual` |
| 29 | the schema dump's `unrendered.sql` | the database holds extended statistics, forced row security, non-default column storage, an unpopulated materialized view, or a trigger on a partitioned table. `x db gen`, commit the directory |
| 30 | a temporary flag's `expiresAt` | it is not ISO-8601 (`'December 1, 2026'`, `'12/01/2026'`) or names a day its month lacks. `X_FLAG_EXPIRY_INVALID` at declaration. Write `'2026-12-01'` |
| 31 | `configureFlags({ reportEveryMs })` | the value can be `NaN`, infinite, negative or a fraction — `Number(process.env.X)` unset. `X_INVARIANT`. Parse and default it first; `0` is legal |
| 32 | `Surrogate-Key` / `Cache-Tag` from `cacheHeaders()` | a test asserts the header value, or a CDN rule parses it: each tagged response adds `e:<entity>`. After the deploy a collection bust misses edge copies cached before it until their `s-maxage` passes — purge the CDN once if that window matters |
| 33 | cache tags | a tag's entity or id holds whitespace or a comma. `X_CACHE_PURGE_FAILED` from `cacheHeaders()`, `surrogateKeys()` and the CDN tier's bust. Rename it in its `declareTags(...)` call |
| 34 | `responsiveImage()`, `usableWidths` | your image driver encodes AVIF and you relied on the default offering it: pass `{ formats: FORMAT_ORDER }`. A snapshot of `<picture>` markup loses its AVIF `<source>`. A source wider than 8192 gets `?w=8192` as its widest candidate. An intrinsic width that is `NaN`, zero or negative is `X_INVARIANT` |
| 35 | `promoteAttachment` | always: add `policy` (TS2741; `X_INVARIANT` from untyped code) — the `uploadPolicy()` the upload was granted under. An upload over `policy.maxBytes` is `X_STORAGE_TOO_LARGE` and stays under `pending/` |
| 36 | a hand-written `StorageDriver` | you ship one: add `stat(key): Promise<StorageObject \| undefined>` (TS2741) |
| 37 | `StorageListEntry.lastModified`, `StorageObject.lastModified` | you read it without a guard (TS18048), or relied on `sweepOrphans` deleting objects with no reported date. Handle `undefined` |
| 38 | `disk.get(key)` | an object can exceed the disk's `maxPutBytes` (10 MB unless set) — a presigned upload, a file written by another tool. `X_STORAGE_TOO_LARGE`. Use `disk.stream(key)`, or set `maxGetBytes` on the driver |
| 39 | stored image variants | always, if variants exist: keys are `<source key>@<transform>.<ext>` with the source's extension kept. Old-shape variants are orphaned; delete them with the commands below. A stored reference to a variant key is recomputed with `variantKey()` |
| 40 | `variantKey`, `fitDimensions` | a width or height can be `NaN`, zero, negative or a fraction (`X_INVARIANT`): round it first. A caller that relied on `contain` upscaling a small source gets the source's size |
| 41 | signed URLs on the local and memory disks | always: URLs minted before the deploy stop verifying (15 minutes by default) — a client retries with a fresh grant. A caller of `canonicalRequest` / `signConstraints` adds the base path: `signedUrlBasePath(baseUrl)` |
| 42 | upload policies | a policy allowed `video/mp4` and took AVIF, HEIC, MOV, M4A or 3GP under it. Add `image/avif`, `image/heic`, `video/quicktime`, `audio/mp4`, `video/3gpp` as needed |
| 43 | a route's `revalidate.tags` | a tag holds whitespace or a comma. `X_ROUTE_MODE_INVALID` at registration, naming the file. Rename the tag |
| 44 | a hand-built `Driver`, a hand-built `SealedMeta` | you ship or wrap an entity driver: add `transactor()` — a wrapper writes `transactor: () => inner.transactor()` (TS2741). A hand-built `SealedMeta` adds `plaintext`. A hand-built driver kept in a file under the app's own `packages/db/` moves the schema hash when it gains `transactor()`: `x verify` answers `X_DB_DRIFT`, and `x db gen "<name>"` re-records the hash and writes no migration |
| 45 | `dbDrift`, `ENTITY_ERROR_CODES` from `@ultimat3/entity` | you import `dbDrift` from entity: `import { dbDrift } from '@ultimat3/db'` (TS2305). A list built from `ENTITY_ERROR_CODES` no longer holds `X_DB_DRIFT` |
| 46 | `preload(relation)` | one page attaches more than 10,000 related rows. `X_INVARIANT_VIOLATED`; pass the bound the relation really has: `.preload('comments', { max: 50000 })`, or page the relation itself. The last `{ max }` stated on a chain wins. The refusal's `fix:` is `x entities show <entity> --json`; the edit is in its cause |
| 47 | `assertAllowed` | a `catch` or a test expects `X_FORBIDDEN` for a caller with no actor, or for a `denied(reason, code)` with the app's own code. It is `X_UNAUTHENTICATED`, or that code, as a `PolicyDenialError` |
| 48 | `HttpDenial.status`, `problem.status`, `problem.title` | you type either status as `403`, or assert on them: no actor is 401, and the title is the code's registered title. Type them `DenialStatus` |
| 49 | policy predicates, `definePolicy({ check })` | one returns something other than a boolean or a `PolicyDecision` — `{ allowed: 'yes' }`, an object from another library, nothing. It denies now. Return `true`, `false` or `denied(reason, code)` |
| 50 | `ctx.peer` | you read the client certificate identity behind a proxy. It is `null` until `configureHttp({ trustClientCertHeader: true })` — set it only where the proxy strips or overwrites `x-forwarded-client-cert` |
| 51 | anonymous `POST` / `PUT` / `PATCH` / `DELETE` from a browser | a page on another origin posts to this app without a session — an embedded sign-up form, a marketing site on its own domain. `X_CSRF_BLOCKED` (403). List the origin: `configureHttp({ cors: { origins: ['https://www.example.com'] } })`. Requests with no `Origin` and no `sec-fetch-site` are unaffected |
| 52 | repeated 401s | a client, probe or test sends many requests that fail `auth: 'required'` from one address: past `rateLimit.defaultBucket` they are 429 with `Retry-After`. Authenticate the probe, point it at `/healthz`, or raise the bucket |
| 53 | `/healthz`, `/readyz` bodies on the web role | a monitor off the box reads `buildId`, the in-flight count or the check names. It gets `{ state, ready, role }`. `configureHttp({ healthDetailPeers: ['loopback', 'private'] })`, or an exact address. Probes that read only the status need nothing. The sync role's routes are unchanged for now |
| 54 | `cache-control` set by a handler | it says `max-age=N`, `must-revalidate` or `proxy-revalidate` with no `private`: the response is rewritten as a shared-cache offer would be. For a per-user lifetime write `private, max-age=N` |
| 55 | request bodies | a client sends a non-empty body with no `content-type`. `X_BODY_INVALID` (422). Send `content-type: application/json`; a handler that wants raw bytes reads `bodyBytes()` |
| 56 | `defineHttpConfig` | an embedder relied on `HOSTNAME` to choose the bind address: pass `hostname`. A config with `buildId: null` and `BUILD_ID` set now runs with skew detection off |
| 57 | `E2eBrowser.close()`, `LaunchedBrowser.close()`, `CdpLaunchFailedError` | a test or script calls `close()` without `await`, or reads `closed`: `await browser.close()`. A hand-built `CdpLaunchFailedError` passes `{ executable, attempts }` |
| 58 | **operator action** — `x_users.mfa_secret` | any user has MFA enrolled. Do the three steps under the table, in order. Until `x auth seal-mfa` has run, each enrolled user gets `X_MFA_SECRET_UNSEALED` (500) at the second factor. App code that writes the secret calls `saveTotpSecret(auth, userId, secret)` |
| 59 | `x_auth_failures`, `x_auth_lockouts` | always, after the rollout completes: `x doctor` reports `X_FRAMEWORK_TABLE_ORPHANED`; run `psql "$DATABASE_URL" -c 'drop table if exists x_auth_failures'`. The new `x_auth_lockouts` columns are added at boot. Failure counts restart; live lockouts carry over |
| 60 | a custom `AuthLimiter` | you implement one: replace `assertAllowed` / `recordFailure` with `reserve(key)` and `refund(reservation)` (TS2741). A caller of either removed method calls `reserve` before the check and `refund` on success |
| 61 | `verifyApiKey`, `apiKeyActor` | you call either: read `.record` off the result and pass a store with `findUserById`. A key whose owner is disabled or gone stops working. A key issued with a `userId` that is not an `x_users` id: reissue it without `userId` |
| 62 | API key scopes | a key holds `*` or `<res>:*`, or more than its owner is granted: the extra scopes are dropped. `issueApiKey` with a wildcard is `X_CONFIG_INVALID` — list the scopes. Where users hold roles, pass `apiKeyResolver(store, { grantsOf })` or their keys resolve with no scopes |
| 63 | `X_MFA_REQUIRED` | a client or handler reads `meta.userId` to run the second factor. Read `meta.challenge` and call `completeMfa(auth, challenge, code)` within 5 minutes |
| 64 | `redeemRecoveryCode`, `mfaRequired`, `authNotImplemented` | you import any (TS2305). A recovery code goes to `completeMfa`; throw the challenge with `mfaChallengeRequired(auth, userId)` |
| 65 | a custom `AuthAdapter` | you ship one: implement the eight members TS2741 names. `BuiltinAdapter` and `MemoryAdapter` are the references |
| 66 | `BuiltinAdapter` write errors | you match `X_DB_UNIQUE_VIOLATION` on a user write. Match `X_AUTH_WRITE_FAILED` and read `meta.column` (`email`, `external_id`, `id`) |
| 67 | `oauthLogin` | neither `baseUrl` nor `APP_URL` is set. `X_ENV_MISSING` at the start leg: set `APP_URL` to the app's public origin |
| 68 | `E2eSession.offline()` | an e2e test toggles offline while a page refuses the script — a page mid-navigation, a crashed tab. `X_CDP_CALL_FAILED` where the call used to resolve and leave that page online. Wait for the page to settle, or close it, before `offline()` |
| 69 | `compareValues`, `compareRows`, `matchesFilter`, `isAfterKey` from `@ultimat3/query` | you wrote a custom `SqlSource` or matcher: delete `compareValues` (TS2305) and pass `kindsOf(shape.entity)` as the kinds argument of the other three (TS2554) |
| 70 | a read with `.limit()` | a client pages it past the limit with `.page()` / `?_first=` and a cursor. The listing now ends at the limit: the last page inside it answers `hasMore: false` and `nextCursor: null`. Drop the `.limit()` to page to the end. A cursor a client held across the deploy on a limited read is `X_CURSOR_INVALID` once — restart from the first page |
| 71 | the MCP tool of a `single: true` read | an agent, prompt or test reads `{ rows }` from it. It answers the row itself, or `X_NOT_FOUND`; its `outputSchema` is the row |
| 72 | `adminPermissions`, permissions declared by importing `@ultimat3/admin` | you import `adminPermissions` (TS2305), or your own closed permission set relies on the import having declared `admin:*` before `defineAdmin()` runs: add `...ADMIN_PERMISSIONS` to `definePermissions([...])` |
| 73 | `mutator({ … })` | always, for every mutator: add `idempotent: true` (TS2741; `X_MUTATOR_NOT_IDEMPOTENT` from untyped code). `x g mutator` writes it. Running more than one replica: `configureIdempotency({ scope: 'shared' })` |
| 74 | `openapi.json` | always, if the file is committed: the `Problem` schema gained `instance`, `requestId`, `issues`, `meta` and lost the `code` pattern. `x manifest`, commit. A generated client that validated `code` against the pattern regenerates |
| 75 | `packages/db/schema/` | always, if the dump is committed: `x_idempotency` gains `tx_bound`. `x db gen`, commit. The column itself is added at boot |
| 76 | an idempotent action inside `withTransaction` | a test or caller expects the idempotency record `settled` after a rollback: it is `in-flight`, and the key can be retried. A retry that overtakes a slow first attempt makes that attempt fail `X_IDEMPOTENCY_RESERVATION_LOST` (409) |
| 77 | `postgresIdempotencyStore` | you build the store yourself: add `origin: () => client` and `reclaimAfterMs: requestDeadlineMs` (TS2741). The framework's boot needs nothing |
| 78 | `cache.invalidates`, `bustAfterCommit` | a test reads the cache inside the transaction and expects the bust to have happened, or uses `bustAfterCommit`'s return value unguarded: it is `undefined` when deferred to the commit |
| 79 | the `manifest` step, `x manifest --check` | a script reads `X_MANIFEST_DRIFT` for a missing `x.manifest.json` — it is `X_MANIFEST_MISSING`; or relied on `contract-diff` to report a stale `openapi.json` — the `manifest` step does, as `X_MANIFEST_STALE` |
| 80 | a job whose worker can die mid-run | it declares `retry.attempts: 1`, or relied on being re-claimed after every lapse. A lease that lapses on the final attempt buries the row `dead` (`failed` with `retry.deadLetter: false`), `lastError` `LEASE_LAPSED_FINAL_ATTEMPT`. Raise `retry.attempts`; requeue a buried row with `x jobs retry <id>` |
| 81 | a custom `JobDriver`, `SchedulerState` or `Lease`; a caller of `SQL_CLAIM` / `SQL_SCHEDULER_FIRE` | you ship one: `claim` buries a final-attempt lapse and reports it through `onExhausted`, honouring `dropExhausted`; `fire` lands `watermarkMs`; `Lease` adds `abandon()`. `SQL_CLAIM` binds a fifth parameter and returns buried rows; `SQL_SCHEDULER_FIRE` binds `$4` |
| 82 | `cancel` on a finished job | a script or admin action cancels a `dead`, `failed`, `done` or `cancelled` job: `X_JOB_NOT_CANCELLABLE`. To remove a dead letter: `x jobs delete <id>` |
| 83 | step writes after a lapsed lease | a job body runs longer than its visibility timeout: its next `step` write is `X_JOB_LEASE_LOST`. Raise the job's visibility timeout or split the step. A custom `StepStore.put` accepts the second argument |
| 84 | `enqueue(..., { runId })` | you pass a `runId` that is not a lowercase uuid — an upper-case uuid, a slug, a number as text. `X_ID_INVALID` before anything is staged. Pass `crypto.randomUUID()`, or omit it |
| 85 | a custom `EventBus` / `EventLookup` | you ship one: `purgeExpired()` returns `Promise<number>` and rejects on failure; the lookup adds `now(): Promise<number>` (TS2741 / TS2322) |
| 86 | list cursors, `backfills.list({ runId })`, `retry.attempts` | a cursor's id is not a uuid (`X_JOB_PAGE_INVALID`), a ledger `runId` is not one (`X_ID_INVALID`), or `retry.attempts` is `Infinity` or a fraction. Take the cursor from the previous page; declare a whole number of attempts |
| 87 | a hand-built `RetentionStores`, a reader of `PurgeReport.swept` | you build the stores yourself: add `events` (TS2741). `swept` has a sixth entry, `x_job_events` — read it by name, not by position |
| 88 | **operator action** — the sync websocket's origin rule | always, where `APP_URL` is set on the `sync` role or `allowedOrigins` is passed: a declared origin is the whole allow-list, so it must be the origin the pages are served on. Otherwise every socket is `403 X_SOCKET_ORIGIN_REFUSED`; its `fix:` is the `export APP_URL=…` that admits the origin that asked. Compose requires `APP_URL` on `sync`; the Helm chart sets it from `ingress.host`; `x dev` adds its own web origin to the list. Undeclared, the node admits the origin it was reached on. A hand-built `SocketOriginRefusedError` passes `{ reason, asked, admitted }` |
| 89 | **operator action** — channels declared with params | the app declares one: `x db gen "replica identity full"`, then `x db migrate`. Until then deletes on those channels are not announced, and the replicator logs `replication.channel_identity_partial` |
| 90 | **operator action** — the replication connection | the server asks the replication role for a cleartext or md5 password and the URL's `sslmode` is `prefer` (the default) or `allow`: `X_REPLICATION_FAILED`. Append `?sslmode=verify-full` (or `verify-ca`), move the role to scram-sha-256, or state `?sslmode=disable`. `require` also connects, but verifies no certificate — it protects the password from a passive listener only |
| 91 | `parseEnvelope`, `parseChange`, the bus wire | you read change rows off the bus: `timestamp()` columns are `Date`, `bytes()` columns `Uint8Array` (base64 on the wire). A fleet with `replicator` and `sync` on different majors does not interoperate — deploy both together |
| 92 | a custom `Transport`, `AdvisoryLock`, `ChangeFeed`, `UpgradeTarget`, `UpgradeDeps` | you ship one: add `Transport.onReconnect(listener)`, `AdvisoryLock.onLost(listener)` and `AdvisoryLock.abandon()`, `ChangeFeed.abandon()`, `requestIP(request)`, `healthDetailPeers` (TS2741). `abandon()` is synchronous and writes no goodbye |
| 93 | `/healthz`, `/readyz` bodies on the sync role | a monitor off the box, or any reader whose request carries `Forwarded`, `X-Forwarded-For`, `X-Forwarded-Host`, `X-Forwarded-Proto`, `X-Real-IP` or `Via`, reads more than `{ state, ready, role }`. List the reader in `healthDetailPeers` and reach the node directly. Status codes are unchanged |
| 94 | channel guards and subscribes | a subscriber with no actor sat on a channel that declares `row`, or a guard throws a tenancy refusal: the seat is denied (`X_TOPIC_FORBIDDEN`). A guard that throws anything else suspends the seat until a later pass. A channel param value over 128 characters is refused, and a socket holding `maxTopicsPerSocket` latched denials gets no new topic |
| 95 | the offline queue | a write is queued, in flight or being flushed while the principal changes — sign-out, switch of org. It rejects with `X_OFFLINE_QUEUE_ABANDONED` and is not sent. Handle the rejection as any outbox refusal; `useMutation` already rolls its optimistic row back. Not covered: the server does not bind a replayed write to its principal — see [Known gaps](/known-gaps) |
| 96 | decoded `records.adopt` / `records.remove` / channel `params` | you call `.hasOwnProperty(…)` or rely on `Object.prototype` members on them: use `Object.hasOwn(value, key)` |
| 97 | a catch-all route (`/docs/*path`) | a test or client expects a 404 at the bare prefix (`/docs`, `/docs/`), or the handler assumes a non-empty rest: it is reached with `path: ''`. Handle `path === ''`, or declare a static `/docs`, which still wins the prefix |
| 98 | `revalidate: { ttl }` | an `isr` route's only trigger is a zero-length string — `'0s'`, `'0ms'`, `'0m'`. `X_ROUTE_MODE_INVALID` at registration; the page never expired. Write a positive `ttl`, or `tags`. `parseTtlMs('0s')` is `null` |
| 99 | `NativeReason` | you `switch` over it exhaustively: add `case 'unparsable':` — an `href` or form action no URL parses, left to the browser |
| 100 | CSS modules (`*.module.css`, `*.module.scss`) | always, after a rebuild: every scoped class name changes once — the suffix hashes the compiled CSS. A snapshot holding one updates. An escaped class (`.w-1\.5`) is one key, `styles['w-1.5']`; it was `styles['w-1']`. A word in a declaration or comment (`local(Inter.Regular)`) is no longer a key of `classes` |
| 101 | `CompiledPattern.specificity`, `compilePattern` | you store or compare against a number: it is a positional rank, every value changed, higher still wins — compare two patterns' `specificity`. The regex now also matches a catch-all's bare prefix and a percent-encoded literal |
| 102 | push notification `url` | a sender links a tap to another host or a non-http scheme: the app root opens instead. Link to a page of the app. Only an app handing `generateServiceWorker` a `vapid` key emits the handler |
| 103 | a hand-built `ThemeEnv` | you build one — a test fake, a server env: add `current()` (the document's `data-theme`) and `appDefault()` (its `data-theme-default`), TS2741. A fake that answers `null` for both keeps the OS rule; a wrapper spreads `browserThemeEnv()` |
| 104 | `clearTheme()`, `watchOsTheme()`, `resolveTheme()`, `ThemeToggle` | `theme.defaultMode` is `'light'` or `'dark'` and a user or test expected "System" or an OS change to follow the OS: it returns to that mode and ignores the OS. Set `defaultMode: 'system'` for an OS-following app. A `ThemeEnv` that overrode `prefersDark` with the booted theme — the `bootedEnv()` an `x new` app carries in `apps/web/shared/theme-toggle.island.tsx` — must drop it: under `'system'` it freezes the OS answer at mount. Delete it and render `<ThemeToggle mode="toggle" />` with no `env` |
| 105 | `THEME_INLINE_SCRIPT`, `themeInlineScriptTag`, `themeInlineScriptHash`, `themeInlineScriptCspSource` | you import any (TS2305). Delete the import, the hand-inlined `<script>` and its `sha256-…` in `script-src`; the boot already inlines `themeScript` and admits its hash |
| 106 | `Popover`'s `trigger` | the callback annotates `'aria-controls': string`, or hands it to a prop typed `string` (TS2322). It is `string \| undefined`, absent while closed; spread the control object, or widen the type |
| 107 | `Dropzone`, `FileInput` | a server, an e2e test or a handler expects to receive a file the control refused by `accept`, `maxBytes` or `maxFiles`: the input holds only accepted files, and a pick that refused everything empties it — a `required` control then blocks the submit. Read the refusal from the `onSelect` selection's `rejected` |
| 108 | a hand-written `RateLimitStore` or `RateLimiter` | you ship one: add `peek` (TS2741). A store answers `peek(key, bucket, nowMs)` from the state it keeps — `rateLimitPeek(bucket, refilledTokens(bucket, stored, lastMs, nowMs))`, or `rateLimitPeek(bucket, bucket.capacity)` for a key it holds nothing for — and writes nothing. A limiter forwards `peek(key, bucketName)` to its store |
| 109 | `auth: 'required'` routes behind one address | a probe, a test or a client sends failing credentials from an address others share. Once `rateLimit.defaultBucket` under `unauthenticated\|ip:<address>` is spent, every request from that address to a required route is `429 X_RATE_LIMITED` until it refills — a valid credential included, signed-in users behind the same NAT included. Point the probe at `/healthz`, authenticate it, give it its own address, or raise the bucket |
| 110 | MCP error text from `defineAppMcp` (`errorAudience: 'caller'`) | an agent, prompt or test reads the `cause` of a 5xx code core's `hasPublicCause` does not list — `X_DB_STATEMENT_FAILED` above all. It reads a fixed sentence pointing at the logs, and `fix: x errors explain <CODE> --json` unless the error carries a `callerFix`; `resources/read` error data likewise. Read the cause in the log line or the error monitor. `createMcpServer` defaults to `'developer'` and is unchanged |
| 111 | `db.query` | an agent's query quotes an identifier as `U&"…"`, or calls `query_to_xml*`, `cursor_to_xml*`, `table_to_xml*`, `schema_to_xml*`, `database_to_xml*`, `ts_stat`, `ts_rewrite` or `pg_import_*`: `X_MCP_QUERY_REJECTED`. Spell the identifier plainly; run the inner statement as the query |
| 112 | `surface: 'meta'` and a hand-registered tool | the tool omits `destructive` and writes: it is listed as `query`, was `action`. Declare `destructive: true` |
| 113 | `manage_resource` list calls | a call passes an optional input key the query's `listParams` leaves out (`includeDeleted`, `status_in`): an `isError` result carrying `X_INPUT_INVALID`. Add it to `mcp.listParams`, or call the query's own tool. Required input keys are still admitted |
| 114 | the MCP endpoint, `mcpHttpRoute`, `x mcp serve --transport http` | a client sends missing or wrong bearer tokens from an address an agent also uses: past 20 failures a minute the address is `429 X_MCP_RATE_LIMITED` before `resolveToken` runs, a right token included. Set `rateLimits: { read, write, unauthenticated: <n> }` on `defineAppMcp` or `mcpHttpRoute`. A host of its own that calls `route.handle(request)` without `{ address }` is not metered |
| 115 | `send()` inside `withTransaction` | a handler relied on the mail going out though the transaction rolled back, or a test reads the queue before the commit: the `mail.send` job is staged and published after `COMMIT`, and a rollback sends nothing. A facade installed with `mode: 'required'` refuses a `send()` outside a transaction (`X_OUTBOX_NO_TX`) — wrap the call in a transaction, or pass `{ sync: true }` |
| 116 | mail recipients, `unsubscribeUrl`, the `mail.send` job | a `to`, `cc`, `bcc` or `replyTo` holds a control character, `<`/`>` in the mailbox, or a non-ASCII mailbox: `X_MAIL_ADDRESS_INVALID` at `send()` on every driver — Resend and memory delivered it. Write the domain as punycode; a non-ASCII local part needs another mailbox. An unparseable `unsubscribeUrl` is `X_VALIDATION_FAILED`. Always, where `x.manifest.json` is committed and the app sends mail: `contract-diff` reports `jobs.mail.send.input` until `x manifest` |
| 117 | `packages/db/schema/` | always, if the dump is committed: the boot creates `x_notify_digests` and its indexes. `x db gen`, commit |
| 118 | `x.manifest.json` | always, if the file is committed: every query gains `input`. `x manifest`, commit. The first diff against a file without it reports no change |
| 119 | manifest facts | a declaration publishes `NaN`, `Infinity` or `-0` — a budget, a rate limit, a retry count read from an unset variable: `X_MANIFEST_FACT_INVALID` at `x manifest`, naming `meta.path`. Replace `-0` with `0`, and `NaN` or `Infinity` with a finite number |
| 120 | an entity column's default | a release drops the default of a NOT NULL column: `contract-diff` reports `…hasDefault` as breaking, so the gate is `X_MANIFEST_BREAKING` without a major bump. Keep the default, or bump the major in `package.json` |
| 121 | a hand-written `BudgetStore` | you ship one: add `take(key, tokens, limit)` (TS2741) — add `tokens` only when the total stays at or under `limit`, in one atomic step on the store (a Redis `EVAL`, one conditional `update`), answering `{ taken, spent }` with `spent` read before the take. Never `spent` then `add`: that pair is the race. Another answer is `X_INVARIANT` at the reservation |
| 122 | a hand-written or wrapping `Gateway` | you ship one: implement `callLedger(keys: BudgetKeys)` (TS2741), or pass keys at a `callLedger()` call (TS2554). A wrapper forwards `keys` to the wrapped gateway's |
| 123 | `createGateway({ budget: { actor, org } })` | the ceiling is declared: every `llm()`, `agent()` and hive member with no `scope()` open counts against `actor:<kind>:<id>` and `org:<orgId>` of its caller, so a caller past it is `X_AI_BUDGET_EXCEEDED` where the call ran. Every anonymous caller is one counter. Raise the ceiling or drop it; key each `gateway.scope()` with `budgetKeysFor(actor)` so a scoped and an unscoped call of one actor share a counter. A hive's members also run under the gateway's `request` ceiling now |
| 124 | `agent()` tool failures | the model, a prompt or a test reads the cause of a tool's 5xx code that core's `hasPublicCause` does not list — an app code with no `registerErrorStatus` row counts as 500. It reads `CODE: ` and a fixed sentence, plus a declared `callerFix`. Give a refusal the model should act on a 4xx (`registerErrorStatus({ X_ORDER_LOCKED: 409 })`) or `registerProblemMeta({ X_ORDER_LOCKED: { publicCause: true } })` |
| 125 | `fnv1a` | you import it from `@ultimat3/ai` (TS2305). Use core's `fingerprint`; a 32-bit number is `Number.parseInt(fingerprint(text).slice(0, 8), 16)` |
| 126 | `HashEmbedder` | a vector store, a seeded database or a fixture holds vectors it wrote, or a test pins a ranking or a vector: every vector changes. Re-run `indexDocument` over each document; update the snapshot |
| 127 | an OpenAI-format `finish_reason` | a provider, fake or recorded fixture sends one this build does not know: it reads `max_tokens`, was `end_turn` (or no finish, streamed). An answer that also fails its schema is `X_LLM_TRUNCATED` with no repair turn; `.stream()` throws it. Send `stop` from a fake |
| 128 | vector reads in an org's request | a `search`, `searchText` or `hybrid` runs on a store with no tenant bound — no `scope`, or an `allow` list alone — while the ambient actor has an `orgId`: `X_VECTOR_UNSCOPED` (500). `store.scoped({ tenant: ctx.actor.orgId })`; a deliberate cross-tenant read opens the store with `scope: UNSCOPED` or calls `.scoped(UNSCOPED)`. A comparison against `UNSCOPED`'s old `{}` fails — it is `{ crossTenant: true }` |
| 129 | a hand-written `VectorStore` | you ship one: add `prune(filter, keep)` (TS2741) — delete every in-scope row `filter` matches except the ids in `keep` |
| 130 | `chunk()` metadata | a caller passes its own `metadata.source`: it is overwritten with the document id, the key `indexDocument` prunes by. Store the value under another key |
| 131 | `numericTolerance` | the bound is `NaN`, infinite or negative — often read from an unset variable: `X_INVARIANT` where it is called. Pass a finite bound of at least 0; `0` is exact match |
| 132 | `describeApp`'s `app()` | a test reads it while the `describe` block registers, before `beforeAll`, and asserts `toBeInstanceOf(ReferenceError)`: it is `X_TEST_APP_NOT_BOOTED`. Assert `toBeUltimateError('X_TEST_APP_NOT_BOOTED')`, or read `app()` inside a `test` body |
| 133 | `assertDeterministic` | a test relied on two results differing only in key order failing `X_TEST_NONDETERMINISTIC`: it passes. Results compare by core's `canonicalJson`; a BigInt, Date or Map result compares instead of throwing `TypeError`, a cyclic one by `Bun.deepEquals`. A top-level `undefined`, function or symbol compares by its type: `undefined` then `null` is `X_TEST_NONDETERMINISTIC`. Assert the order itself |
| 134 | `frozenClock(now, body)` | a test counts `onClockMoved` announcements: each call adds two, entering and restoring, so frozen-scheduler lease renewals fire inside and after the body. A listener that throws on entry rejects the call, the body does not run, and the earlier instant is restored. Update the count |
| 135 | `mountIsland` | a test relied on module state carrying from one mount to the next: each mount loads from its own directory, a fresh module instance. Set the state up in each mount |
| 136 | `localBrowser({ options: { args } })` | `args` holds `--proxy-server`, `--proxy-bypass-list`, `--proxy-pac-url`, `--proxy-auto-detect` or `--no-proxy-server` (`-` or `--`, any case), or is not a list of strings: `X_SCRAPE_LAUNCH_ARGS_INVALID` before anything launches, exit configured or not. Set the exit with `localBrowser({ proxy })` or `scrape({ egress })`; the caller's other args are kept and the exit appended after them |
| 137 | `CdpBrowserLike.target()` | you ship a launcher or a test double that hands back a browser without it (TS2741): add `target()` answering `{ createCDPSession() }`, whose session has `send` and `on` (`CdpBrowserSessionLike`). Puppeteer has it. A browser that refuses browser-level `Fetch.enable` fails `open()` with `X_SCRAPE_BROWSER_UNREACHABLE` |
| 138 | a CDP browser with no `cookies()` or `setCookie()` | `session()` on it, or `restore()` of a session holding cookies: `X_NOT_IMPLEMENTED`, was an empty jar or dropped cookies. Use a puppeteer-core that exposes both, or set `auth: { reuse: false }` on the `scrape()` definition |
| 139 | `burnSession(plan, seen)` | you call it with one argument (TS2554): pass `recordVersion(restored)`, the version of the record the run used. A record another run saved since is kept. A hand-written `ScrapeSessionStore` keeps the new optional `SessionState.version` it is handed. A refused login now also tombstones over a record the run found but did not reuse (`reuse: false`, past `maxAge`) |
| 140 | `expect.maxDrop` | the value is `1` or above (`50` for 50%) or negative: `X_INVARIANT` at `scrape()`. Write a fraction in `[0, 1)` — `0.5` |
| 141 | `urlSecretValues`, `redactSecrets` | a log, artifact or snapshot expected every query value and the whole path concealed: only credential-shaped ones are — a query key whose words (split on `_`, `-`, `.`, camelCase) include a credential word (`apiKey` does, `keyboard` does not), a 16+ character value mixing letters and digits, or the id after `/devtools/<browser\|page>/`. `redactSecrets` also redacts each secret's percent-encoded (hex in either case), form-encoded and HTML-escaped spellings. Update the snapshot |
| 142 | headers on the HTTP leg | a declared header shares a name with the session's in another case and a request relied on the two being joined (`BrowserUA, Mine`): the declared one replaces it. Declare the whole value |
| 143 | `allowHosts` wildcards on the HTTP leg and the robots read | a host admitted only by a wildcard resolves to an address that is not public: `X_SCRAPE_HOST_BLOCKED`, and the connection is pinned to the approved address. Add an exact rule for the internal host. Exact rules and proxied runs are unchanged; the browser leg is not pinned |
| 144 | the robots read, `createRobotsGate`, `robotsFetcher` | a site's `robots.txt` redirects more than 5 times, or to a host off `allowHosts` — every hop screened, the first included — or, on a gate built by hand with no `allowHosts`, to any hostname other than the starting one (a subdomain included; a scheme upgrade, its port change and a moved path are followed): the hop is never requested and the read is "no robots", so no rule applies. List the host it redirects to; pass `allowHosts` to a gate built by hand. A `scrape()` run always passes its own |
| 145 | `X_SCRAPE_FIXTURE_STALE` | a recorded fixture's `recordedAt` is not a date and `maxAgeMs` is set: the recording is stale, was fresh. Re-record it |
| 146 | `KeysetBound.value`, `AdminCursor.value` | you ship an `AdminRepo` whose `list` reads `keyset.value` as a string (TS2345 / TS18047): it is `string \| null`, `null` for a NULL sort value, was `''`. NULL sorts as the largest value — last ascending, first descending |
| 147 | the admin edit form, `_version` | a script, a test or a hand-built page posts an edit without the `_version` the form rendered, or after the row changed: `409`, nothing written, a `failed` entry with `admin.error.row-changed`. Post `rowVersion(row)` of the row the edit was made against. `adminUpdate` with no `version` — MCP, a script — is unchanged |
| 148 | `AdminFormProps.version` | you render `AdminForm` yourself (TS2741): pass `rowVersion(row)` for an edit, `null` for a create |
| 149 | `CrudResult` | a `switch` over `kind` is exhaustive: handle `'stale'` — `row` as it is now, `version`, `audit` — which `adminUpdate(…, { version })` answers when the version no longer matches |
| 150 | create and update policies | a rule reads the subject's `input`: an admin create or update is decided again after validation with the written values as `input` (an update beside `row`), so the rule now fires there. A write it refuses is denied and audited as one |
| 151 | `/admin/audit`, a row's history card | an actor with an `orgId` — a test, an operator — expected another org's entries: only its own org's are listed. An actor with no `orgId` still reads every tenant's |
| 152 | an admin action, a set-based `matching` call, a queued batch | a test expects the work to stay written when its audit entry cannot be: on `postgresAuditLog` the work rolls back with the entry, and the log holds one `failed` entry. A queued batch's id is derived from its content, so a retry dedupes onto chunks still live |
| 153 | `?scope=*`, `listHref({ scope: null })` | a list scope is declared as `*`: `X_ADMIN_FILTER_INVALID`, rename it. `listHref({ scope: null })` writes `?scope=*` and opens every row; it opened the default scope |
| 154 | admin list filters | a link or bookmark sends a number filter as a blank, hex (`0x10`), an exponent (`1e3`) or `Infinity`, or a date the calendar lacks (`2026-02-30`): `X_ADMIN_FILTER_INVALID`. Send a decimal or a real day |
| 155 | split sitemap parts | a CDN, an object store, a search console or a link holds `/sitemap-<n>.xml`: the parts are `/sitemaps/<n>.xml` (`SITEMAP_PARTS_DIR`), served by the web role, the index unchanged at `/sitemap.xml`. Rebuild, upload `sitemaps/`, delete or redirect the old files, re-submit the index |
| 156 | `restorePermissions(names, declaredAt)` | you call it with one argument (TS2554): capture `permissionDeclarationSites()` with `knownPermissions()` and pass both back. The one-argument call restored names with no declaration site, which the `policy` step never judges — `X_PERMISSION_BORROWED` was silently off for them. Pass `{}` only when you mean no provenance |
| 157 | `x build --target static --out <dir>` | the directory exists, is not empty and holds no `.x-export`: `X_BUILD_OUT_UNSAFE`, was deleted. An `--out` that is the app root or holds it is `X_BUILD_OUT_UNSAFE`, was `X_CLI_BAD_FLAG`. Empty it once or build to `.x/static` (exempt); every build writes the marker |
| 158 | a static build | a module of the app fails to import: `X_BUILD_FAILED`, the last export left as it was; the build succeeded without that page. `x verify --only manifest --json` lists each module |
| 159 | `x build` flags | `--tag` off the docker target, `--out` on it, or any of `--tag`, `--out`, `--no-preflight` on prebuilt: `X_CLI_BAD_FLAG`, was ignored. Drop the flag |
| 160 | `x g --dry-run` | a script reads `ok` or `data.files`: `ok: false` with the real run's conflicts, and `data.files` lists only what would be written |
| 161 | `x g` and `apps/web/api/index.ts` | the primitive's camelCased file name is already bound in the index — another feature's module, `api`, `defineApi`, `health`: `X_GENERATE_CONFLICT` before anything is written. Run the `fix:` — the same command under a feature-prefixed name |
| 162 | `x g` names | a plural `resource` or `entity` name, a non-ASCII letter, or a name whose type the generated code uses (`promise`, `omit`, `partial`, `row`): `X_CLI_BAD_FLAG`. Run the `fix:` — `x g resource post`, `x g entity uber`, `x g resource promise-resource` |
| 163 | `x affected`, `x test --affected` | a CI job relies on a change under `scripts/`, `guards/`, `bin/` or to `tsconfig.base.json` selecting nothing: any non-doc path no workspace owns is root-wide — every workspace, every test file |
| 164 | test discovery | an app has a directory named `build`, `examples` or `dummy` below its root: its tests run under `x test` and on the gate's steps, `live` and `e2e` included, and may fail. Fix them; only the root's are skipped |
| 165 | `filesize`, `errors`, `i18n` | a `guards/*.ts`, `apps/*/server.ts` or `apps/*/prerender.ts` is over 500 lines or has a `fix:` that is not one runnable command: the gate fails. Split it or make the fix a command. The `i18n` step no longer reads `guards/` |
| 166 | `/robots.txt`, `/sitemap.xml`, `/sitemaps/<n>.xml` on a running web role | a test or an operator expects a change to the app's routes or SEO settings on the next request: each origin's answer is kept an hour, 8 origins at most. Restart the process |
| 167 | `x new`'s `solid-js` | a fresh scaffold pins `1.9.15`, was `1.9.14`. An existing app moves its pin by hand |
| 168 | `x new`'s app `typecheck` scripts | a fresh scaffold's `apps/web` and `apps/admin` run `tsc --noEmit -p ../../tsconfig.json`, the root program: slower, and it reports a type error another workspace's declaration causes. An existing app keeps its scripts until it changes them |
| 169 | the `policy` step | an app action, query or route guard requires a permission only a package declared — `defineAdmin()`'s `<entity>:read\|write\|delete`: `X_PERMISSION_BORROWED`. Add the name to the app's own `definePermissions([...])`; the finding names the file |
| 170 | the `drift` step | a params channel's `records` table or a live query's `subscribes:` table needs `REPLICA IDENTITY FULL` and no migration records it: `X_DB_SCHEMA_UNMIGRATED`. `x db gen "record replica identity full"`, then `x db migrate`. A `subscribes:` name no entity declares is `X_QUERY_SUBSCRIBES_UNKNOWN` |
| 171 | `ProcessRegistrySnapshot` | you build one by hand (TS2741): add `permissionSites` — `permissionDeclarationSites()`, or `{}` |
| 172 | a handler's own `content-security-policy` | a handler sets one — to loosen the app's, or a test reads the header: it is kept and the app's is added beside it, both enforced. Loosen through `security.csp.extend`; read both policies in the test. The report-only header follows the same rule |
| 173 | the `fix:` of `X_ROUTE_NOT_FOUND`, the duplicate-route and mount-conflict errors | a monitor, script or test matches `x routes list --json`: it reads `x routes --json`. Codes unchanged |
| 174 | a serving role's boot, `ROLE=migrate` | a deploy starts `web`, `sync`, `worker`, `scheduler` or `replicator` on an external `DATABASE_URL` before `ROLE=migrate` ran this build, or never runs it: `X_FRAMEWORK_SCHEMA_UNAPPLIED`, the pod refuses to boot. Run `ROLE=migrate` (`x db migrate`) first — the shipped chart's hook and `docker-compose.prod.yml` already do. An embedded database applies its own |
| 175 | `TRUSTED_PROXY_HOPS` | a malformed value: `X_TRUSTED_PROXY_HOPS_INVALID`, was `X_PORT_INVALID`. Write decimal digits, 1 to 64; repoint an alert on the old code |
| 176 | `PORT`, `METRICS_PORT`, `ROLE` | a port written `0x1F90`, `8e3` or `+80`: `X_PORT_INVALID`, was read as a number. Write decimal digits. A blank `ROLE` now boots `web`, where it was refused |
| 177 | `/_storage`, `/media` | a link opens an uploaded file that is not a raster image, audio or video — PDF, SVG, HTML, XML — in a tab: it downloads, with `content-security-policy: sandbox`. Embed images, audio and video as before; link to a viewer page for the rest |
| 178 | `worker`, `scheduler`, `replicator` on SIGTERM | a process relied on `drain.readinessGraceMs` before it stopped claiming: it stops at once. `web` and `sync` keep the grace |
| 179 | the replicator's readiness, the chart | an alert reads a not-ready replicator as an outage, or a chart was written by `x new`: the replicator is ready only while its stream runs (`/readyz?deep=1` on the metrics port), and its metrics Services publish not-ready addresses. Copy the two edits from `docker/helm/templates/_helpers.tpl` and `service.yaml` into a scaffolded chart |
| 180 | `x help <name>` | a script asks for a name no command has: `X_CLI_UNKNOWN_COMMAND`, exit 1 — was the catalogue and 0. Use the name the fix names |
| 181 | `x i18n add`, `x i18n sync`, `x db seed`, `x db backfill` | a run reports a finding — an app module that will not import, included: exit 1, was 0. Fix the module the finding names |
| 182 | `x doctor`, `DoctorProbe` | `PORT` is set and `--port` is not: the probe reads `PORT`, was always 3000. A loopback `APP_URL` on another port is `X_APP_URL_PORT_MISMATCH`, a malformed one `X_CONFIG_INVALID`. A hand-built `DoctorProbe` adds `appUrl: string \| undefined` (TS2741) |
| 183 | the `/_x` dev dashboard | a script runs `GET /_x/db?sql=`, posts from another origin, or reaches `x dev` under a Host that is not loopback: `405 X_METHOD_NOT_ALLOWED`, `X_CSRF_BLOCKED`, `421 X_DEV_HOST_REFUSED`. `curl -sS -H 'accept: application/json' --data-urlencode 'sql=select 1' http://localhost:3000/_x/db` |
| 184 | `x jobs show\|retry\|cancel\|rm\|promote <id>` | the id is not a uuid: `X_JOB_UNKNOWN`, was `X_DB_STATEMENT_FAILED` on Postgres. Pass the job's id |
| 185 | `x shot --out <relative path>` | a script ran it outside the app root: the file lands relative to the cwd, was the app root. Pass the path from where you run it |
| 186 | `x dev`'s `X_PORT_IN_USE` | a script parses the fix: it suggests `--port N+2`, was `N+1` |
| 187 | `x verify merge` | a part's step has no numeric `durationMs` or malformed findings: `X_VERIFY_MERGE_INPUT`. Sharded `files` that repeat a file or do not hash to `corpusHash`: red. Shard every job by one rule — all with `--timings`, or none |
| 188 | MCP `tests.run` | an agent or test read `failed: 0` from a run with an error outside any test, or a non-zero exit: each counts as a failure |
| 189 | `x mcp serve --transport http` | the port is taken: `X_PORT_IN_USE`, with a neighbouring port as the fix, was `X_CLI_UNEXPECTED` |
| 190 | `x routes` | a script passes a word — `x routes list --json`: `X_CLI_UNKNOWN_COMMAND`, exit 1. `x routes --json`, `--surface <s>` to filter |
| 191 | `checkErrorCodesThrown` (`@ultimat3/cli`) | a repository of your own calls it and relied on a reference row saying "not thrown" or "thrown by nothing" to waive a code: the code is `X_ERROR_CODE_UNTHROWN`. Pass the retired codes as the third argument — `checkErrorCodesThrown(root, page, new Set(['X_MY_RETIRED']))`; it defaults to this repository's `UNTHROWN_CODES`. A listed code that is constructed again is `X_ERROR_CODE_UNTHROWN_STALE` |
| 192 | the chart's NetworkPolicy | the cluster's CNI enforces policies and something outside the cluster scrapes `/metrics`, or reaches a port other than `http`: refused. List the scraper in `networkPolicy.metricsFrom`, narrow `networkPolicy.httpFrom` to the ingress controller if you want, or set `networkPolicy.enabled: false` |
| 193 | `tmp.sizeLimit` | a role writes more than 512Mi to `/tmp`: the pod is evicted. A values file that drops `tmp.sizeLimit` does not render. Raise it per your measurement |
| 194 | `existingSecret`, `roles.<role>.existingSecret` | the release-wide `existingSecret` is empty and a role names none of its own: `helm template` fails naming the role. Set one or the other for every role; the migrate Job reads `migrate.existingSecret` |
| 195 | the sync listener at port 0 | a test or tool runs `x dev --port 0`, `startRoles({ port: 0 })` or a scratch server and dials the web port + 1: the sync node is on its own kernel-chosen port. Read `syncUrl` from the boot's answer; pages dial `/_x/sync` on their own origin and are unaffected |
| 196 | `budget.lcp` on `defineRoute` | a route declares `lcp`: a type error, and `X_ROUTE_MODE_INVALID` at `registerRoute`. Delete it from every `budget: { … }`. A reader of `lcp` in `x routes --json`, of `budget.lcp` in `x.manifest.json` or of `lcpMs` in `.x/build-stats.json` drops it |
| 197 | `x test --worker I` | a script passes it: `X_CLI_BAD_FLAG`, exit 1. `x verify --only unit --shard i/n` |
| 198 | `toWireSchema`, `toOutputSchema` (`@ultimat3/mcp`), `mcpSchemaOf` (`@ultimat3/action`), `toMcpInputSchema` (`@ultimat3/schema`) | you import one: not exported. `import { toWireSchema, toWireOutputSchema } from '@ultimat3/schema'`; for a full draft-07 document `toJsonSchema(s, { dialect: 'draft-07', includeDialect: false })` |
| 199 | a tool schema: `.tool().inputSchema`, `@ultimat3/ai`'s `JsonSchema` | you read `format` off it, or build a `ProjectableAction` with `additionalProperties: <schema>`: no `format` is published, and `additionalProperties` is a boolean — write `true`. The action's parse still enforces the format |
| 200 | `defineConfig`'s `name` | the config is JavaScript or untyped and has no string `name`: `X_CONFIG_INVALID` at load. `defineConfig({ name: 'my-app', … })` |
| 201 | `storageNotImplemented`, `dbNotImplemented`, `scrapeNotImplemented`, `JobsNotImplementedError`, `CliNotImplementedError`, `NotImplementedError` (`@ultimat3/pwa`, `@ultimat3/realtime`) | you import one, or catch `X_NOT_IMPLEMENTED` as `instanceof StorageError` or another package base: not exported, and no longer an instance. `import { NotImplementedError } from '@ultimat3/core'`, or match `err.code === 'X_NOT_IMPLEMENTED'` |
| 202 | `readCookie` (`@ultimat3/auth`), `escapeAttribute` (`@ultimat3/seo`) | you import one: not exported. `import { escapeHtml, readCookie } from '@ultimat3/core'` |

Entry 58, **the one step an operator must not skip.** A deployment with MFA-enrolled users:

| Order | Do | Until it is done |
|---|---|---|
| 1 | make the master key exist: `x secrets init`, or set `ULTIMATE_SECRETS_KEY` in the deploy | `X_SEAL_KEY_MISSING` at `login()` for an enrolled user |
| 2 | deploy this release | — |
| 3 | `x auth seal-mfa --json`, once | `X_MFA_SECRET_UNSEALED` (500) at the second factor for every enrolled user; `x doctor` reports the count |

Step 3 answers `{ sealed, alreadySealed, skipped }`. It is idempotent, and `skipped` counts rows
that changed while it ran — run it again and they are `alreadySealed`. A custom adapter calls
`sealMfaSecrets({ adapter })`. A deployment with no MFA users needs none of this.

Entry 39, the orphaned variants. These commands were **not run against a real disk or bucket** for
this page — list first, read the list, then delete. The pattern matches a variant of either shape,
so new-shape variants already written are deleted too and regenerate on the next request. A key
of your own that ends in such a suffix (`photos/team@w640.png`) matches as well.

```sh
# local disk — list, then repeat with -delete in place of -print
find <root> -type f -regextype posix-extended -regex '.*@(full|((w|h|q)[0-9]+|cover|contain)(-((w|h|q)[0-9]+|cover|contain))*)\.(avif|webp|jpg|png)' -print
# their sidecars live under <root>/.meta/ as <key>.json — the same pattern with \.json appended
find <root>/.meta -type f -regextype posix-extended -regex '.*@(full|((w|h|q)[0-9]+|cover|contain)(-((w|h|q)[0-9]+|cover|contain))*)\.(avif|webp|jpg|png)\.json' -print
# s3 — list, then `aws s3 rm s3://<bucket>/<key>` for each key
aws s3 ls s3://<bucket> --recursive | grep -E '@(full|[whq][0-9]+|cover|contain)[^/]*\.(avif|webp|jpg|png)$'
```

### Not breaking, but you will see it

| Surface | What changed |
|---|---|
| a union in HTTP coercion | every member is tried: `t.union(t.literal('auto'), t.number)` reads `'12'` as `12`. A string a member accepts as a string is never converted |
| `SchemaError#toJSON()` | carries `retry: 'terminal'`; a `bigint` or a cycle in `meta` no longer throws |
| image errors | a header declaring a zero, negative, fractional or `NaN` size is `X_IMAGE_DECODE_FAILED`, was `X_IMAGE_TOO_LARGE`. A HEIC (`mif1` only) is no longer sniffed as AVIF |
| OTLP export | an endpoint with a query string keeps it after the signal path; a `NaN` or infinite attribute is dropped |
| `X_VERIFY_STEP_TIMEOUT` | names the test file still running and its `fix:` runs it; `--json` findings gain `meta` |
| `fix:` lines | `X_REGISTRAR_MISSING` / `X_REGISTRAR_CONFLICT` name the owning package; `X_SECRETS_KEY_INVALID` names the key file when the file is what is wrong |
| `x db gen` and a changed `primaryKey` | the migration is written — drop `<table>_pkey`, add the new key, `drop not null` for a declared-nullable column leaving it. `X_MIGRATION_IRREVERSIBLE` for a key another table's foreign key references, and for a new key over a column the same migration adds with no default, or a `null` one: add and backfill the column in one migration, change the key in the next |
| `x tasks` | `next`, `last` and `upcoming` are unchanged in form; they come from `isoInZone` in `@ultimat3/time` |
| db error codes | a syscall error (`EPIPE`, `E2BIG`) is `X_DB_UNAVAILABLE`, was `X_DB_STATEMENT_FAILED`; a ragged array or Invalid Date parameter is `X_INVARIANT` on both drivers, was `X_DB_UNAVAILABLE` |
| `addBusinessDays`, `businessDaysBetween` | the wall time survives a DST day; a date the zone skipped is not counted |
| `Accept-Language` | a `q` that is not a plain decimal ranks 0, not 1 |
| `formatMoney({ trimZeroFraction: true })` | 1250 USD is `$12.50`, was `$12.5`; a whole amount still drops `.00` |
| `robots.txt` | `seo.robots.disallow` is in every group, and a `User-agent: *` group is emitted when none is declared |
| ISR documents | carry `Surrogate-Key` and `Cache-Tag` from `revalidate.tags`; a response rewritten to `private` or `no-store` drops both |
| feeds | Atom gains the channel `<author>`, `<rights>`, `<icon>`; RSS an item author and `<media:content>` for an item `image` |
| storage error codes | a refused write is `X_STORAGE_PUT_FAILED`, a refused read `X_STORAGE_READ_FAILED`, a local-disk key under another key `X_STORAGE_KEY_CONFLICT` — each was a bare error. An empty org is `X_STORAGE_ORG_MISMATCH` (404), was `X_STORAGE_PATH_UNSAFE` (400) |
| `GET /_storage/:disk/*key` | no `Last-Modified` when the disk reports no date |
| the Redis cache tier | one extra round trip before each `load()` and one after the `SET`; a fill whose load outlives 60 s is not written to Redis |
| `transition(column, id, move)` | an `undefined` or `null` id is `X_NOT_FOUND` and moves no row; on Postgres it moved every row in the `from` state |
| entity tests on `memoryDriver()` | a unique declared as `invariant(name, c.unique([...]))` is enforced (`X_DB_UNIQUE_VIOLATION`); `bigint()` refuses a value outside int8; `trimmed()` strips spaces only; `url()` refuses what the column's CHECK refused — each as Postgres already did |
| `update` / `updateWhere` under an app-only invariant | on Postgres a refused write is rolled back; it used to stay written |
| seed `dryRun` | runs every verb in a transaction and rolls back, so its metrics are a real run's and a seed that would fail fails |
| drift findings | each `fix:` is one runnable command, mostly `psql "$DATABASE_URL" -c '…'` then `x db migrate`; for a table outside `public` it sets `search_path` first, and unexpected tables and objects are schema-qualified |
| `X_CSRF_BLOCKED` | title is "an unsafe request that did not prove same-origin"; code and status unchanged |
| 5xx problem documents | a withheld cause carries `callerFix` or `x errors explain <CODE> --json`, never the developer `fix:` |
| `requestTimeoutMs`, `x-request-timeout-ms` | above 2,147,483,647 the config is `X_CONFIG_INVALID` and the header is ignored; both used to time every request out at once |
| redirects | a non-http(s) target is never handed to the client router; the locale-prefix redirect stays on this origin |
| `Set-Cookie` | two set in one request both reach the wire |
| `X_CDP_LAUNCH_FAILED` | Chrome gets a 60 s launch deadline and one retry; the error lists each attempt in `cause` and `meta.attempts` |
| `disableUser` | also revokes the user's live API keys; the result gains `apiKeysRevoked` |
| `updatePrivileges` with a `passwordHash` | ends the user's other sessions; the result gains `sessionsRevoked` |
| OAuth route failures | the body carries one fixed `cause` and `x errors explain <CODE> --json`; the detail is the `auth.oauth.refused` log line |
| OAuth discovery and JWKS | a document answering for another issuer is refused; a key set with no importable key keeps the cached keys |
| `x doctor` | two auth probes: unsealed second-factor secrets, and an orphaned framework table |
| `X_CDP_TIMEOUT` | `meta.reading` says `target-gone`, `lost-in-transport` or `no-answer`, with the frames seen since the call |
| `search()` refusals | `X_INPUT_INVALID` (400), were `X_INVARIANT` (500) |
| entity tests on `memoryDriver()`, again | decimal kinds compare by value: `'2.5'` matches a stored `'2.50'`, `10` matches a `bigint()` column — as Postgres does |
| the typed read client | a `Date` input goes as its ISO instant; an omitted required array reads `[]` |
| `transition()` | declared idempotent; its `id` input follows the entity's key instead of always `t.uuid`, so the published schema of a transition on a non-uuid key changes |
| `.contract()` | checks the registry-wide OpenAPI document; an unregistered `.named()` twin of a registered route is `X_CONTRACT_DRIFT` |
| `ERROR_STATUS` | unchanged for importers; the table now lives in per-tier files |
| `requeue` of an unknown id | `X_JOB_NOT_FOUND` (404) on both drivers; was `X_INVARIANT` on memory, `X_DRIVER_UNAVAILABLE` on Postgres |
| `BulkResult.remaining`, `stats()` | `remaining` counts only what a second call would move; queues are ordered by code unit on both drivers |
| worker logs | new lines `jobs.claim.exhausted`, `jobs.worker.slot-renewal-failed`, `jobs.step.failure-unrecorded`; the claim loop no longer spins during a database outage |
| `x_job_events` | swept by the hourly `x.purge`; no DDL changed in this slice |
| the replicator | a stream that ends is restarted on `retryDelayMs`; a web+replicator process answers `/readyz` 503 while its stream is down or a change is still being refused by the bus; `ReplicatorStats` gains `restarts` and `failure` |
| the page socket | beats every 10 s by default, was 15 s; the node names the real beat in `hello.heartbeatMs` |
| realtime `fix:` lines | nine Postgres-client fixes are `x doctor --json`; a bad `NATS_URL` is named without echoing it |
| records channels | a bus reconnect or a sequence gap sends `replay-gap` on every open records channel, not only to live queries |
| the outbox | a store that cannot be opened or wiped warns `X_LOCAL_STORE_UNAVAILABLE` and queues in memory instead of rejecting for the rest of the tab |
| a zone that is not a string | `X_TIMEZONE_INVALID` from every zoned function in `@ultimat3/time`, was a bare `TypeError` |
| a static route outside ASCII | matches its percent-encoded request (`/precios-espa%C3%B1a`), was a 404; a server declaring one starts, where `Bun.serve` threw |
| ISR regeneration | a render that never settles frees its path after `regenerateDeadlineMs` (default 30,000); a render slower than that overlaps the next one, and its late page is dropped when a newer one started |
| ISR route lookup | a stored path is filed under the most specific matching route, as the request router picks it, not the first in table order |
| the client router | a reload or full-load Back lands at the saved offset; the page can enter the back/forward cache; `<a href="#">` is the browser's; form line breaks go as CRLF; an aborted navigation's stylesheets are retired. Router script +774 B raw |
| render refusals | a non-text `prerender()` item or param is `X_PRERENDER_FAILED`, `asset()` of a non-string `X_ASSET_MISSING`, a non-string island resolver answer `X_ISLAND_INVALID` — each was a bare `TypeError` |
| `display_override` | follows `display`: `fullscreen` → `fullscreen, standalone, minimal-ui`; `minimal-ui` → `minimal-ui`; `browser` → none; `standalone` unchanged |
| the service worker | a non-JSON push body is the notification text; the install fill skips another build's stamped answer; route rules match encoded literals and a catch-all's bare prefix, in the request router's order — `sw.js` changes once |
| `personalPages: 'last-member'` | a personal page declaring `offline: 'precache'` is kept in the member's partition; it was never kept |
| the boot's theme script | stamps `data-theme-default` too: 52–54 B longer, a new `script-src` hash the boot admits itself. `BUILD_STATS_RULES` is 4, so an old `.x/build-stats.json` is stale until `x build --target static` |
| `<QrCode>` | every code changes its modules — it now decodes; each was unreadable. A snapshot of its markup updates |
| `DataTable` | leaves its error or empty state when the query recovers; those states now render inside the table's wrapper, which carries `class` |
| `defineTheme()` | a role overridden in light only is answered with the shipped dark channels in the dark media block; that brand's CSS and `brandStyleCspSource()` hash change |
| `createFormBinding` | a local `validate` that throws or rejects is a failed submit, not a `submit()` that rejects and stays `submitting` |
| `CommandPalette`, `Tooltip`, `Popover` | centred under `dir="rtl"`; `Popover`'s inline placements sit beside the anchor, and `align` applies to `block-*` placements only; the vertical `Tabs` indicator mirrors |
| `useId` in an island | `<prefix>-<scope>-<n>` with a per-bundle random scope; server ids unchanged |
| `Textarea` in an island | sets `.value`; the leading newline is written on the server only |
| `RelativeTime`, `BarChart`, `linkTarget`, `Meter` | "1 hour ago", was "60 minutes ago"; bars still drawn from 201 points; `//host`, `HTTPS://…` and a leading space are external; `aria-valuenow` clamped to the drawn bar |
| the icon generator | a non-JSON body or a malformed node is `X_UI_INVALID_VALUE`; `bun run --filter @ultimat3/ui icons --bump` moves the pin and regenerates |
| a `.pattern()` with flags | its JSON Schema carries `x-ultimate-pattern-flags` beside `pattern`, in OpenAPI and MCP `inputSchema`; the input's fact in `x.manifest.json` moves with it, so step 45 covers it. MCP argument validation applies the flags — `/^[a-z]+$/i` accepts `ABC` |
| `db.query` | a column named like a keyword plus digits (`set2`) is read, was refused as `SET` |
| outgoing mail headers | `Reply-To` encodes a non-ASCII display name; an ASCII one holding a special is quoted (`"Doe, Jane" <jane@x.test>`); `List-Unsubscribe` is punycode and percent-encoded. `X_MAIL_HEADER_INVALID` gains `meta.reason` |
| `SMTP_URL` | a `%` in the user or password that starts no escape is `X_CONFIG_INVALID`, was a bare `URIError` |
| `registeredMails()` | ordered by code unit, as `registeredMailIds()` is |
| notify fan-out | a recipient id named twice is delivered once; it failed `X_STEP_DUPLICATE` part-way through the audience |
| `x.purge` | sweeps `x_notify_digests` windows closed longer ago than `PgDigestStore.retentionMs` (7 days); `PurgeReport.swept` has seven entries — read them by name |
| `BudgetStore` traffic | an `actor` or `org` counter is written only where that ceiling is declared — an app with no `budget` writes nothing. Under a declared ceiling the default `MemoryBudgetStore` holds one counter per caller who spent, per process (no window; a counter that returns to zero is deleted; `size()` reports the count). A daily or monthly ceiling needs a shared store whose keys expire |
| `llm()` / `agent()` with a non-object `output` | `respond`'s schema is `{ value: <output> }`; the answer comes back unwrapped, and a prose answer that is not JSON is taken as the text |
| the gateway response cache | every key is a 16-hex `fingerprint` over the whole request, tools included, so every entry misses once; a failed `cache.set` is the log line `ai.cache.write_failed`, the answer still returned |
| the semantic cache | an entry is written under `<prompt hash>:<fingerprint>`; lookup is by embedding, so stored entries still answer — unless `HashEmbedder` wrote them (entry 126) |
| `estimateTokens`, `reserve` | the completion is capped at the model's `maxOutput`, so a `maxTokens` above it reserves less |
| provider failure details | read to 4 KiB, 300 characters shown, on every transport; `RemoteEmbedder` scrubs its key from them |
| eval regressions, `numericTolerance(0)` | both sides and the drop rounded to 3 decimals, so a drop of exactly `tolerance` passes; an exact match at `tolerance: 0` scores 1, was 0 |
| `chunk()` | no chunk exceeds `size`: the carried overlap yields to a unit it does not fit beside, so a re-index may cut differently |
| the install | `@ultimat3/ai` depends on `@ultimat3/http` |
| island mount cleanup | the `disposeLiveIslands` file-boundary hook ships with the island fixture, not `@ultimat3/testing/preload`: any preload that installs the leak guard runs it. Each mount's scratch directory goes on dispose, on a throw, or at the next file boundary — none left in the temp root |
| island template parsing | `<!>` and `` are one empty node; `&amp;`, `&lt;`, `&gt;`, `&quot;`, `&apos;` and numeric references are decoded once in text and attributes — a reference to 0, a surrogate or past U+10FFFF is U+FFFD |
| a failed scrape session save | after a successful login it is the log line `scrape.session.write_failed` (`step: 'session.save'`) and the attempt goes on; it failed the run |
| admin list paging | rows sharing a sort value are neither skipped nor repeated at a page boundary, and a backward page answers rows; relation labels resolve every id on the page |
| an admin datetime edit | a field left untouched is left out of the patch, so its stored seconds and sub-seconds survive |
| admin search | a repo that throws lists its resource under `skipped` (`admin.search.skipped.failed`), not a failed search |
| the timeline and cache dev panels | only `DevSourceUnavailableError` is drawn as "unavailable"; any other error from a source is thrown |
| `BATCH_QUEUED_REASON` | declared in `batch-queue.ts`, still exported from `@ultimat3/admin` |

### Where the sites are

```sh
grep -rnE "\.default\(" apps packages --include=*.ts --include=*.tsx
grep -rnE "retry\(|retryDecision\(|createFlightGate\(|withChildContext\(|allowHosts" apps packages --include=*.ts --include=*.tsx
grep -rnE "LOG_LEVEL|ULTIMATE_CURSOR_SECRET|OTEL_(EXPORTER_OTLP_(TRACES|METRICS)_HEADERS|TRACES_SAMPLER)" docker .github apps packages
grep -rnE "(ssl|enabled|expose): *process\.env" apps packages --include=*.ts
grep -rnE "withTransaction\(" -A12 apps packages --include=*.ts | grep -E "catch|Promise\.all|isolation|readOnly|deferrable|client:"
grep -rnE "formatRelative\(|addDaysInZone\(|formatDuration(Iso)?\(|cron: |expiresAt|reportEveryMs|introspect\(|\.generated\b|DriftKind" apps packages --include=*.ts --include=*.tsx
grep -rnE "promoteAttachment\(|\.lastModified|implements StorageDriver|: StorageDriver = |canonicalRequest\(|signConstraints\(|variantKey\(|fitDimensions\(|\.get\(|allowedContentTypes|responsiveImage\(|usableWidths\(|declareTags\(|revalidate:" apps packages --include=*.ts --include=*.tsx
grep -rnE "assertAllowed\(|X_FORBIDDEN|definePolicy\(|status: 403|\.preload\(|dbDrift|ENTITY_ERROR_CODES|: Driver = |implements Driver" apps packages --include=*.ts --include=*.tsx
grep -rnE "ctx\.peer|trustProxy|cache-control|defineHttpConfig\(|healthz|readyz|\.close\(\)|\.closed\b|CdpLaunchFailedError" apps packages docker .github --include=*.ts --include=*.tsx --include=*.yml --include=*.yaml
grep -rnE "mfaSecret|redeemRecoveryCode|mfaRequired|authNotImplemented|verifyApiKey\(|apiKeyActor\(|actorFromApiKey\(|issueApiKey\(|apiKeyResolver\(|assertAllowed\(|recordFailure\(|implements AuthAdapter|: AuthAdapter = |: AuthLimiter = |X_DB_UNIQUE_VIOLATION|oauthLogin\(|meta\.userId" apps packages --include=*.ts --include=*.tsx
grep -rnE "compareValues|compareRows\(|matchesFilter\(|isAfterKey\(|adminPermissions|single: true|\.limit\(|_first=" apps packages --include=*.ts --include=*.tsx
grep -rnE "mutator\(|postgresIdempotencyStore\(|configureIdempotency\(|bustAfterCommit\(|invalidates:|X_MANIFEST_DRIFT" apps packages scripts .github --include=*.ts --include=*.tsx --include=*.yml
grep -rnE "attempts: 1\b|cancelJob\(|\.cancel\(|runId:|purgeExpired|EventLookup|SQL_CLAIM|SQL_SCHEDULER_FIRE|RetentionStores|\.swept|implements JobDriver|: JobDriver = " apps packages scripts --include=*.ts --include=*.tsx
grep -rnE "APP_URL|SYNC_URL|sslmode|allowedOrigins" docker .env.production .github 2>/dev/null
grep -rnE "NativeReason|\.specificity|compilePattern\(|ttl: *'0|/\*[A-Za-z]+|notificationclick|showNotification" apps packages --include=*.ts --include=*.tsx
find apps -type d -name '[[]...*[]]'
grep -rnF '\.' apps --include='*.module.css' --include='*.module.scss'
grep -rnE "channel\(|parseEnvelope\(|parseChange\(|implements Transport|: Transport = |AdvisoryLock|UpgradeTarget|hasOwnProperty\(|OfflineQueue" apps packages --include=*.ts --include=*.tsx
grep -rnE "ThemeEnv|prefersDark|clearTheme\(|watchOsTheme\(|THEME_INLINE_SCRIPT|themeInlineScript|aria-controls|<Dropzone|<FileInput|defaultMode" apps packages --include=*.ts --include=*.tsx
grep -rnE "RateLimitStore|RateLimiter|defaultBucket|rateLimits|errorAudience|mcpHttpRoute\(|\.handle\(|destructive|listParams|createJobsFacade\(" apps packages --include=*.ts --include=*.tsx
grep -rnE "send\(|replyTo|cc:|bcc:|unsubscribeUrl|SMTP_URL" apps packages docker --include=*.ts --include=*.tsx --include=*.yml --include=*.env*
grep -rnE "BudgetStore|callLedger|createGateway\(|\.scope\(|fnv1a|HashEmbedder|finish_reason|VectorStore|\.scoped\(|UNSCOPED|chunk\(|indexDocument\(|numericTolerance\(|agent\(" apps packages --include=*.ts --include=*.tsx
grep -rnE "describeApp\(|ReferenceError|assertDeterministic\(|frozenClock\(|onClockMoved|mountIsland\(" apps packages --include=*.ts --include=*.tsx
grep -rnE "localBrowser\(|args:|CdpBrowserLike|burnSession\(|maxDrop|urlSecretValues|redactSecrets|allowHosts|createRobotsGate\(|robotsFetcher\(|recordedAt" apps packages --include=*.ts --include=*.tsx
grep -rnE "existingSecret|sizeLimit|networkPolicy|metricsFrom|port: 0|--port 0|syncUrl" docker deploy helm charts apps packages --include=*.yaml --include=*.yml --include=*.ts --include=*.tsx
grep -rnE "ROLE=|migrate|TRUSTED_PROXY_HOPS|METRICS_PORT|PORT=|readinessGraceMs|readinessProbe|content-security-policy|/_storage|/media|/_x/db|x routes list|x help |DoctorProbe|tests\.run|x verify merge|x shot" docker .github apps packages scripts --include=*.ts --include=*.tsx --include=*.yml --include=*.yaml --include=*.tpl --include=*.sh
grep -rnE "sitemap-[0-9]|restorePermissions\(|ProcessRegistrySnapshot|--out|--tag|--no-preflight|--dry-run|--affected|solid-js|\"typecheck\"" apps packages scripts .github --include=*.ts --include=*.tsx --include=*.json --include=*.yml
grep -rnE "keyset\.value|AdminCursor|<AdminForm|CrudResult|adminUpdate\(|_version|listHref\(|scope=|scopes:|definePolicy\(" apps packages --include=*.ts --include=*.tsx
```

Tier 0: the `typecheck` step finds none of 1–16 — every entry is a value, not a type. A typed
`app.config.ts` already refused most of entry 6 at compile time; the ones it did not are
`'/'`-less paths, empty lists and a locale spelled twice. Entries 4 and 6 throw at the first import;
7 and 16 at boot; 8 and 9 where the call is made. Entries 1–3, 5, 10 and 11 need the unit,
contract and e2e suites; 12–15 need a read of the deploy environment and the monitor.

Tier 1: the `typecheck` step finds 20, and 18, 26 and 28 where a literal, a `switch` or a string
read exists. Entries 19 and 30 throw at the first import of the declaring file; 24 and 31 at the
call. It finds none of 17, 21–23, 25, 27 or 29 — run the unit, contract, job and live suites, and
read every `catch` and every `Promise.all` inside a `withTransaction` body.

Slice 03: the `typecheck` step finds 35, 36, 37 and the changed signatures of 41. Entry 43 throws
at registration and 33 where the response is built. It finds none of 32, 34, 38, 39, 40 or 42 —
run the unit, contract and e2e suites, and do step 13 by hand after the deploy.

Slice 04, entity and policy: the `typecheck` step finds 44, 45 and a `403`-typed status in 48. It
finds none of 46, 47 or 49, nor a test asserting on 48's status or title — run the unit, contract
and policy suites.

Slice 04, http and testing: the `typecheck` step finds a read of `closed` and a
`CdpLaunchFailedError` built with `detail` (57). It finds none of 50–56 and no unawaited
`close()` — run the unit, contract and e2e suites, and read the deploy's proxy, probe and monitor
configuration for 50, 52 and 53.

Slice 05, auth: the `typecheck` step finds 60, 64, 65 and the changed shapes of 61. `x doctor`
finds 58 (the count of unsealed rows) and 59. Nothing finds 62, 63, 66 or a missing `APP_URL` (67)
before a request does — run the unit, contract and e2e suites and sign in once with an MFA user
after step 20. The e2e suite finds 68.

Slice 06, query, mcp and admin: the `typecheck` step finds 69 and the removed `adminPermissions`
of 72. It finds neither 70, 71 nor a permission set that leaned on the import — run the unit,
contract and e2e suites.

Slice 06, action and cli: the `typecheck` step finds 73 and 77. The gate finds 74 and 79 (the
`manifest` step) and 75 (the `drift` step). It finds neither 76 nor 78 — run the unit, contract
and job suites.

Slice 07, jobs: the `typecheck` step finds the interface members of 81, 85 and 87. It finds none
of 80, 82, 83, 84 or 86, nor an unbound statement parameter in 81 — run the unit and job suites,
and do step 30 by reading.

Slice 08, realtime: the `typecheck` step finds 92. Nothing finds 88–90 before the deploy does —
they are steps 32–34, done by reading the deploy's environment. Entries 91 and 93–96 need the unit,
live and e2e suites; 91 also needs both roles deployed together.

Slice 09, http, render and pwa: the `typecheck` step finds 99. Entry 98 throws at registration. It
finds none of 97, 100, 101 or 102 — run the unit, contract and e2e suites after a fresh `x build`,
and read each push sender's `url` for 102.

Slice 09, ui: the `typecheck` step finds 103, 105 and an annotated or `string`-typed trigger in 106.
It finds neither 104 nor 107 — run the unit and e2e suites after `x build --target static`, and
read `theme.defaultMode` in `app.config.ts` for 104.

Slice 10, first half — http, mcp, mail, notify and manifest: the `typecheck` step finds 108. The
gate finds 116's job schema and 118 (the `manifest` and `contract-diff` steps), 117 (the `drift`
step), 119 at the first manifest build and 120 at `contract-diff` the day a default is dropped.
It finds none of 109–115 or 116's recipient rule — run the unit, contract, job and e2e suites,
and read the deploy's probes and agents for 109 and 114.

Slice 10, ai: the `typecheck` step finds 121, 122, 125 and 129. Entry 131 throws where
`numericTolerance` is called — an eval file's import — and 128 at the first unscoped read in an
org's request. It finds none of 123, 124, 126, 127 or 130 — run the unit, eval and e2e suites, read
the gateway's `budget` before the deploy for 123 (step 49), each agent tool's thrown codes for
124, and re-index what `HashEmbedder` wrote for 126.

Slice 11, testing and scraping: the `typecheck` step finds 137 and a one-argument `burnSession`
(139). Entry 140 throws at `scrape()` — the declaring file's import — and 136 when the browser
launches. It finds none of 132–135, 138 or 141–145 — run the unit, job and e2e suites, and read
each `allowHosts` wildcard and each site's `robots.txt` redirect for 143 and 144.

Slice 11, admin: the `typecheck` step finds 146, 148 and 149. Entry 153's `*` scope throws where
the resource is declared. It finds none of 147 or 150–152, nor 154's links — run the unit, contract
and e2e suites, read each policy rule over `input` for 150, and each script that posts an edit form
for 147.

Slice 12a, seo, policy, cli and testing: the `typecheck` step finds 156 and 171. The gate finds
169 (the `policy` step), 170 (the `drift` step) and 165 (`filesize`, `errors`); 164 is a test that
now runs. `x build` finds 157–159 and `x g` 161–162 at the call. Nothing finds 155, 160, 163 or
166 — read the sitemap's host, each script that reads `x g --dry-run` or `--affected`, and each
check that expects a fresh SEO answer. 167 and 168 change only a fresh scaffold.

Slice 12b, http and cli: the `typecheck` step finds 182's `DoctorProbe`. Entries 174–176 stop the
boot — 174 on every serving pod until `ROLE=migrate` ran, which is step 65, done before the deploy.
Each command in 180, 181, 183–187, 189 and 190 refuses at the call. Nothing finds 172, 173, 177,
178, 179 or 188 — run the unit, contract and e2e suites, and read each handler's CSP, each link to
an upload, the chart's replicator and each monitor that matches a fix string.

Slice 13a, cli: nothing finds 191 but the call — a repository of your own that runs
`checkErrorCodesThrown` reads its findings once after the upgrade.

Slice 13b, the chart: `helm template` finds 193's missing `tmp.sizeLimit` and 194's role with no
Secret — before the deploy, if it runs in CI. Nothing finds 192 but a scraper that stops reading,
nor 195 but a dial that is refused — read each values file and each port-0 test.