Known gaps
Defects and unfinished seams named here rather than left to be discovered. Each row carries its own As of stamp.
A reference manual that hides these is lying to the reader.
Read the section headings first. Open is what is wrong with the code you can install today. Open by decision will not be fixed and says why. Not built yet is unfinished work, not a defect. Carried backlog is owed work with a tracking issue each. Deferred by plan 101 is ranked work a later plan owns. Closed is history — every row there carries the release that fixed it and the workaround for anyone still pinned below it.
Only the footer stamps the installable version. Resolve it yourself rather than reading it here:
npm view @ultimat3/core versionA Closed row whose release is newer than the footer's is fixed in the repository and in no published release yet; if you
are on an earlier version, treat those rows as open and take the workaround. [Unreleased] in
CHANGELOG.md is the source of
truth for what the next release carries, and most of what it fixes was never a row here. Read that section rather than this sentence: grep -n '^## ' CHANGELOG.md finds its bounds, and it changes on any commit.
Publication is not a gap. All 31 workspaces are on the registry, every one with a provenance attestation, As of 2026-11-10 by bun run scripts/registry-audit.ts --json, which registry-audit.yml re-runs daily;
@ultimat3/scraping was the last never-published package and was bootstrapped by hand at 2.0.0
(PUBLISHING.md step 1).
bun add @ultimat3/scraping resolves — check it with npm view @ultimat3/scraping version.
Open
| Gap | Symptom | Work around it by |
|---|---|---|
pdf() prints with no session of its own | As of 2026-11-09. The Chrome that pdf() drives opens the app's route as nobody: no cookie jar is handed across, so a route behind a session answers exactly as it would a signed-out visitor — its 4xx is X_SCRAPE_PDF_ROUTE_FAILED, a sign-in page it renders is what gets printed. pin: none — the gap is an absence no assertion can hold (nothing hands the browser a cookie); the workaround is held by packages/scraping/src/pdf-render-fence.test.ts, 'a same-origin request carries the headers; a cross-origin one is blocked without them' | issue a short-lived token the print route accepts and send it through pdf({ headers }) — Scraping |
A purge that a replica never hears is bounded by ttl, not by the next request | As of 2026-10-10. The purge reaches the other replicas through the invalidation broadcast. With the bus down, a replica with its own store answers its copy as a fresh hit until that copy's ttl (and tells an edge the ttl's remainder), then renders. 'purge' therefore requires a ttl | choose the ttl as the longest a missed purge may last; or share one runtime.isrStore with a tagFence, where the delete is every replica's at once |
| A bust purges by the route's tags | As of 2026-10-10. A stored isr page records no tags of its own, so post:1 reaches every stored page of a route tagged post — one render too many per page, never one too few. Each bust reads the store's keys once; the route of each key is remembered, not re-matched pin | tag a route as narrowly as its data allows; a very large shared store pays one paths() per bust per replica |
A declared isr query parameter's values are unbounded | As of 2026-10-10. query: ['page'] puts page in the key, and ?page=1 … ?page=999999 are each a key: the framework cannot know which values name a page. The store is bounded (least recently generated out first), so it thrashes rather than grows pin | answer withStatus(404, …) from load for a value that names nothing, and rate-limit the route at the ingress; put a small closed set in the path instead (/blog/page/[n] with prerender()) |
A grantUpload with a target is never sniffed | As of 2026-11-02. A presigned PUT signs host and the size, so on an s3 disk the uploader chooses the stored type and the bytes. promoteAttachment is what checks them (X_STORAGE_TYPE_REJECTED), and a grant with a target lands straight on the row's key with no promotion step. With an uploadScanner installed such a grant is refused outright (X_STORAGE_TARGET_UNSCANNED, As of 31.0.0), so the gap is only an app's with no scanner. Served, it is contained: objectResponse sandboxes a type a browser would run, and a false image/png is shown under nosniff pin | grant with no target and call promoteAttachment({ …, policy }); for a targeted grant, read the head yourself with validateStoredHead(…) before writing the key to the row → Storage and uploads |
| A queued offline write is not bound to its principal on the server | As of 2026-10. The fence is in the page: a principal change abandons the running replay, refuses a write in flight (X_OFFLINE_QUEUE_ABANDONED) and wipes the queue. A second TAB whose replay is already past its store read when another tab changes the session cookie still sends its remaining entry under the new session — one entry, one in-flight pass, on a shared browser | sign out by full navigation (signOutHeaders() clears site data, and every boot wipes other principals' queues); do not share a browser profile between accounts with offline writes queued |
| Presence leaves past the lease roster cap | the sweep leader ships at most 4,096 member ids on its lease (PRESENCE_LEASE_ROSTER_LIMIT). When the leader's node dies, members beyond that whose TTL expired before the successor's first leading pass are never announced as left. A successor cannot diff against the shared store: the expired members are already gone from it | nothing to do: each client's next beat is answered with the whole roster, so it heals within one beat (10 s by default) |
An offline scrape reads /robots.txt over the real network | every fakeBrowser / fixtureBrowser run takes the default robots: 'obey', and the gate reads https://<host>/robots.txt with the platform fetch before the first navigation, so outside bun test one request still leaves the process. The silent half is closed As of 2026-10-04: an unreachable robots.txt (5xx, 429, network error, timeout) now refuses the origin with X_SCRAPE_ROBOTS_DISALLOWED (RFC 9309 §2.3.1.4 for 5xx, network error and timeout; refusing on 429 is Ultimate's stricter policy — the RFC calls a 4xx unavailable), so under bun test the sealed network's refusal turns the run red instead of reading as allow-everything | declare the policy in the definition under test — robots: { ignore: 'fixture host, no live origin' } — which returns a gate that reads nothing at all. There is no boolean and no ambient off switch, by design — see Scraping |
| The cache fill fence crosses nodes for tag busts through Redis only | As of 2026-10. A read-through fill re-checks a fence before it writes. Inside one process that covers everything. Across nodes: the Redis tier keeps leased generation keys a bust writes and a fill samples before load() and re-reads after its SET (packages/cache/src/redis-fence.ts), so a tag bust on another pod withdraws the fill even when the broadcast is lost — at one extra round trip before each load() and one after the SET. Three things still do not cross. Key busts: CacheStack.write and .drop mark { key } locally and the wire form carries tags only. A deployment with no Redis tier: there is no shared store to hold a generation, so the window is the broadcast's. Another pod's LRU: a value it cached before the bust stays until its TTL when the broadcast is lost | give a cache: query a TTL you can afford to be wrong for on one pod, keep redis in cache.tiers on a multi-pod deploy, and prefer a tag bust over cache.drop(key): only the first one is broadcast and fenced |
| A replicated table with no replica identity is warned about, never refused | As of 2026-10-02. A channel declared with params does need REPLICA IDENTITY FULL on its records tables — a DELETE is routed by the old row's params, which a key-only image does not carry, so members keep the deleted record; x db gen grants it since 24.0.0 (one migration for an app that already declares such a channel), and the replicator warns (replication.channel_identity_partial) about a table that still lacks it. Live queries do not need REPLICA IDENTITY FULL on a keyed table: the shared window holds the whole row, so a row leaving the result set and a delete are decided from the window — proved on real WAL by pg-identity-window.live.test.ts (out-of-filter update, into-filter update, delete with a key-only before-image, updates to rows never held). What still breaks is a table with no identity at all — NOTHING, or DEFAULT with no primary key — whose UPDATE/DELETE Postgres refuses once published. X_LIVE_REPLICA_IDENTITY names exactly those at preflight, and warns rather than throws; no x verify step reads it. The REPLICA IDENTITY FULL the app's declarations need is checked at the gate: since 24.0.0 the drift step reports a params-channel table whose FULL no migration recorded (X_DB_SCHEMA_UNMIGRATED) — against the migrations, never the live database. A live query's subscribes: tables are not granted FULL, As of 2026-10 (#518): x db gen validates the names and emits no ALTER; a table already on FULL keeps it, and the drift step reads that as agreement | give the table a primary key, or ALTER TABLE <t> REPLICA IDENTITY FULL; — the warning names every table that needs it |
HPAs read <unknown> without a metrics adapter | the chart half is done — values.yaml declares metricsPort: 9090, every role but migrate emits the container port, service.yaml publishes it and templates/servicemonitor.yaml ships the scrape target, rendered wherever the cluster serves the Prometheus operator's kind (serviceMonitor.enabled: auto, As of 2026-11). One thing still stands and it is not the chart's to fix: turning scraped series into the Pods metrics an HPA reads needs a custom-metrics adapter, which is the cluster's pin: none — what is missing is the cluster's custom-metrics adapter, outside this repository | install a metrics adapter (and the Prometheus operator, if the cluster has none; an app chart from before the scrape object gets it from x g chart-alerts); until then disable the HPAs and pin replicas. Do not hand-add a metrics container port — the chart emits one and a duplicate is rejected by the API server → Observability |
--target binary is a launcher for an app tree, and has never been served from a bare VM | As of 2026-10-06. It boots: x build --target binary compiles apps/web/server.ts with --compile-autoload-package-json, --compile-autoload-tsconfig and @ultimat3/* external (binaryArgs, packages/cli/src/cmd-build.ts), and the required windows CI job starts the result from a scaffold and gets 200 from /healthz; packages/cli/e2e/binary-launch.e2e.test.ts proves the same on Linux. It is not a self-contained artifact: at run time it reads the app's package.json and tsconfig.json, imports app.config.ts and every module under apps/*, and loads the framework from the app's node_modules — one instance, so the app's registrations reach the server. Unmeasured: a scaffold compiled, copied with its tree to a VM with no Bun, and served under systemd pin | ship the app root, its installed node_modules (installed for the target OS and CPU) and the binary together, and start the binary from that root. Prove it for your own app first — ./app --version, then ROLE=web PORT=3000 ./app and curl /readyz. Prefer --target docker, the self-contained artifact → Deployment |
A local typecheck step can report green on a tree CI fails, on the same commit — one sighting, unreproduced | As of 2026-10. Observed once (2026-09-17, TypeScript 7.0.2): after a git stash -u → branch switch → x verify on the other branch → switch back → stash pop sequence, a type error that surfaces in an UNCHANGED file (a widened index signature in a file it imports) passed locally and failed CI; deleting tsconfig.tsbuildinfo then reproduced CI's red locally. Five faithful reproduction attempts against 7.0.2 all stayed correctly red (#450). #450 itself is closed As of 2026-09-23 — read from the shipped fix, not from the tracker: plan 101 row 14 r ("Close #450", docs/plans/2026/09/23/101-framework-deep-sweep/14-guards-gate.md:37; plan status.yml complete), recorded in CHANGELOG.md under 22.0.0 and pinned by packages/cli/src/verify-typecheck.test.ts. tsc -b decides "up to date" by mtime, and Bun installs a dependency as hardlinks carrying its cache's old mtimes, so an upgrade to an already-cached version left every input older than the buildinfo and a root without references — every scaffolded app — skipped the program and reported green while a cold CI was red. A scaffolded app's typecheck is now tsc -p . (packages/cli/src/templates/scaffold-repo.ts, its "typecheck" script), which is incremental by content hash, and x verify's typecheck step picks -p . over -b for any such root (packages/cli/src/verify-typecheck.ts); the framework root keeps -b. Still open: a second tsc racing the same buildinfo is the one suspect not excluded, and no cause for it has been shown. x verify deliberately takes two flags and no --cold pin: none — one sighting, and five faithful reproductions stayed red, so no test can hold it | CI is always cold and is the judge of record. If a local typecheck disagrees with it again, capture the command, its argv and its exit code before deleting any tsconfig.tsbuildinfo |
A stream page with a realtime island keeps every island on the page inert until its slowest hole closes | As of 2026-11-02. The runtime goes async so a shell island answers while a hole is open, but only with no realtime boot in the document: the boot (script defer) has to run first (bootScript, packages/cli/src/route-islands.ts), and a deferred script waits for parsing to end. Bound by the hole's 15 s deadline | keep the slow read out of such a page's holes, or give it a short deadline of its own: fetch(url, { signal: AbortSignal.timeout(2_000) }) |
| An open tab can meet a dead island across a deploy | As of 2026-11-03. A web process serves only the hashed files it built, so with assets.retain unset (the default) a tab holding the previous build's document gets a no-store 404 for its stylesheet, or for its island chunk on the first click, after every deploy; during a rolling update a new document that reaches an old pod gets no stylesheet either pin. Declared, a tab survives six deploys with build 0's island alive evidence | declare assets: { retain: { disk, builds } } over a disk every web pod reads → Deployment |
| A soft visit waits for the slowest hole | As of 2026-11-02. The client router reads response.text() (packages/render/src/navigation-fetch.ts) and swaps the whole answer, so holes fill through the re-run scripts and the first paint of an in-app navigation is as late as a full ssr render. A full load streams. No incremental parse here pin | nothing to do in an app; a page that must paint first on a soft visit declares navigation: 'document' |
| A pod evaluates the framework from source, one module at a time — ~28 Mi of resident memory per pod that a bundle would not hold | As of 2026-10-01, measured on the framework serve graph alone: 1,071 modules from source settle at 80.4 Mi; the same graph as one bun build --target=bun file is 5 modules and settles at 51.8 Mi — ~34 kB of RSS per source module, the runtime's cost and not the code's. It is the largest saving left after the per-role load and the prebuilt store (a web pod of the demo app settles at 97–102 Mi, a worker at 80–85 Mi). Not a patch: a registry is a module instance, so the framework bundled apart from the app is two @ultimat3/jobs, two @ultimat3/entity — the app registers into one and the boot reads the other. The unit has to be the app and the framework as one graph, per role, built inside the image build from a generated entry that imports what scanAppModules would, with the .tsx/.scss loaders as build plugins. Containers only either way (axiom 7). Design note: docs/architecture/13-topology-runtime.md pin: none — a resident-memory measurement of a running pod, not a behaviour a unit test can assert | nothing to do in an app. Size a pod's memory request from process_resident_memory_bytes on /metrics, not from a guess |
Every role evaluates all of @ultimat3/jobs — 96 modules (As of 2026-10-02, by file) — whatever it runs | As of 2026-10-01, read off the serve graph by the names each role's code uses: a web pod needs 43 of them (enqueue, the outbox relay, the purge task), a scheduler 46, a worker 62. The rest ride the barrel: worker.ts and its eighteen dependencies in a web pod, the backfill() / exportJob() / webhook() factories (17 modules, 123 kB of source) in an app that declares none, and the memory driver (3 modules) outside a test. ~1.6 Mi per web pod at the measured 34 kB a module. Closing it means subpath entries (@ultimat3/jobs/worker, one per factory) and every importer rewritten — a breaking move for a saving the per-role bundle above makes moot, so it waits on that decision. The operator surface is not in this list: JobIntrospection is a member of the driver, and the worker announces itself and counts settles through it pin: none — a module count read off a role's serve graph, measured rather than asserted | nothing to do in an app |
A job's runAt is stamped from the enqueuer's clock | As of 2026-10. An enqueuer running ahead of the database delays its own "immediate" jobs; up to 1 s ahead still wakes a worker, beyond that the job starts on the poll. A delayed job enqueued by another process starts within the idle ceiling after its runAt — 5 s with a proven wake, 2 s without; only a retry or a sleep the worker itself handed back gets a due-time timer | NTP on every pod; for a deadline tighter than the ceiling, enqueue at the moment the work is due rather than with runAt |
onSettled runs at most once across a crash | As of 2026-10, by decision of the first cut: the hook runs after the settle, inline, by the worker whose settle landed. A worker killed between the settle and the hook does not run it, and nothing re-drives it | record what must be recorded for certain inside the body, in a step.run, and use onSettled for what may be lost |
| The CDP driver cannot PIN a name, and a page's WebSocket is not screened at all | As of 2026-11. Measured on Chrome 150 with localBrowser(): interception (page-level and the browser-level Fetch that now covers popups) never pauses a WebSocket handshake, so new WebSocket('ws://off-list/…') from a scraped page connects. Since 31.0.0 a host a wildcard admitted is resolved before the browser leg releases it, and an inward or unresolvable answer is refused (packages/scraping/src/browser-dial.ts). But the browser then resolves again for itself, so a name that answers public to the screen and private to Chrome (DNS rebinding) is reached — the HTTP leg and the robots read resolve and pin (packages/scraping/src/pinned-host.ts), the browser cannot be pinned through this port — pin | list exact hosts rather than '*' on a scrape that runs where private addresses are reachable, and dial a scrape through an egress proxy outside that network |
ScrapeReport.usage.bytesIn counts the HTTP leg only | As of 2026-10. The CDP port subscribes to requests, not to their sizes, so the browser leg's bytes are not in the figure (packages/scraping/src/usage.ts) | treat bytesIn as a lower bound; a provider's own byte count, returned through the resolver's cost, is the billable one |
An org-scoped operator's runs list reads x_jobs with no index on tenant_id | As of 2026-10. The jobs dashboard keeps an actor with an orgId to its own org's runs through JobFilter.tenantId, and x_jobs.tenant_id is a plain column: the only index naming it is the idempotency one, led by name (packages/jobs/src/driver-pg-ddl.ts). A platform operator's list is unaffected pin | on a large queue, add create index on x_jobs (tenant_id, created_at desc, id desc) in an app migration, or operate the dashboard as a platform operator (an actor with no orgId) |
| Retry-from-step on the jobs dashboard is a free-text field | As of 2026-10. job.retry-from-step takes { step: t.string.min(1) } (packages/admin/src/jobs/job-actions.ts); the run's detail page shows its steps, but the form does not offer them as a choice. | copy the step name from the run's steps section on the same page |
The admin MCP batch tool takes ids only | As of 2026-10. A batch action is one MCP tool, admin.action.<name>, and it takes the rows by id; "all matching" — the list's own filter — is a screen control only | list the rows with admin.<entity>.list and its where, then pass their ids |
| No counts on the jobs dashboard's state tabs | As of 2026-10. stats() counts per queue across every tenant, so a count on a tab would show an org-scoped operator other orgs' numbers; the tabs carry none and the overview's tiles carry the counts, for a platform operator only. A per-tenant count is a second query nothing issues yet | a platform operator reads the overview tiles; an org-scoped operator reads a tab's own page |
x drill proves one node per role on one machine, and drills no object store and no CDN | As of 2026-11-06. The fleet is one web, one worker (and one sync for nats-down) on one host, every dependency a local container behind a Bun TCP proxy: a multi-node NATS cluster, a Postgres failover to a real standby, a real S3 bucket and a real CDN in front of the web role are never faulted. nats-down's sync lane is proven on a stand-in node (packages/testing/src/drill-sync.test.ts), not on examples/dummy, whose transport is in-memory; its client subscribes to nothing by default, so a gap counts only what the node volunteers. CI runs three of the eight (fault-drills) pin | the rest by hand: x drill --list --json, then x drill <name> --json with docker/test-services.env loaded. A multi-node or object-store fault is a staging drill of your own, against docs/ops/06-runbooks.md |
Open by decision
Each of these is a defect somebody has already argued about, and the reasoning is why it stays.
| Gap | Why it stays |
|---|---|
| A local write that died after its last rename is listed with no content type until the key is written again | As of 2026-10. localDriver commits in four steps (marker, sidecar, bytes, clear marker) and a marker left behind means "re-check this pair". get() and stat() do re-check against the bytes and answer correctly; list() never reads bytes, so it reports that object with no contentType and etag: '' — honest, and it stays that way until the next put() of that key. stream() returns bytes only and checks nothing. Two files cannot be committed in one step without changing the on-disk layout |
A delete_resource preview checks only the policy's actor half | As of 2026-10-18. Without live: true the framework answers a preview without running the action, so the part of a policy that reads the input never runs; a preview can be granted for a call the commit then refuses. Running the policy would mean running the action. An action that needs an exact preview declares live in its input and answers its own |
localDriver cannot hold a key and a key beneath it (a and a/b) | As of 2026-10. An object is a file at its key, and a POSIX path is a file or a directory. The second put() is refused by code (X_STORAGE_KEY_CONFLICT) where it used to be a bare ENOTDIR; s3Driver and memoryStorageDriver hold both. Storing objects under suffixed names would lift it, at the price of a layout migration for every existing root — and the keys grantUpload and scopedKey build never nest an object under another, so the limit is met only by hand-built keys |
insert({ ...row }) of a row whose nullable .sealed() column was read from the repository stores NULL, silently | a sealed property on a repository row is not enumerable, so a spread leaves it behind (As of 2026-10). A required one is refused by name (X_INVARIANT_VIOLATED, fix { ...row, <col>: row.<col> }); an absent nullable column is a legal insert, and nothing tells a spread from an omission without making every omission an error. Copy a sealed column by name |
A drift fix: for a table in public does not name its schema | As of 2026-10-02. A fix for a table read from any other schema carries set search_path = "<schema>";; one for public is left unqualified, so it relies on the pasting session resolving public first. Postgres' default path is "$user", public, so a schema named after the connecting role that holds a same-named table shadows it. Check show search_path in that session, or prefix the pasted statement with set search_path = public; |
Drift never compares an index's where text | the catalog answers its own rewriting of the expression ((deleted_at IS NULL)) where the snapshot holds the author's spelling (deleted_at is null), so comparing the strings reports drift on a database that is exactly right. Normalising them means shipping an expression parser to compete with the server's. Three of the four parts of an index are compared now — columns, uniqueness, direction, and the predicate's presence — so a partial index recreated as a total one and a desc index rebuilt ascending are both caught (packages/db/src/drift.ts, compareIndexes). x db gen compares the text, where both sides are generated. A predicate rewritten by hand into a different predicate of the same presence is the residue, and it is invisible: regenerate and read the diff |
A long backfill() retains one step name per batch | stepRunner's claimed Set is the authority for X_STEP_DUPLICATE and is deliberately unbounded: a 5M-row sweep at batch: 250 holds 20,000 short strings (~1–2 MB) for the life of the attempt, released when it ends. MAX_TRACE_NAMES bounds the reported trace, not the membership set. Bounding the set would let a duplicate step name through after 200 batches, and silently replaying a step is the worse failure. Raise batch if the attempt is long enough for the retention to matter |
| The admin has no one-click select-all | As of 2026-10, by decision: the admin ships zero JavaScript (hydrate: 'never'), so every control is a link or a native form. A row checkbox joins the batch form through its form attribute, "all matching" is a radio whose meaning is the list's URL, and an action's input is its own page. Selecting every row on one page is one click per row. An island would buy that click and cost the admin its zero-script rule, and "all matching" already covers the whole list |
Not built yet
| Open | Where it stands |
|---|---|
| Multi-node realtime | the forced-restart benchmarks are measured, on one sync node over InProcessTransport — neither run crossed NATS and neither subscribes to a live query, so no cursor, snapshot or gap-repair path is under test. Fanout, throughput and per-node socket capacity across nodes remain targets → Realtime |
x db studio | planned. X_DB_STUDIO_FAILED is reserved and not thrown; the command exits X_NOT_IMPLEMENTED pointing at x dev → the /_x db panel. Keeping a bunx drizzle-kit studio shell-out would have meant a second schema engine for one subcommand |
api/**/route.ts | not a file kind — owner decision 1 (#648), As of 2026-10. api/ holds the action and query projections defineApi() collects; a route file there is refused at registration (X_ROUTE_FILE_INVALID), and 26.0.0 deleted ROUTE_FILENAME's api row (#709). A wire format an action cannot speak — a webhook receiver, an OAuth endpoint — is a plain HTTP route in the routes runtime override (Configuration) |
Deferred, As of 2026-10-07 | the plugin API and multi-region replication. The Redis job driver shipped (redisJobDriver(), #710) without the operator surface, backfill bookkeeping or fleet-wide concurrency — those answer on Postgres only; the NATS job driver stub that threw X_NOT_IMPLEMENTED was deleted in 25.0.0. |
Carried backlog
Rows plan 101 carried from superseded plans and sweeps 10a–10e left open. The
backlog row is the record — cause, evidence and fix live there and in the issue; a row here says
only what you meet and how to get past it. Backlog: 10-carried-backlog.md.
None open, As of 2026-10-07: sweep 13 closed the last four — B24 (agentJob output, #685), B25
(webhook's endpoints and ledger, #686), B26 (t.array bounds and transition()'s row loader, #687)
and B27 (one PgExecutor builder, dbExecutor(), #688), in PRs #701, #703 and #706.
Deferred by plan 101
As of 2026-10. Gaps the platform-readiness work ranked and did not take, with the evidence that
ranked them — a later plan's, not this release's. Evidence is from read-only surveys of a
downstream app on Ultimate and of systems built without it; no row is a defect in what ships.
| Gap | Evidence | Precedent |
|---|---|---|
a native form's refusal answer — setRedirect covers success (packages/http/src/redirect.ts) | 234 wrapper calls in 142 files | Rails' form convention: 303 on success, 422 re-rendering the form with its errors; flash |
| route-group defaults | 111 routes × ~25 repeated lines | layouts, before_action, controller inheritance |
a view-scoped lazy lookup for t() inside an island — the subset itself ships (catalogSubset / subsetTranslator, #710), named key by key | 94 label types, 3,092 lines of label files | lazy lookup t('.title') scoped to the view |
| a server-first dialog, menu and combobox with a small enhancer | ~2,200 app-side lines; 13 catalog components at 0 imports | HTML first, a small controller second |
QueryRef derived from the declaration | Queries and live queries "Not derived" row | — |
a task that runs without a companion job; a boot hook | 35 task/job pairs, 2 boot stand-ins | recurring tasks that name a command; initializers |
| batches with completion callbacks, debounce, priority, job expiry, argument encryption | each present in a surveyed job system; none asked for by the proving case | batches and limiters in established job systems; queue priorities |
| proxy inventory, captcha, OTP relay | product features of a scraping service — the app's, by axiom 8 | — |
The test-kit row of that plan's table shipped in 23.0.0 work (renderView, runJobs({ actor }),
mountIslandState) and is not repeated here.
Closed
Historical: every row is a defect that shipped; the third column is for readers still pinned below the release that fixed it. Upgrading is the fix, and the breaking entries are the cost (Upgrading).
A row is published once the footer's version reaches its release; 30.0.0 rows land with that major.
| Gap | Fixed in | If you are pinned below that |
|---|---|---|
bun run error-render could not see a catch (error) binding | 8.0.0 | The gate matched a parameter annotated unknown/any; a catch (error) binding is unknown by inference and carries no annotation, which is how String(error) echoing a caller's request body into a 422 cause: stayed green in packages/http/src/request.ts through 3.0.0. error-render now joins catchRenderFindings (scripts/catch-render.ts), which reads the catch binding and a .catch((error: unknown) => …) callback; its header lists what it still cannot see (a helper in a third file, a field laundered by another package). Below it: review every catch by hand — a caught value reaches a cause:/fix: only through renderThrowable(error) from @ultimat3/core |
The manifest did not record an admin action's readonly or matching, a scope's count, or a resource's permission noun | 25.0.0 | B29, #691. AdminActionFact records readonly and matching and x manifest diff reports the gate moving between admin:write and admin:read (packages/manifest/src/diff-admin.ts); a scope's count is diffed; the permission noun reaches the diff through each admin route's permissions (<noun>:read). Below it: read them from the declaration in apps/admin/app/admin/admin.ts |
| The two-platform deploy proof (milestone 11) was undemonstrated | 22.1.0 | Not a defect in what shipped: the build targets, compose files and chart all shipped, the proof did not. .github/workflows/ci.yml's deploy-proof job (docker/deploy-proof/run.sh) installs a scaffolded app into kind with x deploy --method helm and upgrades it under mixed GET/POST load, failing on one non-2xx (roadmap). Nothing to work around below it |
A withdrawn isr page was served once more, however old | 27.8.0 | A tag bust marked the stored page stale and the next request was answered that copy — per stored key, per replica — so a withdrawn article reached its next visitor, days later if nobody asked sooner. revalidate: { tags, ttl, onInvalidate: 'purge' } deletes the route's stored pages on a bust and evicts their renders in flight — the next request on that replica, and on each replica the broadcast reaches; one that misses it serves its copy until ttl and nothing stale after. maxStale bounds a copy served past its ttl (Routes). Below it: serve the route as render: 'ssr' with a cache age |
Any query string minted a stored isr page | 30.0.0 | The key carried the whole query, so /blog?x=1, ?x=2, … were each a render and an entry in a store of 1,000, and utm_* split one page into many. A route with no revalidate.query now keys on none of its query, and load, meta and the page receive the narrowed URL. Below it: declare revalidate: { query: ['page'] } (or []) on every isr route, which keys the page on those parameters only; or strip the query at the ingress, or take render: 'ssr' |
An isr page stored a 5xx | 27.7.0 | A load answering withStatus(503) was stored for the whole ttl plus one stale serve, and replaced the good page it was regenerating. A 5xx is now never stored and never replaces a stored page; noStore(data) is the loader's own "do not keep this". Below it: throw from load on a failed read — a throw was never stored |
A page another replica rendered into a shared isrStore was not invalidated | 27.8.0 | The invalidation graph only held the pages this process rendered, so a bust arriving on a replica that had merely served a hit found nothing. A bust now also asks the store which keys it holds under the tags, and a store that implements IsrStore.tagFence refuses a write from a render that began before the bust on another replica. Below it: give every isr route a ttl |
/sitemap.xml answered a 500 when prerender() failed at the hourly refresh | 27.7.0 | The running web role forgot the previous sitemap before it re-enumerated. The hourly refresh now runs behind the last good answer, which every request keeps getting; a failed or hung one (30 s) is asked again a minute later, and the third in a row is logged as an error. A prerender() that catches its failed read and returns [] still shrinks the sitemap — let it throw |
| The edge was purged before the origin dropped the page | 27.7.0 | invalidateTags cleared the cdn tier first, so a request between the CDN purge and the origin's own delete was a public hit the edge cached again for a whole s-maxage. The order is now read tiers → ISR pages → broadcast → cdn last. Below it: follow a withdrawal with a second x cache bust |
A withdrawn isr page stayed readable at another spelling of its URL | 27.8.1 | The store key was the raw pathname and the router matches a normalised one. /blog//a and //blog/a were answered with the page and stored under a key the route table could not read back: no ttl, no tags, s-maxage=60 with a day of stale-while-revalidate and no purge key — a tag bust, 'purge' included, never reached it, at the origin or at the edge, for the life of the process. The key is now the address the router matched (Routes), and a key no route owns is never stored. Below it: normalise the path at the ingress (collapse //, strip the trailing slash) or 404 a path containing //; restart the pods after a withdrawal |
Every spelling of an isr URL was its own stored page | 27.8.1 | /blog/%61bc, /blog/abc/ and /blog/abc were three entries of one page: a 20-letter slug has a million percent-encodings, against a store of 1,000 with the least recently generated evicted first — any visitor could push the real pages out — and the document stored for a spelling carried that spelling as its canonical. A served static page's memo (1,024 documents) filled the same way. One entry per address now. Below it: normalise or reject non-canonical paths at the ingress |
Every isr page under a basePath was tag-only and untagged | 27.8.1 | The key kept the mount (/shop/blog/a) and the route table has never heard of it, so no route matched: the declared revalidate.ttl and tags were ignored (s-maxage=60, no purge key, no tag edge) and a bust reached nothing. The key now carries no base path. Below it: serve the app at / behind a path-rewriting proxy, or take render: 'ssr' with a cache age |
A locale-prefixed isr page never revalidated | 27.7.0 | The ISR key keeps the /en prefix and the route lookup did not strip it, so /en/blog/a matched no route: no revalidate.ttl (s-maxage=60, the tag-only default), no tag edge. A tag bust and the TTL both expired /blog/a and left /en/blog/a — a withdrawn post, or a 404 stored before publication — served for the life of the process. Below it: serve the affected route as render: 'ssr' with a cache age, or restart the pods after a write that should reach a prefixed page |
| The page outbox did not expose how many writes it holds | 25.2.0 | useOutbox().size (@ultimat3/realtime) counts the writes queued and not yet taken by the server, and follows every enqueue, ack, refusal and principal change. Below it: count useMutation's undefined answers yourself and clear on the browser's online event |
| A tag-only ISR page never went stale | 24.0.0 | No boot attached its ISR controller, so invalidateTags reached no revalidator and a revalidate: { tags } page with no ttl served its first render for the life of the process — on every pod, while report.isr listed it as revalidated. x dev, the container's web role and appRoutes() now attach the controller they build (packages/cli/src/runtime-isr.ts) and detach it on stop; isr-tag-bust.e2e.test.ts proves x-ultimate-isr: stale then the new body. Below it, give every isr route a ttl as well as its tags |
| The ISR cache key was the pathname, so a query string was ignored | 4.0.0 | the key is isrKey(url, locale) — pathname, the negotiated locale, then the sorted query — so ?q=bob and ?q=alice are two documents (declare the parameter: revalidate: { query: ['q'] }). Below it, put the varying part in the path (/search/[q]) or take render: 'ssr' |
invoke()'s cache bust was not transaction-aware | 24.0.0 | An action invoked inside withTransaction busted cache.invalidates when its handler returned — before the commit — so a concurrent read refilled the entry from the pre-commit row and the stale value survived to its TTL, and a rollback busted for a write nobody made. The bust is now handed to the root transaction's onCommit (packages/action/src/cache-gate.ts): it fires once the root COMMIT is answered, never for a rollback or X_DB_TRANSACTION_ABORTED. Below it: do the transactional work inside the handler, or drop cache.invalidates from the declaration and call invalidateTags yourself after the transaction returns |
On an s3 disk a grant's maxBytes bounded nothing at the upload | 30.0.0 | grantUpload requires request.size and an s3 disk signs it as the exact Content-Length, so the provider answers any other length, chunked included, with 403 SignatureDoesNotMatch (proven on the test stack's gateway, driver-s3.live.test.ts). signedUrl({ maxBytes }) with no contentLength is refused there (X_STORAGE_UPLOAD_SIZE_UNDECLARED) instead of ignored |
X_ROUTE_MODE_INVALID's fix offered a <Suspense> boundary that does not work | 30.0.0 | It read "wrap the data-dependent part in <Suspense fallback={…}> or change render to 'ssr'", and Solid's <Suspense> throws getContextId cannot be used under non-hydrating context under this renderer. The boundary check is deleted: a stream page needs no boundary, a page with no <Hole> is one flush, and the slow part streams through <Hole fallback> from @ultimat3/render. Below it: take render: 'ssr' |
x g wrote hand-wrapped source and never formatted it | 30.0.0 | x g runs the app's own node_modules/.bin/biome check --write over the TypeScript files that run created, under the app's config, best effort (generate-format-written.ts). A template wrapping wrong for a name no fixture picks reached the app's lint as a red gate over code nobody typed. Pinned below it: bunx biome check --write over the files the run listed |
| A deployed scaffold issued no session, so every policy-protected route answered 401 | 30.0.0 | Nothing x new wrote called login(), and the development viewer answers only under x dev — anonymous everywhere else. x g sign-in --password or x g sign-in --oidc <provider> now writes the sign-in page, the session actions, auth.signInPath and a boot guard (X_AUTH_SIGN_IN_UNCONFIGURED) → Auth. Below it: write the action yourself — policy: allow('public'), const signedIn = await login(auth, { email, password, ip }) from @ultimat3/auth over the app's defineAuth(…), then useRequestContext('createSession').headers.append('set-cookie', signedIn.cookie) — and post a native form at it |
| Nothing pinned the documented generator file counts | 4.0.0 | scripts/generator-counts.ts reads every page that states how many files x new or x g resource writes and answers X_DOC_FILE_COUNT_STALE on the gate's manifest step. Below it, derive rather than quote: x new myapp --dry-run --json | jq '.data.files | length' |
SessionInit.proxy was declared on the scrape seam and read by nothing | 23.0.0 | scrape({ egress: (input) => proxyUrl }) now reaches the driver as SessionInit.proxy and wins over the driver's own proxy: the launched browser, the HTTP leg and the robots read all dial it, an offline session reports it, and a driver that cannot dial it refuses with X_SCRAPE_EGRESS_UNSUPPORTED. On 22.x the exit is a driver option — localBrowser({ proxy }) / remoteBrowser({ proxy }) — one for every session → Scraping |
| No test file was typechecked | 24.0.0 | All 31 package tsconfig.jsons carry "exclude": ["src/**/*.test.ts"] — they still do — so bun run typecheck, a tsc -b, reads none of the 2,195 test files (both counts As of 2026-10-06) under packages/*/src and the gate's typecheck step was green over every one. Re-derive both: grep -l 'src/\*\*/\*.test.ts' packages/*/tsconfig.json | wc -l and find packages -path '*/src/*' -name '*.test.ts*' -not -path '*/dist/*' | wc -l. Closed by a second program, not by editing the 31: tsconfig.tests.json compiles the tests with noEmit, scripts/test-typecheck-gate.ts runs it on a per-package ratchet that may only fall, and it rides the gate's manifest step (scripts/verify.ts:314) rather than typecheck, which takes no host findings. It landed at 446 errors over 161 files in 27 packages (measured 2026-08-19) and TEST_TYPECHECK_PINS in scripts/lib/test-typecheck-pins.ts is now {} As of 2026-10-06: every package's tests compile at zero errors, so a new test compiles the day it is written. On any published release: nothing to work around at runtime, the tests run either way. Typecheck one package's tests yourself with bunx tsc --noEmit over a copy of its config with the exclude dropped |
| A live query re-delivered the raw table row, and mis-ordered a projected window | 5.0.1 | Two defects with one cause: a ChangeEvent carries the whole TABLE row, and a live query's result set is whatever its sql returned. The leak — every patch forwarded the change row unnarrowed, so a column the projection dropped went out on the socket the moment it CHANGED; examples/dummy's feed projects ten columns and one publish delivered updatedAt, and a column like a salary or a private note would have gone the same way. The per-subscriber gate could not help: it decides whether a ROW is delivered, never which of its columns. The mis-ordering — match() decided position by comparing the change row against the rows the WINDOW holds, so an orderBy on a column the projection omits measured a real value against nothing: every update read as a move, and an arriving row landed wherever undefined sorted. Now a patch row is narrowed to the columns the query actually returned, and a position the window cannot answer for is a refill — one re-read and a re-snapshot — rather than a guess. A DELETE still patches incrementally, because it decides no position. #230. On 5.0.0 and below: give a live query's rows the key they are ordered by, and do not rely on a projection to withhold a column from a live subscriber |
jobs.driver selected no driver | 5.0.0 | JobsConfig.driver accepted 'postgres' | 'redis' | 'nats' and had no reader anywhere — boot always built postgresJobDriver, and packages/jobs/src/driver.ts's own header already said so. jobs: { driver: 'redis' } therefore did not boot-and-then-throw as this wiki once claimed: it changed nothing and you silently got Postgres — the same shape as realtime.heartbeatMs, and worse, because it failed silently in the dangerous direction. Five shipped fix: lines named it as the repair for X_NOT_IMPLEMENTED, which is a fix: that is a no-op; those were corrected in 4.1.0 and the field itself is deleted in 5.0.0, along with the JobsDriver type nothing else used. On 4.1.0 and below the field still typechecks and still does nothing — swap the driver with setJobDriver(postgresJobDriver({ executor })), or setJobDriver(memoryJobDriver()) in a test, and never through app.config.ts |
A caller-controlled string could add a line to the 3-line error format | 5.0.0 | bun run error-render refuses a parameter typed unknown/any; a value already typed string renders without throwing, so nothing objected — while a newline in one writes a second line an operator, a CI log or the dev overlay's <pre> reads as a genuine framework message. Three holes shipped in @ultimat3/auth under a green check, the worst reachable by an unauthenticated stranger with one crafted OIDC token. The first fix escaped at each of the six RENDERERS, which could not hold: six is a number that only goes up, and it covered none of the renderers an app writes. Now UltimateError's and SchemaError's constructors escape code, title, cause, fix and docs — so .message, .cause, format(), toJSON() and any renderer anyone writes are one line by construction, and singleLine() is idempotent so a call site that already escaped is unharmed. On 4.1.0 and below, pass error.cause through singleLine() from @ultimat3/core before you render it yourself |
on delete was declared and reached no SQL | 4.0.0 | references(() => orgs.id, { onDelete: 'cascade' }) type-checked and the rule was dropped one layer below the declaration, so no generated add constraint ever spelled one and a drift check had nothing truthful to compare. Now: ColumnDescription/ReferenceDescription carry onDelete, addForeignKey writes the clause, and a rule changed on either side is changed-foreign-key drift whose fix: is the drop/add pair. On 3.0.0, add the clause to the add constraint the generator emitted, before applying the migration: … references "orgs" ("id") on delete cascade;. Editing it after it applies moves the checksum → X_MIGRATION_CONFLICT |
A references() removed from a column emitted nothing | 4.0.0 | the key stayed on the database, the snapshot beside it recorded foreignKeys: [] — actively denying a constraint the catalog held — and compareForeignKeys judges the declared side, so no check could see it. Now foreignKeyPlan emits the drop constraint in up and the add in down, naming the constraint the previous snapshot recorded rather than the name the generator would have picked. On 3.0.0, write the drop into the next generated migration by hand before applying it: alter table "posts" drop constraint "posts_org_id_fkey"; |
| Drift ignored an index's direction and its predicate's presence | 4.0.0 | a desc index rebuilt ascending served a feed's newest page off the wrong end, and a partial index recreated as a total one silently widened the constraint — both read ok: true. Both are compared now; asc is normalised to null first, since Postgres stores an ascending index as not-descending. The predicate text is still uncompared and always will be — see Open by decision. On 3.0.0, x db gen compares all five fields, so regenerate and inspect the diff |
| An unknown flag was refused before a planned command could answer honestly | 4.0.0 | the parser read flags against the spec first, so x logs tail --follow reported X_CLI_BAD_FLAG instead of X_NOT_IMPLEMENTED. A planned command now answers with its own status whatever flags it was given; a shipped command still refuses an unknown flag, which is correct — x env is shipped and declares --json, --help, --cwd and --verbose, so anything else is X_CLI_BAD_FLAG for the ordinary reason that no such flag exists. On 3.0.0, run the flagless form of a planned command to see the real message |
x deploy --method helm threw X_NOT_IMPLEMENTED in a scaffolded app | 4.0.0 | x new wrote no docker/helm, and the command carried a "does this build implement helm?" branch over a build that implemented it completely. x new now writes the chart — 8 files, Chart.yaml + values.yaml + 6 templates — and the lying branch is deleted, so a missing chart is helm's own error. The framework repo's own docker/helm carried two more templates the scaffold did not (pdb.yaml, servicemonitor.yaml); As of 2026-11 only pdb.yaml — the scaffold writes servicemonitor.yaml and alert-rules.yaml, and x g chart-alerts backfills them. On 3.0.0, copy docker/helm from the framework repo, or use --method compose |
.env.development and .env.production shipped in the image | 4.0.0 | the scaffold's .dockerignore excluded .env and .env.*.local — neither pattern matches .env.production, which is the file docker-compose.prod.yml's env_file: tells the operator to create. This page said the leak was "harmless as generated"; that was false, and a real docker build proved both files land in a layer. Now **/.env + **/.env.* + !**/.env.example, in the framework's file, both tracked apps' and the one x new writes. On 3.0.0, add those three lines to docker/Dockerfile.dockerignore yourself — and rebuild, because an image already built still carries them |
realtime.heartbeatMs was read by nothing | 4.0.0, and BREAKING | the key sat in RealtimeConfig with a default of 15 000 and no reader anywhere. It is deleted: RealtimeConfig is { enabled, tier, transport, urlEnv }, the socket beat is the client's new LiveClient({ heartbeatMs }) and the presence beat is derived (PresenceRegistry.heartbeatMs is max(1000, floor(ttlMs / 3))). Refused at boot since 25.0.0: defineConfig throws X_CONFIG_INVALID naming realtime.heartbeatMs and its replacement (REMOVED_CONFIG_KEYS, packages/core/src/config-removed.ts), and any other key it does not declare is refused too (refuseUnknownKeys, packages/core/src/config-keys.ts). From 4.0.0 until 25.0.0 there was no runtime refusal — section() copied the key through and only TS2353 at typecheck caught it. On 3.0.0, delete the key; setting it changes no behaviour either way |
@ultimat3/action's stableStringify folded -0, NaN and Infinity together | 3.0.0 | this page asked for work that was already done. The hash canonicalizer was split out and moved down: canonicalJson and fingerprint live in @ultimat3/core (tier 0), are injective, and give NaN, ±Infinity and -0 bare tokens of their own plus tagged forms for Date, Map and Set. requestHash is fingerprint(input), the job dedupe key is action:<name>:<fingerprint>, and @ultimat3/query's copy was deleted rather than fixed. stableStringify survives as the OpenAPI document serializer alone, where emitting valid JSON is the requirement |
.job() produced a handle nothing could enqueue | 3.0.0 | agentJob() in @ultimat3/ai is the shipped bridge — an agent as durable, resumable, budgeted background work, with the queue accepting the handle. Pinned by packages/ai/src/agent-job.test.ts, whose test is named for this gap: "agentJob() produces a handle the queue accepts, where .job() never could" |
| A branch reaper could drop another app's databases | 4.0.0 | listBranches() walks pg_database for the whole server, so two Ultimate apps on one Postgres plus one nightly sweep was the other app's branches dropped by a DROP DATABASE nobody asked for. The marker is now <base>:<iso> — BranchInfo carries base — and reapBranches skips any branch whose base is not this database. A pre-3.x marker records no base, so it is skipped too, never dropped: self-healing with no migration, because the next createBranch writes the base down. On 3.0.0, run the reaper only against a server this app owns |
x db branch list could not see a branch made before the psql shell-out was removed | 2.0.0 | the old path issued CREATE DATABASE … TEMPLATE through psql and wrote no comment, so a branch made by a 1.2.0 or earlier CLI is absent from list — and delete may only remove what list shows. Nothing back-fills the comment. Drop it by hand from the string the refusal names: psql "$DATABASE_URL" -c 'DROP DATABASE "<source>_branch_<slug>"', where <slug> is the branch name with every character outside [A-Za-z0-9_] replaced by _ |
MCP tool names were published in snake_case and served verbatim | 2.0.0, and BREAKING | openapi.json carried "mcpTool": "publish_post" while the only name tools/call accepts is the export name, publishPost — 15 of the 17 mcpTool values in the two tracked apps' committed specs named no served tool. toToolName is deleted from @ultimat3/action and @ultimat3/query, all three publishers spell the export name, and packages/mcp/src/cross-surface.test.ts drives a tools/call with the name OpenAPI published. On 1.2.0, ignore the published value and call the export name — x actions list --json prints it as name → MCP and AI |
Shared cache tier invalidation DELed keys it never declared in KEYS | 2.0.0 | it failed on Dragonfly and on Redis Cluster — a cluster cannot route a key it was not told about. The script returns the member list and the tier deletes value keys client-side, one key per DEL, so every delete is slot-local. On 1.2.0, single-node Redis, or a cache tier that is not the shared one → Caching and invalidation |
resolveEnvironment existed twice, with different return types | 2.0.0, and BREAKING | @ultimat3/core and @ultimat3/seo both exported it. seo's is deleted along with SeoEnvironment; RobotsConfig.environment and isIndexable() take core's Environment, 'preview' is spelled 'staging', and core gained tryResolveEnvironment() for callers that must answer rather than throw — ULTIMATE_ENV is in no env schema, so a robots.txt render is routinely its first reader. On 1.2.0, import one with an alias → Configuration |
| Two head serializers, one of them weaker | 2.0.0, and BREAKING | @ultimat3/seo's renderHeadTags escaped </ only — not ` |