Caching and invalidation

Four tiers, one invalidation graph. You declare what a write touches; the framework decides what to evict.

As of 2026-08. Stable API — semver from here (Upgrading).

The four tiers

Read order is 1 → 2 → 3 → origin. A tier is never consulted for a request whose policy has not already passed.

A cached value is JSON + Date + bigint + Map + Set, and every tier answers the same shape. As of 2026-10, tiers 1–3 store one encoding (@ultimat3/cache's value-codec.ts) and decode a fresh copy per hit: a Date is a Date whether the memo, the LRU or Redis answered, and mutating a hit never reaches the next reader. A class instance comes back as a plain object; a cycle is X_CACHE_VALUE_UNENCODABLE at the write.

A cache key is framework-generated, never hand-built. As of 2026-11 it is the query name, the read's authority, a fingerprint of the parsed input and the read's sorted tag keys — cacheKeyFor in @ultimat3/query. The authority is cache: { scope }: actor by default (one key per caller — the narrowest, always correct), tenant (one per org; an org-less actor narrows back to itself) or global. Widening is a written statement that every member of the org, or everyone, sees the same rows. Policy still runs on every read before a tier is consulted; tier 1 is keyed by Ctx identity.

A null holds for its own lease. query({ single: true, cache: { ttlMs, negativeTtlMs } }): a single read that matched no row is stored as null and kept negativeTtlMs, so a lookup that missed 40 ms before its row landed says "does not exist" for that long and not for the whole ttlMs. Only single: true answers null — an empty list is a list and keeps ttlMs. Non-positive or non-finite is X_QUERY_CACHE_TTL_INVALID at query(), the tiers' own rule. As of 2026-11

A read may answer 304. query({ etag: true }) adds tier 4 to a read: private, no-cache, a strong ETag, and a bodiless 304 for a held validator — after the policy and the rate limit (Queries).

Entity tags are the invalidation graph

A tag is a typed handle derived from an entity. There is no string-keyed invalidation API.

export const tag = tags({
  post: entityTag(posts),                       // tag.post, tag.post.id(x)
  feed: derivedTag('feed', [tags.post]),        // invalidating post cascades to feed
});

The graph is a build-time artifact in x.manifest.json, so it records exactly what a write will evict — before you run it. x cache graph --json is the planned reader of it; today it is x dev → the /_x cache panel. Tag typing comes from a generated registry augmentation; x manifest regenerates it.

One hop, all tiers

// action
export const publishPost = action({
  input:  t.object({ postId: t.uuid, notify: t.boolean.default(true) }),
  output: PostView,
  policy: can('post:publish', ({ input, actor }) => ownsPost(actor, input.postId)),
  cache:  { invalidates: [tag.post, tag.feed] },
  mcp:    { expose: true, description: 'Publish a draft post' },
  async handle({ input, ctx }) {
    const post = await ctx.posts.publish(input.postId);
    if (input.notify) await notifySubscribers.enqueue({ postId: post.id, orgId: post.orgId });
    return post;
  },
});

invalidates: [tag.post, tag.feed] fans out on commit:

Fan-out runs after the handler resolves, in the same call — bustAfterCommit awaits invalidateTags() directly (packages/action/src/cache-gate.ts), never through the outbox As of 2026-08. A handler that throws never reaches it, so a rolled-back write never purges; a process that dies between the commit and the fan-out leaves those entries until their TTL. An action invoked inside withTransaction — an app's own unit of work, or another action's — hands the fan-out to the root transaction's onCommit instead As of 2026-10-02: it runs once the root COMMIT is answered and never for a rollback, so a concurrent read cannot refill an entry from rows that are not yet durable.

Tier 3 invalidation is slot-local as of 2.0.0, and was not through 1.2.0. The Lua script DELed keys it never declared in KEYS, which single-node Redis tolerates and Dragonfly and Redis Cluster reject — a cluster cannot route a key it was not told about. The script now returns the member list and the tier deletes the value keys client-side, one key per DEL, so every delete is slot-local. On 1.2.0 and earlier: use single-node Redis, or drop the shared tier → Known gaps.

As of 31.0.0 a bust runs no script at all, and every command names one key. Tag buckets are sorted sets (t2:/e2:) scored by the member's expiry plus 60 s; every write prunes members past their score, so a bucket holds live keys only. Before, only a bust removed members: an entity written constantly and never invalidated collected every key it ever wrote, and its first bust read millions in one reply. A value over redisTier({ maxValueBytes }) (1 MiB) is refused with X_CACHE_TOO_LARGE.

Across replicas without NATS. As of 2026-11-09. The tier-2 message rides whichever bus reaches the other processes, decided once at boot (runtime-cache-bus.ts):

Never both: a second hop beside NATS would deliver every bust twice through two systems that fail independently. A replica whose Redis subscription dropped flushes its own tiers on reconnect rather than serve what it may have missed. This is what makes rung 1 of the scale ladder — roles split, Redis, no NATS — safe with two web replicas.

There is exactly one fan-out entry point in the implementation (invalidateTags()); no caller reaches a tier directly. Tier failures are collected into an invalidation report — a cache tier may never fail a business write.

The write side holds the same rule one layer up: a fan-out that refuses outright — a tag no entity declared, X_CACHE_TAG_UNKNOWN — is absorbed rather than raised: one action.invalidate.failed error line, entries live until their TTL, and the caller keeps the write it already made. A replayed idempotent call (idempotent: true + a repeated Idempotency-Key) busts nothing at all: no handler ran, and the first call already did.

N concurrent misses on one key are ONE load(). The stack holds a single-flight map, per stack rather than per module — two stacks are two ladders and must not join each other's loads. A load() may hold its key for loadDeadlineMs, default 30 s, after which a later reader is allowed to start its own instead of joining a promise that may never resolve. The deadline frees the key and nothing else: load() is your function and the stack holds no signal that could abort it, so the wedged load runs on and the readers already holding its promise still get whatever it answers. Worst case is one duplicate fill, which the ladder's last-write-wins set already tolerates — against a key pinned for the life of the process. 30 s because @ultimat3/http abandons the request waiting on that read at the same bound; a load still running past it has no reader left to serve. As of 2026-08-23. Set it per stack: cacheStack(tiers, { loadDeadlineMs }).

Who shares that load, and who does not As of 2026-11. A read joins it unless the load's fence was moved: after a drop, a write or a tag bust covering the key or the reader's own tags, the next read leads a fresh load, so a read-your-write never answers from data loaded before the write. A reader that joined before the bust keeps the load it joined. And a load whose leader was aborted (ctx.signal, a closed tab) does not fail the readers who joined it: the first one with a live signal loads again under its own context and the rest join that. A load that fails with its leader still live fails its joiners too.

Redis down is a slow miss, never a hang As of 2026-11. A paused server accepts a command and never answers it; unbounded, that hung a read 45 s and a bust 20 s on an action's commit path.

A missed deadline or a park is a cache.tier_failure event with X_CACHE_DRIVER_UNAVAILABLE; a stopped server's refusal is recorded with Bun's ERR_REDIS_CONNECTION_CLOSED message and no code. While the store is gone each pod still collapses its own concurrent misses to one load(); across pods, Postgres takes one load per pod per key — the rung's known cost. The jobs queue on the same client refuses an enqueue with X_DRIVER_UNAVAILABLE instead of waiting (Jobs).

The read ladder keeps the same rule with no report to hand back: a tier that throws on get, set or del is a tier that did not answer, so the walk continues and the source is still returned. A value too large for the in-process LRU or the Redis tier (X_CACHE_TOO_LARGE) costs the entry, never the read. Every absorbed refusal is one cache.tier_failure event on core's instrumentation bus — subscribe('cache.tier_failure', …), carrying the tier, the operation, the key and the X_* code — plus one cache.tier.failed warn; a subscriber that throws is contained by the bus and logged once, never the read's failure. Every rung a read asks is a cache.hit or cache.miss event, and every bust a cache.invalidate event. As of 2026-11 (v31). The one call left to throw is the load itself: it is the business read.

Failure modes removed

The bug is never "the cache is wrong". The bug is that invalidation is a decision made at a distance: the developer editing publishPost must remember which of nine cached things this write affects, including two added last month by someone else.

Agents are measurably bad at distant invariants — "edit here, remember to also edit there" is where LLM-written code regresses most. Declaring invalidates at the write site is local, checkable, and typed.

The CDN leg

The CDN is the one tier Ultimate never reads back from, so the emitted header and the purge call are the whole contract. An edge purges by exact key, so the two-way rule every other tier keeps — a collection bust reaches its rows, a row bust reaches the lists that held it — is built from two key lists, As of 2026-10:

e:<entity> is the entity index: len(entity) + 3 bytes per distinct entity in each header (7 for post, 14 across both). A key carrying whitespace or a comma is refused at emission (X_CACHE_PURGE_FAILED), not on the purge — a CDN splits it into keys nothing ever names.

Which responses carry keys, As of 2026-10-02:

Before this date no shipped response carried a key at all: the purge drivers were called with keys nothing at the edge had ever been tagged with, and answered success. An unpurgeable tag in revalidate.tags is refused when the route registers (X_ROUTE_MODE_INVALID, naming the file).

Which one a process installs is decided from the environment — FASTLY_API_TOKEN + FASTLY_SERVICE_ID, or CLOUDFLARE_API_TOKEN + CLOUDFLARE_ZONE_ID. See Configuration → CDN purge. With neither pair set, nothing is purged and no cdn line appears in the invalidation report: a tier that reported keys an edge that does not exist had accepted would be worse than no tier at all.

A refusal is X_CACHE_PURGE_FAILED with meta.retryable, collected into report.errors — a dead CDN never fails the write that triggered the bust, and the entry expires by TTL instead.

Semantic cache for LLM calls

Model calls are slow and metered; exact-match caching almost never hits because prompts differ by a word.

export const summarize = llm({
  model: 'claude-sonnet-5-5',
  cache: {
    semantic: { threshold: 0.97, ttl: '7d' }, // partitions on the ACTOR by default
    invalidates: [tag.post],
  },
  prompt: summarizePrompt,      // versioned artifact, see 09-ai-first.md
});

The locale is part of the key, and not part of the scope — As of 2026-08-24. A prompt that takes locale as a var differs by one token between languages while carrying a whole document, so the two rendered prompts are near-identical to an embedder: measured over the reference app's summarize template, 0.9986, against the threshold: 0.97 in the snippet above. Before this, whichever language asked first was served to everyone — an English reader got the Spanish summary, and the model had done nothing wrong.

No threshold fixes that, because the same number has to keep an honest repeat above it. So the locale sits in the unconditional half of the store key, beside the prompt version: a scope answers who may share this answer, and a locale is part of what the answer is. A written-down scope: () => 'global' is still partitioned by locale — declaring a shared store says your callers may read one another's summaries, never that a Spanish one will do for an English reader. Pages key by locale for the same reason (ISR).

Also cached exactly (tier 3, not semantic): embeddings themselves, keyed by content hash + model. Re-embedding unchanged text is pure waste. See MCP and AI.

CLI — x cache is planned

Every row below exits X_NOT_IMPLEMENTED As of 2026-08, with x dev → the /_x cache panel as its fix:. A planned command declares no flags either, so x cache graph --tag post dies at the parser with X_CLI_BAD_FLAG before the honest message — call the bare form.

Today: x dev then the /_x cache panel, and invalidateTags([...]) from @ultimat3/cache for a bust from code.

Errors

Verbatim shapes: packages/cache/src/errors.ts. Full index: Error codes.

Rules

• Cache keys are framework-generated. A hand-built key is a rejected PR.
• Every cached query carries at least one tag. Review catches it As of 2026-08, not the gate — X_CACHE_UNTAGGED_QUERY is reserved and no x verify step reads a query's tags.
• Never cache a value whose policy scope is not in its key.
• flushAll has no shipped caller: x cache clear is planned, and there is no runtime API for it.
• Cache misses must be correct and merely slower — no code path may depend on a hit.
• Prefer tag.post.id(x) over tag.post. Narrow eviction is the default, not an optimization.
• A cache tier may never fail a business read or write. Tier errors land in the invalidation report, or in a cache.tier_failure event on the read ladder.
• Realtime is not a cache tier. Live queries flow from logical replication on their own path (Realtime).