# Caching and invalidation

Four tiers, one invalidation graph. You declare what a write touches; the framework decides what to evict.

`As of 2026-08`. Stable API — semver from here ([Upgrading](/upgrading)).

## The four tiers

| Tier | Store | Lifetime | Hit cost | Use |
|---|---|---|---|---|
| 1 | **Request memo** | one request (AsyncLocalStorage) | ~0 | the same `query` called by three components resolves once |
| 2 | **In-process LRU** | process lifetime, size-bounded | ~microseconds | hot rows, policy lookups, compiled templates; per-instance, so treat as a probabilistic hit |
| 3 | **Redis** (`redisClient()`, `@ultimat3/core/redis`) | cross-instance, TTL + tag sets | ~1ms | shared query results, rendered fragments, session-adjacent data |
| 4 | **CDN / HTTP headers** | client + edge | `Cache-Control`, `ETag`, `stale-while-revalidate` | static + ISR pages, images, assets |

Read order is **1 → 2 → 3 → origin**. A tier is never consulted for a request whose `policy` has not already passed.

**A cached value is JSON + `Date` + `bigint` + `Map` + `Set`, and every tier answers the same shape.** `As of 2026-10`, tiers 1–3 store one encoding (`@ultimat3/cache`'s `value-codec.ts`) and decode a fresh copy per hit: a `Date` is a `Date` whether the memo, the LRU or Redis answered, and mutating a hit never reaches the next reader. A class instance comes back as a plain object; a cycle is `X_CACHE_VALUE_UNENCODABLE` at the write.

**A cache key is framework-generated, never hand-built.** `As of 2026-11` it is the query name, the read's **authority**, a fingerprint of the parsed input and the read's sorted tag keys — `cacheKeyFor` in `@ultimat3/query`. The authority is `cache: { scope }`: `actor` by default (one key per caller — the narrowest, always correct), `tenant` (one per org; an org-less actor narrows back to itself) or `global`. Widening is a written statement that every member of the org, or everyone, sees the same rows. Policy still runs on every read before a tier is consulted; tier 1 is keyed by `Ctx` identity.

**A null holds for its own lease.** `query({ single: true, cache: { ttlMs, negativeTtlMs } })`: a `single` read that matched no row is stored as `null` and kept `negativeTtlMs`, so a lookup that missed 40 ms before its row landed says "does not exist" for that long and not for the whole `ttlMs`. Only `single: true` answers `null` — an empty list is a list and keeps `ttlMs`. Non-positive or non-finite is `X_QUERY_CACHE_TTL_INVALID` at `query()`, the tiers' own rule. `As of 2026-11`

**A read may answer 304.** `query({ etag: true })` adds tier 4 to a read: `private, no-cache`, a strong `ETag`, and a bodiless 304 for a held validator — after the policy and the rate limit ([Queries](/queries-and-live-queries#fields)).

| Tier | Opt-out / requirement |
|---|---|
| 1 | always on; no configuration |
| 2 | optional per entry — `local: false` for large or per-tenant-unbounded values |
| 3 | **required** for any entry an ISR page depends on, because regeneration happens on a different instance than the write |
| 4 | emitted from the route's render mode; never hand-set on a response |

## Entity tags are the invalidation graph

A tag is a typed handle derived from an [entity](/entities-and-migrations). There is no string-keyed invalidation API.

```ts
export const tag = tags({
  post: entityTag(posts),                       // tag.post, tag.post.id(x)
  feed: derivedTag('feed', [tags.post]),        // invalidating post cascades to feed
});
```

| Tag form | Scope | Example |
|---|---|---|
| `tag.post` | all posts | a schema-level change |
| `tag.post.id(postId)` | one row | the common case; narrowest eviction |
| `tag.feed` | derived; declares `[tag.post]` as an upstream | list views that a post membership affects |

| Declared where | By |
|---|---|
| Queries | **automatic** — acquired from the tables their `sql` touches |
| Routes | `revalidate: { tags: [...] }` |
| Actions / mutators | `cache: { invalidates: [...] }` |
| LLM calls | `cache: { invalidates: [...] }` |

The graph is a build-time artifact in `x.manifest.json`, so it records exactly what a write will evict — before you run it. `x cache graph --json` is the planned reader of it; today it is `x dev` → the `/_x` cache panel. Tag typing comes from a generated registry augmentation; `x manifest` regenerates it.

## One hop, all tiers

```ts
// action
export const publishPost = action({
  input:  t.object({ postId: t.uuid, notify: t.boolean.default(true) }),
  output: PostView,
  policy: can('post:publish', ({ input, actor }) => ownsPost(actor, input.postId)),
  cache:  { invalidates: [tag.post, tag.feed] },
  mcp:    { expose: true, description: 'Publish a draft post' },
  async handle({ input, ctx }) {
    const post = await ctx.posts.publish(input.postId);
    if (input.notify) await notifySubscribers.enqueue({ postId: post.id, orgId: post.orgId });
    return post;
  },
});
```

`invalidates: [tag.post, tag.feed]` fans out on commit:

| Target | Mechanism | Timing |
|---|---|---|
| Tier 1 request memo | drop entries carrying the tag | immediate, same request |
| Tier 2 in-process LRU (**all instances**) | tag-invalidation message on NATS | ~ms, best-effort; a missed message costs a stale read until TTL, never a wrong write |
| Tier 3 Redis | paged `ZSCAN` over the tag's sorted set, `UNLINK` per value key, `ZREM` of what died | immediate, inside the fan-out; 10k keys in 25 round trips |
| ISR pages | routes whose `revalidate.tags` include the tag are marked stale → regenerated in background | next request serves stale, regen enqueued as a job |
| CDN | purge by surrogate key through the configured `PurgeDriver` — a row bust purges `post:1` and `post`, a collection bust `e:post` and `post` | seconds; `stale-while-revalidate` covers the gap |
| Live queries | the same commit already flows through logical replication | **independent path** — realtime does not depend on cache invalidation |

Fan-out runs **after the handler resolves, in the same call** — `bustAfterCommit` awaits `invalidateTags()` directly (`packages/action/src/cache-gate.ts`), never through the outbox `As of 2026-08`. A handler that throws never reaches it, so a rolled-back write never purges; a process that dies between the commit and the fan-out leaves those entries until their TTL. An action invoked **inside `withTransaction`** — an app's own unit of work, or another action's — hands the fan-out to the root transaction's `onCommit` instead `As of 2026-10-02`: it runs once the root `COMMIT` is answered and never for a rollback, so a concurrent read cannot refill an entry from rows that are not yet durable.

> **Tier 3 invalidation is slot-local as of 2.0.0, and was not through 1.2.0.** The Lua script `DEL`ed keys it never declared in `KEYS`, which single-node Redis tolerates and **Dragonfly and Redis Cluster reject** — a cluster cannot route a key it was not told about. The script now returns the member list and the tier deletes the value keys client-side, one key per `DEL`, so every delete is slot-local. On 1.2.0 and earlier: use single-node Redis, or drop the shared tier → [Known gaps](/known-gaps).
>
> **As of 31.0.0 a bust runs no script at all**, and every command names one key. Tag buckets are sorted sets (`t2:`/`e2:`) scored by the member's expiry plus 60 s; every write prunes members past their score, so a bucket holds live keys only. Before, only a bust removed members: an entity written constantly and never invalidated collected every key it ever wrote, and its first bust read millions in one reply. A value over `redisTier({ maxValueBytes })` (1 MiB) is refused with `X_CACHE_TOO_LARGE`.

**Across replicas without NATS.** `As of 2026-11-09`. The tier-2 message rides whichever bus reaches the other processes, decided once at boot ([`runtime-cache-bus.ts`](https://github.com/developerz-ai/ultimate/blob/main/packages/cli/src/runtime-cache-bus.ts)):

| Configured | Invalidations ride | `cache.broadcast` log line |
|---|---|---|
| `NATS_URL`, or a `transport` in `runtime.ts` | that transport | `bus: 'nats'` (the transport's name) |
| no NATS, `REDIS_URL` (else `VALKEY_URL`) | Redis pub/sub (`redisPubSubAt`) | `bus: 'redis', source: 'REDIS_URL'` |
| neither | the in-process bus: this replica only | `bus: 'in-process'` |

Never both: a second hop beside NATS would deliver every bust twice through two systems that fail independently. A replica whose Redis subscription dropped flushes its own tiers on reconnect rather than serve what it may have missed. This is what makes rung 1 of the [scale ladder](https://github.com/developerz-ai/ultimate/blob/main/docs/idea/17-scale-ladder.md) — roles split, Redis, no NATS — safe with two web replicas.

There is exactly one fan-out entry point in the implementation (`invalidateTags()`); no caller reaches a tier directly. Tier failures are collected into an invalidation report — **a cache tier may never fail a business write.**

The write side holds the same rule one layer up: a fan-out that refuses outright — a tag no entity declared, `X_CACHE_TAG_UNKNOWN` — is absorbed rather than raised: one `action.invalidate.failed` error line, entries live until their TTL, and the caller keeps the write it already made. A replayed idempotent call (`idempotent: true` + a repeated `Idempotency-Key`) busts nothing at all: no handler ran, and the first call already did.

**N concurrent misses on one key are ONE `load()`.** The stack holds a single-flight map, per stack
rather than per module — two stacks are two ladders and must not join each other's loads. A `load()`
may hold its key for `loadDeadlineMs`, default **30 s**, after which a later reader is allowed to
start its own instead of joining a promise that may never resolve. The deadline frees the **key**
and nothing else: `load()` is your function and the stack holds no signal that could abort it, so
the wedged load runs on and the readers already holding its promise still get whatever it answers.
Worst case is one duplicate fill, which the ladder's last-write-wins `set` already tolerates —
against a key pinned for the life of the process. 30 s because `@ultimat3/http` abandons the request
waiting on that read at the same bound; a load still running past it has no reader left to serve.
`As of 2026-08-23`. Set it per stack: `cacheStack(tiers, { loadDeadlineMs })`.

**Who shares that load, and who does not** `As of 2026-11`. A read joins it unless the load's fence was
moved: after a `drop`, a `write` or a tag bust covering the key or the reader's own tags, the next
read leads a fresh load, so a read-your-write never answers from data loaded before the write. A
reader that joined before the bust keeps the load it joined. And a load whose leader was aborted
(`ctx.signal`, a closed tab) does not fail the readers who joined it: the first one with a live
signal loads again under its own context and the rest join that. A load that fails with its leader
still live fails its joiners too.

**Redis down is a slow miss, never a hang** `As of 2026-11`. A paused server accepts a command and never answers it; unbounded, that hung a read 45 s and a bust 20 s on an action's commit path.

| Redis is | A read | A bust (`invalidateTags`) |
|---|---|---|
| up | answers in milliseconds | answers |
| paused | one command misses `redisTier({ commandTimeoutMs })` (default 150 ms) → a miss, the source answers; the value path parks 5 s, every read there an immediate miss with nothing sent | its own 1,000 ms deadline, never parked — a skipped bust is stale rows until TTL; past it, `report.errors` |
| stopped | refused at once → a miss | refused at once → `report.errors`; the same tags again once it answers |
| restarted | every pod reconnects on its own, no restart | answers |

A missed deadline or a park is a `cache.tier_failure` event with `X_CACHE_DRIVER_UNAVAILABLE`; a stopped server's refusal is recorded with Bun's `ERR_REDIS_CONNECTION_CLOSED` message and no `code`. While the store is gone each pod still collapses its own concurrent misses to one `load()`; across pods, Postgres takes one load per pod per key — the rung's known cost. The jobs queue on the same client refuses an enqueue with `X_DRIVER_UNAVAILABLE` instead of waiting ([Jobs](/jobs-and-workflows)).

The read ladder keeps the same rule with no report to hand back: a tier that throws on `get`, `set` or `del` is a tier that did not answer, so the walk continues and the source is still returned. A value too large for the in-process LRU or the Redis tier (`X_CACHE_TOO_LARGE`) costs the entry, never the read. Every absorbed refusal is one `cache.tier_failure` event on core's instrumentation bus — `subscribe('cache.tier_failure', …)`, carrying the tier, the operation, the key and the `X_*` code — plus one `cache.tier.failed` warn; a subscriber that throws is contained by the bus and logged once, never the read's failure. Every rung a read asks is a `cache.hit` or `cache.miss` event, and every bust a `cache.invalidate` event. `As of 2026-11` (v31). The one call left to throw is the load itself: it *is* the business read.

## Failure modes removed

The bug is never "the cache is wrong". The bug is that invalidation is a *decision made at a distance*: the developer editing `publishPost` must remember which of nine cached things this write affects, including two added last month by someone else.

| Failure mode | Cause | Removed by |
|---|---|---|
| Stale page after publish | forgot one `revalidate` call | tags are declared on the route, resolved from the graph |
| Purging too much | uncertainty → `flushAll` | narrow `tag.post.id(x)` is the ergonomic default |
| Tier drift | Redis purged, CDN not | one fanout, all tiers |
| Leak across tenants | hand-built cache key missing the tenant | keys are framework-generated from the query name, the read's authority (`cache: { scope }`, default `actor`), its parsed input and its tags |
| Stale forever | a query whose tables no tag covers | the tag rule — **not yet a gate**: `X_CACHE_UNTAGGED_QUERY` is reserved `As of 2026-08` and nothing raises it, so a cached query no tag covers is cached and never invalidated ([Error codes → Reserved codes](/error-codes#reserved-codes)) |
| Silent typo | `invalidates: [tag.pots]` | a compile error against the generated registry; at runtime `X_CACHE_TAG_UNKNOWN`, which a write logs as `action.invalidate.failed` rather than failing the commit it followed |

Agents are measurably bad at *distant* invariants — "edit here, remember to also edit there" is where LLM-written code regresses most. Declaring `invalidates` at the write site is local, checkable, and typed.

## The CDN leg

The CDN is the one tier Ultimate never reads back from, so the emitted header and the purge call
are the whole contract. An edge purges by exact key, so the two-way rule every other tier keeps —
a collection bust reaches its rows, a row bust reaches the lists that held it — is built from two
key lists, `As of 2026-10`:

| | Row tag `post:1` | Collection tag `post` |
|---|---|---|
| a response carries (`cacheHeaders()`, `surrogateKeys()`) | `post:1` `e:post` | `post` `e:post` |
| a bust purges | `post:1` `post` | `e:post` `post` |

`e:<entity>` is the entity index: `len(entity) + 3` bytes per distinct entity in each header (7
for `post`, 14 across both). A
key carrying whitespace or a comma is refused at emission (`X_CACHE_PURGE_FAILED`), not on the
purge — a CDN splits it into keys nothing ever names.

Which responses carry keys, `As of 2026-10-02`:

| Response | `Surrogate-Key` / `Cache-Tag` |
|---|---|
| an `isr` document whose route declares `revalidate.tags`, answered as a shared (`public`) response | yes — those tags through `surrogateKeys()`, on a miss, a hit and a served-stale answer |
| an `isr` document with `revalidate.ttl` only | no — nothing purges it; `s-maxage` is its whole lease |
| the same `isr` document answered to a **signed-in** visitor | no — the `cache-headers` stage rewrites it to `private` and strips both headers |
| `static`, `ssr`, `stream` documents; assets; action and query responses | no — none is a tagged shared response (`ssr`'s `cache` takes no `tags`; actions and queries are `no-store`) |
| a response your own handler tags (`ctx.cache = { mode: 'public', tags }`, or `cacheHeaders()`) | whatever you pass — hand `CacheHint.tags` the output of `surrogateKeys(tags)`, which `@ultimat3/http` emits verbatim |

Before this date **no shipped response carried a key at all**: the purge drivers were called with
keys nothing at the edge had ever been tagged with, and answered success. An unpurgeable tag in
`revalidate.tags` is refused when the route registers (`X_ROUTE_MODE_INVALID`, naming the file).

| Driver | Purge | Purge all | Per call |
|---|---|---|---|
| `noopPurgeDriver()` | echoes the keys back | resolves | — |
| `fastlyPurgeDriver({ apiToken, serviceId })` | `POST /service/<id>/purge` with `surrogate_keys` | `POST /service/<id>/purge_all` | 256 keys |
| `cloudflarePurgeDriver({ apiToken, zoneId })` | `POST /zones/<id>/purge_cache` with `tags` | same call with `purge_everything` | 30 tags |

Which one a process installs is decided from the environment — `FASTLY_API_TOKEN` +
`FASTLY_SERVICE_ID`, or `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ZONE_ID`. See
[Configuration → CDN purge](/configuration#cdn-purge). With neither pair set, nothing is purged and
no `cdn` line appears in the invalidation report: a tier that reported keys an edge that does not
exist had accepted would be worse than no tier at all.

A refusal is `X_CACHE_PURGE_FAILED` with `meta.retryable`, collected into `report.errors` — a dead
CDN never fails the write that triggered the bust, and the entry expires by TTL instead.

## Semantic cache for LLM calls

Model calls are slow and metered; exact-match caching almost never hits because prompts differ by a word.

```ts
export const summarize = llm({
  model: 'claude-sonnet-5-5',
  cache: {
    semantic: { threshold: 0.97, ttl: '7d' }, // partitions on the ACTOR by default
    invalidates: [tag.post],
  },
  prompt: summarizePrompt,      // versioned artifact, see 09-ai-first.md
});
```

| Aspect | Rule |
|---|---|
| Store | pgvector table, one row per (prompt version, embedded input, scope) |
| Key | embedding of the rendered prompt + model id + **prompt version** + **request locale** |
| Hit | cosine similarity >= `threshold`; default 0.97, never below 0.9 |
| Scope | optional — the calling **actor** by default, so one org never reads another's completion. Widening is written down: `scope: () => 'global'` |
| Bypass | `temperature > 0` results are cached but flagged; `cache: false` for anything user-visible-and-unique |
| Invalidation | participates in the same tag graph; bumping the prompt version invalidates wholesale |
| Metrics | hit rate, tokens saved, cost saved — in `/_x` and `x ai cache --json` |

**The locale is part of the key, and not part of the scope** — `As of 2026-08-24`. A prompt that takes `locale` as a var differs by one token between languages while carrying a whole document, so the two rendered prompts are near-identical to an embedder: measured over the reference app's summarize template, **0.9986**, against the `threshold: 0.97` in the snippet above. Before this, whichever language asked first was served to everyone — an English reader got the Spanish summary, and the model had done nothing wrong.

No threshold fixes that, because the same number has to keep an honest repeat *above* it. So the locale sits in the unconditional half of the store key, beside the prompt version: a `scope` answers *who may share this answer*, and a locale is part of what the answer **is**. A written-down `scope: () => 'global'` is still partitioned by locale — declaring a shared store says your callers may read one another's summaries, never that a Spanish one will do for an English reader. Pages key by locale for the same reason ([ISR](/routes-and-render-modes)).

Also cached exactly (tier 3, not semantic): embeddings themselves, keyed by content hash + model. Re-embedding unchanged text is pure waste. See [MCP and AI](/mcp-and-ai).

## CLI — `x cache` is **planned**

Every row below exits `X_NOT_IMPLEMENTED` `As of 2026-08`, with `x dev` → the `/_x` cache panel as its `fix:`. A planned command declares no flags either, so `x cache graph --tag post` dies at the parser with `X_CLI_BAD_FLAG` before the honest message — call the bare form.

| Command | Will do |
|---|---|
| `x cache graph --json` | print the tag → dependents graph: cache keys, ISR routes, CDN paths, live queries. Build-time truth, no runtime call |
| `x cache graph --tag post --json` | the blast radius of one tag |
| `x cache bust <tag>` | run the real fanout for one tag, all tiers, and print the invalidation report |
| `x cache clear` | **dev only.** The only `flushAll` there would be; there is no runtime API for it |
| `x cache stats --json` | per-tier hit rate, byte usage against budget, evictions |

Today: `x dev` then the `/_x` cache panel, and `invalidateTags([...])` from `@ultimat3/cache` for a bust from code.

## Errors

| Code | Cause | Fix |
|---|---|---|
| `X_CACHE_UNTAGGED_QUERY` | **reserved, nothing raises it** `As of 2026-08` — a query's tables are covered by no tag, so it could never be invalidated ([Error codes → Reserved codes](/error-codes#reserved-codes)) | declare the entity tag, then `x manifest` |
| `X_CACHE_TAG_UNKNOWN` | `tag "<name>" is not declared by any entity` | `x manifest` |
| `X_CACHE_TOO_LARGE` | `entry "<key>" is <n>B, over the <tier> budget of <m>B` | `lruTier({ maxBytes: <n> })` / `redisTier({ maxValueBytes: <n> })` where the tier is built, or cache a projection instead of the row |
| `X_CACHE_DRIVER_UNAVAILABLE` | `cache tier "<driver>" is unavailable` — a Redis command missed its deadline, or the tier is parked after one; or a purge driver built without its token | the error carries the exact config or command to fix |
| `X_CACHE_PURGE_FAILED` | `<driver> refused the purge (HTTP <status>)` — a wrong token, a zone without tag purge, a throttle, or a key a CDN would split | `meta.retryable === true` → the identical purge can land again; otherwise set the env key the `fix` names |

Verbatim shapes: [`packages/cache/src/errors.ts`](https://github.com/developerz-ai/ultimate/blob/main/packages/cache/src/errors.ts). Full index: [Error codes](/error-codes).

## Rules

- Cache keys are framework-generated. A hand-built key is a rejected PR.
- Every cached `query` carries at least one tag. Review catches it `As of 2026-08`, not the gate — `X_CACHE_UNTAGGED_QUERY` is reserved and no `x verify` step reads a query's tags.
- Never cache a value whose policy scope is not in its key.
- `flushAll` has no shipped caller: `x cache clear` is planned, and there is no runtime API for it.
- Cache misses must be correct and merely slower — no code path may depend on a hit.
- Prefer `tag.post.id(x)` over `tag.post`. Narrow eviction is the default, not an optimization.
- A cache tier may never fail a business read or write. Tier errors land in the invalidation report, or in a `cache.tier_failure` event on the read ladder.
- Realtime is not a cache tier. Live queries flow from logical replication on their own path ([Realtime](/realtime)).