Storage and uploads

Call sites name a disk, never a driver. Package @ultimat3/storage (tier 1) — the full reference is packages/storage/README.md, and the upload design is docs/architecture/17-uploads.md.

import { defineStorage, disk, localDriver, s3Driver, scopedKey } from '@ultimat3/storage';

defineStorage({
  disks: {
    uploads: localDriver({ root: '.storage/uploads' }),
    media: s3Driver({ bucket: 'media', endpoint: process.env.S3_ENDPOINT, forcePathStyle: true }),
  },
  default: 'uploads',
});

await disk('media').put(scopedKey(orgId, 'avatars', 'a.png'), bytes, { contentType: 'image/png' });

Swapping local for s3 changes no call site, and x dev needs no S3 server.

A driver of your own is the storage adapter port: StorageDriver from @ultimat3/storage/adapter, memoryStorageDriver to copy, behavesLike(storageDriverConformance, () => yourDriver()) to prove it — Adapters. It is used in defineStorage({ disks }), or as storage: in apps/<app>/runtime.ts for the boot's one default disk.

A declaration holds a disk by THUNK, never by value. disk('sessions') resolves through defineStorage(), which boot runs after an app's modules were evaluated — so a scrape() or any other declaration built at import time takes () => disk('sessions') and reads it when it is used (Scraping). Called at module scope, disk() is X_CONFIG_INVALID: storage was not defined yet.

A real S3 on a laptop is docker/docker-compose.dev.yml's s3 service — Versity S3 Gateway over a volume, with the bucket already there (a directory under its root is a bucket, and the volume is mounted at one). Point the app at it with S3_ENDPOINT=http://127.0.0.1:9000, S3_BUCKET=<app>, S3_FORCE_PATH_STYLE=1 and the key pair the file ships. That file chooses a server for development only — production is whichever S3-compatible endpoint you run.

S3_FORCE_PATH_STYLE is 1 (on) or 0 (off); unset or blank is off. Anything else refuses the boot with X_CONFIG_INVALID, because true used to read as off and send every request to <bucket>.<host>.

S3_REGION is the region requests are signed for. Unset, they are signed for auto: what R2 wants and what the dev gateway is started with. An endpoint that expects another region refuses the first write, list or delete with X_CONFIG_INVALID, and the fix line names the value to set.

A listing is the framework's one page

disk(name).list({ prefix, cursor, limit }) answers Page<StorageListEntry> — { rows, nextCursor, hasMore }, the shape findMany and a query's .page() answer. nextCursor is a string exactly when another object exists past the page, on every driver: a full last page is hasMore: false, never a cursor to an empty one. cursor: null (or absent) is the first page, so cursor = page.nextCursor until it is null walks the prefix. limit is a positive integer or X_INVARIANT. Until 25.0.0 it was { objects, truncated, cursor? } (Upgrading).

A ranged read — get(key, { range: [first, last] })

As of 2026-10-31. Both offsets INCLUSIVE, as HTTP's Range: is. s3 sends ONE signed GET with Range: and reads the window, the size and the etag off that same answer, the local disk slices the file, memory slices its copy. A last past the end is clamped (read.range says what was served; read.object.size is the whole object's), a first at or past the end is X_STORAGE_RANGE_UNSATISFIABLE (416), and maxGetBytes bounds the RANGE — an object refused whole is readable a page at a time.

A sealed object is not range-readable. seal() writes one AES-256-GCM envelope whose tag covers every byte, and there is no chunked form of it: a slice neither opens nor means anything. Page a sealed log by sealing each page as its own object; range-read plain objects.

A ranged read is what importRows() reads a large upload with: a StorageDriver IS an ImportSource (stat + get(key, { range })), one page-sized window per step, never the object whole (Jobs and workflows).

A large object — streamRange and putStream

import { disk } from '@ultimat3/storage';

declare const key: string, first: number, last: number, contentType: string;
declare const request: Request & { body: ReadableStream<Uint8Array> };

export const window = await disk('media').streamRange(key, [first, last]);
export const object = await disk('media').putStream(key, request.body, {
  maxBytes: 2e9,
  contentType,
});

Both are required StorageDriver methods, so a driver lacking one is a compile error. A third-party driver reuses assertPutStream, pumpBounded, streamLimit, assertStreamRange and streamedRange.

Keys are refused, never sanitised

assertSafeKey() runs on every key: .., absolute keys, backslashes, control bytes, encoded separators, empty segments and over-long keys are X_STORAGE_PATH_UNSAFE. scopedKey(org, …) is org/<org>/…; a surface guards a client key with isWithinOrg(key, ctx.actor.orgId) and isTenantScoped(key) (X_STORAGE_ORG_MISMATCH).

The direct-upload round trip

User uploads never go through put() (which buffers, and stops at maxPutBytes, 10 MB by default). They go straight to the disk, in three calls — and the client never names the key:

The mounted PUT (As of 2026-09-24, #523) requires a signed-in actor, like the GET. It needs no permission: the grant is the authorization, and the action that minted it ran its own policy. The route checks the signature and the expiry, and it checks that the key is inside the actor's org. It validates the upload against what the grant signed (the content type and maxBytes), so a PDF granted under uploadPolicy({ allowedContentTypes: ['application/pdf'] }) is accepted, and it does not use uploadPolicy()'s image-only default. The signed maxBytes caps how much of the body is read. bodyLimitBytes does not, and neither does an unverified query value.

The signature covers the constraints (the disk's base path, method, key, expiry, maxBytes, content type), so editing ?x-max= invalidates it — and so does moving the URL to another disk: every local disk signs with the one STORAGE_SIGNING_SECRET, and the base path (/_storage/<disk>) is what keeps a grant for uploads from verifying on evidence. Uploads are sniffed: validateUpload() reads magic bytes and refuses a .png that is really HTML (X_STORAGE_TYPE_REJECTED). ISO base media files are told apart by their major brand, so image/avif, image/heic, video/quicktime, audio/mp4 and video/mp4 each pass when the policy allows them; audio/x-m4a and video/x-m4v are read as audio/mp4 and video/mp4.

The grant binds the size at the PUT, on every disk (As of 2026-11-02). request.size is required: a grant with no whole byte count of at least 1 is X_STORAGE_UPLOAD_SIZE_UNDECLARED, and uploadFile sends the file's own. An s3 disk signs it as the exact Content-Length, so the provider answers any other length, a chunked body included, with 403 SignatureDoesNotMatch and stores nothing. The local disks sign it as the ceiling acceptSignedUpload counts against, so a shorter body passes there and not on s3. grant.maxBytes is that declared size. A bare disk.signedUrl(key, { method: 'PUT' }) on s3 still binds nothing; with a maxBytes and no contentLength it is refused rather than ignored. Proven on the test stack's gateway. A provider that skips SigV4's signed-header check is the one thing that would unbind it, so promoteAttachment still measures. get() refuses an object over the disk's maxGetBytes (default: its maxPutBytes, 10 MB) with X_STORAGE_TOO_LARGE; read anything larger with stream().

The local disk signs with STORAGE_SIGNING_SECRET. The shipped development secret is accepted only in development or test; a process that names no environment is treated as production and refused at construction (X_ENV_MISSING).

A large upload, in parts, that resumes

As of 2026-11-06. Past partThresholdBytes (64 MiB) a file goes up in parts, and a dropped connection costs one part, not the file. One PUT restarts from byte 0 and S3 refuses a single PUT over 5 GiB. Five server functions, each wrapped in the app's own action() as grantUpload is, every one held to the actor's org:

policy is required on signPart and completeUpload (As of 2026-11-07; no policy is X_INVARIANT). The server keeps no record of a grant, and the grant's size is the client's own claim, so the policy the app passes on its server is the bound: pass the one the grant was made under, never a value off the request.

import type { MultipartUploadClient, UploadGrant, UploadRequest, UploadSource } from '@ultimat3/storage';
import { uploadFile } from '@ultimat3/storage';

declare const file: UploadSource;
declare const grantPoster: (request: UploadRequest) => Promise<UploadGrant>;
declare const video: MultipartUploadClient; // the app's five action calls, plus `resume`/`onGrant`

const { key } = await uploadFile({ file, grant: grantPoster, multipart: video });

Four parts in flight, each retried with core's backoff. onProgress counts confirmed bytes. resume: grant (the one onGrant handed over, persisted by the caller) lists what landed and sends only what is missing. A cancelled signal aborts the session; any other failure leaves it open for the resume. The upload stays under pending/ until promoteAttachment, which sniffs a head window, so a completed 2 GiB upload is never read whole. On s3 no part passes through the app; on the local disks the one PUT /_storage route takes a part URL as well as a grant's.

A disk whose driver has no multipart (a third-party one) is X_STORAGE_MULTIPART_UNSUPPORTED; a session that was completed, aborted or never opened is X_STORAGE_UPLOAD_UNKNOWN.

Which disks /_storage and /media serve

As of 2026-09-24 (#524): the app's own. Both routes read the process's one registry per request. That registry is the last defineStorage() call, which is the app's when an app module declares its disks. The boot's env-selected disk (object on S3_ENDPOINT, else local) is served only when nothing declared one. A defineStorage at module scope in any file under apps/*/ is enough:

// apps/web/shared/storage.ts
export const storage = defineStorage({
  disks: { uploads: localDriver({ root: '.storage/uploads' }), evidence: s3Driver({ bucket }) },
  default: 'uploads',
});

Who may read a stored object

As of 2026-10-08 (27.0.0): GET /_storage/:disk/*key and GET /media/*key decide on the session, the policy and the key — in that order — and a key no tenant owns is refused unless the app shared it.

Both refusals are decided from the key alone, before a disk is read, so neither says whether an object exists. Until 26.x the third row was served to every holder of storage:read: "not another tenant's" was read as "everybody's".

import { defineStorage, localDriver, s3Driver } from '@ultimat3/storage';

defineStorage({
  disks: {
    uploads: localDriver({ root: '.storage/uploads' }),
    evidence: s3Driver({ bucket: 'evidence' }),
  },
  shared: { uploads: ['brand/'] }, // whole segments, ending in '/'; never 'org/'
});

A prefix is refused at boot (X_CONFIG_INVALID) when it names an undeclared disk, lacks its trailing /, would share the whole disk, or sits inside org/. storage.isShared(disk, key) is the question both routes ask.

The GET does not read x-sig or x-exp. A signed GET URL localDriver mints points at this route, and is served on the terms above whatever its signature says: it neither opens an unshared key nor outlives the session. To serve an object by signature alone (no session), mount your own route around readSignedObject({ url, disk, orgId }), or return the bytes from an action.

Serving an object to a stranger

As of 2026-11-02. A file of the site is an asset(); a stored object is your route and objectResponse. A shared prefix still needs a signed-in reader, so it cannot express an object whose authorization is app state (a published row): that decision is your route's, and the answer is the framework's.

import { byteWindow, objectResponse, type UltimateRequest } from '@ultimat3/http';
import { objectNotFound, storage } from '@ultimat3/storage';

declare function publishedPost(id: string): Promise<{ imageKey: string }>;

export const handler = async (request: UltimateRequest): Promise<Response> => {
  // Your rule first, from your own row. Nothing here reads a session.
  const post = await publishedPost(request.params['id'] ?? '');
  const disk = storage().disk();
  const object = await disk.stat(post.imageKey);
  if (object === undefined) throw objectNotFound(disk.name, post.imageKey);
  return objectResponse(
    request,
    {
      object,
      open: async (range) =>
        range === undefined
          ? disk.stream(post.imageKey)
          : byteWindow(await disk.stream(post.imageKey), range),
    },
    { cache: { mode: 'public', maxAgeSeconds: 300 } },
  );
};

request needs only header(name), and object only { contentType, size, etag?, lastModified? }: @ultimat3/http imports no storage, so a StorageObject fits and so does anything else.

Test the real route. storageRoutes({ storage }) from @ultimat3/cli returns the mounted GET and PUT (servedStorage(storage) is the registry they read), as assetRoutes returns /media, so an app's contract test drives the route its deploy serves rather than a copy.

The body is streamed (stat + stream), never buffered: an object past the disk's maxGetBytes is served, and a Range reads and drops the bytes before its start rather than holding the object. /media streams the object the same way, and buffers a variant only: it decodes images.

One cache posture on both routes (As of 2026-11-02): private, max-age=0 plus an etag, for every key. A key is not content-addressed, since an overwrite changes the bytes under it, so nothing here answers immutable: /media once did on a shared key, and a CDN then served it to strangers for a year and never refreshed it. A public image is an asset(), or your own route over objectResponse.

Attachments and orphans

An upload lands under org/<org>/pending/… before its row exists, and is promoted once it has one:

An erasure of one object must call deleteVariants. disk.delete(key) removes the source only. Once it is gone /media and /_storage answer X_STORAGE_NOT_FOUND for the variant too, but its bytes stay on the disk, and a variant minted before revisions (hero.png@w640.webp) names no revision and stays readable by its own key. Delete the source and its variants, and report failed.

Promotion also reads the object (As of 2026-11-02). A presigned PUT on an s3 disk signs host and the size, so the uploader chose the stored type and the bytes. promoteAttachment reads the first SNIFF_WINDOW_BYTES (64 KiB) with one ranged read and judges them by the rule the upload path uses (validateStoredHead); a contradiction is X_STORAGE_TYPE_REJECTED, and the key stays pending.

The upload URL stays valid after that read, so the copy names the revision judged (ifMatch: on s3, x-amz-copy-source-if-match), and another is X_STORAGE_OBJECT_CHANGED (409). The copy is then checked: another size is refused and deleted; an etag that differs or is unknown (a disk with none, or SSE-KMS, which mints one per copy) means the copy's own head is judged, and refused and deleted if it does not pass. Bytes past the window are not sniffed on any path through promotion.

A grant with a target has no promotion step: Known gaps. With an uploadScanner installed it is refused outright, by grantUpload and grantMultipart alike (X_STORAGE_TARGET_UNSCANNED, 500): grant with no target and promote.

Scanning uploads — the uploadScanner port

Rule: an uploadScanner returned from apps/<app>/runtime.ts is asked about every upload promoteAttachment attaches. As of 2026-11-09. It scans a copy THIS call made: under pending/scan/, server-named — so no presigned URL can rewrite it after the scan — and its own, so a doubled confirm never scans or deletes another attempt's bytes. That copy is what is attached; one scan is final.

// apps/web/runtime.ts
import type { RuntimeOverrides } from '@ultimat3/cli/serve';
import type { UploadScanner } from '@ultimat3/storage/adapter';

declare function clamdScanner(options: { host: string; port: number }): UploadScanner;

export const runtime = {
  uploadScanner: clamdScanner({ host: 'clamav', port: 3310 }),
} satisfies Partial<RuntimeOverrides>;

Images

A variant's key is the whole source key plus the transform — photos/hero.png at width 640 is photos/hero.png@w640.webp — so two sources that differ only by extension never share a cached variant. width and height are whole numbers of at least 1 (X_INVARIANT otherwise).

/media keys what it stores on the bytes and the encoding as well: photos/hero.png@w640-r0123abcd.png, r<8 hex> being variantRevision(object.etag) and the format the ?f= asked for, else the source's (variantFormatOf). It reads the source first, so a deleted source is X_STORAGE_NOT_FOUND whatever is still stored, and an overwrite misses every old variant. The variant's own key answers for its source too, on both routes: once the source is gone or holds another revision, it is X_STORAGE_NOT_FOUND. A source no key can state, a gif or an unknowable etag, is served and never stored. Old-revision variants are bytes nobody reads: deleteVariants or a bucket lifecycle rule removes them, and when is yours to choose.

transformImage() and blurPlaceholder() run on core's pipeline (Bun.Image, no sharp) and encode png, jpeg and webp; avif is key and srcset math only. variantKey(), srcsetDescriptors() and fitDimensions() are pure, which is how SEO builds srcset without decoding a byte.

One kept variant — variant()

variant(key, { disk, images, width, format? }) → { key, object, created }. As of 2026-11-09. Made by the images driver you pass (any @ultimat3/seo ImageTransformDriver — runtime.ts's images, builtinImageDriver({ read }), a CDN's), stored under variantKey(key, { width, format, revision }). A second call is a stat, never a decode; an overwritten source is a new revision, hence a new key. format omitted keeps a png/jpeg/webp source's encoding.

import { builtinImageDriver } from '@ultimat3/seo';
import { disk, variant } from '@ultimat3/storage';

declare const coverKey: string;

const images = builtinImageDriver({ read: async (src) => (await disk().get(src)).bytes });
export const thumbnail = await variant(coverKey, { disk: disk(), images, width: 640, format: 'webp' });

Warm the variants after the upload — warmVariants

The first request for each width otherwise pays the decode on the request path. A job the upload enqueues pays it once instead, under the exact keys /media reads:

import { disk, warmVariants } from '@ultimat3/storage';

const warmed = await warmVariants({
  disk: disk(),
  key: 'org/o1/member/m1/avatar/a.png',
  widths: [48, 96, 192],
  formats: ['webp'],
});

{ written, failed, skipped }: the revision is read off the source it decodes, so a variant is never keyed to bytes it was not made from; a width above the source's is skipped, never an upscale; an avif fails alone with core's X_IMAGE_UNSUPPORTED and the rest are still written. The source is read and probed once and held to MAX_IMAGE_PIXELS before a pixel is decoded. As of 2026-11. examples/dummy's app/orgs/jobs/warm-avatar.ts is the worked job.

Expiry — expireObjects and abortStaleUploads

As of 2026-11-06. The window is the app's, at the call; the sweep runs on every disk. A local disk has no bucket rule, and an abandoned upload's parts bill until something aborts them.

A person's stored objects are a subject target: storageSubjectTarget(() => disk(), keysOf, { name? }) lists the keys keysOf(subject) names (a list, or a prefix the app lists) for subjectExport() and deletes them for subjectErase(). Which keys are the person's is the app's; there is no listing by owner. Jobs and Workflows.

Every X_STORAGE_* code is in Error codes. Encryption at rest, bucket lifecycle rules and storage classes stay bucket configuration, the operator's (axiom 7): expireObjects is the sweep an app runs, not a rule the framework writes to a bucket.