From d70b3d637901360a72c8d0fb90d7e98765e90ae2 Mon Sep 17 00:00:00 2001 From: Tom McKenzie Date: Mon, 29 Jun 2026 17:50:57 +1000 Subject: [PATCH 1/3] feat(registry): runtime Standard Schema validation + uniform error surfacing (ADR-0014) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wire the per-op schema slots to validate at runtime: insert.schema checks the full-row cols, update.schema the partial patch, a command's schema its args — each in authorize, before the write, throwing to reject (fail loud). A delete carries no cols. It is a gate, not a parser: the original value flows to handlers, so transforms/defaults/coercion are not applied. Validation failures throw `ValidationError` and surface to the client with a `VALIDATION` code. Error surfacing is now uniform for mutations and commands: authorize and validation errors surface (the reason reaches the client), execute errors stay sanitized. handleCall's authorize is split from its execute, revising ADR-0012 D3, which had sanitized a command's authorize too. Tests drive a `validated` collection and a `requireBody` command via a hand-rolled Standard Schema (no validator dep): bad insert/update/args are rejected with the reason + VALIDATION code, good ones commit; a command execute throw stays generic. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_018DxmkLhbtb5w7oHHiJKprr --- CHANGELOG.md | 21 ++++ docs/adr/0014-object-sync-schema.md | 31 ++++-- recipes/commands-vs-mutations.md | 10 +- src/server/registry.ts | 119 ++++++++++++-------- src/server/standard-schema.ts | 164 ++++++++++++++++++++++++++++ src/server/sync-do.ts | 25 +++-- tests/registry-types.ts | 32 ++++++ tests/schema-validation.test.ts | 161 +++++++++++++++++++++++++++ tests/test-worker.ts | 94 +++++++++++++++- 9 files changed, 589 insertions(+), 68 deletions(-) create mode 100644 src/server/standard-schema.ts create mode 100644 tests/schema-validation.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index d4e3255..e67c9cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). While pre-1.0, the public API may change between 0.x releases. +## [Unreleased] + +### Changed + +- **Rejection reasons are surfaced uniformly for mutations and commands (revises + ADR-0012 D3).** An `authorize` throw or a schema validation failure now reaches + the client with its reason (validation failures carry a `VALIDATION` code); + only `execute` errors stay sanitized. Previously a command's `authorize` error + was sanitized like its `execute`, unlike a mutation's. + +### Added + +- **Optional Standard Schema validation (ADR-0014).** A collection's + `insert.schema` (the row schema, which also infers the collection's Row) and + `update.schema` (a partial patch schema), and a command's schema, are checked + at runtime and rejected loudly on failure. Any `~standard` library works (zod, + valibot, arktype) and the framework adds no validator dependency. It is a gate, + not a parser: the original value flows to handlers, so schemas must not rely on + transforms, defaults, or coercion. See + `recipes/zod-standard-schema-collections.md`. + ## [0.4.0] — 2026-07-01 ### Changed diff --git a/docs/adr/0014-object-sync-schema.md b/docs/adr/0014-object-sync-schema.md index 8e672a2..bbcfa49 100644 --- a/docs/adr/0014-object-sync-schema.md +++ b/docs/adr/0014-object-sync-schema.md @@ -3,8 +3,10 @@ **Status:** Accepted. Supersedes [ADR-0001](./0001-sync-architecture.md) D11's `defineMutation`/`defineCommand` builder and the [ADR-0007](./0007-author-owned-schema-register-sync.md) `new SyncRegistry().defineCollection(…)` chain; closes the ADR-0010 manifest and -its "typed command args" out-of-scope follow-up. Hard breaking change to the DO -authoring API and the typed client surface (pre-1.0, clean break). +its "typed command args" out-of-scope follow-up; revises [ADR-0012](./0012-wire-input-hardening.md) +D3 so a command's `authorize`/validation errors surface like a mutation's. Hard +breaking change to the DO authoring API and the typed client surface (pre-1.0, +clean break). ## Context @@ -28,8 +30,11 @@ We also had two genuinely different shapes wearing one builder. A **mutation** mirrors a row write — TanStack DB already names exactly `onInsert`/`onUpdate`/`onDelete`, and a synced write *is* one of those three. A **command** is an RPC: free-form args, a return value, its own atomicity, no collection. ADR-0010 lumped them as -sibling `defineX` methods; ADR-0012 D3 had already found they diverge at runtime -(a command's `authorize` throw is sanitized, a mutation's stays user-facing). +sibling `defineX` methods, but they stay distinct in what's structural: a command +owns its atomicity and may be async; a mutation is synchronous inside the +transaction. (ADR-0012 D3 had also split their error surfacing — command +`authorize` sanitized, mutation user-facing — but D2 below drops that, so +surfacing is now uniform.) ## Decision @@ -95,11 +100,12 @@ This is the open counterpart to the closed trio: anything that isn't one-row insert/update/delete (multi-row, cross-collection, compute-and-return, no write at all) is a command. -The two shapes stay distinct at runtime exactly as ADR-0012 D3 found: a command's -`authorize` throw is logged and surfaced as a generic rejection; a mutation's -`authorize` throw is a distinct user-facing message. Keeping them separate -helpers, rather than ADR-0010's sibling `defineX` methods, makes that divergence -structural instead of incidental. +They stay separate helpers because their shapes differ — a command is RPC with +free args and a return value, a mutation is a typed row op — not because their +errors differ. Error surfacing is now the **same** for both (revising ADR-0012 +D3, which had sanitized a command's `authorize`): a mutation and a command both +surface their `authorize` and validation errors to the client, and both sanitize +their `execute` errors. See D3. ### D3: The row schema lives on the insert mutation; it infers the row and validates writes @@ -203,8 +209,9 @@ mut/call handler, `runSyncedWrite` is still the path (ADR-0006). patch, a command schema validates `args`; a `delete` is unvalidated (no cols, pk validated at insert). It validates but does not transform — the original value flows to handlers. Dependency-free via Standard Schema; no zod runtime pulled in. -- **`authorize` divergence is now structural** (ADR-0012 D3): mutation-authorize - throws stay user-facing, command-authorize throws are sanitized — the two - helpers make that explicit rather than a shared-code accident. +- **Uniform error surfacing (revises ADR-0012 D3).** Mutations and commands both + surface `authorize` and validation errors (a schema failure carries a + `VALIDATION` code) and both sanitize `execute` errors. ADR-0012 D3's + command-`authorize` sanitization is dropped, so the two behave the same. - **Per-DO contract.** Multi-DO apps run one transport and one `Api` per DO; there is no global command registry. Consistent with the per-connection cursor (0002). diff --git a/recipes/commands-vs-mutations.md b/recipes/commands-vs-mutations.md index 7edb4b0..97699ee 100644 --- a/recipes/commands-vs-mutations.md +++ b/recipes/commands-vs-mutations.md @@ -84,10 +84,12 @@ returns the count. ## Notes -- A mutation's `authorize` denies a write by throwing, and the client receives - that error message. A command's `authorize` denies a call the same way, by - throwing, but a command's errors are sanitized — the client receives a generic - rejection rather than the thrown text. +- Denial is uniform across mutations and commands: an `authorize` throw, or a + schema validation failure, surfaces its reason to the client (validation + failures carry a `VALIDATION` code), so you can tell the user why. Only an + `execute` error is sanitized to a generic message ("mutation failed" / + "command failed") — by then the call is authorized, and a failure there may + carry internal detail. - You can type a command's args, either with a generic (`sync.command<{ before: number }>()(fn)`) or from a schema (`sync.command(zArgs, fn)`). diff --git a/src/server/registry.ts b/src/server/registry.ts index e611e45..7a3cdff 100644 --- a/src/server/registry.ts +++ b/src/server/registry.ts @@ -23,36 +23,31 @@ import type { SqlStorage } from "@cloudflare/workers-types" import type { MutOp, RowOp } from "../wire/frames.ts" import { SYNC_PREFIX } from "./changes.ts" +import type { StandardSchemaV1 } from "./standard-schema.ts" + +export type { StandardSchemaV1 } const IDENT = /^[A-Za-z_][A-Za-z0-9_]*$/ // --------------------------------------------------------------------------- // Standard Schema — the dependency-free `~standard` interface that zod/valibot/ -// arktype all satisfy. We never import a validator. +// arktype all satisfy, vendored verbatim in `./standard-schema.ts` (we import no +// validator and add no dependency). +// +// A schema does double duty: it infers types (a row schema on `insert.schema` +// infers a collection's Row, `command(schema, …)` infers a command's Args) AND it +// validates at runtime. `compileSchema` calls `~standard.validate` before the +// handler runs and throws on issues, rejecting the frame (fail loud). // -// On THIS branch a schema is used for TYPE inference only: a row schema on -// `insert.schema` infers a collection's Row, and `command(schema, …)` infers a -// command's Args. Runtime validation against these schemas is NOT wired here; it -// lands in the stacked validation PR (see the tombstone notes in -// `compileMutation`/`compileCommand`). When it lands it is a validation GATE, not -// a parser: the handler receives the original wire value, never the schema's -// parsed output, so a schema must not rely on transforms/defaults/coercion (input -// must equal output). Rewriting a row the client already applied optimistically -// would manufacture divergence, and a pk rewrite would break optimistic-id == +// It is a validation GATE, not a parser: the handler receives the original wire +// value — the schema's INPUT — never its parsed output. We infer from input +// (`StandardSchemaV1.InferInput`), so `op.cols`/`args` describe exactly what the +// handler gets; a transforming schema's output is validated but discarded, never +// applied. Applying it would rewrite a row the client already holds optimistically +// (manufacturing divergence), and a pk rewrite would break optimistic-id == // confirmed-id (ADR-0001 D9). // --------------------------------------------------------------------------- -export interface StandardSchemaV1 { - readonly "~standard": { - readonly version: 1 - readonly vendor: string - readonly validate: (value: unknown) => StandardResult | Promise> - readonly types?: { readonly input: unknown; readonly output: Output } - } -} -type StandardResult = - | { readonly value: Output; readonly issues?: undefined } - | { readonly issues: ReadonlyArray<{ readonly message: string }> } -type InferSchema = S extends StandardSchemaV1 ? O : never +type InferSchema = StandardSchemaV1.InferInput // --------------------------------------------------------------------------- // Collection identity (runtime) + per-op shapes. @@ -96,21 +91,23 @@ export interface CommandCtx { // Authoring types — the closed mutation trio + collection/command entries. // // `insert` and `update` each carry an optional `schema` (the row schema, and the -// patch schema). On this branch the schema types `op.cols`; the stacked PR wires -// runtime validation against it. +// patch schema). The schema validates `op.cols` at runtime AND types it — for +// `insert`, it is also the inference source for the collection's Row. // --------------------------------------------------------------------------- export interface InsertDef { - /** Row schema. Validates the full-row `cols` (in the validation PR) and, when - * used as the inference source, types the collection's Row. */ - schema?: StandardSchemaV1 + /** Row schema. Validates the full-row `cols` and, when used as the inference + * source, types the collection's Row. ``: the schema's INPUT is + * the Row (what the handler receives); its output is discarded (gate, not parser). */ + schema?: StandardSchemaV1 authorize?: (ctx: MutationCtx>) => void | Promise execute: (ctx: MutationCtx>) => void afterCommit?: (ctx: MutationCtx>) => unknown | Promise } export interface UpdateDef { /** Patch schema. The author supplies a PARTIAL schema (e.g. `Row.partial()`), - * since an update carries a top-level partial patch, not a full row. */ - schema?: StandardSchemaV1> + * since an update carries a top-level partial patch, not a full row. Output is + * discarded — only the INPUT (the patch shape the handler receives) is typed. */ + schema?: StandardSchemaV1, unknown> authorize?: (ctx: MutationCtx>) => void | Promise execute: (ctx: MutationCtx>) => void afterCommit?: (ctx: MutationCtx>) => unknown | Promise @@ -175,7 +172,9 @@ export type CommandInput = /** A command entry: carries Args and the (awaited) Result in the type. The * phantoms let the client recover both from `typeof schema`. */ export interface CommandEntry { - schema?: StandardSchemaV1 + /** Args schema. ``: the schema's INPUT is the args the handler + * receives; its output is discarded (gate, not parser). */ + schema?: StandardSchemaV1 authorize?: (ctx: CommandCtx) => void | Promise execute: (ctx: CommandCtx) => Result | Promise /** phantoms — type-only carriers for client inference. */ @@ -317,10 +316,10 @@ const ROW_OPS = ["insert", "update", "delete"] as const * Compile an authored schema value into the flat dispatch tables, validating each * collection's identifiers (ADR-0007/0008). * - * The per-op Standard Schema slots (`insert.schema`, `update.schema`, command - * `schema`) type the author's `cols`/`args`, but on this branch they are NOT - * enforced at runtime — runtime validation is wired in the stacked validation PR - * (see the tombstone notes in `compileMutation`/`compileCommand`). + * When a per-op Standard Schema is present it is enforced at runtime: + * `insert.schema` validates the full-row `cols`, `update.schema` validates the + * partial patch, and a command `schema` validates `args` — each before the + * handler runs, throwing on issues (fail loud). A `delete` carries no cols. */ export function compileSchema(schema: SyncSchema): CompiledSync { const collections = new Map() @@ -355,19 +354,51 @@ function compileMutation( m: RuntimeMutationDef, schema: StandardSchemaV1 | undefined, ): RuntimeMutationDef { - // TOMBSTONE (base branch): the per-op `schema` (insert full-row, update partial) - // types `op.cols` for the author, but runtime validation is NOT wired here. It - // lands in the stacked validation PR, where this gates the write on `schema`. - // On this branch the schema is inert at runtime. - void type - void schema - return m + // Gate the write on the per-op schema when present: insert.schema validates the + // full-row cols, update.schema the partial patch. A delete carries no cols. + // Validation runs in authorize (before the transaction) and throws on issues. + if (!schema || type === "delete") return m + const userAuthorize = m.authorize + return { + authorize: async (ctx) => { + await validateStandard(schema, ctx.op.cols) + if (userAuthorize) await userAuthorize(ctx) + }, + execute: m.execute, + afterCommit: m.afterCommit, + } } function compileCommand(entry: CommandEntry): RuntimeCommandDef { - // TOMBSTONE (base branch): the command `schema` types `args`; runtime validation - // is wired in the stacked validation PR. - return { authorize: entry.authorize, execute: entry.execute } + const userAuthorize = entry.authorize + const schema = entry.schema + if (!schema) return { authorize: userAuthorize, execute: entry.execute } + return { + authorize: async (ctx) => { + await validateStandard(schema, ctx.args) + if (userAuthorize) await userAuthorize(ctx) + }, + execute: entry.execute, + } +} + +/** Thrown by the validation gate on a schema failure. Carries the issue detail + * and is surfaced to the client with a `VALIDATION` code, since bad input is the + * caller's to fix. An `execute` error, by contrast, stays sanitized. */ +export class ValidationError extends Error { + constructor(message: string) { + super(message) + this.name = "ValidationError" + } +} + +/** Run a Standard Schema's validator and throw `ValidationError` on issues. */ +async function validateStandard(schema: StandardSchemaV1, value: unknown): Promise { + const result = await schema["~standard"].validate(value) + if ("issues" in result && result.issues) { + const detail = result.issues.map((i) => i.message).join("; ") + throw new ValidationError(`validation failed: ${detail}`) + } } /** diff --git a/src/server/standard-schema.ts b/src/server/standard-schema.ts new file mode 100644 index 0000000..a06761b --- /dev/null +++ b/src/server/standard-schema.ts @@ -0,0 +1,164 @@ +// Vendored verbatim from https://github.com/standard-schema/standard-schema/blob/main/packages/spec/src/index.ts +/** The Standard Typed interface. This is a base type extended by other specs. */ +export interface StandardTypedV1 { + /** The Standard properties. */ + readonly "~standard": StandardTypedV1.Props; +} + +export declare namespace StandardTypedV1 { + /** The Standard Typed properties interface. */ + export interface Props { + /** The version number of the standard. */ + readonly version: 1; + /** The vendor name of the schema library. */ + readonly vendor: string; + /** Inferred types associated with the schema. */ + readonly types?: Types | undefined; + } + + /** The Standard Typed types interface. */ + export interface Types { + /** The input type of the schema. */ + readonly input: Input; + /** The output type of the schema. */ + readonly output: Output; + } + + /** Infers the input type of a Standard Typed. */ + export type InferInput = NonNullable< + Schema["~standard"]["types"] + >["input"]; + + /** Infers the output type of a Standard Typed. */ + export type InferOutput = NonNullable< + Schema["~standard"]["types"] + >["output"]; +} + +/** The Standard Schema interface. */ +export interface StandardSchemaV1 { + /** The Standard Schema properties. */ + readonly "~standard": StandardSchemaV1.Props; +} + +export declare namespace StandardSchemaV1 { + /** The Standard Schema properties interface. */ + export interface Props + extends StandardTypedV1.Props { + /** Validates unknown input values. */ + readonly validate: ( + value: unknown, + options?: StandardSchemaV1.Options | undefined, + ) => Result | Promise>; + } + + /** The result interface of the validate function. */ + export type Result = SuccessResult | FailureResult; + + /** The result interface if validation succeeds. */ + export interface SuccessResult { + /** The typed output value. */ + readonly value: Output; + /** A falsy value for `issues` indicates success. */ + readonly issues?: undefined; + } + + export interface Options { + /** Explicit support for additional vendor-specific parameters, if needed. */ + readonly libraryOptions?: Record | undefined; + } + + /** The result interface if validation fails. */ + export interface FailureResult { + /** The issues of failed validation. */ + readonly issues: ReadonlyArray; + } + + /** The issue interface of the failure output. */ + export interface Issue { + /** The error message of the issue. */ + readonly message: string; + /** The path of the issue, if any. */ + readonly path?: ReadonlyArray | undefined; + } + + /** The path segment interface of the issue. */ + export interface PathSegment { + /** The key representing a path segment. */ + readonly key: PropertyKey; + } + + /** The Standard types interface. */ + export interface Types + extends StandardTypedV1.Types {} + + /** Infers the input type of a Standard. */ + export type InferInput = + StandardTypedV1.InferInput; + + /** Infers the output type of a Standard. */ + export type InferOutput = + StandardTypedV1.InferOutput; +} + +/** The Standard JSON Schema interface. */ +export interface StandardJSONSchemaV1 { + /** The Standard JSON Schema properties. */ + readonly "~standard": StandardJSONSchemaV1.Props; +} + +export declare namespace StandardJSONSchemaV1 { + /** The Standard JSON Schema properties interface. */ + export interface Props + extends StandardTypedV1.Props { + /** Methods for generating the input/output JSON Schema. */ + readonly jsonSchema: StandardJSONSchemaV1.Converter; + } + + /** The Standard JSON Schema converter interface. */ + export interface Converter { + /** Converts the input type to JSON Schema. May throw if conversion is not supported. */ + readonly input: ( + options: StandardJSONSchemaV1.Options, + ) => Record; + /** Converts the output type to JSON Schema. May throw if conversion is not supported. */ + readonly output: ( + options: StandardJSONSchemaV1.Options, + ) => Record; + } + + /** + * The target version of the generated JSON Schema. + * + * It is *strongly recommended* that implementers support `"draft-2020-12"` and `"draft-07"`, as they are both in wide use. All other targets can be implemented on a best-effort basis. Libraries should throw if they don't support a specified target. + * + * The `"openapi-3.0"` target is intended as a standardized specifier for OpenAPI 3.0 which is a superset of JSON Schema `"draft-04"`. + */ + export type Target = + | "draft-2020-12" + | "draft-07" + | "openapi-3.0" + // Accepts any string: allows future targets while preserving autocomplete + | ({} & string); + + /** The options for the input/output methods. */ + export interface Options { + /** Specifies the target version of the generated JSON Schema. Support for all versions is on a best-effort basis. If a given version is not supported, the library should throw. */ + readonly target: Target; + + /** Explicit support for additional vendor-specific parameters, if needed. */ + readonly libraryOptions?: Record | undefined; + } + + /** The Standard types interface. */ + export interface Types + extends StandardTypedV1.Types {} + + /** Infers the input type of a Standard. */ + export type InferInput = + StandardTypedV1.InferInput; + + /** Infers the output type of a Standard. */ + export type InferOutput = + StandardTypedV1.InferOutput; +} diff --git a/src/server/sync-do.ts b/src/server/sync-do.ts index deef9eb..6bb3d1a 100644 --- a/src/server/sync-do.ts +++ b/src/server/sync-do.ts @@ -31,7 +31,7 @@ import { } from "./changes.ts" import { Broadcaster } from "./broadcast.ts" import { decodeResult, encodeResult, lookupTx, recordTx, type SeenTx, sweepDedup } from "./dedup.ts" -import { compileSchema, type CompiledSync, type SyncSchema } from "./registry.ts" +import { compileSchema, type CompiledSync, type SyncSchema, ValidationError } from "./registry.ts" import { andPredicates, compileSubsetQuery, UnsupportedPredicateError } from "./sql-compiler.ts" import { SubscriptionRegistry, type Sub } from "./subscriptions.ts" @@ -358,6 +358,10 @@ export abstract class SyncDurableObject extends if (def.authorize) await def.authorize({ user, op, sql: this.sql, env: this.env }) } } catch (e) { + // authorize and validation errors surface to the client: a schema failure + // carries a VALIDATION code, an authz "throw to deny" keeps its message. The + // execute catch below stays sanitized. + if (e instanceof ValidationError) return this.rejectTx(ws, f.txId, e.message, "VALIDATION") return this.rejectTx(ws, f.txId, errorMessage(e)) } @@ -426,16 +430,23 @@ export abstract class SyncDurableObject extends if (!def) return this.rejectTx(ws, f.txId, `unknown command '${f.name}'`, "UNKNOWN_COMMAND") const user = this.userFor(ws) - let result: unknown + // authorize and validation errors surface like a mutation's: a schema failure + // carries a VALIDATION code, an authz "throw to deny" keeps its message. This + // matches mutation surfacing, revising ADR-0012 D3 (which sanitized a + // command's authorize too). try { if (def.authorize) await def.authorize({ user, args: f.args, sql: this.sql, env: this.env }) + } catch (e) { + if (e instanceof ValidationError) return this.rejectTx(ws, f.txId, e.message, "VALIDATION") + return this.rejectTx(ws, f.txId, errorMessage(e)) + } + // execute runs arbitrary, often async code, so its errors are sanitized like a + // mutation's execute — internal detail never leaks. + let result: unknown + try { result = await def.execute({ user, args: f.args, sql: this.sql, env: this.env }) } catch (e) { - // Log full detail server-side; send only a generic message to the client - // (ADR-0012). Both authorize and execute errors for commands go through - // this path — command authorize errors are NOT currently user-facing API - // (unlike mutation authorize, which is "throw to deny"). Log + generic. - console.error(`command '${f.name}' failed: ${errorMessage(e)}`) + console.error(`command '${f.name}' execute failed: ${errorMessage(e)}`) return this.rejectTx(ws, f.txId, "command failed", "EXECUTE_FAILED") } diff --git a/tests/registry-types.ts b/tests/registry-types.ts index cd02900..33f2594 100644 --- a/tests/registry-types.ts +++ b/tests/registry-types.ts @@ -136,3 +136,35 @@ sync.collection({ archive: { execute: () => {} }, }, }) + +// --- gate-not-parser: `op.cols` is typed from the schema's INPUT, not its parsed +// OUTPUT. The handler receives the original wire value (ADR-0014), so a +// transforming schema must not lure the author into treating `cols` as the parsed +// shape. Here input.count is a string, output.count a number. --- +declare function transformSchema(): StandardSchemaV1 + +sync.collection({ + pk: "id", + mutations: { + insert: { + schema: transformSchema<{ id: string; count: string }, { id: string; count: number }>(), + execute: ({ op }) => { + const c: string = op.cols.count // INPUT (string) — the value the handler gets + void c + }, + }, + }, +}) +sync.collection({ + pk: "id", + mutations: { + insert: { + schema: transformSchema<{ id: string; count: string }, { id: string; count: number }>(), + execute: ({ op }) => { + // @ts-expect-error op.cols.count is the INPUT (string), never the parsed output (number) + const n: number = op.cols.count + void n + }, + }, + }, +}) diff --git a/tests/schema-validation.test.ts b/tests/schema-validation.test.ts new file mode 100644 index 0000000..bc0774d --- /dev/null +++ b/tests/schema-validation.test.ts @@ -0,0 +1,161 @@ +import { SELF } from "cloudflare:test" +import { describe, expect, it } from "vitest" +import { createFrameCodec } from "../src/wire/frame-codec.ts" +import type { ClientFrame, ServerFrame } from "../src/wire/frames.ts" + +// WHY: a per-op Standard Schema is a validation GATE (ADR-0014). A write whose +// `cols`/`args` fail the schema must be REJECTED before anything is applied, and +// a write that passes must commit unchanged. These pin that gate on the wire: +// - insert validates the full row; a bad row is rejected with the validation +// message (a mutation's authorize error is user-facing) and writes nothing. +// - update validates the PARTIAL patch (the reason updates need their own +// schema — a full-row schema would reject every valid partial). +// - a command validates its args the same way, and its rejection carries the +// validation reason and a VALIDATION code too (ADR-0014 unifies command and +// mutation surfacing: authorize and validation errors surface, execute errors +// stay sanitized). +// The `validated` collection + `requireBody` command in test-worker carry a +// hand-rolled schema that rejects an empty `body`. + +const codec = createFrameCodec() + +async function openWs(path: string): Promise { + const res = await SELF.fetch(`https://example.com${path}`, { headers: { Upgrade: "websocket" } }) + expect(res.status).toBe(101) + const ws = res.webSocket + if (!ws) throw new Error("no webSocket on 101 response") + ws.accept() + return ws +} + +function send(ws: WebSocket, frame: ClientFrame): void { + ws.send(codec.encode(frame)) +} + +function collectUntil(ws: WebSocket, done: (f: ServerFrame) => boolean, timeoutMs = 2000): Promise> { + return new Promise((resolve, reject) => { + const out: Array = [] + const timer = setTimeout(() => reject(new Error(`timeout; got [${out.map((f) => f.t).join(",")}]`)), timeoutMs) + ws.addEventListener("message", function onMsg(e: MessageEvent) { + out.push(codec.decode(e.data as ArrayBuffer) as ServerFrame) + if (done(out[out.length - 1]!)) { + clearTimeout(timer) + ws.removeEventListener("message", onMsg) + resolve(out) + } + }) + }) +} + +const last = (frames: Array): ServerFrame => frames[frames.length - 1]! + +describe("per-op Standard Schema validation gate (ADR-0014)", () => { + it("rejects an insert whose row fails the schema, applying nothing", async () => { + const ws = await openWs("/sync/v-ins-bad") + send(ws, { t: "mut", txId: "t1", collection: "validated", ops: [{ type: "insert", key: "a", cols: { id: "a", body: "" } }] }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || f.t === "committed") + const l = last(frames) + expect(l.t).toBe("rejected") + if (l.t === "rejected") { + expect(l.error.message).toMatch(/validation failed/) + expect(l.error.code).toBe("VALIDATION") + } + expect(frames.some((f) => f.t === "d")).toBe(false) // nothing applied + ws.close() + }) + + it("commits an insert whose row passes the schema", async () => { + const ws = await openWs("/sync/v-ins-ok") + send(ws, { t: "mut", txId: "t1", collection: "validated", ops: [{ type: "insert", key: "a", cols: { id: "a", body: "hi" } }] }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || f.t === "committed") + expect(last(frames).t).toBe("committed") + ws.close() + }) + + it("rejects an update whose PARTIAL patch fails the schema (the partial-schema reason)", async () => { + const ws = await openWs("/sync/v-upd-bad") + send(ws, { t: "mut", txId: "t1", collection: "validated", ops: [{ type: "insert", key: "a", cols: { id: "a", body: "ok" } }] }) + await collectUntil(ws, (f) => f.t === "committed") + // A patch carrying only { body: "" } — a full-row schema would wrongly reject + // any partial; the update schema validates exactly the patch. + send(ws, { t: "mut", txId: "t2", collection: "validated", ops: [{ type: "update", key: "a", cols: { body: "" } }] }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || (f.t === "committed" && f.txId === "t2")) + const l = last(frames) + expect(l.t).toBe("rejected") + if (l.t === "rejected") { + expect(l.error.message).toMatch(/validation failed/) + expect(l.error.code).toBe("VALIDATION") + } + ws.close() + }) + + it("commits an update whose partial patch passes the schema", async () => { + const ws = await openWs("/sync/v-upd-ok") + send(ws, { t: "mut", txId: "t1", collection: "validated", ops: [{ type: "insert", key: "a", cols: { id: "a", body: "ok" } }] }) + await collectUntil(ws, (f) => f.t === "committed") + send(ws, { t: "mut", txId: "t2", collection: "validated", ops: [{ type: "update", key: "a", cols: { body: "edited" } }] }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || (f.t === "committed" && f.txId === "t2")) + expect(last(frames).t).toBe("committed") + ws.close() + }) + + it("rejects a command whose args fail the schema, surfacing the reason (ADR-0014)", async () => { + const ws = await openWs("/sync/v-cmd-bad") + send(ws, { t: "call", txId: "c1", name: "requireBody", args: { body: "" } }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || f.t === "committed") + const l = last(frames) + expect(l.t).toBe("rejected") + // A command's validation rejection now carries the reason + code, like a + // mutation's — not the old generic "command failed". + if (l.t === "rejected") { + expect(l.error.message).toMatch(/validation failed/) + expect(l.error.code).toBe("VALIDATION") + } + ws.close() + }) + + it("runs a command whose args pass the schema and returns its result", async () => { + const ws = await openWs("/sync/v-cmd-ok") + send(ws, { t: "call", txId: "c1", name: "requireBody", args: { body: "hello" } }) + const frames = await collectUntil(ws, (f) => f.t === "committed" || f.t === "rejected") + const l = last(frames) + expect(l.t).toBe("committed") + if (l.t === "committed") expect(l.result).toEqual({ echoed: "hello" }) + ws.close() + }) + + // WHY: the gate validates but MUST NOT parse — the handler receives the original + // wire value, never the schema's transformed output. If the transform leaked + // through, the stored/broadcast row would diverge from the row the client already + // applied optimistically, and a pk rewrite would break optimistic-id == + // confirmed-id (ADR-0001 D9). `transformed`'s schema uppercases `body`. + it("gate-not-parser: a transforming schema's output is discarded; the raw wire value is stored and synced", async () => { + const ws = await openWs("/sync/v-transform") + // Subscribe first, so the committed insert broadcasts back as a delta carrying + // the STORED cols (what the handler actually wrote). + send(ws, { t: "sub", subId: "s1", collection: "transformed" }) + await collectUntil(ws, (f) => f.t === "snap-end") + send(ws, { t: "mut", txId: "t1", collection: "transformed", ops: [{ type: "insert", key: "a", cols: { id: "a", body: "hi" } }] }) + const frames = await collectUntil(ws, (f) => f.t === "committed" && f.txId === "t1") + expect(last(frames).t).toBe("committed") // the transform doesn't reject; it commits + // The row that synced back carries the RAW "hi", not the schema's "HI" — proving + // the handler got the wire value, not the parsed output. + const delta = frames.find((f): f is Extract => f.t === "d" && f.sub === "s1") + expect(delta).toBeDefined() + expect((delta!.cols as { body: string }).body).toBe("hi") + ws.close() + }) + + // WHY: uniform surfacing (ADR-0014 revises ADR-0012 D3) — a command's authorize + // "throw to deny" reaches the client verbatim, exactly like a mutation's; only an + // execute error stays sanitized (that path is covered by `boom`). + it("a command's authorize throw surfaces its reason, not a generic rejection", async () => { + const ws = await openWs("/sync/v-cmd-authz") + send(ws, { t: "call", txId: "c1", name: "denyCall", args: undefined }) + const frames = await collectUntil(ws, (f) => f.t === "rejected" || f.t === "committed") + const l = last(frames) + expect(l.t).toBe("rejected") + if (l.t === "rejected") expect(l.error.message).toMatch(/call denied by authorize/) + ws.close() + }) +}) diff --git a/tests/test-worker.ts b/tests/test-worker.ts index 7ea64a0..02a4ec7 100644 --- a/tests/test-worker.ts +++ b/tests/test-worker.ts @@ -2,7 +2,7 @@ // miniflare.durableObjects, and routes WebSocket upgrades to the sync DO. import { DurableObject } from "cloudflare:workers" -import { defineSync } from "../src/server/registry.ts" +import { defineSync, type StandardSchemaV1 } from "../src/server/registry.ts" import { SyncDurableObject } from "../src/server/sync-do.ts" /** Bare DO for the M1 CDC tests; they drive storage via runInDurableObject. */ @@ -25,6 +25,49 @@ interface FileRow { name: string } +interface ValidatedRow { + id: string + body: string +} + +/** A hand-rolled Standard Schema (no validator dependency): rejects when `body` + * is present but not a non-empty string. Used as both the insert (full-row) and + * update (partial) schema for the `validated` collection, to exercise the + * runtime validation gate. */ +function nonEmptyBody(): StandardSchemaV1 { + return { + "~standard": { + version: 1, + vendor: "test", + validate: (value) => { + const v = value as { body?: unknown } + if ("body" in v && (typeof v.body !== "string" || v.body.trim() === "")) { + return { issues: [{ message: "body must be a non-empty string" }] } + } + return { value: value as T } + }, + }, + } +} + +/** A TRANSFORMING Standard Schema — its `validate` returns a MUTATED value + * (uppercased `body`). The gate must DISCARD that output and hand the handler the + * ORIGINAL wire value (ADR-0014 gate-not-parser); if the transform ever leaked + * through, the stored/broadcast row would diverge from the row the client applied + * optimistically (ADR-0001 D9). Used by the `transformed` collection. */ +function upcasingBody(): StandardSchemaV1 { + return { + "~standard": { + version: 1, + vendor: "test", + validate: (value) => { + const v = value as { body?: unknown } + return { value: { ...(v as object), body: typeof v.body === "string" ? v.body.toUpperCase() : v.body } as T } + }, + }, + } +} + const sync = defineSync() // The same collections/mutations/commands as before, authored via the @@ -82,6 +125,39 @@ const testSchema = sync.schema({ }, }, }), + // Carries Standard Schemas so the runtime validation gate is exercised: the + // insert schema validates the full row, the update schema the partial patch. + validated: sync.collection({ + pk: "id", + mutations: { + insert: { + schema: nonEmptyBody(), + execute: ({ op, sql }) => { + sql.exec("INSERT INTO validated(id, body) VALUES (?, ?)", op.cols.id, op.cols.body) + }, + }, + update: { + schema: nonEmptyBody>(), + execute: ({ op, sql }) => { + sql.exec("UPDATE validated SET body = ? WHERE id = ?", op.cols.body, op.key) + }, + }, + }, + }), + // Its insert schema TRANSFORMS (uppercases `body`), but the gate discards that + // output and the handler stores the RAW `op.cols.body` — a subscriber sees the + // wire value, not the transform (ADR-0014 gate-not-parser; ADR-0001 D9). + transformed: sync.collection({ + pk: "id", + mutations: { + insert: { + schema: upcasingBody(), + execute: ({ op, sql }) => { + sql.exec("INSERT INTO transformed(id, body) VALUES (?, ?)", op.cols.id, op.cols.body) + }, + }, + }, + }), }, commands: { echo: sync.command()(({ args }) => ({ echoed: args })), @@ -100,6 +176,20 @@ const testSchema = sync.schema({ boom: sync.command()(() => { throw new Error("command boom") }), + // Carries an args Standard Schema — exercises command-args validation. As for + // a mutation, a validation failure surfaces to the client with its reason and a + // VALIDATION code (uniform surfacing; ADR-0014 revises ADR-0012 D3). + requireBody: sync.command(nonEmptyBody<{ body: string }>(), ({ args }) => ({ echoed: args.body })), + // A command whose `authorize` THROWS — its reason must surface to the client + // (ADR-0014 unifies command + mutation authorize surfacing; ADR-0012 D3 had + // sanitized a command's authorize to a generic message). An `execute` throw + // stays sanitized — that's `boom`. + denyCall: sync.command()({ + authorize: () => { + throw new Error("call denied by authorize") + }, + execute: () => ({ ok: true }), + }), }, }) @@ -114,6 +204,8 @@ export class SyncTestDO extends SyncDurableObject { // The author owns table creation; the framework wires sync after (ADR-0007). this.sql.exec(`CREATE TABLE IF NOT EXISTS messages (id TEXT PRIMARY KEY, body TEXT)`) this.sql.exec(`CREATE TABLE IF NOT EXISTS files (id TEXT PRIMARY KEY, name TEXT)`) + this.sql.exec(`CREATE TABLE IF NOT EXISTS validated (id TEXT PRIMARY KEY, body TEXT)`) + this.sql.exec(`CREATE TABLE IF NOT EXISTS transformed (id TEXT PRIMARY KEY, body TEXT)`) this.registerSync(testSchema) }) } From ee6584a81932912a15eced7c41731fee02ae1a02 Mon Sep 17 00:00:00 2001 From: Tom McKenzie Date: Mon, 29 Jun 2026 17:50:57 +1000 Subject: [PATCH 2/3] docs: add the Standard Schema validation recipe recipes/zod-standard-schema-collections.md: how to attach a schema to a collection (insert plus a partial update) and a command, what is checked, and the gate-not-parser rule. Written in the project's plain style. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_018DxmkLhbtb5w7oHHiJKprr --- recipes/zod-standard-schema-collections.md | 124 +++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 recipes/zod-standard-schema-collections.md diff --git a/recipes/zod-standard-schema-collections.md b/recipes/zod-standard-schema-collections.md new file mode 100644 index 0000000..ba03d9f --- /dev/null +++ b/recipes/zod-standard-schema-collections.md @@ -0,0 +1,124 @@ +# Validate collections and commands with a schema + +Use this when you want the Durable Object to reject a malformed write at the edge, +such as a bad insert row or bad command arguments, using a schema library you +already have. It works with zod, valibot, or arktype, and the framework does not +depend on any of them. + +The schema is optional. If you leave it off, you pass the row type as a generic +(`collection({ ... })`) and nothing is checked at runtime. Add a schema +when you want the runtime check as well. + +## Recipe + +```ts +// server +import { z } from "zod" // 3.24 or later; any Standard Schema library works +import { defineSync } from "tanstack-do-db-collection/server" + +const Message = z.object({ + id: z.string(), + author: z.string(), + content: z.string().min(1).max(4000), + created_at: z.number().int(), +}) + +const sync = defineSync() + +export const schema = sync.schema({ + collections: { + messages: sync.collection({ + pk: "id", + mutations: { + // The insert schema checks the full row. Its type also becomes the + // collection's Row, so you do not pass a generic. + insert: { + schema: Message, + execute: ({ op, sql }) => + sql.exec( + "INSERT INTO messages(id, author, content, created_at) VALUES (?, ?, ?, ?)", + op.cols.id, + op.cols.author, + op.cols.content, + op.cols.created_at, + ), + }, + // An update sends a partial patch, so pass a partial schema. + update: { + schema: Message.partial(), + execute: ({ op, sql }) => { + if (op.cols.content !== undefined) { + sql.exec("UPDATE messages SET content = ? WHERE id = ?", op.cols.content, op.key) + } + }, + }, + delete: { execute: ({ op, sql }) => sql.exec("DELETE FROM messages WHERE id = ?", op.key) }, + }, + }), + }, + commands: { + // A command checks its arguments against the schema you pass first. + clearOlderThan: sync.command(z.object({ before: z.number().int() }), ({ args, sql }) => { + sql.exec("DELETE FROM messages WHERE created_at < ?", args.before) + return { ok: true } + }), + }, +}) +export type Api = typeof schema +``` + +## What gets checked + +- **Insert.** The insert schema checks the full row before the write runs. +- **Update.** The update schema checks the partial patch. You pass a partial + schema because an update sends only the fields that changed, not the whole row. + A full row schema would reject every valid partial. +- **Delete.** A delete needs no schema. It carries only the key, the framework + already checks that the key is a non-empty string, and the primary key was + checked when the row was inserted. +- **Command.** A command checks its arguments against the schema you pass. + +When a check fails, the write is rejected and nothing is applied. + +## The schema is a gate, not a parser + +The schema checks the value and rejects it on failure. It does not change the +value. Your handler receives the original value from the wire, not the schema's +parsed output. So do not rely on `.transform()`, `.default()`, `z.coerce`, or +unknown-key stripping to reshape what gets written, because the parsed result is +thrown away. Use schemas where the input equals the output. Constraints like +`.min()`, `.max()`, and `.refine()` are fine, because they check the value and do +not reshape it. + +The reason is that a synced row is applied on the client the moment it is sent. If +the server changed the row during a write, the stored row would no longer match +the client's copy, and the correction would overwrite it. Changing the primary +key would also break the rule that the optimistic id equals the confirmed id +(ADR-0001 D9). + +## Any Standard Schema works + +The slot accepts any library that implements the +[Standard Schema](https://github.com/standard-schema/standard-schema) interface, +such as zod 3.24 or later, valibot, or arktype. The framework imports no +validator, so you bring your own. To switch libraries, replace `z.object({ ... })` +with the equivalent from your library and change nothing else. + +## Notes + +- **Types follow the schema.** The row type and the argument type come from the + schema you pass, so `op.cols` and `args` are typed exactly as the schema + describes. +- **A failed validation tells the client why.** For both mutations and commands, + a schema failure (or an `authorize` throw) is surfaced to the client with its + reason and a `VALIDATION` code, so you can show the user what was wrong. Only an + `execute` error is sanitized to a generic message. +- **It runs on every matching write,** so keep schemas cheap. + +## See also + +- ADR-0014 for the object schema and why the slot is a gate and not a parser. +- ADR-0012 for wire input hardening; ADR-0014 for how authorize/validation errors + are surfaced (revising ADR-0012 D3). +- `examples/chat` for a collection and command with no schema, as a baseline to + compare against. From 8142df33e2be3fa3518551431e5bb11c92484fdc Mon Sep 17 00:00:00 2001 From: Tom McKenzie Date: Tue, 30 Jun 2026 11:36:01 +1000 Subject: [PATCH 3/3] docs(types): reflect that runtime validation is live on this branch end-to-end-types.md was authored on feat/sync-schema, where the schema only infers types. On this branch runtime Standard Schema validation is wired (5031f7c), so the "runtime check is a follow-up" note no longer holds: the schema infers the type and validates the value. Re-link the validation recipe. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_018DxmkLhbtb5w7oHHiJKprr --- recipes/end-to-end-types.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/recipes/end-to-end-types.md b/recipes/end-to-end-types.md index 603093c..84e2526 100644 --- a/recipes/end-to-end-types.md +++ b/recipes/end-to-end-types.md @@ -45,9 +45,9 @@ Pick whichever you prefer. - Pass the row type as a generic: `sync.collection({ pk: "id", mutations })`. - Infer it from a schema on the insert mutation: `sync.collection({ pk: "id", - mutations: { insert: { schema: Message, execute } } })`. On this branch the - schema only infers the type. A runtime check against the schema is a separate - follow-up. + mutations: { insert: { schema: Message, execute } } })`. The schema infers the + type and also validates the value at runtime. See + `recipes/zod-standard-schema-collections.md`. ## Notes @@ -62,3 +62,5 @@ Pick whichever you prefer. - ADR-0014 describes the schema as the shared contract. - `examples/chat` types its transport and collection from `ChatApi`. +- `recipes/zod-standard-schema-collections.md` covers validating rows and command + args with a schema.