From 918c6ffc226d62bc600bb4f12a571406c5e79375 Mon Sep 17 00:00:00 2001 From: Mario Zechner Date: Fri, 30 Oct 2026 22:57:58 +0000 Subject: [PATCH] docs(chord): slice 10 spec review folded (R1-R6), no automatic worker restarts, app kit malleable by plugins --- packages/chord/docs/next/decisions.md | 49 +++- packages/chord/docs/next/design.md | 31 ++- packages/chord/docs/next/slice-10.md | 332 ++++++++++++++------------ 3 files changed, 250 insertions(+), 162 deletions(-) diff --git a/packages/chord/docs/next/decisions.md b/packages/chord/docs/next/decisions.md index e3061f3c1..de24791fc 100644 --- a/packages/chord/docs/next/decisions.md +++ b/packages/chord/docs/next/decisions.md @@ -1911,7 +1911,7 @@ validation and copies the v2 kernel added around Delta. numbers, e.g. Durable's active tasks) whenever they change; the server stops a worker whose counts were all zero for `idleMs` (default 5 min, checked every second), never while it starts; a stop sends `shutdown` (the host closes), the process tree killed after 5 s. - (D10, "a") A failed worker start uses the kernel's route backoff (1 s doubling to 60 s while holds exist, + (D10, "a"; **superseded** by R2 below: no automatic restarts, no route backoff) A failed worker start uses the kernel's route backoff (1 s doubling to 60 s while holds exist, `resolve_failed` reported); no sticky failure; `workers.state` shows the error and when it is tried again. (D10b, "b") `host.route(Kind, options, ctx)` returns a `RouteHandle` with `retry(address)`: it ends that address's backoff, so `resolve` runs now if the address has holders (a Retry button for a worker that failed to start, wired @@ -1926,6 +1926,47 @@ validation and copies the v2 kernel added around Delta. the in-process server step returns a list of one); the workers step reloads every running worker in parallel; a worker that closed during it is listed with its error and does not stop the rollout; a plugin's dirty mark clears only when it applied on every host of every step. + Spec review (Fable, Opus): (R1, "a") `RouteHandle.suspend(address)`: while suspended the route does not resolve the + address, its holders stay `loading`; `retry(address)` ends the suspension; `workers.stop` suspends, then stops (an + archived session stays stopped while a tab still holds it). + (R2, owner: "take it"; **supersedes D10 and the kernel's route backoff**) **Workers start on demand and never + restart by themselves** (owner: "if it crashes, chances are it had a good reason to crash, and restarting it won't + fix that"; D10 as recommended was a restart loop whenever a consumer holds). A worker starts for a hold of an address + that has not failed (the first, or one after an idle stop or `stop`). A crash or a failed start marks the address + failed with its error until `workers.retry(address)` (`RouteHandle.retry`) or a rollout that reaches workers (new + code may fix it, as a reload retries failed objects); its holders and new holds see `failed` with that error, nothing + forks. Kernel: a route has no backoff timer; it resolves only for a hold, on `retry`, or after its link dropped while + held, and a rejected `resolve` fails the route's holders with its error (no endless `loading`); a network route that + wants to reconnect keeps that policy in its own `resolve`. The workers component owns its route + (`createWorkers(host, Kind, options)` calls `host.route` and keeps the handle) and answers a resolve of a failed + address with the stored failure at once. A start finding the lock still held by a draining old worker waits for it in + the child (bounded by the start timeout). R1's `suspend` and D10b's `retry` stay. + Other timed retries (owner, 2026-10-30: "keep them as is for now"): objects' transient failures (1 s to 60 s while + held), `self.on` handlers that throw (1 s to 60 s while live) and `self.onLive` restarts (the same, reset after a + minute's run) stay; they heal real transient failures (a network down for a while) and cost at most one try a minute + each; revisit if one shows up as a loop in practice. + (R3, "a"; refines D9) The child reports only the flip: `{ idle: true }` when every count (`host.activity` and `busy`) + becomes zero, `{ idle: false }` when one becomes nonzero; the server's idle clock starts at `true` (a status message + costs about 6 µs on each side, a local call 2.9 µs: no report per call). + (R4, "a if configurable at runtime so allowed domains can be added and removed by a plugin") Before `authenticate`, the + component refuses (403) a `Host` that is neither loopback nor allowed, and an upgrade (`/chord`, a plugin socket) + whose `Origin` is not the request's own origin; `request.origin` comes only from an accepted `Host` (`http://` on + loopback, `https://` for an allowed host); `headers(request, principal)` gets the principal. Allowed hosts are the + server option `allowedHosts` plus every `hosts` list in `Http` contributions: a plugin adds and removes its domains + at runtime by contributing a writable (`self.writable`, design §17), the router recomposed on each change (a tunnel or + relay plugin names its public host). + **The app kit is malleable by plugins** (owner, 2026-10-30: "all of this should be malleable via plugins"; "yes"): + in `http`, `workers` (and slice 11's components) every option that is not bootstrap is either a contribution to + `Http` (routes, wraps, sockets, hosts; writables where it changes at runtime) or a function the component calls per + use (`authenticate`, `headers`, a worker's `start`, `idleMs`), which the app backs with its own registries; nothing is + copied out of a contribution or an option at startup. Bootstrap stays base code, changed by a restart: the listen + address and port, the links path, the worker entry module with its `execArgv` and `env`, the host kind workers serve. + (R5, "agree") `workers.state` per address key: `{ status: "starting" | "running" | "stopping", pid, startedAt }` or + `{ status: "failed", error: { message, code?, reason? }, at, stderrTail? }`; no entry when stopped or never started + (a clean stop removes it); `failed` stays until Retry or a rollout; plain JSON (a core object may serve it). + (R6, "b, common enough") `bytes` may omit `size`: chunked transfer encoding, no ranges (no `Accept-Ranges`), `read` + called with growing offsets until it returns an empty chunk (a zip built on the fly, a growing log, an export paged + from a worker). - **A plugin's npm dependencies are bundled; external only by opt-in (owner, 2026-10-29: "explicit opt in + (a)"):** a plugin's `dependencies` are installed into its own `node_modules` (`npm install --ignore-scripts`) and bundled into each facet that imports them (host packages, its `peerDependencies`, stay external and are passed in). On Node a @@ -2626,6 +2667,12 @@ Everything else from earlier lists is decided (see the entries above and `packag - **Future goal: deploy Chord apps to Cloudflare Durable Objects and similar platforms.** Hot reload there is not needed; composition through plugins is (plugin set fixed at deploy). Research in `research/chord-on-durable-objects.md`. - Deferred until a concrete case: jobs as objects and where a flow lives (when login is ported); project plugins. +- **Chord from code mode scripts** (owner, 2026-10-30: "something we want"; p3): the agent's code mode scripts get a + Chord call scope (`host.root`) to call objects and change things: `chord.get("notes.store", {})` by name, every + argument checked by the provider's closed schema. Needs, to design with p3: a principal for the agent (acting as who + runs the session, so views apply their authority; never the owner's powers in a guest's session), a by-name `get` + and a list of served contracts with their members and schemas (the agent's declarations), values crossing the + sandbox boundary as plain data, the scope sealed when the script ends. - **Every object may contribute** (owner, 2026-10-29: "note it down as something to consider for the future; can be changed later without a need for migrations"): lift R21 so objects with params and `remote` objects contribute too; the entry lives while the object is live, its identity is the object's (type, params, principal key); `contribute` diff --git a/packages/chord/docs/next/design.md b/packages/chord/docs/next/design.md index f10c3b56c..f0ff3136b 100644 --- a/packages/chord/docs/next/design.md +++ b/packages/chord/docs/next/design.md @@ -226,8 +226,10 @@ server.route(Session, { Do not pass `context.abortSignal` to `fork`: `host.close` aborts it, which kills the worker and makes Node emit an `error` event that crashes an app without a handler. Chord closes the link at `host.close`; the worker exits when its -parent link closes. `resolve` returns the link before the worker has loaded (the worker connects first, W6): a -`resolve` waiting for the worker's `load` while the worker's facets hold server types would deadlock. +parent link closes. `resolve` returns the link before the worker has loaded (the worker connects first, W6): made +later, the link would leave the worker's `hello` unread and the heartbeat would close it during a long `load` (slice +10's spec review; the deadlock once feared here does not happen: 7a's barrier ends its first pass on a wait on another +host). The workers component (§23) does this. | Call | Meaning | | --- | --- | @@ -236,7 +238,7 @@ parent link closes. `resolve` returns the link before the worker has loaded (the | `reload([{ name, facet }], context)` | the complete new plugin set after `load` (a facet passed as the same object is an unchanged plugin): staging, then one commit, then every object of a changed or removed plugin takes its fate; resolves with a report (a plugin whose new code fails keeps its old code, listed in `failed`); one at a time (section 21, slice 7b) | | `connect(link, { principal, trusted?, codec? }, context)` | the link to the parent, at most one open; once it closed (a drop), `connect` takes the next, and every remote reference over the parent is held again on it. `principal` = who the parent is | | `accept(link, { principal, trusted?, codec? }, context)` | a link from a child; holds and calls on it act as `principal` | -| `route(Kind, { authorize, resolve }, context)` | how to reach hosts of a kind (below); called before `load`, once per kind; work arriving on its links runs under `context`; returns a `RouteHandle` whose `retry(address)` ends that address's backoff (slice 10 D10b) | +| `route(Kind, { authorize, resolve }, context)` | how to reach hosts of a kind (below); called before `load`, once per kind; work arriving on its links runs under `context`; returns a `RouteHandle`: `retry(address)` resolves the address again (D10b), `suspend(address)` stops resolving it until `retry` (R1) | | `get(Type, params, context, { readyTimeout? })` | a reference once its object is live, waiting through `loading`, `failed { transient }` and `failed { waiting }` up to its `readyTimeout` (section 7) | | `hold(Type, params, context, { readyTimeout? })` | a reference at once that follows its object until released (section 7) | | `close(context)` | closes the links (children, routes, then the parent), then ends every object at once and tears them down, the last to go live first (reason `close`); rejects with an `AggregateError` of the native closes that threw | @@ -304,12 +306,14 @@ host.route(Session, { aborted only when the host closes; a link it returns after that is closed at once. A `resolve` whose holds were all released meanwhile finishes and its link is kept (a worker start that produced a link finishes even if its first requester left). A `resolve` that never settles is the app's (it bounds its own `fork`); getters still time out. -- **Failure and re-resolve:** a rejected `resolve` (reported `resolve_failed`), a returned link already closed, or the - address's link closing while holds over it exist, starts the **per-address backoff** (1 s doubling to 60 s; a map - keyed by address, separate from objects' identity backoff). Meanwhile the holds are `loading`. Only a link that - completed its handshake resets the backoff. Without holds for the address nothing is resolved again, so passive - viewers recover without a user action while they hold. `RouteHandle.retry(address)` (slice 10 D10b) ends an - address's backoff, so `resolve` runs at once if it has holders (a Retry button for a worker that failed to start). +- **Failure and re-resolve** (slice 10 R2, owner, 2026-10-30; replaces the per-address backoff): a route has no + timer. A rejected `resolve` (reported `resolve_failed`) or a returned link already closed fails the route's holders + with its error (`failed`, not an endless `loading`); the route resolves again only for a new hold, on + `RouteHandle.retry(address)`, or once after the address's link dropped while holds exist (a drop of a healthy link + recovers at once; whether a crashed worker comes back is the `resolve`'s answer). `RouteHandle.suspend(address)` + (R1) stops resolving an address (its holders stay `loading`) until `retry`. A network route that wants to reconnect + keeps its own policy in its `resolve`. The workers component answers a failed address's `resolve` with its stored + failure at once: workers never restart by themselves (§23). - Chord never closes a route link itself except at `host.close`; stopping idle workers is the app's (`host.activity`). - Work arriving on a route's links runs under the context given to `host.route(Kind, options, context)` (slice 4b). @@ -2149,12 +2153,13 @@ not apply. ### Workers (`@earendil-works/chord/next/workers`) -`createWorkers({ entry, start, plugins, idleMs?, onOutput? }, ctx)` gives a route's `resolve` (a forked worker per +`createWorkers(host, Kind, { entry, start, plugins, idleMs?, onOutput? }, ctx)` routes `Kind` itself (a forked worker per address, IPC, a start message with the app's start data and the worker kind's built scripts, `ready` within 60 s), a `state` watchable per address and `stop(address)`; `workerMain(fn)` is the child's side. One live worker per address -(a kernel lock taken by the child before it opens its store). Keep-alive (D9): the child reports `host.activity` and -the app's `busy` counts; the server stops a worker idle (all zero) for `idleMs` (5 min). A failed start uses the -route's backoff (D10); `RouteHandle.retry(address)` (D10b) ends it. Workers never build (D11): the server's plugins +(a kernel lock taken by the child before it opens its store). Keep-alive (D9, R3): the child reports when `host.activity` +and the app's `busy` counts flip between all zero and not; the server stops a worker idle (all zero) for `idleMs` (5 min). Workers start on demand and +never restart by themselves (R2): a crash or a failed start leaves the address failed with its error until +`workers.retry(address)` or a rollout reaching workers; `workers.stop` suspends the address (R1). Workers never build (D11): the server's plugins component builds every plugin; workers evaluate shipped scripts, and `workers.step(name)` ships a rollout's scripts to every running worker (D12: one report per host). diff --git a/packages/chord/docs/next/slice-10.md b/packages/chord/docs/next/slice-10.md index 5d9eb88d3..d0d63866c 100644 --- a/packages/chord/docs/next/slice-10.md +++ b/packages/chord/docs/next/slice-10.md @@ -1,6 +1,7 @@ # Slice 10: HTTP and workers (spec) -Status: **points decided** (owner, 2026-10-30: D1-D12 in the decision "Slice 10's points"); next: spec reviews. +Status: **ready for the owner's approval** (owner, 2026-10-30: D1-D12, D10b and the spec review's R1-R6 in the decision +"Slice 10's points"; spec reviews by Fable and Opus, `/tmp/s10-spec-review-{fable,opus}.md`, folded in). Slice 10 builds two Node components of the app kit (decision "Slice 7b's open points" (6)): `chord/next/http` (the server's HTTP and WebSocket entry, routes contributed by plugins through a registry, responses as descriptions) and `chord/next/workers` (a worker process per host address, started on demand, stopped when idle, its plugins shipped from @@ -23,30 +24,47 @@ p2's server host (`src/hosts/server.ts`, 742 lines; `src/runtime/server.ts`, 720 | a Node `http` server on loopback, requests parked until `load` resolved, upgrades captured before listening | `http` | | `Routes` (`"METHOD /path"`, `/prefix/*`), handlers writing Node's `ServerResponse` themselves (ranges, download headers, `HEAD` by hand in `files`) | `http`: routes returning descriptions | | `Upgrades` (a WebSocket by path for terminals and voice), `/chord` (`acceptWebSockets` with per-socket limits) | `http` | -| `Authenticators`, `RequestResolvers` (cookies, devices, the Radius relay), loopback and allowed hosts, Fetch Metadata, CSP and the loopback cookie | the app (p3's base code), through `http`'s one `authenticate` hook (D3) | -| `SessionHosts`: a worker per session, `createHostManager` + `forkDriver`, a kernel lock per session, idle stop from reported counts, crash notices, ready timeout | `workers` | +| `Authenticators`, `RequestResolvers` (cookies, devices, the Radius relay), Fetch Metadata, CSP and the loopback cookie | the app (p3's base code), through `http`'s `authenticate` and `headers` (D3) | +| loopback and allowed hosts, the `Origin` check on upgrades | `http` (R4), allowed hosts contributable by plugins | +| `SessionHosts`: a worker per session, `createHostManager` + `forkDriver`, a kernel lock per session, idle stop from reported counts, crash notices, sticky failed starts, ready timeout | `workers` (failures stay until Retry or a rollout, R2) | | relocation and pause of a worker, the catalog, `WorkerInitValues` | the app (built on `workers`' `stop` and start data) | | `hosts/worker.ts`: the child's control channel, lock, start data, connect, load, report counts, shutdown | `workers`' child side | | the dev reload across server and workers, worker plugin lists | `plugins` (7c) plus `workers`' rollout step (§4) | +Every option that is not bootstrap is changeable by plugins (decision "The app kit is malleable by plugins"): either a +contribution to `Http` or a function the component calls per use, which the app backs with its own registries; +nothing is copied out of a contribution or an option at startup. Bootstrap (changed by a restart): the listen address +and port, the links path, the worker entry with its `execArgv` and `env`, the host kind workers serve. + ## 2. `chord/next/http` ```ts const http = await httpServer(server, { - listen: { host: "127.0.0.1", port: 8788 }, - authenticate: (request, ctx) => app.principalOf(request, ctx), // the app's policy: cookies, loopback, relays - headers: (request) => app.securityHeaders(request), // every response's headers (CSP, nosniff, framing) - links: { path: "/chord", limits: { part: 16 * MiB, decoded: 64 * MiB } }, // host.accept for every socket there + listen: { host: "127.0.0.1", port: 8788 }, // bootstrap + allowedHosts: () => app.allowedHosts(), // besides loopback and contributed `hosts` (R4) + authenticate: (request, ctx) => app.principalOf(request, ctx), // D3: the app's policy, per request and upgrade + headers: (request, principal) => app.securityHeaders(request, principal), // every response, also a 404 or a 403 + links: { path: "/chord", limits: { part: 16 * MiB, decoded: 64 * MiB }, maxPayload: 64 * MiB }, + onError: (error) => log(error), // `HttpError`, below }, ctx); -await server.load([{ name: "@app/core", facet: core }, { name: "chord.http", facet: http.facet }, ...plugins], ctx); +// the component's facet goes into the plugins component's core, so every reload passes it unchanged (7c) +await plugins.load(server, { kind: "server", core: [{ name: "@app/core", facet: core }, { name: "chord.http", facet: http.facet }] }, ctx); … -await http.close(ctx); +await http.close(ctx); // stops accepting, closes sockets, waits for running requests (bounded by `closeTimeout`) ``` -- **Order:** the server listens first (plugins see the real port; a page reconnecting while plugins load is not lost), - holds every request and upgrade until `load` resolved, then serves. -- **`Http`**, a registry the component defines and serves through its facet (D1): plugins contribute routes and - sockets; the composed router is followed, each request uses the router as it was when it started. +- **Order:** `httpServer` listens at once (plugins see the real port; a page reconnecting while plugins load is not + lost). A request or upgrade waits in `host.registry(Http, ctx)` (the publication and the router's first `compose`; + 503 after `readyTimeout`); nothing waits for the start barrier (a route's `ctx.get` waits for its object, and `/chord` + holds are placed by the publication gate). +- **Checks before anything else** (R4), in order: the `Host` header is loopback (`127.0.0.1:`, `localhost:`) + or allowed (`allowedHosts()` plus every contribution's `hosts`), else 403; an upgrade's `Origin`, when sent, equals + the request's origin, else 403; `request.origin` is built only from an accepted `Host` (`http://` on loopback, + `https://` for an allowed host). Then `authenticate`. +- **`Http`** (D1): the registry `"chord.http"` (version 1), defined and served by the component's own facet; its + `compose` builds the router from every contribution `{ routes?, wraps?, sockets?, hosts? }`; a contribution may be a + writable (a plugin that adds and removes its tunnel host at runtime). Each request uses the router as it was when it + started. ```ts // plugins/notes/server.ts @@ -58,183 +76,201 @@ export default defineFacet((host, self) => { const notes = await ctx.get(NoteStore, {}); return { kind: "json", value: await notes.export({ id: request.params.id }, ctx) }; }, - "GET /notes/template.md": async (_request, ctx) => fileResponse(pluginFile("template.md"), { download: "template.md" }), + "GET /notes/template.md": async () => fileResponse(pluginFile("template.md"), { download: "template.md" }), }, - sockets: { "/terminal": (socket, request, ctx) => terminals.attach(socket, ctx) }, // D6 + wraps: [{ name: "audit", phase: "observe", wrap: async (request, next, ctx) => audited(await next(request), ctx) }], + sockets: { "/terminal": (socket, request, ctx) => terminals.attach(socket, ctx) }, }); }); ``` -- **A route** is `(request: HttpRequest, ctx: Ctx) => Promise`, run in a `host.root` scope with the - request's principal (`ctx.principal`; `undefined` when `authenticate` found none) and an abort signal that fires when - the client disconnects. `HttpRequest`: `method`, `path`, `params` (from the pattern), `query` - (`URLSearchParams`), `headers` (lowercased, read-only), `origin` (scheme, host, port), and the body (D4). -- **Patterns** (D8): `"METHOD /a/b"`, `:name` segments (`params.name`), a trailing `/*` (`params["*"]`, the rest); - the most specific match wins, segment by segment from the left (exact before `:name` before `*`); two contributions with - one pattern are a conflict reported through `compose` (`registry_conflict`), the first by plugin name wins. `HEAD` - falls back to the `GET` route. -- **Responses are descriptions** (D5): the component writes status, headers and body, so every plugin gets ranges, - `HEAD`, download headers and abort on disconnect from one place: +- **A request's scope:** one `host.root` scope with the request's principal (`ctx.principal`, `undefined` when + `authenticate` found none) spans matching, wraps, the route and the whole write of its response; its signal aborts + when the client disconnects. A socket handler's scope lasts until the socket closes. +- **`HttpRequest`:** `method`, `path`, `params`, `query` (`URLSearchParams`), `headers` (lowercased, read-only), + `origin`, `json(limit?)`, `body(limit?)` (D4). `json` parses at most `limit` bytes (1 MiB by default) into a + `JsonValue`; `body` is an async iterable of `Uint8Array` chunks, failing past `limit`. Both throw an `HttpFailure` + carrying its status (400 malformed, 413 too large); if it escapes the route, that status is the answer (not 500). A + body nobody read is drained up to 64 KiB after the response, else the connection closes (`Connection: close`). +- **Patterns** (D8): exact segments, `:name` (one non-empty segment, percent-decoded) and a trailing `/*` (the rest, + possibly empty: `/files/*` matches `/files`); `params["*"]` is decoded per segment, and a segment that decodes to + contain `/` or is `.` or `..` is answered 400. The most specific match wins, segment by segment from the left (exact + before `:name` before `*`); one pattern from two contributions is a reported conflict (`registry_conflict`), the first + by plugin name wins. `HEAD` falls back to the `GET` route. No route: 404. +- **Wraps** (D2): `{ name, phase, scope?, route?, wrap(request, next, ctx) }` in the phases `observe`, `headers`, + `transform` (outermost first), within a phase in contributor order (plugin name, then type); `scope: "own"` (default: + around the contributing plugin's own routes) or `"all"`; `route` narrows to one pattern (a missing one is a reported + conflict). A wrap adds a boundary on top of a route's checks, never instead of them. A throwing wrap is a 500, + reported naming its plugin. +- **Responses** (D5, R6): ```ts type HttpResponse = { readonly headers?: Readonly> } & ( | { readonly kind: "json"; readonly value: JsonValue; readonly status?: number } - | { readonly kind: "html"; readonly body: string; readonly status?: number } // its CSP via `headers` - | { readonly kind: "bytes"; readonly mime: string; readonly size: number; + | { readonly kind: "html"; readonly body: string; readonly status?: number } + | { readonly kind: "bytes"; readonly mime: string; readonly size?: number; readonly read: (offset: number, length: number, ctx: Ctx) => Promise; - readonly download?: string; // Content-Disposition: attachment with this file name, made safe - readonly ranges?: boolean } // 206/416 and Accept-Ranges + readonly download?: string; readonly ranges?: boolean } | { readonly kind: "redirect"; readonly to: string; readonly status?: 302 | 303 | 307 } | { readonly kind: "error"; readonly status: number; readonly message?: string }); ``` - `bytes` is read in chunks (64 KiB) with the socket's backpressure, stopped when the client disconnects; a route - that throws is a 500 (reported, never its message); one that returns after a disconnect writes nothing. -- **`fileResponse(path, { download?, ranges?, mime? })`**: a `bytes` response over a file (its size from `stat`, `read` through a file - handle opened once and closed when the response ends), for plugin files and downloads. -- **The links endpoint** (`links.path`): an upgrade there is authenticated, then `host.accept(webSocketLink(ws), { - principal, limits })`; a socket without a principal is refused (403). Per-socket output cap as p2's - `limitedSocket` (a client that stops reading is cut off). -- **Wraps** (D2): middleware around routes, below. **Not built:** request resolvers as a registry (D3). - -> **Decided D1 (owner, 2026-10-30: "a"):** the component exports the token `Http` (`"chord.http"`) and a facet the -> app loads with its plugins (`http.facet`, plugin name `chord.http`), which serves the registry (`compose`: the -> router); the server reads it with `host.registry(Http, ctx)`. - -> **Decided D2 (owner, 2026-10-30: "a"):** wraps are built as designed (§23): `wraps: [{ name, phase, scope?, route?, -> wrap(request, next, ctx) }]` in the component's phases `observe`, `headers`, `transform` (outermost first), within a -> phase in contributor order (plugin name, then type), `scope: "own"` by default (around the contributing plugin's own -> routes), `"all"` explicit, `route` narrowing to one pattern (a missing one is a reported conflict). A wrap adds a -> boundary on top of a route's own checks, never instead of them. A wrap that throws is a 500 (reported, naming its -> plugin). The `headers` option stays for every response, also those no route makes (a 404, a refused request). - -> **Decided D3 (owner, 2026-10-30: "a"):** one app function `authenticate(request, ctx) => P | undefined`, called -> once per request and per upgrade; no Chord registry of resolvers: who a request is, and which credential wins, is the -> app's policy (p3 composes it from its own registry). - -> **Decided D4 (owner, 2026-10-30: "k"):** `request.json(limit?)` (parsed JSON, at most 1 MiB by default, malformed: -> 400) and `request.body(limit?)` (an async iterable of `Uint8Array` chunks, for uploads streamed to a file); nothing is -> read before the route asks; past its limit the read stops and the request is answered 413; a client that disconnects -> aborts `ctx`, which ends the iteration. - -> **Decided D5 (owner, 2026-10-30: "a"):** the five kinds above, `headers` on every kind (cookies, a preview's own -> CSP), `html` a full document instead of the design's `page { title, body }`. - -> **Decided D6 (owner, 2026-10-30: "a"):** `sockets: { "/terminal": (socket, request, ctx) => … }`: after the -> origin check and `authenticate` (a socket without a principal is refused, 403, before the plugin sees it), the -> component upgrades and hands the plugin a `ws` WebSocket (its own protocol and flow control, as p2's terminals and -> voice); the plugin authorizes further; patterns as routes; wraps do not apply. The design's channel component waits -> for a second user and would be built on top of this. - -> **Decided D7 (owner, 2026-10-30: "a"):** `ws` is an optional peer dependency of Chord (pinned exactly, the version -> its tests use), imported only by `next-node/http`, as esbuild. - -> **Decided D8 (owner, 2026-10-30: "a"):** `:name` matches one non-empty segment (percent-decoded), a trailing `/*` -> the rest (possibly empty); the most specific match wins, segment by segment from the left (exact before `:name` -> before `*`). + Headers: the `headers` option's first, then the response's own, replacing same-named ones (case-insensitive: an + artifact preview sets its own CSP). `bytes` with a `size`: `Content-Length`; with `ranges`, one range gets 206 and an + unsatisfiable one 416, a multi-range request gets 200 with the whole body; `HEAD` sends headers only; `download` + sets `Content-Disposition: attachment` with an ASCII fallback and `filename*`. `bytes` without a `size` (R6): chunked + encoding, no ranges, `read` called with growing offsets until it returns an empty chunk. Reads are 64 KiB with the + socket's backpressure (`drain`), stopped when the client disconnects; a `read` returning fewer bytes than asked with a + `size` set destroys the connection (reported). A throwing route is a 500 without its message (reported). +- **`fileResponse(path, { download?, ranges?, mime? })`:** `stat`s at once, opens its file handle on the first `read` + and closes it when the request's scope seals, so an unwritten response (`HEAD`, a 416, a disconnect, a wrap that + replaced it) opens nothing or leaks nothing. Without `mime` it picks from a small fixed table by extension (text, + images, audio, video), else `application/octet-stream`; active types (HTML, SVG) only when the route names them. +- **Sockets** (D6): `sockets: { "/terminal": (socket, request, ctx) => … }`, patterns as routes. After the checks and + `authenticate` (no principal: 403 before plugin code), the component upgrades and hands the handler a `ws` + WebSocket with an `error` listener already attached (an oversized frame closes it, never the server); per contribution + options `maxPayload` (1 MiB by default) and no `perMessageDeflate`. Wraps do not apply. +- **The links endpoint:** `links.path` upgrades to `host.accept(webSocketLink(socket), { principal, limits }, ctx)`, + with a per-socket output cap (p2's `limitedSocket`: a client that stops reading past 4 MiB unsent is cut off) and the + transport's `maxPayload`. +- **Errors** (`onError`, `HttpError`): `route_failed { pattern, plugin, error }`, `wrap_failed { name, plugin, error }`, + `socket_failed { pattern, plugin, error }`, `response_failed { pattern, error }` (a short `read`, a write error). +- `ws` is an optional peer dependency (D7), imported only here. ## 3. `chord/next/workers` ```ts // server.ts -const workers = createWorkers({ - entry: "./worker.ts", // the child's entry module - start: (address, ctx) => app.startData(address, ctx), // the app's start data (JSON), before each start - plugins, // the plugins component: the worker kind's scripts - idleMs: 300_000, +const workers = createWorkers(server, Session, { + entry: "./worker.ts", execArgv: ["--import", resolver], env: { … }, // bootstrap + authorize: (principal, address) => app.mayReach(principal, address), // the route's `authorize` + principal: (address) => ({ kind: "worker", session: address.params.id }), + start: (address, ctx) => app.startData(address, ctx), // JSON, before each start + lock: (address) => `app:${realpath(dataDir)}:worker:${address.key}`, // one live worker per address + idleMs: () => app.idleMs(), + plugins, // the plugins component onOutput: (address, stream, chunk) => logs.append(address.key, stream, chunk), + onError: (error) => log(error), // `WorkersError`, below }, ctx); -const sessions = server.route(Session, { authorize: app.authorizeSession, resolve: workers.resolve }, ctx); -sessions.retry(Session({ id })); // a Retry button: ends the address's backoff (D10b) -workers.state // JSON watchable: { [address key]: { status, pid?, startedAt?, error? } } -await workers.stop(Session({ id }), ctx); // archive; the next hold starts it again +workers.state // JSON watchable (R5) +await workers.stop(Session({ id }), ctx); // archive: suspended, then stopped; a held reference just waits +await workers.retry(Session({ id }), ctx); // the Retry button: a failed or suspended address resolves again +await workers.close(ctx); // after `server.close`: no new starts, every child stopped ``` ```ts // worker.ts await workerMain(async ({ address, start, parent, plugins }, ctx) => { + const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(join(start.dir, "session.sqlite"))), ctx); const worker = createHost

({ address, principal: { kind: "worker", session: start.id }, store, onError }); worker.connect(parent, { principal: { kind: "server" }, trusted: true }, ctx); // W6: connect, then load await plugins.load(worker, { kind: "worker", core: [{ name: "@app/core", facet: coreWorker(start) }] }, ctx); - return { host: worker, busy: harness.activeTasks }; // extra keep-alive counts (Durable tasks) -}); + return { host: worker, busy: harness.activeTasks, close: (ctx) => store.close(ctx) }; +}, { onError: (error) => log(redact(error)) }); ``` -- **`resolve(address, ctx)`** (a route's `resolve`): the address's running worker's link, or starts one: forks `entry` - (`serialization: "advanced"`, detached process group, killed as a tree), sends the start message (`address`, the - app's start data, the worker kind's built scripts), waits for `ready` within `readyTimeout` (60 s), and returns - `{ link: ipcLink(child), principal, trusted: true }`. One start at a time per address; a failed start rejects (the - route's backoff and `resolve_failed` are the kernel's: D10). -- **One live worker per address** across processes (a server restart while an old worker drains): the child takes a - kernel lock named after the app's data and the address before it opens its store (v1's `processLock`, ported); a - second fails its start `unavailable/locked`. -- **Keep-alive** (D9): the child reports `host.activity` (`references`, `calls`, `jobs`, `lingering`) and the app's - `busy` counts whenever they change; the server stops a worker whose counts were all zero for `idleMs`, and never one - that is starting. A stop sends `shutdown`, the child closes its host (`host.close`) and exits; after a grace period - (5 s) the process tree is killed. -- **Exits:** a worker that exits unasked (a crash, an OOM) is reported (`worker_exited { address, code, signal, stderr - tail }`) and its link closes; its holders go `loading` and the route resolves again while holds exist (the kernel's - backoff). `state` shows the last error until the next start succeeds. -- **The child** (`workerMain`): reads the start message, takes the lock, makes `ipcLink(process)`, calls the app's - function, sends `ready` once it returned (after `load`), reports counts, handles `shutdown` and the parent's - disconnect (close the host, exit), and reports an uncaught error as `fatal` (redacted by the app's `onError`). -- **Plugins in workers** (D11): workers never run esbuild. The start message carries the worker kind's current scripts - (`Built`, plain data, 7c); the child's `plugins` is the evaluating half of the plugins component (no build, no - watcher, no esbuild import): `load` evaluates them, `reload` evaluates new ones and calls `host.reload` per plugin, - as on the server. +- **The route:** `createWorkers` calls `host.route(Kind, { authorize, resolve }, ctx)` itself and keeps the + `RouteHandle` (`retry`, `suspend`: D10b, R1). +- **A start** (`resolve(address)`): one queue per address orders starts, stops and exits (a start waits for the + previous child's exit; a stop during a start waits for it, then stops; an aborted start kills its child). A start + calls `start(address, ctx)` (a throw rejects the start), forks `entry` (`serialization: "advanced"`, a detached + process group, stdout and stderr piped: the stderr tail kept, mirrored to the server's when there is no `onOutput`), + makes `ipcLink(child)` at once (so the child's `hello` and early holds are queued, never lost), and sends the start + message `{ address, start, scripts, lock }` (`scripts`: the worker kind's complete current scripts, below). The child + takes the lock (waiting for a draining old worker to let it go, within the start timeout), connects and sends + `started`; `resolve` then returns `{ link, principal: principal(address), trusted: true }`, so the handshake and the + worker's server traffic run during its `load`. `ready` (after the app's function returned) ends the start timer (60 + s from the fork; past it the tree is killed and the start fails). +- **Workers never restart by themselves** (R2): a crash (an exit nobody asked for) or a failed start (a `start` that + threw, a lock not taken in time, `fatal`, the start timer) leaves the address `failed` with its error + (`worker_failed` reported once); `resolve` of a failed address rejects at once with that error, so its holders and + new holds see `failed`, and nothing forks. `workers.retry(address)` (`RouteHandle.retry`) or a rollout that reaches + workers clears it. An idle stop or `stop` is no failure: the next hold starts it on demand. +- **Keep-alive** (D9, R3): the child watches `host.activity` and the app's `busy` (a JSON watchable of numbers) and + sends `{ idle: true }` when every count becomes zero, `{ idle: false }` when one becomes nonzero; a rollout applying in + the child counts as busy. The server stops a worker idle since `idleMs()` (5 min by default; checked every second), + never while it starts. A stop (idle, `stop`, `close`): `shutdown`; the child closes its host, runs the app's `close`, + exits; it forces its own exit after 3 s; the server kills the tree after 5 s. +- **`state`** (R5): per address key `{ status: "starting" | "running" | "stopping", pid, startedAt }` or `{ status: + "failed", error: { message, code?, reason? }, at, stderrTail? }`; no entry when stopped or never started. +- **`stop(address, ctx)`:** `RouteHandle.suspend(address)` (holders stay `loading`, nothing resolves it), then the + stop; `retry(address)` lifts the suspension. **`close(ctx)`:** new starts reject `unavailable/closed`, every child is + stopped (grace, then kill), the timers cleared; `createWorkers`'s `ctx` aborting closes it too. +- **The child** (`workerMain(fn, { onError })`): reads the start message, takes the lock, makes `ipcLink(process)`, + calls `fn({ address, start, parent, plugins }, ctx)`; `started` once connected, `ready` once `fn` returned; reports + `idle` flips; handles `shutdown`, the parent's disconnect, `SIGTERM` and `SIGINT` alike (close the host, the app's + `close`, exit); an uncaught error is sent as `fatal { message, code?, reason? }` through `onError` (the app redacts), + then the child exits. +- **Control messages** (JSON beside the link's `Uint8Array` frames; the IPC link skips them): parent to child `start`, + `shutdown`, `reload { scripts }`; child to parent `started`, `ready`, `idle { idle }`, `fatal { error }`, `reloaded + { report } | { error }`. Errors cross as `{ message, code?, reason? }` (a structured clone keeps no `ChordError`). +- **Errors** (`onError`, `WorkersError`): `worker_failed { address, error, stderrTail? }`, `worker_exited { address, + code, signal }` (a clean exit after a stop is not reported). + +### Plugins in workers (D11) + +Workers never build. The plugins component (7c) gets `scripts(kind, ctx)`: the complete current set of one kind (every +discovered plugin's facet of it, built on first use, then updated by every rollout that ships that kind), each with its +plugin record (`name`, `dir`, `chord.external`). The component's per-host loading (its loaded map, `#prepare`, `adopt`, +the queue per host) moves into a file without esbuild or the watcher, `createPluginRuntime({ hostPackages, onError })` +with `load(host, { kind, core, scripts }, ctx)` and `reload(host, shipped, ctx)`; `createPlugins` uses it, and the +child's `plugins` is one (`import` of `chord/next/plugins/runtime` loads no esbuild). ## 4. The rollout across server and workers ```ts const built = await plugins.build(plugins.state.value.dirty, ctx); const result = await plugins.rollout([plugins.step("server", server), workers.step("workers")], built, ctx); +// result.steps: [{ name: "server", hosts: [{ host: "server", report }] }, +// { name: "workers", hosts: [{ host: "session|S1", report }, { host: "session|S2", error }] }] ``` -`workers.step(name)` ships `built` (its worker kind's facets and plugin records) to every running worker over its -control channel; each evaluates and reloads; the step resolves with one report per worker (`RolloutStep.reload` -returns `{ hosts: [{ host, report }] }`, D12). A worker starting during or after it gets the current scripts in its -start message. A worker that fails to apply (closed, aborted) is listed and does not stop the rollout; a plugin kept old -on any worker stays dirty (7c's rule). - -> **Decided D9 (owner, 2026-10-30: "a"):** Chord's `host.activity` plus the app's `busy` counts (a JSON watchable of -> numbers, e.g. Durable's active tasks), reported to the server when they change; `idleMs` default 5 min, checked every -> second; never while the worker starts. - -> **Decided D10 (owner, 2026-10-30: "a"):** the kernel's route backoff (1 s doubling to 60 s while holds exist, -> `resolve_failed`), no sticky failure; `state` shows the error and when the next try is. -> -> **Decided D10b (owner, 2026-10-30: "b"):** a kernel change: `host.route(Kind, options, ctx)` returns a `RouteHandle` -> with `retry(address)`, which ends that address's backoff so `resolve` runs now if the address has holders (nothing -> without holders); the app wires its Retry button to the handle (`sessions.retry(Session({ id }))`); the workers -> component has no `retry` of its own. - -> **Decided D11 (owner, 2026-10-30: "a"):** workers never build; the server discovers and builds every plugin, one set -> for every host; a worker's start message carries its kind's scripts and a reload ships new ones. Plugins are defined -> where the server can read them (owner: "no cwd specific plugins, which might be alright; that way workers can live -> anywhere"). A worker on another machine needs the plugin directories only for `chord.external` packages and -> `pluginFile` (both resolve in the plugin's directory). - -> **Decided D12 (owner, 2026-10-30: "a"):** `RolloutStep.reload` returns `{ hosts: [{ host, report }] }` (7c's -> in-process step returns a list of one); workers reload in parallel; one that closed during the step is listed with its -> error and does not stop the rollout. +`RolloutStep.reload(built, ctx)` returns `{ hosts: readonly ({ host: string; report: ReloadReport } | { host: string; +error: { message, code?, reason? } })[] }` (D12; 7c's in-process step returns a list of one, its `host` the name given to +`step`). `workers.step(name)` first replaces the workers' current set (`plugins.scripts`), then ships the change to +every worker whose start message or last reload carried an older set, starting ones included (a child queues a reload +that arrives before its `ready` until its load is done); workers reload in parallel; one that closed or failed during +the step is listed with its error and does not stop the rollout, and every plugin of its kind in the build counts as +failed there (stays dirty). A rollout that reaches workers clears every failed address (R2). A plugin's dirty mark +clears only when it applied on every host of every step. ## 5. Not in slice 10 The web page host, the browser's evaluation of scripts and serving them (slice 11); the TUI (slice 11); relocation and -pause of workers (the app's); request resolvers and authenticators as registries (the app's); chunked -transfers as a Chord helper (an owner pattern, design §23, unchanged); telemetry spans for requests (slice 9). +pause of workers (the app's: `start(address)` is where the app awaits its own barrier); request resolvers and +authenticators as registries (the app's); chunked transfers as a Chord helper (an owner pattern, design §23); the +channel component (D6); telemetry spans for requests (slice 9). -## 6. Tests, examples, cost +## 6. Kernel changes -- One test file per source file. HTTP: listen-before-load holding requests and upgrades; authentication per request and - upgrade; patterns and precedence; conflicts reported; wraps (phases, order, scopes, a missing route, a throwing wrap); every response kind (ranges 206/416, `HEAD`, download names - made safe, a disconnect stopping `read`); bodies past their limit; a throwing route (500, reported); the links - endpoint (`host.accept`, a refused socket, the output cap); plugin sockets; a reload swapping a route while a request - runs on the old router; close. Workers: a start on the first hold, the lock, `ready` timeout, idle stop and a hold - keeping it, `busy` counts, a crash (reported, re-resolved while held), `stop`, `RouteHandle.retry` (D10b), the rollout step (a worker - starting during it gets the new scripts), the child's shutdown and disconnect. -- GC: requests (their root scopes and bodies) and stopped workers' state collected; no listener left on a closed socket. -- Examples (each a folder, as 7c's): `40-http` (routes, a download with ranges, a socket), `41-workers` (a server - routing a session kind to forked workers, idle stop, a crash and recovery, a rollout reaching the workers). -- Benchmark: a route's overhead per request against a plain Node handler; a worker's cold start (fork to `ready`) with - shipped scripts against building in the child. +- `host.route(Kind, options, ctx)` returns a `RouteHandle { retry(address), suspend(address) }` (D10b, R1). +- A route has no backoff timer (R2): a rejected `resolve` or a link closed at once fails the route's holders with its + error (`failed`, not `loading`); the route resolves again only for a new hold, on `retry`, or once after the + address's link dropped while holds exist; a suspended address resolves nothing until `retry`. +- Design §3's paragraph on a `resolve` waiting for the worker's `load` (a deadlock) is updated: 7a's barrier ends its + first pass on a wait on another host, and `resolve` returns the link before `load` anyway. + +## 7. Tests, examples, cost + +- One test file per source file. HTTP: listening before the publication (a request waits, then is served; 503 past + `readyTimeout`); `Host` and `Origin` checks, allowed hosts from the option and from a writable contribution changing + at runtime; authentication per request and upgrade; patterns, decoding, precedence, `HEAD`, 404; conflicts reported; + wraps (phases, order, scopes, a missing route, a throwing wrap); every response kind (ranges 206/416, multi-range, + `HEAD`, download names, chunked without `size`, a short `read`, a disconnect stopping `read`, header merging); + `json`/`body` limits and their statuses, an unread body; a throwing route; the links endpoint (`host.accept`, a + refused socket, the output cap); plugin sockets (an oversized frame closes only that socket); a reload swapping a + route while a request runs on the old router; `fileResponse` closing its handle on every path; `close`. Workers: a + start on the first hold; the link made at the fork (a load longer than the heartbeat's 30 s binds); the lock (a + draining old worker waited for); the start timer; `start` throwing; a crash and a failed start staying failed (no + fork for a new hold), `retry` and a rollout clearing it; idle stop and a hold keeping it; `busy`; the `idle` flips (no + message per call); `stop` with holders (suspended: no restart); stop during start and start during stop; `close`; + the rollout step (a worker starting during it gets the new set; a failed worker listed); the child's shutdown, + disconnect and signals. Kernel: `RouteHandle.retry` and `suspend`; no re-resolve without a hold, a retry or one drop. +- GC: requests (their scopes, bodies, file handles) and stopped workers' state collected; no listener left on a closed + socket; 1,000 addresses started and stopped leave no `state` entries. +- Examples (each a folder, as 7c's): `40-http` (routes, a wrap, a download with ranges, a socket, an allowed host + added at runtime), `41-workers` (a server routing a session kind to forked workers, idle stop, a crash staying failed + until Retry, a rollout reaching the workers). +- Benchmark: a route's overhead per request against a plain Node handler (bar: within 10% of a keep-alive round + trip; Opus measured 2.5 µs of 54 µs); `bytes` throughput for a 100 MiB file against `pipeline`; a worker's cold start + (fork to `ready`) with shipped scripts; memory per idle worker on the server.