diff --git a/packages/chord/docs/next/http-workers/book.md b/packages/chord/docs/next/http-workers/book.md new file mode 100644 index 000000000..9b9992337 --- /dev/null +++ b/packages/chord/docs/next/http-workers/book.md @@ -0,0 +1,377 @@ +--- +title: Serving and Spawning: HTTP and workers in Chord v2 +cover: Serving / and Spawning +kicker: An explainer · Chord v2 slice 10 · 2026-10-31 +subtitle: How a request reaches a plugin's route, who it acts as, and how a worker process per session starts, stops and fails +stats: + - 1 | registry plugins contribute routes, wraps, sockets and hosts to + - 0 | automatic worker restarts + - 5 min | idle before a worker stops (default) + - 6 | security rules on by default +note: | + Written by the redesign session from `slice-10.md`, the decision entries "Slice 10's points", "Origins only from + relays", "The HTTP component's security baseline" and their neighbours in `decisions.md`, design §3 and §23, and the + build in `/home/badlogic/pi-7b`. The build exists; its fix round (the security baseline among it) is next. Where the + build and the decisions differ, this book says what is decided. +colophon: Serving and Spawning · first edition · 2026-10-31 · Chord v2 slice 10 +--- + +# Before we begin {.preface} + +Mario runs a small chat app on his laptop. Each conversation is a **session**, and each session runs in its own +process, a **worker**, so a session that crashes or eats memory takes nothing else down. A browser tab talks to the +server; the server talks to the workers. Plugins add features: one adds a chat panel, one adds file downloads, one +adds a terminal. Mario edits plugins while the app runs, and sometimes an agent edits them for him. + +Three things have to work for him without thought. A plugin adds an HTTP route, and it is served safely: no other +website open in his browser can read his data or act as him through it. A session's worker exists exactly while +someone uses it. And when a worker crashes, nothing restarts it in a loop: he sees why it failed and decides. + +Slice 10 builds two components for this: `chord/next/http` (the server's HTTP and WebSocket entry) and +`chord/next/workers` (a worker process per session). Both are Node code on top of Chord's kernel; the kernel itself +stays portable. + +## Eight things to know + +:::findings +1. **Plugins contribute routes, Chord serves them.** The HTTP component defines one registry, `Http`; plugins + contribute routes, wraps (middleware), WebSocket handlers and allowed hosts to it, and every reload swaps them. +2. **A route returns a description.** JSON, HTML, bytes, a redirect or an error; the component writes it, with ranges, + `HEAD`, download headers and backpressure done once for every plugin. +3. **Security is built in.** Where a request came from, whether another website sent it, and which headers every + response carries are decided by the component before any plugin code runs. The app only says who someone is. +4. **A worker starts on the first use of its session** and stops after five idle minutes. The server forks it; the + worker connects back and loads its plugins. +5. **A worker never restarts by itself.** A crash or a failed start leaves the session failed with its error until a + person presses Retry or rolls out new code. +6. **Workers never build plugins.** The server builds every plugin once; a worker evaluates the scripts it is sent. +7. **A rollout reaches every host.** The server reloads first, then every running worker, each reporting its result. +8. **Everything but the bootstrap is changeable by plugins.** Routes, hosts, start data, idle times: contributions or + functions read at each use. +::: + +## Words used throughout + +A **route** is a function a plugin contributes for a method and path pattern. A **wrap** runs around routes. A +**socket** is a WebSocket a plugin handles with its own protocol; a **link** is Chord's own WebSocket at `/chord`, +over which tabs hold objects. **Admission** is what the component checks before any plugin code. **Provenance** is +where a request came from (local, an allowed host, a relay). A request is **foreign** when a browser sent it for +another website. A **worker** is a process serving one **address** (`Session({ id })`). A worker's **start message** +carries its start data and its plugins' **scripts**. A **rollout** applies a build to the server and then the workers. + +# The App We Build {#app lede="A server, workers, two plugins: the files a reader needs to follow the rest."} + +The whole app is four files of its own plus its plugins. The server creates its host, the HTTP component and the +workers component, then loads its plugins: + +```ts +// server.ts +const server = createHost
({ address: Server(), principal: { kind: "server" }, onError }); +const plugins = createPlugins({ + roots: ["./plugins"], + kinds: { + server: { platform: "node", hostPackages: ["@earendil-works/chord/next", "@app/core"] }, + worker: { platform: "node", hostPackages: ["@earendil-works/chord/next", "@app/core"] }, + }, + onError, +}, ctx); +const http = await httpServer(server, { listen: { host: "127.0.0.1", port: 8788 }, authenticate, onError }, ctx); +const workers = createWorkers(server, Session, { + kind: "worker", entry: "./worker.ts", plugins, principal: (a) => ({ kind: "worker", session: a.params.id }), + start: (address, ctx) => ({ id: address.params.id, dir: sessionDir(address) }), + lock: (address) => `chat:${dataDir}:${address.key}`, authorize: mayReach, onError, +}, ctx); +await plugins.load(server, { kind: "server", core: [core, { name: "chord.http", facet: http.facet }], modules }, ctx); +``` + +```ts +// worker.ts: one per session +await workerMain(async ({ address, start, parent, plugins, onClose }, ctx) => { + const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(join(start.dir, "session.sqlite"))), ctx); + onClose((ctx) => store.close(ctx)); + const worker = createHost
({ address, principal: { kind: "worker", session: start.id }, store, onError });
+ onClose((ctx) => worker.close(ctx));
+ worker.connect(parent, { principal: { kind: "server" }, trusted: true }, ctx);
+ await plugins.load(worker, { kind: "worker", core: [workerCore(start)] }, ctx);
+ return { busy: { tasks: 0 } };
+}, { hostPackages: modules, redact: (error) => ({ message: "worker failed" }), onError });
+```
+
+A plugin is a directory with a `package.json` and one file per kind it needs. The chat plugin keeps its messages in a
+worker (each session has its own) and contributes one route on the server for exporting them:
+
+```ts
+// plugins/chat/server.ts
+export default defineFacet((host, self) => {
+ self.contribute(Http, {
+ routes: {
+ "GET /s/:session/export": async (request, ctx) => {
+ if (!(await mayView(ctx.principal, request.params.session, ctx))) return { kind: "error", status: 403 };
+ const chat = await ctx.get(Chat, { session: request.params.session }); // reaches the session's worker
+ return { kind: "json", value: await chat.export(ctx) };
+ },
+ },
+ });
+});
+```
+
+`ctx.get(Chat, { session })` is the one line that connects the two halves: `Chat`'s host address is the session's
+worker, the server's route for `Session` finds (or starts) that worker, and the call goes over its link.
+
+# A Request, Start to End {#request lede="What happens between a TCP connection and the route's response, in order."}
+
+Every request takes the same path. The order matters: nothing a plugin wrote runs before the component decided that
+the request may run at all, and as whom.
+
+:::flow wide animate | One request through the HTTP component. Everything left of `authenticate` is the component's own; plugins start at the router.
+row "Admission" accent
+box prov "Provenance" "local, allowed host or relay; else 403"
+box foreign "Foreign?" "another website's browser request"
+box origin "Origin" "foreign Origin 403; unsafe methods need ours"
+row "Who" blue
+box auth "authenticate" "the app: cookies, devices"
+box anon "anonymous" "foreign or no credential" muted
+row "Plugins"
+box router "Router" "pattern → route, as of now"
+box wraps "Wraps" "observe, headers, transform"
+box route "Route" "returns a description"
+row "Answer" muted
+box write "Write" "headers, ranges, chunks, backpressure"
+arrow prov -> foreign
+arrow foreign -> origin
+arrow origin -> auth
+arrow origin -> anon "foreign"
+arrow auth -> router
+arrow router -> wraps
+arrow wraps -> route
+arrow route -> write
+:::
+
+**Provenance** answers "where from". A request is local only if three things hold: the socket arrived on a loopback
+address, the `Host` header names loopback with the port, and no proxy header (`Forwarded`, `X-Forwarded-*`) is present.
+Anything else must name a host the app allowed (a tunnel's public name) and is then treated as remote https. Anything
+else is refused before a single plugin line runs. The security audit found that checking the `Host` header alone let
+anyone reaching the port claim to be local: the socket address is the part a remote client cannot fake.
+
+**Foreign** answers "did another website make the browser send this". Browsers say so in `Sec-Fetch-Site`. A foreign
+request never reaches `authenticate`: it runs anonymous, whatever cookies it carries. That one rule stops a page on
+another site from reading Mario's data through an `` or `
+