mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
feat(chord): Chord v2 slice 10, HTTP and workers
chord/next/http: a Web fetch handler built from the registry Http (routes, wraps, sockets, hosts, relays, CSP sources) and Authenticators, responses as descriptions, a security baseline on by default, Chord's own Node listener as a host-lifetime object, nodeRequest for apps mounting it, links and plugin sockets. chord/next/workers: a forked worker per address, started on demand, stopped when idle, never restarted by itself, plugins built by the server and shipped. Plugins: kinds with platform and host packages, createPluginRuntime, scripts(kind), rollout results per host, imports of other plugins by name. Kernel: RouteHandle (retry, suspend), no route backoff, anonymous host.root, origins only from relays. ws is an optional peer dependency.
This commit is contained in:
Mario Zechner
parent
81f3cd4127
commit
8959ad6e68
143 changed files with 11066 additions and 561 deletions
6
package-lock.json
generated
6
package-lock.json
generated
|
|
@ -5568,11 +5568,15 @@
|
|||
"node": ">=22.19.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"esbuild": "0.28.2"
|
||||
"esbuild": "0.28.2",
|
||||
"ws": "8.21.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"esbuild": {
|
||||
"optional": true
|
||||
},
|
||||
"ws": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
|
|
|
|||
|
|
@ -17,6 +17,7 @@
|
|||
- Chord v2 has a package export again, `@earendil-works/chord/next` (experimental): every layer's API for apps (contracts, data, documents, hosts, jobs, links, state, the store), and, marked internal, the store's surface for adapters (`StoreOwnerTables`, `OwnerTable`, the documents engine's reads and writes, the commit step, `applyBatch`). v2's store gains one id allocator for every id in it, claims per table and per document namespace, Durable's Storage semantics for a refused commit (it rejects that commit only; an unknown outcome stops the store), document histories `none`, `latest` and `rewindable` with reads at a point, archives carrying document revisions, and batch apply; v2 objects gain `self.follow` (another runtime's synchronous observers as sources) and persisted jobs (`persist: { resume }`, a stored `abort(id, ctx)`, deferred state changes).
|
||||
- Chord v2 (`@earendil-works/chord/next`) gains registries, writables, wake signals and root scopes (slice 8): `defineRegistry<C, V>(name, { version })` tokens shared by plugins, served by the defining plugin's facet with `host.serve(R, { compose, accepts? })`; contributions from facets (`defineFacet((host, self) => self.contribute(R, value))`) and host-lifetime objects (`StoreSelf.contribute`), plain values or Chord watchables, in a table sorted by plugin name and composed lazily (once at the end of `load`'s setups, then when read or followed, in the batch that changed the table); readers through `self.registry(R)` (a declared dependency: an object activates once the registry has a value, and fails `transient` with the new reason `unavailable/undeclared` when nobody serves it) and `host.registry(R, ctx)`, a `RegistryValue<V>` with `observe(observer, ctx)`; older contributions and readers adapted by `accepts`; the helpers `keyedEntries` and `orderedEntries`; the `onError` kinds `registry_conflict`, `registry_failed`, `contribution_failed` and `contribution_refused`. `self.writable(initial)` is an in-process value an attempt sets (identity compared); native sources (writables, registry values, `ref.state`) are derive sources. `self.retryOn(...sources)` retries a failed object when a registry value changes. `host.root(fn, context, { principal? })` runs code outside objects with a handler's `ctx`, sealed when `fn` settles, `context` aborts or the host closes.
|
||||
- Chord v2 gains plugins (slice 7c): `src/next-node/plugins` (Node, to be exported as `@earendil-works/chord/next/plugins`) finds plugin directories, builds each facet with esbuild (now an optional peer dependency of `@earendil-works/chord`, imported only there), evaluates it with `vm.Script`, maps plugin stack traces (`mapStack`), installs plugin dependencies, watches plugin and base code, and rolls reloads out across hosts. `host.load` and `host.reload` work per plugin: a plugin whose setup, factory or declarations fail is left out at `load` (`{ failed }`) or kept on its old code at `reload` (`failed` in the report, `reload(facets, ctx, { failed })` for build and evaluation failures), with a closure rule keeping a consumer old when its provider was kept old; an abort after a reload's commit rejects with `details.committed`.
|
||||
- Chord v2 gains HTTP and workers (slice 10): `src/next-node/http` (to be exported as `@earendil-works/chord/next/http`) builds a Web-standard `fetch` handler from the routes, wraps, sockets, hosts, relays and CSP sources plugins contribute to the registry `Http`, with authenticators from the registry `Authenticators`, responses as descriptions (`json`, `html`, `bytes` with ranges, `redirect`, `error`; `fileResponse`), and a security baseline on by default (provenance by the socket and `Host`, foreign requests anonymous, `Origin` checks, default headers with a per-response CSP nonce, signed cookies, safe downloads); Chord's own Node listener as a host-lifetime object, `nodeRequest` for apps mounting it on their own Node server, the links endpoint and plugin sockets with a heartbeat, closed at once when their contribution leaves. `ws` is an optional peer dependency, imported only there. `src/next-node/workers` (`createWorkers`, `workerMain`) runs a forked worker per address of a host kind, started on demand, stopped when idle, never restarted by itself (a failure stays until `retry` or a rollout), its plugins built by the server and shipped. The plugins component defines one definition per facet kind (`kinds: { name: { platform, hostPackages } }`, modules given at `plugins.load(host, { kind, core, modules })`), splits loading into `createPluginRuntime()` (no esbuild), offers `scripts(kind)`, rolls out with one result per host (`RolloutStep` returns `{ hosts }`), and resolves cross-plugin imports by plugin name (`@app/notes/contract`). Kernel: `host.route` returns a `RouteHandle` (`retry`, `suspend`) and a route has no backoff timer (a failed `resolve` fails its holders `waiting`); `host.root(fn, ctx, { principal: undefined })` runs anonymous; a link reads `origin` only on parent and route links (a protocol error on a child link) and drops a per-origin record once it is idle.
|
||||
- Chord v2, experimental and not exported yet: the next object runtime in `src/next` (contracts, hosts and objects, documents, derives, links and relays, routines and jobs, the store), with its tests and runnable examples in `test/next` and its design in `docs/next` (`design.md`, the decision log `decisions.md`, the slice plan `slices.md`). Its API changes with every slice until it gets an entry point.
|
||||
- `NativeWatchable<T>`: an in-process value that changes and can be watched (`value`, `watch(context)` returning a `NativeWatchHandle<T>`), the in-process counterpart of `Watchable<T>`.
|
||||
- `WatchHandle<T>` and `WatchEnd`, the one watch API, moved from Durable: a serialized observation with `value`, `start(listener)`, `stop()`, and `closed`; the listener gets a `WatchChange<T>` (`{ value, ops, retired }`) and the context. `WatchEnd` reasons are `stopped`, `cancelled`, `ended`, `listener_error`, and `failed`.
|
||||
|
|
|
|||
|
|
@ -914,3 +914,46 @@ derive commit 10.34-10.36 → 10.43-10.49 µs (14,899 → 14,901 B), activation
|
|||
0.026-0.030 → 0.028-0.033 ms (pinned 0.030-0.035 → 0.030-0.032 ms); retained per hold equal (one binary hold 176.2 →
|
||||
170.1 KiB). `reload.ts`: swap of 50 / 1,000 objects 1.44-1.73 / 30.83-32.71 → 1.52-1.53 / 30.55-33.25 ms, restart
|
||||
1.01-1.05 / 15.54-16.87 → 0.95-0.99 / 15.90-16.02 ms until the reload resolves.
|
||||
|
||||
## Slice 10: HTTP and workers (2026-10-31)
|
||||
|
||||
`test/next/bench/http.ts` and `test/next/bench/workers.ts`, Node 24.4.1, CPU 6 (the forked workers inherit the pin).
|
||||
|
||||
| Measurement | Result |
|
||||
| --- | --- |
|
||||
| keep-alive round trip, a JSON body: plain Node handler / a route (same process as the client) | build: 42.0-42.5 / 46.4-47.0 µs: +4.2-4.9 µs (9.9-11.7%); fix round 1: 42.5-42.9 / 62.8-64.4 µs: +20.3-21.5 µs |
|
||||
| the same after 30,000 warm-up requests (fix round 1) | 30.9-31.3 / 43.4-44.0 µs: +12.1-12.9 µs; with an authenticator reading `request.headers`: +15.9 µs |
|
||||
| the same, server and client in two processes (CPU 6 and 8; build) | 36.4-37.0 / 41.7-42.4 µs: +5 µs |
|
||||
| 100 MiB file: `pipeline` of a read stream / `fileResponse` | build (64 KiB reads): 602-618 / 487-565 MiB/s; fix round 1 (256 KiB, pulled): 477-751 / 560-611 MiB/s |
|
||||
| a worker's cold start with one shipped script: fork to `started` / to `ready` (median of 7) | 531-532 / 538-539 ms |
|
||||
| the server's heap per idle worker (20 workers, one reference each) | 49.5 KiB |
|
||||
|
||||
A request's overhead was 16 µs at first: `host.root` cost about 7 µs of it per request (a linked abort signal, a
|
||||
`Promise.race` and an `awaitWithContext` per scope). Root scopes now take the caller's signal with one listener and
|
||||
settle one promise (`root.ts`; behaviour and tests unchanged): 4.2-4.9 µs, about the bar of 10% of a round trip. The
|
||||
rest is the request's scope (its abort controller, `CallContext`), admission and the response's header merge. The cold
|
||||
start is mostly Node's startup stripping the types of the source tree it imports; the start timer (60 s) is far from
|
||||
it.
|
||||
|
||||
**Fix round 1** (the `fetch` core, the security baseline, Chord's listener as a native object). A route's overhead in
|
||||
the standard run grew from +4.5 µs to +20-21 µs (+41 µs before the Node fast path: the listener reads Node's parsed
|
||||
headers and makes a Web `Request` only when something reads `request.headers` or the body or a relay verifies; whole
|
||||
bodies are written as strings, in one write with the head). Warmed, +12-13 µs; a CPU profile of the warmed route path
|
||||
(per request, `test/next/bench/http.ts`'s route): the listener's request function 13.6 µs inclusive against the plain
|
||||
handler's 8.9 µs, of which writing the answer 8.3 µs (the plain handler's `writeHead` and `end`: 9.1 µs); `serve` 4.2
|
||||
µs (the root scope 2.1, the URL parse 0.9, default headers 0.6, admission and routing 1.1, the answer 0.7); the root
|
||||
scope's seal in a later microtask 0.9 µs. The baseline's extra headers cost 0.4 µs (a plain handler sending them).
|
||||
The rest, about 5 µs, sits in no function of the component: Node's tick and microtask machinery (33 µs inclusive
|
||||
against 18 µs for the plain handler). Opus's two trims are in (`lookup` decodes only segments with `%`; one abort
|
||||
controller per request: the Node side's own is gone). Kernel rows, HEAD `bc9a20f84` → fix round 1, CPU 6, alternating:
|
||||
binary 1 MiB call median 0.182-0.220 → 0.196 ms; reconnect of 1 / 100 references 1.02-1.10 / 79.1-80.3 → 1.04-1.05 /
|
||||
81.3-81.5 ms; watched `writable.set` 0.836-0.868 → 0.861-0.863 µs; watched derive commit 10.42-10.58 → 10.36-10.60 µs;
|
||||
activation churn 58.9-60.5 → 59.1-60.8 µs.
|
||||
|
||||
**Kernel hot paths** (routes without a timer, `RouteHandle`, `Path` events, root scopes), HEAD `bc9a20f84` → build at
|
||||
paths of equal length, CPU 6, alternating runs. `calls.ts`: local call 2.70-2.83 → 2.70-2.83 µs (6,660 B both),
|
||||
watched derive commit 10.36-10.60 → 10.34-10.79 µs, activation churn 59.43-61.30 → 59.48-60.72 µs. `links.ts`: commit
|
||||
to frame and encode/decode alike; reconnect of 1 / 100 references 0.94-1.15 / 80.8-84.0 → 0.94-1.15 / 80.6-86.5 ms;
|
||||
1,000 parked calls 24.4-25.2 → 24.3-24.6 ms. `reload.ts`: swap of 50 / 1,000 objects 1.44-1.55 / 30.78-31.42 →
|
||||
1.32-1.56 / 31.20-31.83 ms, restart 0.96-0.98 / 15.75-16.34 → 0.95-1.10 / 15.91-15.94 ms until the reload resolves.
|
||||
`registries.ts` alike (one of 100 contributions set 1.47-2.45 → 1.48-3.41 µs, run to run).
|
||||
|
|
|
|||
|
|
@ -94,11 +94,15 @@
|
|||
"typebox": "1.3.27"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"esbuild": "0.28.2"
|
||||
"esbuild": "0.28.2",
|
||||
"ws": "8.21.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"esbuild": {
|
||||
"optional": true
|
||||
},
|
||||
"ws": {
|
||||
"optional": true
|
||||
}
|
||||
},
|
||||
"devDependencies": {
|
||||
|
|
|
|||
166
packages/chord/src/next-node/http/README.md
Normal file
166
packages/chord/src/next-node/http/README.md
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
# http
|
||||
|
||||
The HTTP component (slice 10 §2, design §23; decisions "The HTTP core is a fetch handler", "The HTTP server is a native
|
||||
object", "The HTTP component's security baseline", "Slice 10 fix round, allowed hosts only through contributions",
|
||||
"Settings are arguments, extensions are registries", "A CSP nonce per response"): the server's HTTP and WebSocket
|
||||
entry. Plugins contribute routes, wraps, sockets, allowed hosts, framers, relays and
|
||||
CSP sources to the registry `Http`; routes return response descriptions. The core is one Web-standard handler,
|
||||
`fetch(Request) → Response`; Chord's own Node server (the listener) is the easy path, an app with a server of its own
|
||||
mounts the handler. Node-only; `ws` is an optional peer dependency of `@earendil-works/chord`, imported only here.
|
||||
Exported as `@earendil-works/chord/next/http` once v2 has its entry point.
|
||||
|
||||
## For apps (`index.ts`)
|
||||
|
||||
```ts
|
||||
// the app's settings are arguments, fixed for the component's life (changing one is a restart)
|
||||
const http = createHttp(server, {
|
||||
listen: { host: "127.0.0.1", port: 8788 }, // Chord's own listener; port 0 picks one; none: no listener
|
||||
links: { path: "/chord", options: (request, principal) => ({ limits, maxPayload }) }, // or a value
|
||||
closeTimeout: 30_000,
|
||||
onError: (error) => log(error), // HttpError
|
||||
}, ctx);
|
||||
// the app's core facet: who someone is (a registry: plugins may add authenticators too)
|
||||
const app = defineFacet((_host, self) => {
|
||||
self.contribute(Authenticators, (request, ctx) => principalFromCookie(readCookie(request, "session", secret)));
|
||||
});
|
||||
const core = [{ name: "chord.http", facet: http.facet }, { name: "@app/core", facet: app }];
|
||||
await plugins.load(server, { kind: "server", core, modules }, ctx);
|
||||
const { port } = await http.listening(ctx);
|
||||
…
|
||||
await http.close(ctx); // first: drains running requests (host.close would seal their scopes at once)
|
||||
await server.close(ctx);
|
||||
```
|
||||
|
||||
```ts
|
||||
// a plugin's server facet
|
||||
self.contribute(Http, {
|
||||
routes: {
|
||||
"GET /notes/:id": async (request, ctx) => ({ kind: "json", value: … }),
|
||||
"GET /": { page: true, handler: async (request) => ({ kind: "html", body: `<script nonce="${request.nonce}">…` }) },
|
||||
"POST /hooks/ci": { crossSite: true, handler: async (request) => … }, // always anonymous
|
||||
},
|
||||
wraps: [{ name: "audit", phase: "observe", scope: "all", wrap: async (request, next, ctx) => next(request) }],
|
||||
sockets: { "/terminal/:id": (socket, request, ctx) => attach(socket, ctx) },
|
||||
maxPayload: 1024 * 1024, // its sockets' largest frame
|
||||
hosts: ["notes.example.com"], // a writable (`self.writable`) to change it
|
||||
framers: ["https://shell.example.com"], // who may frame the app's pages
|
||||
relays: [{ name: "radius", verify: (request) => verified(request) }], // { origin, framer? } | undefined
|
||||
csp: { "img-src": ["https://cdn.example.com"] },
|
||||
});
|
||||
```
|
||||
|
||||
```ts
|
||||
// an app's own server instead of the listener (no `listen`)
|
||||
hono.all("/*", (c) => http.fetch(c.req.raw, { localAddress, localPort, remoteAddress }));
|
||||
// plain Node: `nodeRequest` converts with the listener's checks, never throwing (refused as the listener answers:
|
||||
// absolute-form 403, a target that does not parse 400, a `Host` with userinfo 403, TRACE/TRACK/CONNECT 501, any other
|
||||
// failure 400), raw headers with their duplicates, the signal aborted when the client goes
|
||||
nodeServer.on("request", async (message, res) => {
|
||||
// { kind: "request", request, connection } | { kind: "refused", status }
|
||||
const converted = nodeRequest(message, res);
|
||||
…
|
||||
});
|
||||
nodeServer.on("upgrade", http.upgrade);
|
||||
await http.close(ctx); // before host.close
|
||||
```
|
||||
|
||||
- **Order:** `http.facet` (plugin `chord.http`) serves `Http` (`compose`: the router), `Authenticators` (every
|
||||
contribution in order) and, with `listen`, the listener, a host-lifetime object whose native is Chord's Node server:
|
||||
it opens right after the publication. The facet is the same object at every reload, so a reload never closes it.
|
||||
`fetch` resolves the registries once; requests before the publication wait (503 past the host's `readyTimeout`; a
|
||||
`close` meanwhile: 503).
|
||||
- **Admission** (every request and upgrade, first): local only when the connection's local address is loopback, `Host`
|
||||
is a loopback name with the listening port (`127.0.0.1`, `localhost`, `[::1]`; without it on port 80) and no
|
||||
`Forwarded`/`X-Forwarded-*` header; a forwarded request on loopback only when a contributed relay verifies it
|
||||
(`provenance: { kind: "relayed", relay, origin }`, its framer allowed; an `origin` or `framer` that is not exactly an
|
||||
origin refuses the request, reported `hook_failed`); otherwise only an allowed host (a
|
||||
contribution's `hosts`, compared as the client sends it: `notes.example.com:443` is another name), as https; else 403.
|
||||
Two `Host`s, none, an absolute-form target and a `Host` with userinfo are 403; `TRACE`, `TRACK` and `CONNECT` (which a
|
||||
Web `Request` cannot carry) are 501, served for no resource, and a route contributed for one is refused at compose,
|
||||
reported as a conflict. A mounted handler passes what it knows of the connection:
|
||||
without `localAddress` and `localPort` no request is local. `request.origin` and `request.provenance` come only from
|
||||
here.
|
||||
- **Foreign requests** (Fetch Metadata: `Sec-Fetch-Site` neither `same-origin` nor `none`, or a frame not from this
|
||||
origin) are never authenticated, except a top-level navigation (`GET`, `navigate`) to a `page: true` route. A foreign
|
||||
`Origin` is 403 on every request and upgrade, and an unsafe method from a browser needs the app's own `Origin`; a
|
||||
`crossSite: true` route opts out and always runs anonymous. A request without the headers (a script, the TUI) is not
|
||||
a browser's.
|
||||
- **Who:** the authenticators in contributor order (plugin name: an order that means nothing, so an app needing
|
||||
precedence between credentials contributes one authenticator that decides); the first principal wins, a throwing one
|
||||
is reported `hook_failed` and skipped; none: anonymous; once the request is gone or sealed no further one starts.
|
||||
`cookie(name, value, { secret, request, sameSite?, maxAge? })` makes a signed `HttpOnly`, `Path=/`, `SameSite=Strict`
|
||||
(or `Lax`) cookie, `Secure` on a remote origin, its expiry (`maxAge`) signed with it; `readCookie(request, name,
|
||||
secret)` reads it back, its signature and expiry checked. A cookie without `maxAge` holds until the secret changes.
|
||||
Authenticators run raced against the request's signal: a client gone or `close`'s seal answers 503 (or refuses the
|
||||
upgrade) without waiting for them.
|
||||
- **A request's scope:** one `host.root` scope with its principal (an explicit `undefined`: anonymous) spans matching,
|
||||
wraps, the route and its response until the body is written (Chord's listener: flushed) or abandoned; its signal
|
||||
aborts when the client disconnects. An anonymous scope gets no remote object served here (`forbidden`: a `forbidden`
|
||||
escaping a route answers 403, unreported), but a local
|
||||
object it calls acts as its handler is written: a local helper calling remote types runs them as the host. A socket
|
||||
handler's scope lasts until its socket closes.
|
||||
- **`HttpRequest`:** `method`, `path` (dot segments resolved), `params`, `query`, `headers` (Web `Headers`), `origin`,
|
||||
`provenance`, `nonce` (fresh for every response), `json(limit?)` (1 MiB; malformed 400, too large 413),
|
||||
`body(limit?)` (chunks, 413 past the limit; a body cut short, by a disconnect or a broken stream, is a 400: a prefix
|
||||
never parses as the whole). An `HttpFailure` escaping a route answers its status. Chord's listener drains a body
|
||||
nobody read after the response up to 64 KiB, else closes the connection.
|
||||
- **Patterns:** exact segments, `:name` (one non-empty segment, decoded), a trailing `/*` (the rest, possibly empty:
|
||||
`/files/*` matches `/files`); a `:name` or rest segment decoding to one with `/`, `\` or NUL, or to `.` or `..`, is a
|
||||
400; the most specific match wins segment by segment from the left (exact, `:name`, `*`); each route names its params
|
||||
by its own pattern; one pattern from two contributions is a reported conflict (`registry_conflict`), the first by
|
||||
plugin name wins; `HEAD` falls back to the `GET` route of the same path; no route: 404.
|
||||
- **Wraps:** phases `observe`, `headers`, `transform` (outermost first), within a phase in contributor order; around
|
||||
the contributing plugin's routes (`scope: "own"`, default) or every route (`"all"`); `route` narrows to one (a route
|
||||
nobody contributes: a reported conflict). A throwing route or wrap is a 500 without its message (`route_failed`,
|
||||
`wrap_failed` naming the plugin).
|
||||
- **Responses:** `json`, `html`, `redirect`, `error` (204, 205 and 304 without a body), `bytes` (`read(offset, length,
|
||||
ctx)` in 256 KiB pieces as the consumer pulls; once the request's scope is sealed, by a disconnect or `close`, no read
|
||||
starts and the stream errors, which ends the request even if its consumer never reads again; with `size`:
|
||||
`Content-Length`, `ranges` (one range 206, unsatisfiable 416, several 200); without: until an empty read). A `bytes`
|
||||
read that throws or comes back short is reported `response_failed` and destroys the connection. `bytes` refuse a
|
||||
foreign request (403), are an attachment unless `inline` (then with a `sandbox` CSP), and a `download` name drops
|
||||
control and bidirectional formatting characters. `fileResponse(path, { download?, inline?, ranges?, mime?, root? })`
|
||||
stats at once (a missing file is 404, with `root` a real path outside it is 403), opens its handle at the first read
|
||||
and closes it when the request's scope seals; its type comes from a small table (text, images, audio, video), never
|
||||
HTML or SVG unless named.
|
||||
- **Default headers** on every answer, the component's own 403, 404 and 503 included: `Cache-Control: no-store`,
|
||||
`X-Content-Type-Options: nosniff`, `Referrer-Policy: no-referrer`, `X-Frame-Options: DENY` (without framers), a CSP
|
||||
(`default-src 'none'`, scripts `'self'` and the response's own nonce, `connect-src` self and its WebSocket, `base-uri
|
||||
'none'`, `form-action 'self'`, `frame-ancestors` the framers or `'none'`, plus contributed sources). A response's own
|
||||
headers replace same-named ones (a list sends one line per value: several `Set-Cookie`); a header with a CR, LF or
|
||||
NUL, or a value that is no JSON, is a 500 reported `response_failed`.
|
||||
- **Sockets:** after admission, the origin checks and the authenticators (no principal: 403 before plugin code), a
|
||||
`ws` WebSocket (its own server per upgrade, no compression) with an `error` listener attached (an oversized frame
|
||||
closes it, 1009); a throwing handler is reported `socket_failed` and closes it (1011). Pinged after 15 s of silence,
|
||||
terminated 15 s later without an answer. A client gone while its upgrade authenticates aborts that work.
|
||||
- **Links:** `links.path` upgrades to `host.accept(webSocketLink(socket), { principal, limits })` with what
|
||||
`links.options` gives that socket (`maxPayload`: 1 MiB by default); a client that stops reading is cut off once more
|
||||
than 4 MiB beyond the frame being sent waits.
|
||||
- **Revocation:** when a router change removes the host, the relay or the socket handler a link or socket came through,
|
||||
it is terminated at once and its scope sealed (no close handshake a peer could ignore). Identity is the handler's: a
|
||||
writable contribution changing only its hosts keeps its own sockets. An upgrade is checked again against the current
|
||||
router after its authenticators answered.
|
||||
- **`close(ctx)`:** new requests and upgrades refused (503), sockets and links closed (1001, terminated at
|
||||
`closeTimeout`), running requests and upgrades waited for up to `closeTimeout` (30 s; an aborted `ctx`: not at all),
|
||||
then sealed. Run it before `host.close`: the host's close seals every request's scope first, so the listener's native
|
||||
close (which runs it too, then closes its connections) only seals.
|
||||
- **Node's own limits** stay Node's defaults on the listener: `headersTimeout` 60 s, `requestTimeout` 300 s, 16 KiB of
|
||||
headers (431), keep-alive 5 s.
|
||||
|
||||
## Examples
|
||||
|
||||
`test/next/examples/`: `40-http` (the listener, a page with the nonce, a JSON route and a download with a range),
|
||||
`44-http-wraps` (a plugin's own wrap and one around every route), `45-plugin-socket` (a plugin's WebSocket, refused
|
||||
without a principal), `46-allowed-hosts` (a tunnel plugin allowing its host at runtime), `47-http-own-server` (the
|
||||
handler mounted in an app's own Node server), `48-http-baseline` (a signed cookie, foreign requests, a page link, a
|
||||
webhook).
|
||||
|
||||
## Inside the directory
|
||||
|
||||
`http.ts` (`createHttp`: the facet, `fetch`, the upgrade adapter, `close`), `node.ts` (Chord's Node server: Node
|
||||
requests as the component reads them, answers written; `nodeRequest` for apps mounting on Node), `incoming.ts` (one
|
||||
request shape for both), `router.ts` (the registry `Http` and its `compose`), `authenticate.ts` (`Authenticators`),
|
||||
`admission.ts` (provenance, relays, foreign requests, `Origin`), `headers.ts` (the default headers and the CSP),
|
||||
`cookies.ts` (`cookie`, `readCookie`), `request.ts` (`HttpRequest`, bodies, `HttpFailure`), `response.ts` (descriptions
|
||||
as answers: ranges, attachments, pulled bodies), `file.ts` (`fileResponse`), `upgrades.ts` (links and plugin sockets:
|
||||
kept, revoked, heartbeat, closed).
|
||||
178
packages/chord/src/next-node/http/admission.ts
Normal file
178
packages/chord/src/next-node/http/admission.ts
Normal file
|
|
@ -0,0 +1,178 @@
|
|||
// Admission (slice 10 §2, R4; decision "The HTTP component's security baseline" (1)-(3), "Relays"): where a request
|
||||
// comes from, decided before anything else. Local only when the connection arrived on a loopback address, its `Host`
|
||||
// is a loopback name with the listening port, and it carries no forwarded header; a forwarded request on loopback only
|
||||
// when a contributed relay verifies it; otherwise only through an allowed host (https); else refused. Foreign requests
|
||||
// (another site's, by Fetch Metadata) and a foreign `Origin` are classified here; the handler applies them per route.
|
||||
|
||||
import type { Incoming } from "./incoming.ts";
|
||||
import type { HttpRelay } from "./router.ts";
|
||||
|
||||
/** Where an admitted request came from. */
|
||||
export type Provenance =
|
||||
| { readonly kind: "local" }
|
||||
| { readonly kind: "remote"; readonly host: string }
|
||||
| { readonly kind: "relayed"; readonly relay: string; readonly origin: string };
|
||||
|
||||
/** The connection a request arrived on, as its listener knows it (a mounted handler passes what it has). */
|
||||
export type Connection = {
|
||||
readonly localAddress?: string;
|
||||
readonly localPort?: number;
|
||||
readonly remoteAddress?: string;
|
||||
};
|
||||
|
||||
/** An admitted request: its provenance, its own origin, the origin allowed to frame it (a relay's), how it came. */
|
||||
export type Admitted = {
|
||||
readonly provenance: Provenance;
|
||||
readonly origin: string;
|
||||
readonly framer: string | undefined;
|
||||
/** The allowed host it came through: its connections end when that host leaves. */
|
||||
readonly allowed: string | undefined;
|
||||
/** The relay that verified it: its connections end when that relay leaves. */
|
||||
readonly relay: HttpRelay | undefined;
|
||||
};
|
||||
|
||||
/** What admission refuses, and the relay hook that threw (reported by the caller). */
|
||||
export type Refused = {
|
||||
readonly kind: "refused";
|
||||
readonly failed?: { readonly relay: string; readonly error: unknown };
|
||||
};
|
||||
|
||||
/**
|
||||
* Admits `request` arriving on `connection` with the allowed `hosts` and contributed `relays`, or refuses it (403):
|
||||
* no or several `Host`s, a host neither loopback nor allowed, a loopback `Host` on another connection, forwarded
|
||||
* headers no relay verifies.
|
||||
*/
|
||||
export function admit(
|
||||
request: Incoming,
|
||||
connection: Connection,
|
||||
hosts: ReadonlySet<string>,
|
||||
relays: readonly HttpRelay[],
|
||||
): Admitted | Refused {
|
||||
const host = request.header("host")?.toLowerCase();
|
||||
// Several `Host` headers arrive joined with commas.
|
||||
if (host === undefined || host === "" || host.includes(",")) return REFUSED;
|
||||
const { localAddress, localPort } = connection;
|
||||
if (isLoopbackName(host, localPort)) {
|
||||
if (localAddress === undefined || !isLoopbackAddress(localAddress)) return REFUSED;
|
||||
if (!request.forwarded())
|
||||
return {
|
||||
provenance: LOCAL,
|
||||
origin: `http://${host}`,
|
||||
framer: undefined,
|
||||
allowed: undefined,
|
||||
relay: undefined,
|
||||
};
|
||||
return relayed(request, relays);
|
||||
}
|
||||
if (!hosts.has(host)) return REFUSED;
|
||||
return {
|
||||
provenance: { kind: "remote", host },
|
||||
origin: `https://${host}`,
|
||||
framer: undefined,
|
||||
allowed: host,
|
||||
relay: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
/** What the Fetch Metadata and `Origin` checks read: a request's headers by name. */
|
||||
export type HeaderSource = Pick<Incoming, "header">;
|
||||
|
||||
/** Whether `admission` refused its request. */
|
||||
export function refused(admission: Admitted | Refused): admission is Refused {
|
||||
return "kind" in admission;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a request is another site's (Fetch Metadata, p2's rule): `Sec-Fetch-Site` neither `same-origin` nor `none`,
|
||||
* or a frame or embed not from this origin. A request without the headers (a script, the TUI) is not.
|
||||
*/
|
||||
export function isForeign(headers: HeaderSource): boolean {
|
||||
const site = headers.header("sec-fetch-site");
|
||||
if (site !== null && site !== "same-origin" && site !== "none") return true;
|
||||
return site !== "same-origin" && FRAMES.has(headers.header("sec-fetch-dest") ?? "");
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a foreign request is a top-level navigation (a link followed to a page): `GET`, `navigate`, a `document` (or
|
||||
* `empty`: Chrome on Android launching an installed app).
|
||||
*/
|
||||
export function isNavigation(method: string, headers: HeaderSource): boolean {
|
||||
const dest = headers.header("sec-fetch-dest");
|
||||
return (
|
||||
method === "GET" && headers.header("sec-fetch-mode") === "navigate" && (dest === "document" || dest === "empty")
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `headers`' `Origin` is acceptable for a request whose own origin is `origin`: absent or equal, and for an
|
||||
* unsafe method from a browser (one sending Fetch Metadata) present and equal.
|
||||
*/
|
||||
export function originAccepted(method: string, headers: HeaderSource, origin: string): boolean {
|
||||
const sent = headers.header("origin");
|
||||
if (sent !== null) return sent === origin;
|
||||
return SAFE.has(method) || headers.header("sec-fetch-site") === null;
|
||||
}
|
||||
|
||||
const REFUSED: Refused = { kind: "refused" };
|
||||
|
||||
const LOCAL: Provenance = { kind: "local" };
|
||||
|
||||
const LOOPBACK_NAMES = ["127.0.0.1", "localhost", "[::1]"];
|
||||
|
||||
/** Whether `host` (lowercased) names this server on loopback with `port` (without it on port 80). */
|
||||
function isLoopbackName(host: string, port: number | undefined): boolean {
|
||||
if (port === undefined) return false;
|
||||
for (const name of LOOPBACK_NAMES) {
|
||||
if (host === `${name}:${port}`) return true;
|
||||
if (port === 80 && host === name) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Whether `address` is a loopback address: 127.0.0.0/8, ::1, or 127.x mapped into IPv6. */
|
||||
function isLoopbackAddress(address: string): boolean {
|
||||
return address === "::1" || address.startsWith("127.") || address.startsWith("::ffff:127.");
|
||||
}
|
||||
|
||||
/** A forwarded request on loopback: admitted as relayed when a relay verifies it, else refused. */
|
||||
function relayed(request: Incoming, relays: readonly HttpRelay[]): Admitted | Refused {
|
||||
for (const relay of relays) {
|
||||
let verified: ReturnType<HttpRelay["verify"]>;
|
||||
try {
|
||||
verified = relay.verify(request.request());
|
||||
} catch (error) {
|
||||
return { kind: "refused", failed: { relay: relay.name, error } };
|
||||
}
|
||||
if (verified === undefined) continue;
|
||||
// Its origins reach the CSP: each must be one origin and nothing else (Fable's recheck N2).
|
||||
for (const origin of [verified.origin, verified.framer]) {
|
||||
if (origin === undefined || isOrigin(origin)) continue;
|
||||
const error = new Error(`relay ${relay.name}: ${JSON.stringify(origin)} is no origin`);
|
||||
return { kind: "refused", failed: { relay: relay.name, error } };
|
||||
}
|
||||
return {
|
||||
provenance: { kind: "relayed", relay: relay.name, origin: verified.origin },
|
||||
origin: verified.origin,
|
||||
framer: verified.framer,
|
||||
allowed: undefined,
|
||||
relay,
|
||||
};
|
||||
}
|
||||
return REFUSED;
|
||||
}
|
||||
|
||||
/** Whether `text` is exactly an origin (`https://host[:port]`): what the URL parser serializes it to. */
|
||||
function isOrigin(text: string): boolean {
|
||||
try {
|
||||
const url = new URL(text);
|
||||
return (url.protocol === "https:" || url.protocol === "http:") && url.origin === text;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** The `Sec-Fetch-Dest` of a frame or an embed. */
|
||||
const FRAMES: ReadonlySet<string> = new Set(["iframe", "frame", "embed", "object", "fencedframe"]);
|
||||
|
||||
/** The methods that change nothing. */
|
||||
const SAFE: ReadonlySet<string> = new Set(["GET", "HEAD", "OPTIONS"]);
|
||||
54
packages/chord/src/next-node/http/authenticate.ts
Normal file
54
packages/chord/src/next-node/http/authenticate.ts
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
// Who someone is (decision "The HTTP server is a native object", replacing the `authenticate` option): the registry
|
||||
// `Authenticators`, one function per contributor, asked in contributor order (plugin name: an order with no meaning, so
|
||||
// an app needing precedence between credentials contributes one authenticator that decides); the first principal
|
||||
// returned wins, a throwing one is reported and skipped.
|
||||
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import type { Principal } from "../../next/data/index.ts";
|
||||
import { defineRegistry, type RegistryTable } from "../../next/registries/index.ts";
|
||||
import type { HttpRequest } from "./request.ts";
|
||||
|
||||
/** Who `request` is, or `undefined` (anonymous here, maybe another authenticator knows). */
|
||||
export type Authenticator = (
|
||||
request: HttpRequest,
|
||||
context: Context,
|
||||
) => Principal | undefined | Promise<Principal | undefined>;
|
||||
|
||||
/** An authenticator with its plugin, as the component asks it. */
|
||||
export type AuthenticatorEntry = { readonly plugin: string; readonly authenticate: Authenticator };
|
||||
|
||||
/** The registry of authenticators (served by `http.facet`). */
|
||||
export const Authenticators = defineRegistry<Authenticator, readonly AuthenticatorEntry[]>(
|
||||
"chord.http.authenticators",
|
||||
{
|
||||
version: 1,
|
||||
},
|
||||
);
|
||||
|
||||
/** Composes `Authenticators`: every contribution in contributor order. */
|
||||
export function composeAuthenticators(table: RegistryTable<Authenticator>): readonly AuthenticatorEntry[] {
|
||||
return table.map(({ plugin, value }) => ({ plugin, authenticate: value }));
|
||||
}
|
||||
|
||||
/**
|
||||
* The principal of `request`: the first one an authenticator returns, in order; one that throws is reported through
|
||||
* `failed` and skipped; none: `undefined` (anonymous). Once `context` is aborted no further authenticator starts.
|
||||
*/
|
||||
export async function authenticate(
|
||||
authenticators: readonly AuthenticatorEntry[],
|
||||
request: HttpRequest,
|
||||
failed: (plugin: string, error: unknown) => void,
|
||||
context: Context,
|
||||
): Promise<Principal | undefined> {
|
||||
for (const { plugin, authenticate } of authenticators) {
|
||||
// A request gone or sealed starts no new work (Sol's second recheck C).
|
||||
if (context.abortSignal?.aborted) return undefined;
|
||||
try {
|
||||
const principal = await authenticate(request, context);
|
||||
if (principal !== undefined) return principal;
|
||||
} catch (error) {
|
||||
failed(plugin, error);
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
71
packages/chord/src/next-node/http/cookies.ts
Normal file
71
packages/chord/src/next-node/http/cookies.ts
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
// Cookies (decision "The HTTP component's security baseline" (5)): `cookie` makes a signed `Set-Cookie` value with the
|
||||
// safe defaults (`HttpOnly`, `Path=/`, `SameSite=Strict`, `Secure` on a remote origin), `readCookie` reads one back,
|
||||
// its signature checked in constant time.
|
||||
|
||||
import { createHmac, timingSafeEqual } from "node:crypto";
|
||||
import type { HttpRequest } from "./request.ts";
|
||||
|
||||
/** What `cookie` takes besides the name and the value. */
|
||||
export type CookieOptions = {
|
||||
/** The key it is signed with (HMAC-SHA256). */
|
||||
readonly secret: string;
|
||||
/** The request it answers: a remote one gets `Secure`. */
|
||||
readonly request: HttpRequest;
|
||||
/** `Strict` by default; `Lax` lets a top-level navigation from another site carry it. */
|
||||
readonly sameSite?: "Strict" | "Lax";
|
||||
/** Seconds until it expires; none: a session cookie. */
|
||||
readonly maxAge?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* A signed `Set-Cookie` value for `name` and `value` (put it in a response's `headers`, `Set-Cookie`). With `maxAge`
|
||||
* its expiry is signed with it: a copy read back later is refused (Opus's recheck R3); without, it holds until the
|
||||
* secret changes.
|
||||
*/
|
||||
export function cookie(name: string, value: string, options: CookieOptions): string {
|
||||
const encoded = encodeURIComponent(value);
|
||||
const expires = options.maxAge === undefined ? "" : String(Math.floor(Date.now() / 1000 + options.maxAge));
|
||||
const payload = `${encoded}.${expires}`;
|
||||
const parts = [
|
||||
`${name}=${payload}.${signature(name, payload, options.secret)}`,
|
||||
"HttpOnly",
|
||||
"Path=/",
|
||||
`SameSite=${options.sameSite ?? "Strict"}`,
|
||||
];
|
||||
if (options.request.provenance.kind !== "local") parts.push("Secure");
|
||||
if (options.maxAge !== undefined) parts.push(`Max-Age=${Math.floor(options.maxAge)}`);
|
||||
return parts.join("; ");
|
||||
}
|
||||
|
||||
/**
|
||||
* The value of cookie `name` on `request` when its signature with `secret` holds and its signed expiry has not passed;
|
||||
* `undefined` otherwise.
|
||||
*/
|
||||
export function readCookie(request: HttpRequest, name: string, secret: string): string | undefined {
|
||||
for (const pair of (request.headers.get("cookie") ?? "").split(";")) {
|
||||
const equals = pair.indexOf("=");
|
||||
if (equals < 0 || pair.slice(0, equals).trim() !== name) continue;
|
||||
const signed = pair.slice(equals + 1).trim();
|
||||
// From the right: the value may hold dots, the expiry and the signature never do.
|
||||
const dot = signed.lastIndexOf(".");
|
||||
const payload = signed.slice(0, dot);
|
||||
const expiresAt = payload.lastIndexOf(".");
|
||||
if (dot < 0 || expiresAt < 0) return undefined;
|
||||
const given = Buffer.from(signed.slice(dot + 1));
|
||||
const expected = Buffer.from(signature(name, payload, secret));
|
||||
if (given.length !== expected.length || !timingSafeEqual(given, expected)) return undefined;
|
||||
const expires = payload.slice(expiresAt + 1);
|
||||
if (expires !== "" && Number(expires) * 1000 <= Date.now()) return undefined;
|
||||
try {
|
||||
return decodeURIComponent(payload.slice(0, expiresAt));
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** The signature of a cookie's name and payload (its encoded value and expiry). */
|
||||
function signature(name: string, payload: string, secret: string): string {
|
||||
return createHmac("sha256", secret).update(`${name}=${payload}`).digest("base64url");
|
||||
}
|
||||
113
packages/chord/src/next-node/http/file.ts
Normal file
113
packages/chord/src/next-node/http/file.ts
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
// File responses (slice 10 §2; decision "Slice 10 fix round, three small points" (1)): a file as a `bytes` response.
|
||||
// `fileResponse` stats the file at once (with a `root`, its real path must lie inside it); the file handle is opened by
|
||||
// the first `read` and closed when the request's scope seals (its signal aborts), so a response nobody writes (`HEAD`,
|
||||
// a 416, a disconnect, a wrap that replaced it) opens nothing and leaks nothing.
|
||||
|
||||
import { type FileHandle, open, realpath, stat } from "node:fs/promises";
|
||||
import { basename, extname, sep } from "node:path";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import { HttpFailure } from "./request.ts";
|
||||
import type { HttpResponse } from "./response.ts";
|
||||
|
||||
/** What `fileResponse` takes besides the path. */
|
||||
export type FileResponseOptions = {
|
||||
/** Saved under this name. */
|
||||
readonly download?: string;
|
||||
/** Shown in the browser instead of saved (an image, a video), with a `sandbox` CSP. */
|
||||
readonly inline?: true;
|
||||
/**
|
||||
* The directory the file must lie in once its symbolic links are resolved: one outside it (a link out) is refused
|
||||
* 403. Without it the path is trusted (a plugin's own file).
|
||||
*/
|
||||
readonly root?: string;
|
||||
/** Whether a single range is answered 206 (audio and video seek with it). */
|
||||
readonly ranges?: boolean;
|
||||
/** The type; by default from the extension (text, images, audio, video), else `application/octet-stream`. */
|
||||
readonly mime?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* The file at `path` as a `bytes` response; `HttpFailure` 404 for a missing one (anyone may ask for one: no failure
|
||||
* of the route, Fable's recheck N1), 403 for one outside `root`; throws what else `stat` throws.
|
||||
*/
|
||||
export async function fileResponse(given: string, options: FileResponseOptions = {}): Promise<HttpResponse> {
|
||||
let path: string;
|
||||
let size: number;
|
||||
try {
|
||||
path = options.root === undefined ? given : await inside(given, options.root);
|
||||
({ size } = await stat(path));
|
||||
} catch (error) {
|
||||
const code = (error as NodeJS.ErrnoException).code;
|
||||
if (code === "ENOENT" || code === "ENOTDIR") throw new HttpFailure(404, "no such file");
|
||||
throw error;
|
||||
}
|
||||
let handle: Promise<FileHandle> | undefined;
|
||||
const read = async (offset: number, length: number, context: Context): Promise<Uint8Array> => {
|
||||
handle ??= opened(path, context);
|
||||
const file = await handle;
|
||||
// Only the bytes read are handed out.
|
||||
const buffer = Buffer.allocUnsafe(length);
|
||||
const { bytesRead } = await file.read(buffer, 0, length, offset);
|
||||
return buffer.subarray(0, bytesRead);
|
||||
};
|
||||
return {
|
||||
kind: "bytes",
|
||||
mime: options.mime ?? mimeOf(path),
|
||||
size,
|
||||
read,
|
||||
...(options.download === undefined ? {} : { download: options.download }),
|
||||
...(options.inline === undefined ? {} : { inline: options.inline }),
|
||||
...(options.ranges === undefined ? {} : { ranges: options.ranges }),
|
||||
};
|
||||
}
|
||||
|
||||
/** The real path of `path` when it lies inside `root`'s real path; `HttpFailure` 403 otherwise. */
|
||||
async function inside(path: string, root: string): Promise<string> {
|
||||
const [real, base] = await Promise.all([realpath(path), realpath(root)]);
|
||||
if (real !== base && !real.startsWith(base.endsWith(sep) ? base : base + sep)) {
|
||||
throw new HttpFailure(403, "the file lies outside its root");
|
||||
}
|
||||
return real;
|
||||
}
|
||||
|
||||
/** Opens `path`, closed when `context` (the request's scope) aborts; at once when it aborted already. */
|
||||
async function opened(path: string, context: Context): Promise<FileHandle> {
|
||||
const file = await open(path, "r");
|
||||
const close = (): void => void file.close().catch(() => {});
|
||||
const signal = context.abortSignal;
|
||||
if (signal === undefined) throw new Error(`fileResponse(${basename(path)}): read outside a request's scope`);
|
||||
if (signal.aborted) close();
|
||||
else signal.addEventListener("abort", close, { once: true });
|
||||
return file;
|
||||
}
|
||||
|
||||
const MIME: Readonly<Record<string, string>> = {
|
||||
".txt": "text/plain; charset=utf-8",
|
||||
".log": "text/plain; charset=utf-8",
|
||||
".md": "text/markdown; charset=utf-8",
|
||||
".csv": "text/csv; charset=utf-8",
|
||||
".json": "application/json",
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".avif": "image/avif",
|
||||
".mp3": "audio/mpeg",
|
||||
".m4a": "audio/mp4",
|
||||
".ogg": "audio/ogg",
|
||||
".opus": "audio/ogg",
|
||||
".wav": "audio/wav",
|
||||
".flac": "audio/flac",
|
||||
".mp4": "video/mp4",
|
||||
".webm": "video/webm",
|
||||
".mov": "video/quicktime",
|
||||
};
|
||||
|
||||
/**
|
||||
* The type of `path` by its extension: text, images, audio and video only. Active types (HTML, SVG) never come from
|
||||
* here: a route serving one names it (`mime`).
|
||||
*/
|
||||
function mimeOf(path: string): string {
|
||||
return MIME[extname(path).toLowerCase()] ?? "application/octet-stream";
|
||||
}
|
||||
142
packages/chord/src/next-node/http/headers.ts
Normal file
142
packages/chord/src/next-node/http/headers.ts
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
// The default headers (decision "The HTTP component's security baseline" (4), "A CSP nonce per response"): on every
|
||||
// response, the component's own included: no caching, no sniffing, no referrer, no framing but by the contributed
|
||||
// framers, and a CSP with the response's own script nonce. A response's own headers replace same-named ones (a route
|
||||
// widens its own CSP); they are validated as they are set, so a CR or LF never reaches the wire.
|
||||
|
||||
import { randomFillSync } from "node:crypto";
|
||||
import type { Admitted } from "./admission.ts";
|
||||
import type { Router } from "./router.ts";
|
||||
|
||||
/** A response's headers as the component builds them: by lowercased name, each value checked when set. */
|
||||
export class AnswerHeaders {
|
||||
/** By lowercased name: the name as set and its values (several only for a header sent more than once). */
|
||||
readonly #entries = new Map<string, { readonly name: string; readonly values: readonly string[] }>();
|
||||
|
||||
/** The values of `name` joined with `", "`, or `null`. */
|
||||
get(name: string): string | null {
|
||||
return this.#entries.get(name.toLowerCase())?.values.join(", ") ?? null;
|
||||
}
|
||||
|
||||
has(name: string): boolean {
|
||||
return this.#entries.has(name.toLowerCase());
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets `name` to `value` (several values: one header line each, as `Set-Cookie` needs), replacing a same-named one;
|
||||
* throws `TypeError` for a name or value HTTP cannot carry.
|
||||
*/
|
||||
set(name: string, value: string | readonly string[]): void {
|
||||
if (!TOKEN.test(name)) throw new TypeError(`header: invalid name ${JSON.stringify(name)}`);
|
||||
const values = typeof value === "string" ? [value] : value;
|
||||
for (const each of values) {
|
||||
if (!FIELD_VALUE.test(each)) throw new TypeError(`header ${name}: invalid value ${JSON.stringify(each)}`);
|
||||
}
|
||||
this.#entries.set(name.toLowerCase(), { name, values });
|
||||
}
|
||||
|
||||
delete(name: string): void {
|
||||
this.#entries.delete(name.toLowerCase());
|
||||
}
|
||||
|
||||
/** Each header line as `[name as set, value]`. */
|
||||
*lines(): IterableIterator<readonly [string, string]> {
|
||||
for (const { name, values } of this.#entries.values()) for (const value of values) yield [name, value];
|
||||
}
|
||||
}
|
||||
|
||||
/** A header name: an HTTP token. */
|
||||
const TOKEN = /^[!#$%&'*+\-.^_`|~0-9A-Za-z]+$/;
|
||||
|
||||
/** A header value: visible Latin-1, spaces and tabs; no CR, LF or NUL. */
|
||||
const FIELD_VALUE = /^[\t\x20-\x7e\x80-\xff]*$/;
|
||||
|
||||
/** Fresh script nonces, one per response: 16 random bytes each, base64, cut from a pool filled 4 KiB at a time. */
|
||||
export class Nonces {
|
||||
readonly #pool = Buffer.allocUnsafe(4096);
|
||||
#at = 4096;
|
||||
|
||||
/** The next nonce (never one handed out before). */
|
||||
next(): string {
|
||||
// One `randomFillSync` per 256 nonces: `randomBytes(16)` per response costs 1.6 µs.
|
||||
if (this.#at === this.#pool.length) {
|
||||
randomFillSync(this.#pool);
|
||||
this.#at = 0;
|
||||
}
|
||||
const nonce = this.#pool.toString("base64", this.#at, this.#at + 16);
|
||||
this.#at += 16;
|
||||
return nonce;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The default headers of one component's responses. The CSP around the nonce is kept per router and origin for local
|
||||
* and allowed-host requests (a bounded set: the hosts contributions name); a relayed request's origins come from its
|
||||
* relay and are never kept (Fable's and Opus's rechecks: a cache growing with each relayed origin).
|
||||
*/
|
||||
export class DefaultHeaders {
|
||||
/** By router (its change makes new ones), then by origin: the CSP's text before and after the nonce. */
|
||||
readonly #policies = new WeakMap<Router, Map<string, readonly [string, string]>>();
|
||||
|
||||
/**
|
||||
* The default headers of a response to a request admitted as `admitted` (none: refused before admission), with
|
||||
* `router`'s framers and CSP sources (none before the router is resolved), its CSP allowing scripts with `nonce`.
|
||||
*/
|
||||
of(admitted: Admitted | undefined, router: Router | undefined, nonce: string): AnswerHeaders {
|
||||
const headers = new AnswerHeaders();
|
||||
headers.set("Cache-Control", "no-store");
|
||||
headers.set("X-Content-Type-Options", "nosniff");
|
||||
headers.set("Referrer-Policy", "no-referrer");
|
||||
// `X-Frame-Options` names no origin: with framers, only `frame-ancestors` speaks.
|
||||
if ((router?.framers.size ?? 0) === 0 && admitted?.framer === undefined) headers.set("X-Frame-Options", "DENY");
|
||||
const [before, after] = this.#policy(admitted, router);
|
||||
headers.set("Content-Security-Policy", `${before}${nonce}${after}`);
|
||||
return headers;
|
||||
}
|
||||
|
||||
/** The CSP's text around the nonce for `admitted` under `router`. */
|
||||
#policy(admitted: Admitted | undefined, router: Router | undefined): readonly [string, string] {
|
||||
if (router === undefined || admitted === undefined || admitted.relay !== undefined) return csp(admitted, router);
|
||||
let policies = this.#policies.get(router);
|
||||
if (policies === undefined) {
|
||||
policies = new Map();
|
||||
this.#policies.set(router, policies);
|
||||
}
|
||||
let policy = policies.get(admitted.origin);
|
||||
if (policy === undefined) {
|
||||
policy = csp(admitted, router);
|
||||
policies.set(admitted.origin, policy);
|
||||
}
|
||||
return policy;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The CSP's text before and after the nonce: nothing by default, scripts from this origin or with the nonce,
|
||||
* connections to it and its WebSocket, framing by the framers, plus contributed sources.
|
||||
*/
|
||||
function csp(admitted: Admitted | undefined, router: Router | undefined): readonly [string, string] {
|
||||
const framers = [...(router?.framers ?? [])];
|
||||
if (admitted?.framer !== undefined) framers.push(admitted.framer);
|
||||
const directives = new Map<string, string[]>([
|
||||
["default-src", ["'none'"]],
|
||||
["script-src", ["'self'", NONCE]],
|
||||
["connect-src", ["'self'", ...(admitted === undefined ? [] : [admitted.origin.replace(/^http/, "ws")])]],
|
||||
["base-uri", ["'none'"]],
|
||||
["form-action", ["'self'"]],
|
||||
["frame-ancestors", framers.length === 0 ? ["'none'"] : framers],
|
||||
]);
|
||||
for (const [directive, sources] of router?.csp ?? []) {
|
||||
const current = directives.get(directive);
|
||||
// A directive the baseline sets to 'none' is replaced by the sources given for it.
|
||||
directives.set(
|
||||
directive,
|
||||
current === undefined || current[0] === "'none'" ? [...sources] : [...current, ...sources],
|
||||
);
|
||||
}
|
||||
const text = [...directives].map(([directive, sources]) => `${directive} ${sources.join(" ")}`).join("; ");
|
||||
const at = text.indexOf(NONCE);
|
||||
return [`${text.slice(0, at)}'nonce-`, `'${text.slice(at + NONCE.length)}`];
|
||||
}
|
||||
|
||||
/** Where the nonce goes in the CSP's text while it is made. */
|
||||
const NONCE = "\0nonce\0";
|
||||
565
packages/chord/src/next-node/http/http.ts
Normal file
565
packages/chord/src/next-node/http/http.ts
Normal file
|
|
@ -0,0 +1,565 @@
|
|||
// The HTTP component (slice 10 §2; decisions "The HTTP core is a fetch handler", "The HTTP server is a native object",
|
||||
// "The HTTP component's security baseline", "Settings are arguments, extensions are registries"): `createHttp` takes
|
||||
// the app's settings (where to listen, the links endpoint, the close bound) and builds one Web-standard handler,
|
||||
// `fetch`, from the registries its facet serves (`Http`, `Authenticators`), an upgrade adapter for Node (`upgrade`), and
|
||||
// the facet, whose host-lifetime listener object is Chord's own Node server. Every request is admitted (provenance), routed,
|
||||
// checked for foreign origins, authenticated unless foreign, and served in one `host.root` scope with its principal
|
||||
// that lasts until its body is delivered; every answer carries the default headers.
|
||||
|
||||
import type { IncomingMessage } from "node:http";
|
||||
import type { Duplex } from "node:stream";
|
||||
import { awaitWithContext, type Context, withAbortSignal, withoutAbortSignal } from "../../context/index.ts";
|
||||
import { local } from "../../next/contracts/index.ts";
|
||||
import { isChordError, type Principal } from "../../next/data/index.ts";
|
||||
import { type Ctx, defineFacet, type Facet, type Host } from "../../next/host/index.ts";
|
||||
import type { RegistryValue } from "../../next/registries/index.ts";
|
||||
import {
|
||||
type Admitted,
|
||||
admit,
|
||||
type Connection,
|
||||
isForeign,
|
||||
isNavigation,
|
||||
originAccepted,
|
||||
refused,
|
||||
} from "./admission.ts";
|
||||
import { type AuthenticatorEntry, Authenticators, authenticate, composeAuthenticators } from "./authenticate.ts";
|
||||
import { DefaultHeaders, Nonces } from "./headers.ts";
|
||||
import { type Incoming, webIncoming } from "./incoming.ts";
|
||||
import { connectionOf, type Handler, type ListenAddress, NodeIncoming, NodeListener, type Served } from "./node.ts";
|
||||
import { HttpFailure, type HttpRequest, WebRequest } from "./request.ts";
|
||||
import { type Answer, answerOf, errorAnswer, type HttpResponse, webResponse } from "./response.ts";
|
||||
import { composeRouter, Http, type HttpRoute, type RouteMatch, type Router } from "./router.ts";
|
||||
import { type LinkOptions, Upgrades } from "./upgrades.ts";
|
||||
|
||||
/** What `createHttp` takes: the app's settings, fixed for the component's life (changing one is a restart). */
|
||||
export type HttpOptions = {
|
||||
/** Where Chord's own listener listens; none: no listener (an app mounting `fetch`). Port 0 picks one. */
|
||||
readonly listen?: ListenAddress;
|
||||
/** The links endpoint: its path, and each socket's options, a value or per request and principal. */
|
||||
readonly links?: LinksOptions;
|
||||
/** How long `close` waits for running requests and closing sockets: 30 s by default. */
|
||||
readonly closeTimeout?: number;
|
||||
readonly onError: (error: HttpError) => void;
|
||||
};
|
||||
|
||||
/** The links endpoint's path and what it gives each socket (`maxPayload` 1 MiB by default). */
|
||||
export type LinksOptions = {
|
||||
readonly path: string;
|
||||
readonly options?: LinkOptions | ((request: HttpRequest, principal: Principal) => LinkOptions);
|
||||
};
|
||||
|
||||
/** What the component reports. */
|
||||
export type HttpError =
|
||||
| { readonly kind: "route_failed"; readonly pattern: string; readonly plugin: string; readonly error: unknown }
|
||||
| { readonly kind: "wrap_failed"; readonly name: string; readonly plugin: string; readonly error: unknown }
|
||||
| { readonly kind: "socket_failed"; readonly pattern: string; readonly plugin: string; readonly error: unknown }
|
||||
/** A response that could not be made (an invalid header, no JSON) or read whole (a `bytes` read that threw). */
|
||||
| { readonly kind: "response_failed"; readonly pattern: string; readonly error: unknown }
|
||||
/** A contributed hook threw (`plugin`), or the app's `links.options` (no plugin). */
|
||||
| { readonly kind: "hook_failed"; readonly hook: HookName; readonly plugin?: string; readonly error: unknown };
|
||||
|
||||
/** The hooks the component calls. */
|
||||
export type HookName = "authenticate" | "relay" | "links.options";
|
||||
|
||||
/** The running component. */
|
||||
export interface HttpComponent<P extends Principal> {
|
||||
/**
|
||||
* Its facet (plugin `chord.http`): serves `Http` and `Authenticators`, and with `listen` the listener, a
|
||||
* host-lifetime object. The app loads it among its core facets, the same object at every reload.
|
||||
*/
|
||||
readonly facet: Facet<P>;
|
||||
/**
|
||||
* Answers a Web `Request` (an app's own server mounts it: `hono.all("/*", (c) => http.fetch(c.req.raw, …))`);
|
||||
* `connection` is what the server knows of the socket: without `localAddress` and `localPort` no request is local.
|
||||
* Requests before the publication wait for it (503 past the host's `readyTimeout`).
|
||||
*/
|
||||
fetch(request: Request, connection?: Connection): Promise<Response>;
|
||||
/** Node's upgrade adapter (`server.on("upgrade", http.upgrade)`): the links endpoint and plugin sockets. */
|
||||
readonly upgrade: (message: IncomingMessage, socket: Duplex, head: Buffer) => void;
|
||||
/** Where the listener listens, once it does (the real port when `listen.port` was 0). */
|
||||
listening(context: Context): Promise<ListenAddress>;
|
||||
/**
|
||||
* Refuses new requests and upgrades (503), closes sockets and links (1001, terminated at `closeTimeout`), waits for
|
||||
* running requests up to `closeTimeout` (an aborted `context`: not at all), then seals them. Run it before
|
||||
* `host.close`, which seals every request's scope at once (the listener's close runs it then, too late to drain).
|
||||
*/
|
||||
close(context: Context): Promise<void>;
|
||||
}
|
||||
|
||||
/** Builds the HTTP component on `host` (a request's scope is a `host.root` with its principal). */
|
||||
export function createHttp<P extends Principal>(
|
||||
host: Host<P>,
|
||||
options: HttpOptions,
|
||||
context: Context,
|
||||
): HttpComponent<P> {
|
||||
return new Component(host, options, context);
|
||||
}
|
||||
|
||||
/** The listener: a host-lifetime object of plugin `chord.http` whose native is Chord's own Node server. */
|
||||
export const HttpListener = local("chord.http.listener", { version: 1, params: {}, members: {} });
|
||||
|
||||
/** The registries a request reads, resolved once. */
|
||||
type Registries = {
|
||||
readonly router: RegistryValue<Router>;
|
||||
readonly authenticators: RegistryValue<readonly AuthenticatorEntry[]>;
|
||||
};
|
||||
|
||||
/** A request or an upgrade from its arrival to its end: what `close` waits for, then seals. */
|
||||
type Running = { readonly controller: AbortController; done: Promise<unknown> };
|
||||
|
||||
/** `close`'s bound unless `closeTimeout` says otherwise: 30 s. */
|
||||
const CLOSE_TIMEOUT = 30_000;
|
||||
|
||||
/** Who threw out of a route itself (no wrap). */
|
||||
const ROUTE = { kind: "route" } as const;
|
||||
|
||||
class Component<P extends Principal> implements HttpComponent<P>, Handler {
|
||||
readonly facet: Facet<P>;
|
||||
readonly upgrade: (message: IncomingMessage, socket: Duplex, head: Buffer) => void;
|
||||
readonly #host: Host<P>;
|
||||
readonly #options: HttpOptions;
|
||||
readonly #onError: (error: HttpError) => void;
|
||||
readonly #context: Context;
|
||||
/** The component's context without its cancellation: each request's scope adds its own signal. */
|
||||
readonly #base: Context;
|
||||
readonly #defaults = new DefaultHeaders();
|
||||
readonly #nonces = new Nonces();
|
||||
readonly #upgrades = new Upgrades();
|
||||
readonly #running = new Set<Running>();
|
||||
/** The registries while they resolve (once; again after a failed try), then the values themselves. */
|
||||
#resolving: Promise<Registries> | undefined;
|
||||
#registries: Registries | undefined;
|
||||
/** Aborted at `close`: the router's watch. */
|
||||
readonly #closing = new AbortController();
|
||||
#closed: Promise<void> | undefined;
|
||||
readonly #listening = Promise.withResolvers<ListenAddress>();
|
||||
|
||||
constructor(host: Host<P>, options: HttpOptions, context: Context) {
|
||||
this.#host = host;
|
||||
this.#options = options;
|
||||
this.#onError = options.onError;
|
||||
this.#context = context;
|
||||
this.#base = withoutAbortSignal(context);
|
||||
this.#listening.promise.catch(() => {});
|
||||
this.upgrade = (message, socket, head) => {
|
||||
socket.on("error", () => socket.destroy());
|
||||
void this.#upgrade(message, socket, head).catch(() => socket.destroy());
|
||||
};
|
||||
const { listen } = options;
|
||||
this.facet = defineFacet<P>((facet) => {
|
||||
facet.serve(Http, { compose: composeRouter });
|
||||
facet.serve(Authenticators, { compose: composeAuthenticators });
|
||||
if (listen === undefined) return;
|
||||
facet.serve(HttpListener, (_params, self) => {
|
||||
self.open(
|
||||
(opening) => this.#listen(listen, opening),
|
||||
async (listener, _reason, closing) => {
|
||||
listener.stop();
|
||||
await this.close(closing);
|
||||
await listener.closed();
|
||||
},
|
||||
);
|
||||
return {};
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
async fetch(request: Request, connection: Connection = {}): Promise<Response> {
|
||||
const { answer, finished } = await this.serve(webIncoming(request), connection);
|
||||
// A whole body is out of the component's hands once its `Response` is; a pulled one ends when it is read.
|
||||
if (answer.body instanceof ReadableStream)
|
||||
return webResponse({ ...answer, body: tracked(answer.body, finished) });
|
||||
finished();
|
||||
return webResponse(answer);
|
||||
}
|
||||
|
||||
listening(context: Context): Promise<ListenAddress> {
|
||||
return awaitWithContext(this.#listening.promise, context);
|
||||
}
|
||||
|
||||
close(context: Context): Promise<void> {
|
||||
this.#closed ??= this.#close(context);
|
||||
return this.#closed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Serves `request`: its answer, and what its consumer calls once the body is written or abandoned. Never throws:
|
||||
* whatever fails answers (503 before the publication or while closing, 403, 404, 500 reported).
|
||||
*/
|
||||
async serve(request: Incoming, connection: Connection): Promise<Served> {
|
||||
const controller = new AbortController();
|
||||
const finished = Promise.withResolvers<void>();
|
||||
const running: Running = { controller, done: finished.promise };
|
||||
this.#running.add(running);
|
||||
request.cancelled((reason) => controller.abort(reason));
|
||||
let answer: Answer;
|
||||
try {
|
||||
const served = await this.#serve(request, connection, controller, finished.promise);
|
||||
answer = served.answer;
|
||||
// The request is over once its consumer finished and its scope (if it got one) ended.
|
||||
if (served.scope !== undefined) running.done = Promise.all([finished.promise, served.scope]);
|
||||
} catch {
|
||||
answer = this.refusal(500);
|
||||
}
|
||||
void running.done.then(() => this.#running.delete(running));
|
||||
return { answer, finished: () => finished.resolve() };
|
||||
}
|
||||
|
||||
/** The answer for a refused or failed request, outside any scope, with the default headers. */
|
||||
refusal(status: number, admitted?: Admitted, router?: Router, head = false): Answer {
|
||||
return errorAnswer(status, this.#defaults.of(admitted, router, this.#nonces.next()), undefined, head);
|
||||
}
|
||||
|
||||
/** The steps of `serve`: closing, the registries, admission, the route, origins, the principal, then the scope. */
|
||||
async #serve(
|
||||
request: Incoming,
|
||||
connection: Connection,
|
||||
controller: AbortController,
|
||||
finished: Promise<void>,
|
||||
): Promise<{ readonly answer: Answer; readonly scope?: Promise<void> }> {
|
||||
const head = request.method === "HEAD";
|
||||
if (this.#closing.signal.aborted) return { answer: this.refusal(503, undefined, undefined, head) };
|
||||
let registries: Registries;
|
||||
try {
|
||||
registries = this.#registries ?? (await this.#resolve());
|
||||
} catch {
|
||||
return { answer: this.refusal(503, undefined, undefined, head) };
|
||||
}
|
||||
if (this.#closing.signal.aborted) return { answer: this.refusal(503, undefined, undefined, head) };
|
||||
const router = registries.router.value;
|
||||
const admission = admit(request, connection, router.hosts, router.relays);
|
||||
if (refused(admission)) {
|
||||
if (admission.failed !== undefined) this.#hookFailed("relay", admission.failed.relay, admission.failed.error);
|
||||
return { answer: this.refusal(403, undefined, router, head) };
|
||||
}
|
||||
// A fresh script nonce per response (decision "A CSP nonce per response").
|
||||
const webRequest = new WebRequest(request, admission, this.#nonces.next());
|
||||
const match = router.route(request.method, webRequest.path);
|
||||
if (match === "bad_path" || match === undefined) {
|
||||
return { answer: this.refusal(match === undefined ? 404 : 400, admission, router, head) };
|
||||
}
|
||||
webRequest.params = match.params;
|
||||
const foreign = isForeign(request);
|
||||
if (!match.crossSite && !originAccepted(request.method, request, admission.origin)) {
|
||||
return { answer: this.refusal(403, admission, router, head) };
|
||||
}
|
||||
let principal: P | undefined;
|
||||
// Foreign requests are never authenticated but a top-level navigation to a page; a cross-site route never is.
|
||||
const authenticators = registries.authenticators.value;
|
||||
if (
|
||||
authenticators.length > 0 &&
|
||||
!match.crossSite &&
|
||||
(!foreign || (match.page && isNavigation(request.method, request)))
|
||||
) {
|
||||
principal = await this.#principal(registries, webRequest, controller.signal);
|
||||
if (this.#closing.signal.aborted || controller.signal.aborted) {
|
||||
return { answer: this.refusal(503, admission, router, head) };
|
||||
}
|
||||
}
|
||||
const answered = Promise.withResolvers<Answer>();
|
||||
const scope = this.#host
|
||||
.root(
|
||||
async (ctx) => {
|
||||
answered.resolve(await this.#run(request, webRequest, match, admission, router, foreign, ctx));
|
||||
// The scope lasts until the body is written or abandoned (a `bytes` body reads in it).
|
||||
await finished;
|
||||
},
|
||||
withAbortSignal(controller.signal, this.#base),
|
||||
{ principal },
|
||||
)
|
||||
.catch(() => {
|
||||
// The scope was sealed (a disconnect, `close`) before it answered: the request is over.
|
||||
answered.resolve(this.refusal(503, admission, router, head));
|
||||
});
|
||||
return { answer: await answered.promise, scope };
|
||||
}
|
||||
|
||||
/** Runs the wraps and the route in the request's scope `ctx`, and makes its answer (a failure: a reported 500). */
|
||||
async #run(
|
||||
incoming: Incoming,
|
||||
request: WebRequest,
|
||||
match: RouteMatch,
|
||||
admitted: Admitted,
|
||||
router: Router,
|
||||
foreign: boolean,
|
||||
ctx: Ctx<P>,
|
||||
): Promise<Answer> {
|
||||
let response: HttpResponse;
|
||||
try {
|
||||
response = await chainOf(match, ctx)(request);
|
||||
} catch (error) {
|
||||
response = this.#failure(error, match);
|
||||
}
|
||||
// Downloads are never another site's to take (rule 6).
|
||||
if (response.kind === "bytes" && foreign) response = { kind: "error", status: 403 };
|
||||
const failed = (error: unknown): void =>
|
||||
this.#onError({ kind: "response_failed", pattern: match.pattern, error });
|
||||
try {
|
||||
return answerOf(response, incoming, this.#defaults.of(admitted, router, request.nonce), ctx, failed);
|
||||
} catch (error) {
|
||||
failed(error);
|
||||
return this.refusal(500, admitted, router, request.method === "HEAD");
|
||||
}
|
||||
}
|
||||
|
||||
/** What an error escaping the chain answers: its `HttpFailure`'s status, else a reported 500 without its message. */
|
||||
#failure(error: unknown, match: RouteMatch): HttpResponse {
|
||||
const thrown = error instanceof Thrown ? error : new Thrown(ROUTE, match.plugin, error);
|
||||
if (thrown.cause instanceof HttpFailure) return { kind: "error", status: thrown.cause.status };
|
||||
// Chord refused the caller (an anonymous scope's remote object, a route's `authorize`): no failure of the route.
|
||||
if (isChordError(thrown.cause, "forbidden")) return { kind: "error", status: 403 };
|
||||
if (thrown.by.kind === "route") {
|
||||
this.#onError({ kind: "route_failed", pattern: match.pattern, plugin: thrown.plugin, error: thrown.cause });
|
||||
} else {
|
||||
this.#onError({ kind: "wrap_failed", name: thrown.by.name, plugin: thrown.plugin, error: thrown.cause });
|
||||
}
|
||||
return { kind: "error", status: 500 };
|
||||
}
|
||||
|
||||
/**
|
||||
* The principal the authenticators give `request`; one that throws is reported and skipped. Raced against `signal`
|
||||
* (a disconnect, `close`'s seal): an authenticator ignoring it no longer holds the request; the caller sees the
|
||||
* abort and answers 503.
|
||||
*/
|
||||
async #principal(registries: Registries, request: HttpRequest, signal: AbortSignal): Promise<P | undefined> {
|
||||
const failed = (plugin: string, error: unknown): void => this.#hookFailed("authenticate", plugin, error);
|
||||
const context = withAbortSignal(signal, this.#base);
|
||||
const authenticating = authenticate(registries.authenticators.value, request, failed, context);
|
||||
// The registry's values are typed by no principal type: an app's authenticators return its principals.
|
||||
return (await awaitWithContext(authenticating, context).catch(() => undefined)) as P | undefined;
|
||||
}
|
||||
|
||||
#hookFailed(hook: HookName, plugin: string | undefined, error: unknown): void {
|
||||
this.#onError({ kind: "hook_failed", hook, ...(plugin === undefined ? {} : { plugin }), error });
|
||||
}
|
||||
|
||||
/** The registries: resolved once, waiting for the publication; a failed try is tried again later. */
|
||||
#resolve(): Promise<Registries> {
|
||||
this.#resolving ??= Promise.all([
|
||||
this.#host.registry(Http, this.#context),
|
||||
this.#host.registry(Authenticators, this.#context),
|
||||
]).then(
|
||||
([router, authenticators]) => {
|
||||
this.#registries = { router, authenticators };
|
||||
// A close may come first: nothing to follow then (Astra's recheck R4: the stopped watch threw unhandled).
|
||||
this.#follow(router).catch(() => {});
|
||||
return this.#registries;
|
||||
},
|
||||
(error: unknown) => {
|
||||
this.#resolving = undefined;
|
||||
throw error;
|
||||
},
|
||||
);
|
||||
return this.#resolving;
|
||||
}
|
||||
|
||||
/** Follows the router until `close`: connections its changes no longer admit are terminated. */
|
||||
async #follow(router: RegistryValue<Router>): Promise<void> {
|
||||
if (this.#closing.signal.aborted) return;
|
||||
const watch = await router.watch(withAbortSignal(this.#closing.signal, this.#base));
|
||||
if (this.#closing.signal.aborted) return;
|
||||
watch.start(async (call) => {
|
||||
if (call.kind === "value") this.#upgrades.revoke(call.value);
|
||||
});
|
||||
}
|
||||
|
||||
/** One upgrade, tracked from its arrival: refused while closing, else admitted and checked, then handed out. */
|
||||
async #upgrade(message: IncomingMessage, stream: Duplex, head: Buffer): Promise<void> {
|
||||
const controller = new AbortController();
|
||||
// A client gone while its upgrade authenticates aborts that work (Astra's recheck R1, Sol's N5).
|
||||
const gone = (): void => controller.abort(new Error("http: the client disconnected"));
|
||||
if (stream.destroyed) gone();
|
||||
else {
|
||||
// A clean disconnect (FIN) is seen only by reading: nothing reads a detached upgrade socket, so start it
|
||||
// (Fable's second recheck: `end()` while authenticating went unnoticed).
|
||||
stream.once("close", gone);
|
||||
stream.once("end", gone);
|
||||
stream.read(0);
|
||||
}
|
||||
const running: Running = { controller, done: Promise.resolve() };
|
||||
const work = this.#upgradeSteps(message, stream, head, controller.signal);
|
||||
running.done = work;
|
||||
this.#running.add(running);
|
||||
try {
|
||||
await work;
|
||||
} finally {
|
||||
stream.removeListener("close", gone);
|
||||
stream.removeListener("end", gone);
|
||||
this.#running.delete(running);
|
||||
}
|
||||
}
|
||||
|
||||
/** The steps of an upgrade (the same rules as a request's; a socket needs a principal), checked again at hand-out. */
|
||||
async #upgradeSteps(message: IncomingMessage, stream: Duplex, head: Buffer, signal: AbortSignal): Promise<void> {
|
||||
if (this.#closing.signal.aborted) return refuse(stream, 503);
|
||||
const request = NodeIncoming.of(message);
|
||||
if (!(request instanceof NodeIncoming)) return refuse(stream, request.status);
|
||||
let registries: Registries;
|
||||
try {
|
||||
registries = this.#registries ?? (await this.#resolve());
|
||||
} catch {
|
||||
return refuse(stream, 503);
|
||||
}
|
||||
if (this.#closing.signal.aborted || signal.aborted) return refuse(stream, 503);
|
||||
const before = registries.router.value;
|
||||
const admission = admit(request, connectionOf(message), before.hosts, before.relays);
|
||||
if (refused(admission)) {
|
||||
if (admission.failed !== undefined) this.#hookFailed("relay", admission.failed.relay, admission.failed.error);
|
||||
return refuse(stream, 403);
|
||||
}
|
||||
const sent = request.header("origin");
|
||||
if (isForeign(request) || (sent !== null && sent !== admission.origin)) return refuse(stream, 403);
|
||||
const webRequest = new WebRequest(request, admission, this.#nonces.next());
|
||||
const principal = await this.#principal(registries, webRequest, signal);
|
||||
if (this.#closing.signal.aborted || signal.aborted) return refuse(stream, 503);
|
||||
if (principal === undefined) return refuse(stream, 403);
|
||||
// The router now: a host, a relay or a socket removed while authenticating refuses the upgrade.
|
||||
const router = registries.router.value;
|
||||
const { allowed, relay } = admission;
|
||||
if (
|
||||
(allowed !== undefined && !router.hosts.has(allowed)) ||
|
||||
(relay !== undefined && !router.relays.includes(relay))
|
||||
) {
|
||||
return refuse(stream, 403);
|
||||
}
|
||||
const { links } = this.#options;
|
||||
if (links !== undefined && webRequest.path === links.path) {
|
||||
let options: LinkOptions;
|
||||
try {
|
||||
options =
|
||||
(typeof links.options === "function" ? links.options(webRequest, principal) : links.options) ?? {};
|
||||
} catch (error) {
|
||||
this.#hookFailed("links.options", undefined, error);
|
||||
return refuse(stream, 403);
|
||||
}
|
||||
const admitted = { allowed, relay, principal, ...options };
|
||||
return this.#upgrades.link(message, stream, head, admitted, this.#host, this.#context);
|
||||
}
|
||||
const match = router.socket(webRequest.path);
|
||||
if (match === "bad_path") return refuse(stream, 400);
|
||||
if (match === undefined) return refuse(stream, 404);
|
||||
webRequest.params = match.params;
|
||||
const fail = ({ pattern, plugin, error }: { pattern: string; plugin: string; error: unknown }) =>
|
||||
this.#onError({ kind: "socket_failed", pattern, plugin, error });
|
||||
const admitted = { allowed, relay, principal, match, request: webRequest };
|
||||
this.#upgrades.socket(message, stream, head, admitted, this.#host, this.#context, fail);
|
||||
}
|
||||
|
||||
/** The listener's open: Chord's own Node server listening at `listen`. */
|
||||
async #listen(listen: ListenAddress, context: Context): Promise<NodeListener> {
|
||||
const listener = await NodeListener.listen(this, listen, context);
|
||||
this.#listening.resolve(listener.address);
|
||||
return listener;
|
||||
}
|
||||
|
||||
/** `close`: refuses from now on, then waits for sockets and running requests until the deadline, then seals. */
|
||||
async #close(context: Context): Promise<void> {
|
||||
this.#closing.abort();
|
||||
this.#listening.reject(new Error("http: closed"));
|
||||
const timeout = this.#options.closeTimeout ?? CLOSE_TIMEOUT;
|
||||
const signal = context.abortSignal;
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
let aborted: (() => void) | undefined;
|
||||
const deadline = new Promise<void>((resolve) => {
|
||||
// Already aborted: no wait at all (Sol's recheck N9).
|
||||
if (signal?.aborted) return resolve();
|
||||
timer = setTimeout(resolve, timeout);
|
||||
aborted = () => resolve();
|
||||
signal?.addEventListener("abort", aborted, { once: true });
|
||||
});
|
||||
const running = Promise.all([...this.#running].map(({ done }) => done));
|
||||
await Promise.race([Promise.all([this.#upgrades.close(deadline), running]), deadline]);
|
||||
clearTimeout(timer);
|
||||
if (aborted !== undefined) signal?.removeEventListener("abort", aborted);
|
||||
for (const { controller } of this.#running) controller.abort(new Error("http: closed"));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* `stream` as read by its consumer, `finished` called once it is read to its end, failed or cancelled (a mounted
|
||||
* handler's `bytes` body: the request's scope lasts until then).
|
||||
*/
|
||||
function tracked(stream: ReadableStream<Uint8Array>, finished: () => void): ReadableStream<Uint8Array> {
|
||||
const reader = stream.getReader();
|
||||
// The owned stream ending (read whole, or errored by the scope's seal) ends the request too, whether or not its
|
||||
// consumer pulls again (Sol's second recheck B: an aborted `Response` nobody touched kept its request and scope).
|
||||
reader.closed.then(finished, finished);
|
||||
return new ReadableStream<Uint8Array>(
|
||||
{
|
||||
pull: async (controller) => {
|
||||
try {
|
||||
const next = await reader.read();
|
||||
if (next.done) {
|
||||
controller.close();
|
||||
finished();
|
||||
} else {
|
||||
controller.enqueue(next.value);
|
||||
}
|
||||
} catch (error) {
|
||||
controller.error(error);
|
||||
finished();
|
||||
}
|
||||
},
|
||||
cancel: async (reason) => {
|
||||
finished();
|
||||
await reader.cancel(reason);
|
||||
},
|
||||
},
|
||||
{ highWaterMark: 0 },
|
||||
);
|
||||
}
|
||||
|
||||
/** Refuses an upgrade with `status` and closes its connection. */
|
||||
function refuse(stream: Duplex, status: number): void {
|
||||
stream.end(`HTTP/1.1 ${status} ${STATUS[status] ?? ""}\r\nConnection: close\r\nContent-Length: 0\r\n\r\n`);
|
||||
}
|
||||
|
||||
/** The reason phrases of the statuses an upgrade is refused with. */
|
||||
const STATUS: Readonly<Record<number, string>> = {
|
||||
400: "Bad Request",
|
||||
403: "Forbidden",
|
||||
404: "Not Found",
|
||||
501: "Not Implemented",
|
||||
503: "Service Unavailable",
|
||||
};
|
||||
|
||||
/** Who threw out of a request's chain: the route, or a wrap (by name), with its plugin. */
|
||||
class Thrown extends Error {
|
||||
readonly by: { readonly kind: "route" } | { readonly kind: "wrap"; readonly name: string };
|
||||
readonly plugin: string;
|
||||
|
||||
constructor(by: Thrown["by"], plugin: string, cause: unknown) {
|
||||
super("thrown", { cause });
|
||||
this.by = by;
|
||||
this.plugin = plugin;
|
||||
}
|
||||
}
|
||||
|
||||
/** The route of `match` inside its wraps (outermost first), each throw tagged with who threw it. */
|
||||
function chainOf(match: RouteMatch, ctx: Ctx): (request: HttpRequest) => Promise<HttpResponse> {
|
||||
let next: (request: HttpRequest) => Promise<HttpResponse> = tagged(match.handler, match.plugin, ctx);
|
||||
for (let index = match.wraps.length - 1; index >= 0; index--) {
|
||||
const { wrap, plugin } = match.wraps[index]!;
|
||||
const inner = next;
|
||||
next = async (request) => {
|
||||
try {
|
||||
return await wrap.wrap(request, inner, ctx);
|
||||
} catch (error) {
|
||||
if (error instanceof Thrown) throw error;
|
||||
throw new Thrown({ kind: "wrap", name: wrap.name }, plugin, error);
|
||||
}
|
||||
};
|
||||
}
|
||||
return next;
|
||||
}
|
||||
|
||||
/** `route` with its throws tagged as the route's. */
|
||||
function tagged(route: HttpRoute, plugin: string, ctx: Ctx): (request: HttpRequest) => Promise<HttpResponse> {
|
||||
return async (request) => {
|
||||
try {
|
||||
return await route(request, ctx);
|
||||
} catch (error) {
|
||||
throw new Thrown(ROUTE, plugin, error);
|
||||
}
|
||||
};
|
||||
}
|
||||
43
packages/chord/src/next-node/http/incoming.ts
Normal file
43
packages/chord/src/next-node/http/incoming.ts
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
// What the component reads of a request (decision "The HTTP core is a fetch handler"): one shape for a Web `Request`
|
||||
// (`fetch`) and for Chord's own Node server, so admission, routing and the baseline have one implementation. The Node
|
||||
// side reads Node's own parsed headers and builds a Web `Request` only when something asks for it (a route's
|
||||
// `headers` or body, a relay's `verify`).
|
||||
|
||||
/** A request as the component reads it. */
|
||||
export interface Incoming {
|
||||
readonly method: string;
|
||||
/** The URL as the URL parser serialized it (dot segments resolved, `\` as `/`). */
|
||||
readonly url: string;
|
||||
/** A header's value, several of one name joined with `", "` (as `Headers.get`); `null` without it. */
|
||||
header(name: string): string | null;
|
||||
/** Whether a proxy forwarded it: a `Forwarded` or `X-Forwarded-*` header. */
|
||||
forwarded(): boolean;
|
||||
/** Calls `abort` once when the client goes away before its answer is written (at once when it has). */
|
||||
cancelled(abort: (reason: unknown) => void): void;
|
||||
/** The Web `Request`, made at the first call on the Node side. */
|
||||
request(): Request;
|
||||
}
|
||||
|
||||
/** `request` as the component reads it. */
|
||||
export function webIncoming(request: Request): Incoming {
|
||||
return {
|
||||
method: request.method,
|
||||
url: request.url,
|
||||
header: (name) => request.headers.get(name),
|
||||
forwarded: () => {
|
||||
for (const name of request.headers.keys()) if (isForwarding(name)) return true;
|
||||
return false;
|
||||
},
|
||||
cancelled: (abort) => {
|
||||
const { signal } = request;
|
||||
if (signal.aborted) abort(signal.reason);
|
||||
else signal.addEventListener("abort", () => abort(signal.reason), { once: true });
|
||||
},
|
||||
request: () => request,
|
||||
};
|
||||
}
|
||||
|
||||
/** Whether a (lowercased) header name says a proxy forwarded the request. */
|
||||
export function isForwarding(name: string): boolean {
|
||||
return name === "forwarded" || name.startsWith("x-forwarded-");
|
||||
}
|
||||
34
packages/chord/src/next-node/http/index.ts
Normal file
34
packages/chord/src/next-node/http/index.ts
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
// Chord v2's HTTP component (`@earendil-works/chord/next/http`, slice 10 §2): a Web-standard `fetch` handler built
|
||||
// from the routes, wraps and sockets plugins contribute to the registry `Http`, with the security baseline on by
|
||||
// default, Chord's own Node listener, a Node upgrade adapter and the Node request conversion for apps that mount it.
|
||||
// Node-only; `ws` is an optional peer dependency, imported only here.
|
||||
|
||||
export type { Connection, Provenance } from "./admission.ts";
|
||||
export { type Authenticator, Authenticators } from "./authenticate.ts";
|
||||
export { type CookieOptions, cookie, readCookie } from "./cookies.ts";
|
||||
export { type FileResponseOptions, fileResponse } from "./file.ts";
|
||||
export {
|
||||
createHttp,
|
||||
type HookName,
|
||||
type HttpComponent,
|
||||
type HttpError,
|
||||
HttpListener,
|
||||
type HttpOptions,
|
||||
type LinksOptions,
|
||||
} from "./http.ts";
|
||||
export { type ListenAddress, type NodeRequest, nodeRequest } from "./node.ts";
|
||||
export { HttpFailure, type HttpRequest } from "./request.ts";
|
||||
export type { HttpResponse } from "./response.ts";
|
||||
export {
|
||||
Http,
|
||||
type HttpContribution,
|
||||
type HttpRelay,
|
||||
type HttpRoute,
|
||||
type HttpRouteOptions,
|
||||
type HttpSocket,
|
||||
type HttpWrap,
|
||||
type Router,
|
||||
type WebSocketHandle,
|
||||
type WrapPhase,
|
||||
} from "./router.ts";
|
||||
export type { LinkOptions } from "./upgrades.ts";
|
||||
317
packages/chord/src/next-node/http/node.ts
Normal file
317
packages/chord/src/next-node/http/node.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
// Chord's own Node server (decision "The HTTP server is a native object"; "The HTTP core is a fetch handler; Chord's
|
||||
// own Node server is the easy path"): the listener's native, a Node `http` server that hands each request to the
|
||||
// component as it reads requests (origin-form targets only; a Web `Request` made only when something asks for one, its
|
||||
// body streamed with backpressure), and writes its answer: whole, or pulled piece by piece as the socket drains. A body the route left unread is drained after the
|
||||
// response when small, else the connection closes; the request's scope ends once the response is flushed or abandoned.
|
||||
|
||||
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
|
||||
import type { Duplex } from "node:stream";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import type { Connection } from "./admission.ts";
|
||||
import { type Incoming, isForwarding } from "./incoming.ts";
|
||||
import type { Answer } from "./response.ts";
|
||||
import { UNCARRIED } from "./router.ts";
|
||||
|
||||
/** Where a listener listens. */
|
||||
export type ListenAddress = { readonly host: string; readonly port: number };
|
||||
|
||||
/** An answer and what its consumer calls once the body is written or abandoned (the request's scope then ends). */
|
||||
export type Served = { readonly answer: Answer; readonly finished: () => void };
|
||||
|
||||
/** What the listener needs of the component. */
|
||||
export interface Handler {
|
||||
/** Serves a request: never throws. */
|
||||
serve(incoming: Incoming, connection: Connection): Promise<Served>;
|
||||
/** The answer for a request refused before it could be made one (default headers only). */
|
||||
refusal(status: number): Answer;
|
||||
readonly upgrade: (message: IncomingMessage, socket: Duplex, head: Buffer) => void;
|
||||
}
|
||||
|
||||
/** The listener's native value: a listening Node server. */
|
||||
export class NodeListener {
|
||||
readonly address: ListenAddress;
|
||||
readonly #server: Server;
|
||||
readonly #closed: Promise<void>;
|
||||
|
||||
private constructor(server: Server, address: ListenAddress) {
|
||||
this.#server = server;
|
||||
this.address = address;
|
||||
this.#closed = new Promise((resolve) => server.once("close", () => resolve()));
|
||||
}
|
||||
|
||||
/** A Node server for `handler` listening at `listen`; rejects with the listen error (a port in use). */
|
||||
static async listen(handler: Handler, listen: ListenAddress, context: Context): Promise<NodeListener> {
|
||||
const server = createServer((message, res) => {
|
||||
void request(handler, message, res).catch(() => res.destroy());
|
||||
});
|
||||
server.on("upgrade", handler.upgrade);
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const failed = (error: Error): void => reject(error);
|
||||
server.once("error", failed);
|
||||
server.listen(listen.port, listen.host, () => {
|
||||
server.removeListener("error", failed);
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
if (context.abortSignal?.aborted) {
|
||||
server.close();
|
||||
throw context.abortSignal.reason;
|
||||
}
|
||||
const address = server.address();
|
||||
const port = typeof address === "object" && address !== null ? address.port : listen.port;
|
||||
return new NodeListener(server, { host: listen.host, port });
|
||||
}
|
||||
|
||||
/** Stops accepting connections and closes the idle ones (the component's close runs next). */
|
||||
stop(): void {
|
||||
this.#server.close();
|
||||
this.#server.closeIdleConnections();
|
||||
}
|
||||
|
||||
/** Closes every connection still open and resolves once the server closed. */
|
||||
async closed(): Promise<void> {
|
||||
this.#server.close();
|
||||
this.#server.closeAllConnections();
|
||||
await this.#closed;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A Node request as the component reads it: Node's parsed headers, its URL parsed once, the Web `Request` made only
|
||||
* when something asks for it (a route's `headers` or body, a relay).
|
||||
*/
|
||||
export class NodeIncoming implements Incoming {
|
||||
readonly method: string;
|
||||
readonly url: string;
|
||||
readonly #message: IncomingMessage;
|
||||
/** The response, to notice a client gone (none for an upgrade). */
|
||||
readonly #res: ServerResponse | undefined;
|
||||
#request: Request | undefined;
|
||||
|
||||
private constructor(message: IncomingMessage, url: string, res: ServerResponse | undefined) {
|
||||
this.method = message.method ?? "GET";
|
||||
this.url = url;
|
||||
this.#message = message;
|
||||
this.#res = res;
|
||||
}
|
||||
|
||||
/**
|
||||
* `message` as the component reads it, or what refuses it at the wire (the listener and `nodeRequest` alike): an
|
||||
* absolute-form target 403, a target that does not parse 400, a `Host` with userinfo 403 (never a loopback or
|
||||
* allowed name), a method a Web `Request` cannot carry (`CONNECT`, `TRACE`, `TRACK`) 501: served for no resource.
|
||||
*/
|
||||
static of(message: IncomingMessage, res?: ServerResponse): NodeIncoming | WireRefusal {
|
||||
const target = message.url ?? "";
|
||||
// Only origin-form targets: an absolute-form one names an authority of its own, `*` none.
|
||||
if (!target.startsWith("/")) return { kind: "refused", status: 403 };
|
||||
if (UNCARRIED.has(message.method ?? "GET")) return { kind: "refused", status: 501 };
|
||||
let url: URL;
|
||||
try {
|
||||
// The authority is the `Host` the client sent; admission checks it, so a strange one parses but is refused.
|
||||
url = new URL(`http://${message.headers.host ?? "invalid"}${target}`);
|
||||
} catch {
|
||||
return { kind: "refused", status: 400 };
|
||||
}
|
||||
if (url.username !== "" || url.password !== "") return { kind: "refused", status: 403 };
|
||||
return new NodeIncoming(message, url.href, res);
|
||||
}
|
||||
|
||||
header(name: string): string | null {
|
||||
return this.#message.headersDistinct[name]?.join(", ") ?? null;
|
||||
}
|
||||
|
||||
forwarded(): boolean {
|
||||
for (const name in this.#message.headersDistinct) if (isForwarding(name)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
cancelled(abort: (reason: unknown) => void): void {
|
||||
const res = this.#res;
|
||||
if (res === undefined) return;
|
||||
const gone = (): void => abort(new Error("http: the client disconnected"));
|
||||
if (res.closed) gone();
|
||||
else res.once("close", () => res.writableFinished || gone());
|
||||
}
|
||||
|
||||
request(): Request {
|
||||
this.#request ??= webRequestOf(this.#message, this.url);
|
||||
return this.#request;
|
||||
}
|
||||
}
|
||||
|
||||
/** What refuses a Node request at the wire, with its status. */
|
||||
export type WireRefusal = { readonly kind: "refused"; readonly status: 400 | 403 | 501 };
|
||||
|
||||
/** A Node request as an app mounting `fetch` on its own Node server gets it: a Web `Request`, or refused. */
|
||||
export type NodeRequest =
|
||||
| { readonly kind: "request"; readonly request: Request; readonly connection: Connection }
|
||||
| WireRefusal;
|
||||
|
||||
/**
|
||||
* Converts `message` for `http.fetch` on an app's own Node server, the whole conversion being the wire boundary: what
|
||||
* the listener refuses is refused with the same status (`NodeIncoming.of`), any other construction failure 400, never a
|
||||
* throw (Sol's rechecks N6 and A: a hand-made `Request` crashed the app on a malformed target, `TRACE /` or
|
||||
* `Host: user@…`). Raw headers keep their duplicates (two `Host`s: admission refuses); the signal aborts when the
|
||||
* client goes away before `res` finished.
|
||||
*/
|
||||
export function nodeRequest(message: IncomingMessage, res: ServerResponse): NodeRequest {
|
||||
const incoming = NodeIncoming.of(message, res);
|
||||
if (!(incoming instanceof NodeIncoming)) return incoming;
|
||||
const gone = new AbortController();
|
||||
incoming.cancelled((reason) => gone.abort(reason));
|
||||
let request: Request;
|
||||
try {
|
||||
request = webRequestOf(message, incoming.url, gone.signal);
|
||||
} catch {
|
||||
return { kind: "refused", status: 400 };
|
||||
}
|
||||
return { kind: "request", request, connection: connectionOf(message) };
|
||||
}
|
||||
|
||||
/**
|
||||
* The Web `Request` of `message` at `url`: its raw headers (duplicates kept), its body streamed when it has one, and
|
||||
* `signal` when the caller follows the client.
|
||||
*/
|
||||
function webRequestOf(message: IncomingMessage, url: string, signal?: AbortSignal): Request {
|
||||
const headers = new Headers();
|
||||
const raw = message.rawHeaders;
|
||||
for (let index = 0; index < raw.length; index += 2) headers.append(raw[index]!, raw[index + 1]!);
|
||||
const method = message.method ?? "GET";
|
||||
const init: RequestInit & { duplex?: "half" } = { method, headers, ...(signal === undefined ? {} : { signal }) };
|
||||
if (hasBody(message) && method !== "GET" && method !== "HEAD") {
|
||||
init.body = bodyOf(message);
|
||||
init.duplex = "half";
|
||||
}
|
||||
return new Request(url, init);
|
||||
}
|
||||
|
||||
/** What a Node socket knows of its connection. */
|
||||
export function connectionOf(message: IncomingMessage): Connection {
|
||||
const { localAddress, localPort, remoteAddress } = message.socket;
|
||||
return {
|
||||
...(localAddress === undefined ? {} : { localAddress }),
|
||||
...(localPort === undefined ? {} : { localPort }),
|
||||
...(remoteAddress === undefined ? {} : { remoteAddress }),
|
||||
};
|
||||
}
|
||||
|
||||
/** One request: served (an absolute-form target: 403; a URL that does not parse: 400), its answer written. */
|
||||
async function request(handler: Handler, message: IncomingMessage, res: ServerResponse): Promise<void> {
|
||||
const incoming = NodeIncoming.of(message, res);
|
||||
if (!(incoming instanceof NodeIncoming)) {
|
||||
return write(message, res, { answer: handler.refusal(incoming.status), finished: () => {} });
|
||||
}
|
||||
await write(message, res, await handler.serve(incoming, connectionOf(message)));
|
||||
}
|
||||
|
||||
/** What `json` and `body` leave unread that is drained after the response, keeping the connection: 64 KiB. */
|
||||
const DRAIN_LIMIT = 64 * 1024;
|
||||
|
||||
/** Writes `served`'s answer to `res`; what is left of the request's body is drained after it or closes the connection. */
|
||||
async function write(message: IncomingMessage, res: ServerResponse, { answer, finished }: Served): Promise<void> {
|
||||
// A client gone before its answer: nothing to write.
|
||||
if (res.closed) return finished();
|
||||
res.once("close", finished);
|
||||
const left = message.complete || !hasBody(message) ? "none" : declared(message) <= DRAIN_LIMIT ? "drain" : "close";
|
||||
const headers: string[] = [];
|
||||
for (const [name, value] of answer.headers.lines()) headers.push(name, value);
|
||||
if (left === "close") headers.push("connection", "close");
|
||||
res.writeHead(answer.status, headers);
|
||||
const { body } = answer;
|
||||
if (body instanceof ReadableStream) await pump(body, res);
|
||||
else res.end(body ?? undefined);
|
||||
if (left === "drain") drain(message);
|
||||
}
|
||||
|
||||
/** Writes `body` to `res` as the socket drains; a body that fails (reported by the component) destroys the connection. */
|
||||
async function pump(body: ReadableStream<Uint8Array>, res: ServerResponse): Promise<void> {
|
||||
const reader = body.getReader();
|
||||
const cancel = (): void => void reader.cancel().catch(() => {});
|
||||
res.once("close", cancel);
|
||||
for (;;) {
|
||||
let next: ReadableStreamReadResult<Uint8Array>;
|
||||
try {
|
||||
next = await reader.read();
|
||||
} catch {
|
||||
res.destroy();
|
||||
return;
|
||||
}
|
||||
if (next.done) break;
|
||||
if (res.destroyed) return;
|
||||
if (!res.write(next.value)) await drained(res);
|
||||
}
|
||||
res.removeListener("close", cancel);
|
||||
res.end();
|
||||
}
|
||||
|
||||
/** Resolves once `res` drained or closed. */
|
||||
function drained(res: ServerResponse): Promise<void> {
|
||||
return new Promise((resolve) => {
|
||||
const done = (): void => {
|
||||
res.removeListener("drain", done);
|
||||
res.removeListener("close", done);
|
||||
resolve();
|
||||
};
|
||||
res.once("drain", done);
|
||||
res.once("close", done);
|
||||
});
|
||||
}
|
||||
|
||||
/** A request body as a Web stream: read as its consumer pulls (the socket paused meanwhile); a disconnect errors it. */
|
||||
function bodyOf(message: IncomingMessage): ReadableStream<Uint8Array> {
|
||||
// Whether the stream still takes chunks (not ended, errored or cancelled).
|
||||
let open = true;
|
||||
return new ReadableStream<Uint8Array>(
|
||||
{
|
||||
start: (controller) => {
|
||||
message.on("data", (chunk: Buffer) => {
|
||||
if (!open) return;
|
||||
controller.enqueue(chunk);
|
||||
if ((controller.desiredSize ?? 0) <= 0) message.pause();
|
||||
});
|
||||
message.once("end", () => {
|
||||
if (open) controller.close();
|
||||
open = false;
|
||||
});
|
||||
message.once("close", () => {
|
||||
if (open && !message.complete) controller.error(new Error("http: the client disconnected"));
|
||||
open = false;
|
||||
});
|
||||
message.pause();
|
||||
},
|
||||
pull: () => {
|
||||
message.resume();
|
||||
},
|
||||
// The rest stays unread: the response closes the connection (or drains a small rest).
|
||||
cancel: () => {
|
||||
open = false;
|
||||
message.pause();
|
||||
},
|
||||
},
|
||||
new ByteLengthQueuingStrategy({ highWaterMark: DRAIN_LIMIT }),
|
||||
);
|
||||
}
|
||||
|
||||
/** Reads and drops the rest of the body (at most 64 KiB); more than that, or an error, closes the connection. */
|
||||
function drain(message: IncomingMessage): void {
|
||||
let read = 0;
|
||||
message.on("data", (chunk: Buffer) => {
|
||||
read += chunk.byteLength;
|
||||
if (read > DRAIN_LIMIT) message.socket.destroy();
|
||||
});
|
||||
message.on("error", () => message.socket.destroy());
|
||||
message.resume();
|
||||
}
|
||||
|
||||
/** Whether `message` carries a body (a declared length above zero, or chunked). */
|
||||
function hasBody(message: IncomingMessage): boolean {
|
||||
const length = message.headers["content-length"];
|
||||
if (length !== undefined) return Number(length) > 0;
|
||||
return message.headers["transfer-encoding"] !== undefined;
|
||||
}
|
||||
|
||||
/** The declared length of `message`'s body; infinite when chunked. */
|
||||
function declared(message: IncomingMessage): number {
|
||||
const length = Number(message.headers["content-length"] ?? Number.NaN);
|
||||
return Number.isFinite(length) ? length : Number.POSITIVE_INFINITY;
|
||||
}
|
||||
145
packages/chord/src/next-node/http/request.ts
Normal file
145
packages/chord/src/next-node/http/request.ts
Normal file
|
|
@ -0,0 +1,145 @@
|
|||
// Requests (slice 10 §2, D4): what a route, a wrap, a socket handler and an authenticator see of an HTTP request (a
|
||||
// Web `Request` once admitted), and its body, read only when asked: `json(limit?)` (1 MiB by default, malformed 400,
|
||||
// too large 413) and `body(limit?)`, an async iterable of chunks.
|
||||
|
||||
import type { JsonValue } from "../../json.ts";
|
||||
import type { Provenance } from "./admission.ts";
|
||||
import type { Incoming } from "./incoming.ts";
|
||||
|
||||
/** A request as routes see it, once admitted: `origin` and `provenance` come only from admission. */
|
||||
export interface HttpRequest {
|
||||
readonly method: string;
|
||||
/** The path, without the query, percent-encoded, its dot segments resolved (by the URL parser). */
|
||||
readonly path: string;
|
||||
/** The matched pattern's params, percent-decoded (`*` for the rest of a `/*` pattern). */
|
||||
readonly params: Readonly<Record<string, string>>;
|
||||
readonly query: URLSearchParams;
|
||||
/** The request's headers; read only. */
|
||||
readonly headers: Headers;
|
||||
/** `http://<host>` on loopback, `https://<host>` for an allowed host, a relay's origin for a relayed request. */
|
||||
readonly origin: string;
|
||||
/** Where it came from (decision "The HTTP component's security baseline" (1), "Relays"). */
|
||||
readonly provenance: Provenance;
|
||||
/** The server's script nonce: a page's inline scripts carry it (`<script nonce="…">`), the CSP allows them. */
|
||||
readonly nonce: string;
|
||||
/** The body as JSON, at most `limit` bytes (1 MiB by default): `HttpFailure` 400 when malformed, 413 when larger. */
|
||||
json(limit?: number): Promise<JsonValue>;
|
||||
/** The body's chunks, at most `limit` bytes (1 MiB by default): `HttpFailure` 413 past it, 400 when cut short. */
|
||||
body(limit?: number): AsyncIterable<Uint8Array>;
|
||||
}
|
||||
|
||||
/** A failure that answers its request with `status` when it escapes a route (400 malformed, 413 too large). */
|
||||
export class HttpFailure extends Error {
|
||||
readonly status: number;
|
||||
|
||||
constructor(status: number, message: string) {
|
||||
super(message);
|
||||
this.name = "HttpFailure";
|
||||
this.status = status;
|
||||
}
|
||||
}
|
||||
|
||||
/** What `json` and `body` read unless a route says otherwise: 1 MiB. */
|
||||
const DEFAULT_LIMIT = 1024 * 1024;
|
||||
|
||||
/** The params of a request no pattern matched (yet). */
|
||||
const NO_PARAMS: Readonly<Record<string, string>> = {};
|
||||
|
||||
/** A request over what the component reads of it; the component sets its params once it matched a route. */
|
||||
export class WebRequest implements HttpRequest {
|
||||
readonly method: string;
|
||||
readonly path: string;
|
||||
params: Readonly<Record<string, string>> = NO_PARAMS;
|
||||
readonly origin: string;
|
||||
readonly provenance: Provenance;
|
||||
readonly nonce: string;
|
||||
readonly #incoming: Incoming;
|
||||
/** The query string (without `?`), parsed when first read. */
|
||||
readonly #search: string;
|
||||
#query: URLSearchParams | undefined;
|
||||
/** Whether a reader took the body (it is read once). */
|
||||
#taken = false;
|
||||
|
||||
constructor(
|
||||
incoming: Incoming,
|
||||
admitted: { readonly origin: string; readonly provenance: Provenance },
|
||||
nonce: string,
|
||||
) {
|
||||
this.method = incoming.method;
|
||||
// The URL is parsed and serialized already (dot segments resolved, `\` as `/`): only split it here.
|
||||
const { path, search } = splitUrl(incoming.url);
|
||||
this.path = path;
|
||||
this.#search = search;
|
||||
this.origin = admitted.origin;
|
||||
this.provenance = admitted.provenance;
|
||||
this.nonce = nonce;
|
||||
this.#incoming = incoming;
|
||||
}
|
||||
|
||||
/** The headers: on Chord's Node server the Web `Request` is made only when something reads them. */
|
||||
get headers(): Headers {
|
||||
return this.#incoming.request().headers;
|
||||
}
|
||||
|
||||
get query(): URLSearchParams {
|
||||
this.#query ??= new URLSearchParams(this.#search);
|
||||
return this.#query;
|
||||
}
|
||||
|
||||
async json(limit = DEFAULT_LIMIT): Promise<JsonValue> {
|
||||
const chunks: Uint8Array[] = [];
|
||||
for await (const chunk of this.body(limit)) chunks.push(chunk);
|
||||
const text = Buffer.concat(chunks).toString("utf8");
|
||||
try {
|
||||
return JSON.parse(text) as JsonValue;
|
||||
} catch {
|
||||
throw new HttpFailure(400, "the body is no JSON");
|
||||
}
|
||||
}
|
||||
|
||||
async *body(limit = DEFAULT_LIMIT): AsyncIterable<Uint8Array> {
|
||||
if (this.#taken) throw new Error("request body: read twice");
|
||||
this.#taken = true;
|
||||
const declared = Number(this.#incoming.header("content-length") ?? Number.NaN);
|
||||
if (declared > limit) throw new HttpFailure(413, `the body is larger than ${limit} bytes`);
|
||||
const stream = this.#incoming.request().body;
|
||||
if (stream === null) return;
|
||||
const reader = stream.getReader();
|
||||
let read = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
let next: ReadableStreamReadResult<Uint8Array>;
|
||||
try {
|
||||
next = await reader.read();
|
||||
} catch {
|
||||
// A body cut short (a disconnect, a broken stream) is no body: a prefix must never parse as the
|
||||
// whole (Sol's recheck N4: valid JSON then an error ran the route).
|
||||
throw new HttpFailure(400, "the body could not be read whole");
|
||||
}
|
||||
if (next.done) return;
|
||||
read += next.value.byteLength;
|
||||
if (read > limit) throw new HttpFailure(413, `the body is larger than ${limit} bytes`);
|
||||
yield next.value;
|
||||
}
|
||||
} finally {
|
||||
// Past the limit, or a reader that stopped early: the rest stays unread (its server closes the connection
|
||||
// after the response); a cancel could reset the connection before the response is written.
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** A serialized URL's path (from the first `/` after the authority) and query (without `?`, no fragment). */
|
||||
export function splitUrl(url: string): { readonly path: string; readonly search: string } {
|
||||
const authority = url.indexOf("//") + 2;
|
||||
let start = url.indexOf("/", authority);
|
||||
let end = url.indexOf("#", authority);
|
||||
if (end < 0) end = url.length;
|
||||
const mark = url.indexOf("?", authority);
|
||||
if (start < 0 || (mark >= 0 && start > mark)) start = -1;
|
||||
const pathEnd = mark >= 0 && mark < end ? mark : end;
|
||||
return {
|
||||
path: start < 0 ? "/" : url.slice(start, pathEnd),
|
||||
search: mark >= 0 && mark < end ? url.slice(mark + 1, end) : "",
|
||||
};
|
||||
}
|
||||
258
packages/chord/src/next-node/http/response.ts
Normal file
258
packages/chord/src/next-node/http/response.ts
Normal file
|
|
@ -0,0 +1,258 @@
|
|||
// Responses (slice 10 §2, D5, R6; decision "The HTTP component's security baseline" (4), (6)): routes return
|
||||
// descriptions and the component turns them into an answer, a status, headers and a body: the default headers first,
|
||||
// the response's own replacing same-named ones (validated: a CR or LF throws), `json`, `html`, `redirect`, `error`,
|
||||
// and `bytes` read in 256 KiB pieces as the consumer pulls them (ranges 206/416, `HEAD`, an attachment unless inline,
|
||||
// safe download names; without a `size`, until an empty read).
|
||||
|
||||
import { STATUS_CODES } from "node:http";
|
||||
import type { JsonValue } from "../../json.ts";
|
||||
import type { Ctx } from "../../next/host/index.ts";
|
||||
import type { AnswerHeaders } from "./headers.ts";
|
||||
|
||||
/**
|
||||
* A route's answer, as a description; `headers` replace same-named default ones (a route widens its own CSP); a list
|
||||
* sends one line per value (several `Set-Cookie`).
|
||||
*/
|
||||
export type HttpResponse = { readonly headers?: Readonly<Record<string, string | readonly string[]>> } & (
|
||||
| { readonly kind: "json"; readonly value: JsonValue; readonly status?: number }
|
||||
| { readonly kind: "html"; readonly body: string; readonly status?: number }
|
||||
| {
|
||||
readonly kind: "bytes";
|
||||
readonly mime: string;
|
||||
/** The length: `Content-Length` and ranges; without it, read until `read` returns an empty chunk. */
|
||||
readonly size?: number;
|
||||
/** `length` bytes from `offset` (fewer only at the end of a body without `size`), in the request's scope. */
|
||||
readonly read: (offset: number, length: number, ctx: Ctx) => Promise<Uint8Array>;
|
||||
/** The file name it is saved under. */
|
||||
readonly download?: string;
|
||||
/** Shown in the browser instead of saved (an image, a video), with a `sandbox` CSP. */
|
||||
readonly inline?: true;
|
||||
/** Whether a single range is answered 206 (with `size`). */
|
||||
readonly ranges?: boolean;
|
||||
}
|
||||
| { readonly kind: "redirect"; readonly to: string; readonly status?: 302 | 303 | 307 }
|
||||
| { readonly kind: "error"; readonly status: number; readonly message?: string }
|
||||
);
|
||||
|
||||
/** What the component answers: a status, its headers, and a whole body, a pulled one, or none. */
|
||||
export type Answer = {
|
||||
readonly status: number;
|
||||
readonly headers: AnswerHeaders;
|
||||
/** A whole body is its text (UTF-8): Node writes a string body in one write with the head. */
|
||||
readonly body: string | ReadableStream<Uint8Array> | null;
|
||||
};
|
||||
|
||||
/** A `bytes` body that could not be read whole (a throwing or a short `read`). */
|
||||
export class ResponseFailure extends Error {
|
||||
constructor(message: string, options?: ErrorOptions) {
|
||||
super(message, options);
|
||||
this.name = "ResponseFailure";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The answer to `request` for `response` over `defaults` (which it changes): throws when a header is invalid (a CR or
|
||||
* LF) or the value is no JSON. A `bytes` body that cannot be read whole is reported to `failed` (its stream errors).
|
||||
*/
|
||||
export function answerOf(
|
||||
response: HttpResponse,
|
||||
request: { readonly method: string; header(name: string): string | null },
|
||||
defaults: AnswerHeaders,
|
||||
ctx: Ctx,
|
||||
failed: (error: unknown) => void,
|
||||
): Answer {
|
||||
const headers = defaults;
|
||||
for (const name in response.headers) headers.set(name, response.headers[name]!);
|
||||
const head = request.method === "HEAD";
|
||||
switch (response.kind) {
|
||||
case "json":
|
||||
return whole(
|
||||
response.status ?? 200,
|
||||
headers,
|
||||
"application/json; charset=utf-8",
|
||||
JSON.stringify(response.value),
|
||||
head,
|
||||
);
|
||||
case "html":
|
||||
return whole(response.status ?? 200, headers, "text/html; charset=utf-8", response.body, head);
|
||||
case "redirect":
|
||||
headers.set("Location", response.to);
|
||||
return whole(response.status ?? 302, headers, undefined, "", head);
|
||||
case "error":
|
||||
return errorAnswer(response.status, headers, response.message, head);
|
||||
case "bytes":
|
||||
return bytesAnswer(response, request, headers, ctx, failed);
|
||||
}
|
||||
}
|
||||
|
||||
/** The answer for an error `status` with its reason phrase (or `message`) as plain text. */
|
||||
export function errorAnswer(status: number, headers: AnswerHeaders, message?: string, head = false): Answer {
|
||||
return whole(status, headers, "text/plain; charset=utf-8", message ?? STATUS_CODES[status] ?? "", head);
|
||||
}
|
||||
|
||||
/** `answer` as a Web `Response`. */
|
||||
export function webResponse(answer: Answer): Response {
|
||||
// Bodies are bytes over plain `ArrayBuffer`s (never shared memory); the DOM types ask for exactly that.
|
||||
const headers = new Headers();
|
||||
for (const [name, value] of answer.headers.lines()) headers.append(name, value);
|
||||
return new Response(answer.body as BodyInit | null, { status: answer.status, headers });
|
||||
}
|
||||
|
||||
/**
|
||||
* A whole body: `type` unless a header set one, and its length; none for `HEAD`, and neither for a status that
|
||||
* carries no body (204, 205, 304: Sol's recheck N8, `Response` threw on them).
|
||||
*/
|
||||
function whole(status: number, headers: AnswerHeaders, type: string | undefined, text: string, head: boolean): Answer {
|
||||
if (status < 200 || status > 599) throw new RangeError(`response: status ${status}`);
|
||||
if (BODYLESS.has(status)) {
|
||||
headers.delete("Content-Type");
|
||||
headers.delete("Content-Length");
|
||||
return { status, headers, body: null };
|
||||
}
|
||||
if (type !== undefined && !headers.has("content-type")) headers.set("Content-Type", type);
|
||||
headers.set("Content-Length", String(Buffer.byteLength(text)));
|
||||
return { status, headers, body: head || text === "" ? null : text };
|
||||
}
|
||||
|
||||
/** The statuses whose responses carry no body. */
|
||||
const BODYLESS: ReadonlySet<number> = new Set([204, 205, 304]);
|
||||
|
||||
/** One `read` of a `bytes` body: 256 KiB. */
|
||||
const PIECE = 256 * 1024;
|
||||
|
||||
/** A `bytes` answer: its range or all of it, or until an empty read without a `size`. */
|
||||
function bytesAnswer(
|
||||
response: Extract<HttpResponse, { readonly kind: "bytes" }>,
|
||||
request: { readonly method: string; header(name: string): string | null },
|
||||
headers: AnswerHeaders,
|
||||
ctx: Ctx,
|
||||
failed: (error: unknown) => void,
|
||||
): Answer {
|
||||
if (!headers.has("content-type")) headers.set("Content-Type", response.mime);
|
||||
headers.set("Content-Disposition", disposition(response));
|
||||
if (response.inline === true)
|
||||
headers.set("Content-Security-Policy", `${headers.get("content-security-policy") ?? ""}; sandbox`);
|
||||
const { size } = response;
|
||||
let start = 0;
|
||||
let length = size;
|
||||
let status = 200;
|
||||
if (size !== undefined && response.ranges === true) {
|
||||
headers.set("Accept-Ranges", "bytes");
|
||||
const range = rangeOf(request.header("range"), size);
|
||||
if (range === "unsatisfiable") {
|
||||
headers.set("Content-Range", `bytes */${size}`);
|
||||
return whole(416, headers, undefined, "", true);
|
||||
}
|
||||
if (range !== undefined) {
|
||||
({ start, length } = range);
|
||||
status = 206;
|
||||
headers.set("Content-Range", `bytes ${start}-${start + range.length - 1}/${size}`);
|
||||
}
|
||||
}
|
||||
if (length !== undefined) headers.set("Content-Length", String(length));
|
||||
if (request.method === "HEAD" || length === 0) return { status, headers, body: null };
|
||||
return { status, headers, body: pulled(response, start, length, ctx, failed) };
|
||||
}
|
||||
|
||||
/**
|
||||
* The body of a `bytes` response from `start`, `length` bytes (`undefined`: until an empty read), read one piece per
|
||||
* pull, so the consumer's backpressure paces the reads; a `read` that throws or comes back short fails it. It follows
|
||||
* the request's scope: once sealed (a disconnect, `close`) the stream errors, no read starts, a late one is dropped
|
||||
* (Astra's recheck R2, Sol's N3: a held `Response` went on reading after the seal).
|
||||
*/
|
||||
function pulled(
|
||||
response: Extract<HttpResponse, { readonly kind: "bytes" }>,
|
||||
start: number,
|
||||
length: number | undefined,
|
||||
ctx: Ctx,
|
||||
failed: (error: unknown) => void,
|
||||
): ReadableStream<Uint8Array> {
|
||||
const end = length === undefined ? Number.POSITIVE_INFINITY : start + length;
|
||||
const signal = ctx.abortSignal;
|
||||
let offset = start;
|
||||
let sealed: (() => void) | undefined;
|
||||
return new ReadableStream<Uint8Array>(
|
||||
{
|
||||
start: (controller) => {
|
||||
if (signal === undefined) return;
|
||||
// Sealed before the stream exists (an abort while the route returned it): no event will come
|
||||
// (Sol's third recheck: the request never finished, `close` waited its whole bound).
|
||||
if (signal.aborted) return controller.error(signal.reason);
|
||||
sealed = () => controller.error(signal.reason);
|
||||
signal.addEventListener("abort", sealed, { once: true });
|
||||
},
|
||||
pull: async (controller) => {
|
||||
if (signal?.aborted) return controller.error(signal.reason);
|
||||
const want = Math.min(PIECE, end - offset);
|
||||
let chunk: Uint8Array;
|
||||
try {
|
||||
chunk = await response.read(offset, want, ctx);
|
||||
if (length !== undefined && chunk.byteLength !== want) {
|
||||
throw new ResponseFailure(
|
||||
`bytes: read ${chunk.byteLength} of ${want} bytes at ${offset} (size ${response.size})`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
// A read cut by the seal is no failure of the response.
|
||||
if (signal?.aborted) return;
|
||||
failed(error);
|
||||
controller.error(error);
|
||||
return;
|
||||
}
|
||||
// Sealed while reading: the stream errored already, the late chunk is dropped.
|
||||
if (signal?.aborted) return;
|
||||
if (chunk.byteLength === 0 || offset + chunk.byteLength >= end) {
|
||||
if (chunk.byteLength > 0) controller.enqueue(chunk);
|
||||
if (sealed !== undefined) signal?.removeEventListener("abort", sealed);
|
||||
controller.close();
|
||||
return;
|
||||
}
|
||||
controller.enqueue(chunk);
|
||||
offset += chunk.byteLength;
|
||||
},
|
||||
cancel: () => {
|
||||
if (sealed !== undefined) signal?.removeEventListener("abort", sealed);
|
||||
},
|
||||
},
|
||||
{ highWaterMark: 0 },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The single range `header` asks for within `size` bytes; `undefined` for none, a malformed or a multi-range one (the
|
||||
* whole body); `unsatisfiable` for one starting at or past the end.
|
||||
*/
|
||||
function rangeOf(header: string | null, size: number): { start: number; length: number } | "unsatisfiable" | undefined {
|
||||
if (header === null) return undefined;
|
||||
const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
|
||||
if (match === null) return undefined;
|
||||
const [, from = "", to = ""] = match;
|
||||
if (from === "" && to === "") return undefined;
|
||||
if (from === "") {
|
||||
const suffix = Number(to);
|
||||
if (suffix === 0 || size === 0) return "unsatisfiable";
|
||||
const length = Math.min(suffix, size);
|
||||
return { start: size - length, length };
|
||||
}
|
||||
const start = Number(from);
|
||||
if (start >= size) return "unsatisfiable";
|
||||
const last = to === "" ? size - 1 : Math.min(Number(to), size - 1);
|
||||
if (last < start) return undefined;
|
||||
return { start, length: last - start + 1 };
|
||||
}
|
||||
|
||||
/**
|
||||
* `Content-Disposition`: `inline` or `attachment`, with the download name as an ASCII fallback and as UTF-8 (RFC 6266,
|
||||
* RFC 5987), its control and bidirectional formatting characters dropped (a name must read as it is).
|
||||
*/
|
||||
function disposition(response: { readonly download?: string; readonly inline?: true }): string {
|
||||
const kind = response.inline === true ? "inline" : "attachment";
|
||||
if (response.download === undefined) return kind;
|
||||
const name = response.download.replace(/[\u0000-\u001f\u007f\u061c\u200e\u200f\u202a-\u202e\u2066-\u2069]/g, "");
|
||||
const ascii = name.replace(/[^\x20-\x7e]|["\\]/g, "_");
|
||||
const encoded = encodeURIComponent(name).replace(
|
||||
/['()*]/g,
|
||||
(char) => `%${char.charCodeAt(0).toString(16).toUpperCase()}`,
|
||||
);
|
||||
return `${kind}; filename="${ascii}"; filename*=UTF-8''${encoded}`;
|
||||
}
|
||||
412
packages/chord/src/next-node/http/router.ts
Normal file
412
packages/chord/src/next-node/http/router.ts
Normal file
|
|
@ -0,0 +1,412 @@
|
|||
// The router (slice 10 §2, D1, D2, D8): the registry `Http` that plugins contribute routes, wraps, sockets, allowed
|
||||
// hosts, framers, relays and CSP sources to, and its `compose`, which builds the router every request reads: patterns
|
||||
// matched segment by segment (exact before `:name` before a trailing `*`), each route's wraps resolved in phase and
|
||||
// contributor order, conflicts reported with the first by plugin name winning.
|
||||
|
||||
import type { WebSocket } from "ws";
|
||||
import type { Ctx } from "../../next/host/index.ts";
|
||||
import { type ComposeContext, defineRegistry, type RegistryTable } from "../../next/registries/index.ts";
|
||||
import type { HttpRequest } from "./request.ts";
|
||||
import type { HttpResponse } from "./response.ts";
|
||||
|
||||
/** A route: answers a request with a response description, in the request's scope (`ctx.principal`, its signal). */
|
||||
export type HttpRoute = (request: HttpRequest, ctx: Ctx) => Promise<HttpResponse>;
|
||||
|
||||
/**
|
||||
* A route with its options: `page` admits a top-level navigation from another site (a link to it) with its
|
||||
* authentication; `crossSite` admits any foreign request (a webhook), always anonymous.
|
||||
*/
|
||||
export type HttpRouteOptions = {
|
||||
readonly handler: HttpRoute;
|
||||
readonly page?: true;
|
||||
readonly crossSite?: true;
|
||||
};
|
||||
|
||||
/** A wrap's phases, outermost first. */
|
||||
export type WrapPhase = "observe" | "headers" | "transform";
|
||||
|
||||
/** A wrap (D2): runs around routes in its phase, around its own plugin's routes unless `scope` is `"all"`. */
|
||||
export type HttpWrap = {
|
||||
readonly name: string;
|
||||
readonly phase: WrapPhase;
|
||||
/** `"own"` (default): around the contributing plugin's routes; `"all"`: every route. */
|
||||
readonly scope?: "own" | "all";
|
||||
/** Only around this route (`"GET /notes/:id"`); a route nobody contributes is a reported conflict. */
|
||||
readonly route?: string;
|
||||
readonly wrap: (
|
||||
request: HttpRequest,
|
||||
next: (request: HttpRequest) => Promise<HttpResponse>,
|
||||
ctx: Ctx,
|
||||
) => Promise<HttpResponse>;
|
||||
};
|
||||
|
||||
/**
|
||||
* A plugin socket's handler (D6): a `ws` WebSocket after the checks and authentication, its `error` listener attached;
|
||||
* its scope lasts until the socket closes.
|
||||
*/
|
||||
export type HttpSocket = (socket: WebSocketHandle, request: HttpRequest, ctx: Ctx) => void | Promise<void>;
|
||||
|
||||
/** A plugin socket: `ws`'s WebSocket, for the plugin's own protocol. */
|
||||
export type WebSocketHandle = WebSocket;
|
||||
|
||||
/**
|
||||
* A relay (p2's Radius): a forwarder on loopback that sends forwarded headers. `verify` recognizes its requests (a
|
||||
* secret header, a signature) and names the origin the request came from and the origin allowed to frame it.
|
||||
*/
|
||||
export type HttpRelay = {
|
||||
readonly name: string;
|
||||
readonly verify: (request: Request) => { readonly origin: string; readonly framer?: string } | undefined;
|
||||
};
|
||||
|
||||
/** What a plugin contributes to `Http`: a plain value, or a writable of one (its change recomposes the router). */
|
||||
export type HttpContribution = {
|
||||
/** Routes by `"METHOD /pattern"`: exact segments, `:name`, a trailing `/*`. */
|
||||
readonly routes?: Readonly<Record<string, HttpRoute | HttpRouteOptions>>;
|
||||
readonly wraps?: readonly HttpWrap[];
|
||||
/** Plugin sockets by pattern (no method). */
|
||||
readonly sockets?: Readonly<Record<string, HttpSocket>>;
|
||||
/** The largest frame this contribution's sockets accept; 1 MiB by default. */
|
||||
readonly maxPayload?: number;
|
||||
/** Hosts allowed besides loopback, as the client names them (`notes.example.com`): requests through them are https. */
|
||||
readonly hosts?: readonly string[];
|
||||
/** Origins allowed to frame the app's pages (CSP `frame-ancestors`). */
|
||||
readonly framers?: readonly string[];
|
||||
/** Relays whose forwarded requests are admitted (verified) on loopback. */
|
||||
readonly relays?: readonly HttpRelay[];
|
||||
/** Extra CSP sources by directive (`{ "img-src": ["https://cdn.example.com"] }`). */
|
||||
readonly csp?: Readonly<Record<string, readonly string[]>>;
|
||||
};
|
||||
|
||||
/** The router a request uses: as it was when the request started. */
|
||||
export type Router = {
|
||||
/** The route for `method` and `path`, with its params and wraps; `bad_path` for a malformed one. */
|
||||
route(method: string, path: string): RouteMatch | "bad_path" | undefined;
|
||||
/** The plugin socket for `path`. */
|
||||
socket(path: string): SocketMatch | "bad_path" | undefined;
|
||||
/** The hosts contributions allow (lowercased). */
|
||||
readonly hosts: ReadonlySet<string>;
|
||||
/** The origins allowed to frame pages. */
|
||||
readonly framers: ReadonlySet<string>;
|
||||
/** The relays, in contributor order. */
|
||||
readonly relays: readonly HttpRelay[];
|
||||
/** Extra CSP sources by directive. */
|
||||
readonly csp: ReadonlyMap<string, readonly string[]>;
|
||||
/** The socket handlers it serves: a socket handed out by one no longer here is closed. */
|
||||
readonly socketHandlers: ReadonlySet<HttpSocket>;
|
||||
};
|
||||
|
||||
/** A matched route: its pattern, plugin, params, handler, options and wraps (outermost first). */
|
||||
export type RouteMatch = {
|
||||
readonly pattern: string;
|
||||
readonly plugin: string;
|
||||
readonly params: Readonly<Record<string, string>>;
|
||||
readonly handler: HttpRoute;
|
||||
readonly page: boolean;
|
||||
readonly crossSite: boolean;
|
||||
readonly wraps: readonly ResolvedWrap[];
|
||||
};
|
||||
|
||||
/** A matched plugin socket. */
|
||||
export type SocketMatch = {
|
||||
readonly pattern: string;
|
||||
readonly plugin: string;
|
||||
readonly params: Readonly<Record<string, string>>;
|
||||
readonly handler: HttpSocket;
|
||||
readonly maxPayload: number;
|
||||
};
|
||||
|
||||
/** A wrap as a route runs it, with its plugin. */
|
||||
export type ResolvedWrap = { readonly wrap: HttpWrap; readonly plugin: string };
|
||||
|
||||
/** The registry the HTTP component defines and serves (`http.facet`, plugin `chord.http`). */
|
||||
export const Http = defineRegistry<HttpContribution, Router>("chord.http", { version: 1 });
|
||||
|
||||
/** A plugin socket's largest frame unless its contribution says otherwise: 1 MiB. */
|
||||
const DEFAULT_MAX_PAYLOAD = 1024 * 1024;
|
||||
|
||||
/**
|
||||
* The methods a Web `Request` cannot carry (the Fetch standard's forbidden methods): answered 501 at the wire, a route
|
||||
* for one refused at compose.
|
||||
*/
|
||||
export const UNCARRIED: ReadonlySet<string> = new Set(["CONNECT", "TRACE", "TRACK"]);
|
||||
|
||||
/** Builds the router from `table` (the registry's `compose`): pure, conflicts reported, the first by plugin name wins. */
|
||||
export function composeRouter(table: RegistryTable<HttpContribution>, { report }: ComposeContext): Router {
|
||||
const routes = new Map<string, Node<RouteEntry>>();
|
||||
const sockets: Node<SocketEntry> = emptyNode();
|
||||
const entries: RouteEntry[] = [];
|
||||
const hosts = new Set<string>();
|
||||
const framers = new Set<string>();
|
||||
const relays: HttpRelay[] = [];
|
||||
const csp = new Map<string, string[]>();
|
||||
const socketHandlers = new Set<HttpSocket>();
|
||||
for (const { plugin, value } of table) {
|
||||
for (const host of value.hosts ?? []) hosts.add(host.toLowerCase());
|
||||
for (const framer of value.framers ?? []) framers.add(framer);
|
||||
relays.push(...(value.relays ?? []));
|
||||
for (const [directive, sources] of Object.entries(value.csp ?? {})) {
|
||||
csp.set(directive, [...(csp.get(directive) ?? []), ...sources]);
|
||||
}
|
||||
for (const [key, route] of Object.entries(value.routes ?? {})) {
|
||||
const parsed = parseRoute(key);
|
||||
if (parsed === undefined) {
|
||||
report({ message: `route "${key}" is no "METHOD /pattern"`, plugins: [plugin] });
|
||||
continue;
|
||||
}
|
||||
if (UNCARRIED.has(parsed.method)) {
|
||||
report({ message: `route "${key}": ${parsed.method} is served for no resource (501)`, plugins: [plugin] });
|
||||
continue;
|
||||
}
|
||||
let tree = routes.get(parsed.method);
|
||||
if (tree === undefined) {
|
||||
tree = emptyNode();
|
||||
routes.set(parsed.method, tree);
|
||||
}
|
||||
const options = typeof route === "function" ? { handler: route } : route;
|
||||
const entry: RouteEntry = {
|
||||
pattern: key,
|
||||
plugin,
|
||||
handler: options.handler,
|
||||
page: options.page === true,
|
||||
crossSite: options.crossSite === true,
|
||||
segments: parsed.segments,
|
||||
wraps: [],
|
||||
};
|
||||
const taken = insert(tree, parsed.segments, entry);
|
||||
if (taken === undefined) entries.push(entry);
|
||||
else report({ message: `route "${key}" is contributed twice`, plugins: [taken.plugin, plugin] });
|
||||
}
|
||||
const maxPayload = value.maxPayload ?? DEFAULT_MAX_PAYLOAD;
|
||||
for (const [pattern, handler] of Object.entries(value.sockets ?? {})) {
|
||||
const segments = parsePattern(pattern);
|
||||
if (segments === undefined) {
|
||||
report({ message: `socket "${pattern}" is no pattern`, plugins: [plugin] });
|
||||
continue;
|
||||
}
|
||||
const taken = insert(sockets, segments, { pattern, plugin, handler, segments, maxPayload });
|
||||
if (taken === undefined) socketHandlers.add(handler);
|
||||
else report({ message: `socket "${pattern}" is contributed twice`, plugins: [taken.plugin, plugin] });
|
||||
}
|
||||
}
|
||||
resolveWraps(table, entries, report);
|
||||
const get = routes.get("GET");
|
||||
const head = routes.get("HEAD");
|
||||
return {
|
||||
route: (method, path) => {
|
||||
// `HEAD` falls back to `GET` per path: an explicit `HEAD` route elsewhere hides no `GET` route.
|
||||
if (method === "HEAD") return orElse(head === undefined ? undefined : lookup(head, path), get, path);
|
||||
const tree = routes.get(method);
|
||||
return tree === undefined ? undefined : lookup(tree, path);
|
||||
},
|
||||
socket: (path) => lookup(sockets, path),
|
||||
hosts,
|
||||
framers,
|
||||
relays,
|
||||
csp,
|
||||
socketHandlers,
|
||||
};
|
||||
}
|
||||
|
||||
/** One pattern segment: an exact one, `:name`, or the trailing rest (`*`). */
|
||||
type Segment =
|
||||
| { readonly kind: "exact"; readonly text: string }
|
||||
| { readonly kind: "param"; readonly name: string }
|
||||
| { readonly kind: "rest" };
|
||||
|
||||
/** What a route contributes, with its wraps once resolved. */
|
||||
type RouteEntry = {
|
||||
readonly pattern: string;
|
||||
readonly plugin: string;
|
||||
readonly handler: HttpRoute;
|
||||
readonly page: boolean;
|
||||
readonly crossSite: boolean;
|
||||
readonly segments: readonly Segment[];
|
||||
wraps: readonly ResolvedWrap[];
|
||||
};
|
||||
|
||||
/** What a plugin socket contributes. */
|
||||
type SocketEntry = {
|
||||
readonly pattern: string;
|
||||
readonly plugin: string;
|
||||
readonly handler: HttpSocket;
|
||||
readonly segments: readonly Segment[];
|
||||
readonly maxPayload: number;
|
||||
};
|
||||
|
||||
/** A node of a pattern tree: its children by kind, and the entry ending here. */
|
||||
type Node<E> = {
|
||||
readonly exact: Map<string, Node<E>>;
|
||||
/** Any `:name` at this position (the names are each entry's own). */
|
||||
param: Node<E> | undefined;
|
||||
rest: E | undefined;
|
||||
entry: E | undefined;
|
||||
};
|
||||
|
||||
function emptyNode<E>(): Node<E> {
|
||||
return { exact: new Map(), param: undefined, rest: undefined, entry: undefined };
|
||||
}
|
||||
|
||||
/** `"GET /notes/:id"` as its method and segments; `undefined` when it is none. */
|
||||
function parseRoute(key: string): { readonly method: string; readonly segments: readonly Segment[] } | undefined {
|
||||
const space = key.indexOf(" ");
|
||||
if (space <= 0) return undefined;
|
||||
const method = key.slice(0, space);
|
||||
if (!/^[A-Z]+$/.test(method)) return undefined;
|
||||
const segments = parsePattern(key.slice(space + 1));
|
||||
return segments === undefined ? undefined : { method, segments };
|
||||
}
|
||||
|
||||
/** A pattern's segments (`/files/*`, `/notes/:id`); `undefined` for one that is not a path or has `*` before its end. */
|
||||
function parsePattern(pattern: string): readonly Segment[] | undefined {
|
||||
if (!pattern.startsWith("/")) return undefined;
|
||||
const parts = pattern.slice(1).split("/");
|
||||
const segments: Segment[] = [];
|
||||
for (const [index, part] of parts.entries()) {
|
||||
if (part === "*") {
|
||||
if (index !== parts.length - 1) return undefined;
|
||||
segments.push({ kind: "rest" });
|
||||
} else if (part.startsWith(":")) {
|
||||
if (part.length === 1) return undefined;
|
||||
segments.push({ kind: "param", name: part.slice(1) });
|
||||
} else {
|
||||
segments.push({ kind: "exact", text: part });
|
||||
}
|
||||
}
|
||||
return segments;
|
||||
}
|
||||
|
||||
/** Puts `entry` at `segments` in `tree`; returns the entry already there (the first by plugin name keeps it). */
|
||||
function insert<E>(tree: Node<E>, segments: readonly Segment[], entry: E): E | undefined {
|
||||
let node = tree;
|
||||
for (const segment of segments) {
|
||||
if (segment.kind === "rest") {
|
||||
if (node.rest !== undefined) return node.rest;
|
||||
node.rest = entry;
|
||||
return undefined;
|
||||
}
|
||||
if (segment.kind === "exact") {
|
||||
let next = node.exact.get(segment.text);
|
||||
if (next === undefined) {
|
||||
next = emptyNode();
|
||||
node.exact.set(segment.text, next);
|
||||
}
|
||||
node = next;
|
||||
} else {
|
||||
// `:a` and `:b` at one position share the node: one shape, so the same pattern twice is a conflict.
|
||||
node.param ??= emptyNode();
|
||||
node = node.param;
|
||||
}
|
||||
}
|
||||
if (node.entry !== undefined) return node.entry;
|
||||
node.entry = entry;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** `found`, or when it is no route the `GET` route for `path` (`HEAD`'s fallback). */
|
||||
function orElse(
|
||||
found: RouteMatch | "bad_path" | undefined,
|
||||
get: Node<RouteEntry> | undefined,
|
||||
path: string,
|
||||
): RouteMatch | "bad_path" | undefined {
|
||||
if (found !== undefined || get === undefined) return found;
|
||||
return lookup(get, path);
|
||||
}
|
||||
|
||||
/**
|
||||
* The entry `path` matches in `tree` with its params named by the entry's own pattern, the most specific segment by
|
||||
* segment from the left (exact before `:name` before `*`); `bad_path` for a segment that does not decode, or a `:name`
|
||||
* or rest segment decoding to one containing `/`, `\` or NUL, or to `.` or `..` (a param must never be a path).
|
||||
*/
|
||||
function lookup<E extends { readonly segments: readonly Segment[] }>(
|
||||
tree: Node<E>,
|
||||
path: string,
|
||||
): (Omit<E, "segments"> & { readonly params: Readonly<Record<string, string>> }) | "bad_path" | undefined {
|
||||
const raw = path.slice(1).split("/");
|
||||
const decoded: string[] = [];
|
||||
for (const part of raw) {
|
||||
if (!part.includes("%")) {
|
||||
decoded.push(part);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
decoded.push(decodeURIComponent(part));
|
||||
} catch {
|
||||
return "bad_path";
|
||||
}
|
||||
}
|
||||
const found = walk(tree, decoded, 0, []);
|
||||
if (found === undefined) return undefined;
|
||||
const params: Record<string, string> = {};
|
||||
let capture = 0;
|
||||
for (const segment of found.entry.segments) {
|
||||
if (segment.kind !== "param") continue;
|
||||
const value = found.captures[capture++]!;
|
||||
if (!safe(value)) return "bad_path";
|
||||
params[segment.name] = value;
|
||||
}
|
||||
if (found.rest !== undefined) {
|
||||
const rest = decoded.slice(found.rest);
|
||||
if (!rest.every(safe)) return "bad_path";
|
||||
params["*"] = rest.join("/");
|
||||
}
|
||||
const { segments: _segments, ...entry } = found.entry;
|
||||
return { ...entry, params };
|
||||
}
|
||||
|
||||
/** Whether a decoded segment may be a param: no `/`, `\` or NUL, not `.` or `..`. */
|
||||
function safe(segment: string): boolean {
|
||||
return segment !== "." && segment !== ".." && !/[/\\\0]/.test(segment);
|
||||
}
|
||||
|
||||
/** A match found by `walk`: its entry, its `:name` values in order, where the rest starts (for `*`). */
|
||||
type Found<E> = { readonly entry: E; readonly captures: readonly string[]; readonly rest: number | undefined };
|
||||
|
||||
/** Depth first in precedence order: the first match found is the most specific from the left. */
|
||||
function walk<E>(node: Node<E>, parts: readonly string[], index: number, captures: string[]): Found<E> | undefined {
|
||||
if (index === parts.length) {
|
||||
if (node.entry !== undefined) return { entry: node.entry, captures, rest: undefined };
|
||||
// `/files/*` matches `/files`: the rest is empty.
|
||||
return node.rest === undefined ? undefined : { entry: node.rest, captures, rest: index };
|
||||
}
|
||||
const part = parts[index]!;
|
||||
const exact = node.exact.get(part);
|
||||
const viaExact = exact === undefined ? undefined : walk(exact, parts, index + 1, captures);
|
||||
if (viaExact !== undefined) return viaExact;
|
||||
if (node.param !== undefined && part !== "") {
|
||||
const viaParam = walk(node.param, parts, index + 1, [...captures, part]);
|
||||
if (viaParam !== undefined) return viaParam;
|
||||
}
|
||||
return node.rest === undefined ? undefined : { entry: node.rest, captures, rest: index };
|
||||
}
|
||||
|
||||
/** The wrap phases, outermost first. */
|
||||
const PHASES: readonly WrapPhase[] = ["observe", "headers", "transform"];
|
||||
|
||||
/**
|
||||
* Gives every route its wraps: by phase (outermost first), within a phase in contributor (table) order; `own` ones
|
||||
* around their plugin's routes, `all` ones around every route; one naming a route nobody contributes is reported.
|
||||
*/
|
||||
function resolveWraps(
|
||||
table: RegistryTable<HttpContribution>,
|
||||
routes: RouteEntry[],
|
||||
report: ComposeContext["report"],
|
||||
): void {
|
||||
const all: ResolvedWrap[] = [];
|
||||
for (const { plugin, value } of table) for (const wrap of value.wraps ?? []) all.push({ wrap, plugin });
|
||||
all.sort((a, b) => PHASES.indexOf(a.wrap.phase) - PHASES.indexOf(b.wrap.phase));
|
||||
for (const { wrap, plugin } of all) {
|
||||
if (wrap.route !== undefined && !routes.some((route) => route.pattern === wrap.route)) {
|
||||
report({
|
||||
message: `wrap "${wrap.name}" names route "${wrap.route}", which nobody contributes`,
|
||||
plugins: [plugin],
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const route of routes) {
|
||||
route.wraps = all.filter(
|
||||
({ wrap, plugin }) =>
|
||||
(wrap.scope === "all" || plugin === route.plugin) &&
|
||||
(wrap.route === undefined || wrap.route === route.pattern),
|
||||
);
|
||||
}
|
||||
}
|
||||
245
packages/chord/src/next-node/http/upgrades.ts
Normal file
245
packages/chord/src/next-node/http/upgrades.ts
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
// Upgraded connections (slice 10 §2, D6, Q6): the links endpoint (`host.accept` over a WebSocket with a per-socket
|
||||
// output cap) and plugin sockets (a `ws` WebSocket handed to the contribution's handler for its own protocol, in a scope
|
||||
// lasting until it closes, kept alive by a heartbeat). Each connection is kept with what admitted it: when that leaves
|
||||
// (its allowed host, its relay, its socket handler) it is terminated at once and its scope sealed (decision "Slice 10
|
||||
// fix round, allowed hosts only through contributions"); the component's close closes all (1001), terminating at its
|
||||
// timeout those that never acknowledge.
|
||||
|
||||
import type { IncomingMessage } from "node:http";
|
||||
import type { Duplex } from "node:stream";
|
||||
import { type WebSocket, WebSocketServer } from "ws";
|
||||
import { type Context, withAbortSignal, withoutAbortSignal } from "../../context/index.ts";
|
||||
import type { Principal } from "../../next/data/index.ts";
|
||||
import type { Ctx, Host } from "../../next/host/index.ts";
|
||||
import { type PartLimits, type WebSocketLike, webSocketLink } from "../../next/link/index.ts";
|
||||
import type { HttpRequest } from "./request.ts";
|
||||
import type { HttpRelay, HttpSocket, Router, SocketMatch } from "./router.ts";
|
||||
|
||||
/** What admitted an upgrade: the allowed host or the relay it came through (local: neither), and its principal. */
|
||||
export type Admission<P extends Principal> = {
|
||||
readonly allowed: string | undefined;
|
||||
readonly relay: HttpRelay | undefined;
|
||||
readonly principal: P;
|
||||
};
|
||||
|
||||
/** What the links endpoint gives one socket: its parts' limits and its largest frame (1 MiB by default). */
|
||||
export type LinkOptions = { readonly limits?: PartLimits; readonly maxPayload?: number };
|
||||
|
||||
/** A plugin socket's handler failed: reported, the socket closed (1011). */
|
||||
export type SocketFailure = { readonly pattern: string; readonly plugin: string; readonly error: unknown };
|
||||
|
||||
/** One upgraded connection, with what admitted it and how to seal its scope. */
|
||||
type Upgraded = {
|
||||
readonly socket: WebSocket;
|
||||
readonly allowed: string | undefined;
|
||||
readonly relay: HttpRelay | undefined;
|
||||
/** The handler that took it (plugin sockets only). */
|
||||
readonly handler: HttpSocket | undefined;
|
||||
/** Ends its work at once: the link closed, or the handler's scope sealed. */
|
||||
readonly seal: () => void;
|
||||
};
|
||||
|
||||
/** A link socket's largest frame unless `links.options` says otherwise: 1 MiB. */
|
||||
const DEFAULT_LINK_PAYLOAD = 1024 * 1024;
|
||||
|
||||
/** The upgraded connections of one HTTP component. */
|
||||
export class Upgrades {
|
||||
readonly #open = new Set<Upgraded>();
|
||||
|
||||
/**
|
||||
* Upgrades `message` to the links endpoint: `host.accept` over the socket (its output capped) as the admitted
|
||||
* principal, with `options`' limits; `context` is the parent of the work on the link.
|
||||
*/
|
||||
link<P extends Principal>(
|
||||
message: IncomingMessage,
|
||||
stream: Duplex,
|
||||
head: Buffer,
|
||||
admission: Admission<P> & LinkOptions,
|
||||
host: Host<P>,
|
||||
context: Context,
|
||||
): void {
|
||||
const { principal, limits } = admission;
|
||||
upgrade(message, stream, head, admission.maxPayload ?? DEFAULT_LINK_PAYLOAD, (socket) => {
|
||||
const link = webSocketLink(limitedSocket(socket));
|
||||
this.#keep({
|
||||
socket,
|
||||
allowed: admission.allowed,
|
||||
relay: admission.relay,
|
||||
handler: undefined,
|
||||
seal: () => link.close(),
|
||||
});
|
||||
host.accept(link, { principal, ...(limits === undefined ? {} : { limits }) }, context);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Upgrades `message` to the plugin socket `match`: its handler gets the socket and `request` in a `host.root` scope
|
||||
* acting as the admitted principal that lasts until the socket closes; a handler that throws is reported and closes
|
||||
* it (1011).
|
||||
*/
|
||||
socket<P extends Principal>(
|
||||
message: IncomingMessage,
|
||||
stream: Duplex,
|
||||
head: Buffer,
|
||||
admission: Admission<P> & { readonly match: SocketMatch; readonly request: HttpRequest },
|
||||
host: Host<P>,
|
||||
context: Context,
|
||||
failed: (failure: SocketFailure) => void,
|
||||
): void {
|
||||
const { principal, match, request } = admission;
|
||||
upgrade(message, stream, head, match.maxPayload, (socket) => {
|
||||
const closed = new AbortController();
|
||||
const seal = (): void => closed.abort(new Error("http: the socket closed"));
|
||||
socket.once("close", seal);
|
||||
this.#keep({ socket, allowed: admission.allowed, relay: admission.relay, handler: match.handler, seal });
|
||||
heartbeat(socket);
|
||||
const scope = withAbortSignal(closed.signal, withoutAbortSignal(context));
|
||||
const run = async (ctx: Ctx<P>): Promise<void> => {
|
||||
try {
|
||||
await match.handler(socket, request, ctx);
|
||||
} catch (error) {
|
||||
failed({ pattern: match.pattern, plugin: match.plugin, error });
|
||||
socket.close(1011, "internal error");
|
||||
return;
|
||||
}
|
||||
if (!closed.signal.aborted) {
|
||||
await new Promise((resolve) => closed.signal.addEventListener("abort", resolve, { once: true }));
|
||||
}
|
||||
};
|
||||
// The scope sealed (its socket closed, the host closing) is no failure of the handler.
|
||||
void host.root(run, scope, { principal }).catch(() => {});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The router changed: connections through a host or a relay no longer there, and plugin sockets whose handler left,
|
||||
* are terminated at once and their scopes sealed (no close handshake a peer could ignore).
|
||||
*/
|
||||
revoke(router: Router): void {
|
||||
for (const upgraded of this.#open) {
|
||||
const gone =
|
||||
(upgraded.allowed !== undefined && !router.hosts.has(upgraded.allowed)) ||
|
||||
(upgraded.relay !== undefined && !router.relays.includes(upgraded.relay)) ||
|
||||
(upgraded.handler !== undefined && !router.socketHandlers.has(upgraded.handler));
|
||||
if (gone) end(upgraded);
|
||||
}
|
||||
}
|
||||
|
||||
/** Closes every connection (1001, "going away"); those still open at `deadline` are terminated. */
|
||||
async close(deadline: Promise<void>): Promise<void> {
|
||||
const open = [...this.#open];
|
||||
const closed = open.map(
|
||||
({ socket }) =>
|
||||
new Promise<void>((resolve) => {
|
||||
if (socket.readyState === socket.CLOSED) resolve();
|
||||
else socket.once("close", () => resolve());
|
||||
}),
|
||||
);
|
||||
for (const { socket } of open) socket.close(1001, "going away");
|
||||
await Promise.race([Promise.all(closed), deadline]);
|
||||
for (const upgraded of this.#open) end(upgraded);
|
||||
}
|
||||
|
||||
/** Keeps `upgraded` until its socket closes; an error (an oversized frame) terminates only that socket. */
|
||||
#keep(upgraded: Upgraded): void {
|
||||
const { socket } = upgraded;
|
||||
socket.on("error", () => socket.terminate());
|
||||
this.#open.add(upgraded);
|
||||
socket.once("close", () => this.#open.delete(upgraded));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Completes the upgrade with a `ws` server of its own (no listener, no client tracking, no compression), dropped with
|
||||
* the upgrade: nothing is kept per frame size.
|
||||
*/
|
||||
function upgrade(
|
||||
message: IncomingMessage,
|
||||
stream: Duplex,
|
||||
head: Buffer,
|
||||
maxPayload: number,
|
||||
then: (socket: WebSocket) => void,
|
||||
): void {
|
||||
const server = new WebSocketServer({ noServer: true, clientTracking: false, maxPayload, perMessageDeflate: false });
|
||||
server.handleUpgrade(message, stream, head, then);
|
||||
}
|
||||
|
||||
/** Terminates `upgraded` and seals its scope at once. */
|
||||
function end(upgraded: Upgraded): void {
|
||||
// Terminated first: a link's close would send a close frame.
|
||||
upgraded.socket.terminate();
|
||||
upgraded.seal();
|
||||
}
|
||||
|
||||
/** Silence after which a plugin socket is pinged, and after the ping terminated: 15 s each (the link heartbeat's). */
|
||||
const SILENCE = 15_000;
|
||||
|
||||
/**
|
||||
* Keeps a plugin socket honest (decision "plugin sockets get a heartbeat"): after 15 s without a frame from the peer it
|
||||
* is pinged, after 15 s more terminated; browsers and `ws` answer pings by themselves.
|
||||
*/
|
||||
function heartbeat(socket: WebSocket): void {
|
||||
let pinged = false;
|
||||
const timer = setTimeout(() => {
|
||||
if (pinged) {
|
||||
socket.terminate();
|
||||
return;
|
||||
}
|
||||
pinged = true;
|
||||
socket.ping();
|
||||
timer.refresh();
|
||||
}, SILENCE);
|
||||
timer.unref();
|
||||
const heard = (): void => {
|
||||
pinged = false;
|
||||
timer.refresh();
|
||||
};
|
||||
socket.on("message", heard);
|
||||
socket.on("pong", heard);
|
||||
socket.on("ping", heard);
|
||||
socket.once("close", () => clearTimeout(timer));
|
||||
}
|
||||
|
||||
/** Unsent data beyond the frame being sent at which a link socket is cut off: a client that stopped reading. */
|
||||
const MAX_UNSENT_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
/**
|
||||
* `socket` as the link sees it, its output capped (p2's `limitedSocket`): before each send, what is still buffered
|
||||
* beyond the largest frame being sent must stay under 4 MiB, or the socket is terminated (the client reconnects from
|
||||
* snapshots); Chord's flow control keeps a reading client far below it.
|
||||
*/
|
||||
export function limitedSocket(socket: WebSocket): WebSocketLike {
|
||||
const unsent = new Set<{ readonly bytes: number }>();
|
||||
return {
|
||||
get readyState() {
|
||||
return socket.readyState;
|
||||
},
|
||||
get binaryType() {
|
||||
return socket.binaryType;
|
||||
},
|
||||
set binaryType(type: string) {
|
||||
socket.binaryType = type as WebSocket["binaryType"];
|
||||
},
|
||||
send(data) {
|
||||
if (socket.readyState !== socket.OPEN) return;
|
||||
let sending = 0;
|
||||
for (const frame of unsent) sending = Math.max(sending, frame.bytes);
|
||||
if (socket.bufferedAmount - sending > MAX_UNSENT_BYTES) {
|
||||
socket.terminate();
|
||||
return;
|
||||
}
|
||||
// The largest server frame header is 10 bytes.
|
||||
const frame = { bytes: data.byteLength + 10 };
|
||||
unsent.add(frame);
|
||||
socket.send(data, (error) => {
|
||||
unsent.delete(frame);
|
||||
if (error) socket.terminate();
|
||||
});
|
||||
},
|
||||
close: () => socket.close(),
|
||||
addEventListener: (type: string, listener: (event: { readonly data: unknown }) => void) =>
|
||||
socket.addEventListener(type as "message", listener as never),
|
||||
removeEventListener: (type: string, listener: (event: { readonly data: unknown }) => void) =>
|
||||
socket.removeEventListener(type as "message", listener as never),
|
||||
} as WebSocketLike;
|
||||
}
|
||||
|
|
@ -5,17 +5,20 @@ facet built into a script by esbuild and evaluated on Node, loaded and reloaded
|
|||
Chord app modifies itself (the owner or an agent edits a plugin while it runs), so this is core machinery, no dev mode.
|
||||
Node-only (`fs`, esbuild, `vm`); the kernel in `src/next` stays portable and takes facets however they were made.
|
||||
esbuild is an optional peer dependency of `@earendil-works/chord`, imported only here. Exported as
|
||||
`@earendil-works/chord/next/plugins` once v2 has its entry point.
|
||||
`@earendil-works/chord/next/plugins` once v2 has its entry point; the plugin runtime alone (no esbuild, no watcher:
|
||||
what a worker evaluates with) as `@earendil-works/chord/next/plugins/runtime`.
|
||||
|
||||
## For apps (`index.ts`)
|
||||
|
||||
```ts
|
||||
const plugins = createPlugins({ roots: ["./plugins", join(dataDir, "plugins")], kinds: ["server", "worker", "web", "tui"],
|
||||
browserKinds: ["web"], hostPackages: HOST, base: ["./src"], onError }, ctx);
|
||||
await plugins.load(server, { kind: "server", core: [{ name: "@app/core", facet: coreServerFacet }] }, ctx);
|
||||
const HOST = ["@earendil-works/chord/next", "@app/core"];
|
||||
const plugins = createPlugins({ roots: ["./plugins", join(dataDir, "plugins")], base: ["./src"], onError,
|
||||
kinds: { server: { platform: "node", hostPackages: HOST }, worker: { platform: "node", hostPackages: HOST },
|
||||
web: { platform: "browser", hostPackages: [...HOST, "react"] } } }, ctx);
|
||||
await plugins.load(server, { kind: "server", core: [{ name: "@app/core", facet: coreServerFacet }], modules }, ctx);
|
||||
plugins.state // { dirty, install, set, base }, a JSON watchable
|
||||
const built = await plugins.build(plugins.state.value.dirty, ctx);
|
||||
await plugins.rollout([plugins.step("server", server), ...workerSteps], built, ctx); // the app's order
|
||||
await plugins.rollout([plugins.step("server", server), workers.step("workers")], built, ctx); // the app's order
|
||||
```
|
||||
|
||||
- **A plugin** is a directory with a `package.json`: its `name` (npm style) is the plugin's name, `dependencies` are
|
||||
|
|
@ -27,9 +30,12 @@ await plugins.rollout([plugins.step("server", server), ...workerSteps], built, c
|
|||
symlinks to real paths; the first root wins a duplicate name. `{ plugins, problems }`: a directory without a valid
|
||||
`package.json` or name, with two files of one kind, or a later duplicate is a problem, not a plugin.
|
||||
- `buildFacet(plugin, kind, { hostPackages, platform, plugins?, publicPath? }, ctx)`: one esbuild build from cold.
|
||||
Inlined: the plugin's own files, files of other discovered plugins (copies, listed in `copied`; they may import only
|
||||
host packages and files of their own plugin: contracts and side-effect-free modules), packages of its
|
||||
`dependencies`. External: host packages, `@earendil-works/chord/next/plugins`, `chord.external` packages and Node
|
||||
Inlined: the plugin's own files, files of other discovered plugins imported by name (`import { Notes } from
|
||||
"@app/notes/contract"`: the discovered plugin `@app/notes`, its `contract.ts`, wherever its directory is; copies,
|
||||
listed in `copied`; they may import only host packages and files of their own plugin: contracts and side-effect-free
|
||||
modules; decision "Plugins import each other by name"), packages of its `dependencies`. A relative path into another
|
||||
plugin's directory is refused, and an import naming no host package, dependency or discovered plugin is a build
|
||||
error naming it. External: host packages, `@earendil-works/chord/next/plugins`, `chord.external` packages and Node
|
||||
builtins (Node builds; a browser build refuses both). Browser builds collect CSS into one stylesheet (`css`) and
|
||||
assets (`import url from "./icon.svg?url"`, `.wasm`) into content-hashed files (`assets`) under `publicPath`.
|
||||
Refused, with the file and line: `import.meta`, `__dirname`, `__filename` (each use, found through the script's
|
||||
|
|
@ -50,24 +56,36 @@ await plugins.rollout([plugins.step("server", server), ...workerSteps], built, c
|
|||
lives (its facet, on any host, or the error its evaluation threw).
|
||||
- `installDependencies(plugin, ctx, { npmCommand? })`: `npm install --omit=dev --legacy-peer-deps --ignore-scripts
|
||||
--no-audit --no-fund` in the plugin's real directory.
|
||||
- `createPlugins(options, ctx)`: turns V8's compilation cache off (once per process) and starts the watcher.
|
||||
- `load(host, { kind, core }, ctx)`: discovers (once), builds and evaluates the `kind` facet of every plugin and
|
||||
calls `host.load([...core, ...facets])`; a plugin whose build or evaluation fails is left out, reported in the
|
||||
report's `failed` beside the host's own.
|
||||
- `createPluginRuntime()` (`runtime.ts`, slice 10 D11): loads and reloads hosts with shipped scripts, never builds.
|
||||
Per host it keeps each plugin's loaded facet; `load(host, { kind, core, scripts, modules }, ctx)` evaluates the
|
||||
complete set `scripts` (`PluginScript`: `{ plugin, built }`, plain data) with `modules` (the host packages, this
|
||||
process's instances, kept for the host's reloads) and calls `host.load([...core, ...facets])`; a plugin whose build
|
||||
or evaluation fails is left out, reported in the report's `failed` beside the host's own. `reload(host, scripts,
|
||||
ctx)`, one after another per host, passes the complete set: core facets and unchanged plugins (same digest) as the
|
||||
same objects, new scripts evaluated, loaded plugins not in `scripts` left out; a plugin whose build or evaluation
|
||||
failed is passed as its loaded facet (an added one left out) and named to `host.reload`'s `failed` option, so the
|
||||
host's closure rule sees it and its report lists it. What the host committed becomes the loaded set, also when the
|
||||
reload rejects `aborted` after its commit (`details.committed`). Turns V8's compilation cache off (once per process).
|
||||
- `createPlugins(options, ctx)`: `kinds` defines each facet kind: its builds' `platform` (`node`, `browser`) and the
|
||||
host packages they keep external (by specifier; the modules are given where scripts are evaluated). It starts the
|
||||
watcher and uses a plugin runtime.
|
||||
- `scripts(kind, ctx)`: the kind's complete current set, every discovered plugin's facet of it built at first use,
|
||||
then updated only by what ships the kind (a rollout step of that kind, a direct `reload`): what a worker starts
|
||||
with. `build` never changes it.
|
||||
- `load(host, { kind, core, modules }, ctx)`: `kind`'s current set loaded with the runtime.
|
||||
- `build(names, ctx)`: discovers again when the plugin set or a plugin's `package.json` changed, builds every facet
|
||||
of `names` and of plugins discovered since: `{ facets, plugins, removed }`, plain data (`plugins`: the built
|
||||
plugins as discovered, by name; a clone reloads the same).
|
||||
- `reload(host, built, ctx)`, one after another per host: the complete set for the host's kind (the plugins it
|
||||
loaded and those `built` brings): core facets and unchanged plugins (same digest) as the same objects, new scripts
|
||||
evaluated, the plugins `built` removed or rebuilt without a facet of the host's kind left out (its own
|
||||
discovery's, never a later one's); a plugin whose build or evaluation failed is
|
||||
passed as its loaded facet (an added one left out) and named to `host.reload`'s `failed` option, so the host's
|
||||
closure rule sees it and its report lists it. What the host committed becomes the loaded set, also when the
|
||||
reload rejects `aborted` after its commit (`details.committed`).
|
||||
- `step(name, host)`: an in-process rollout step; `rollout(steps, built, ctx)` runs steps in order and stops at one
|
||||
that rejects (`stopped`); failed plugins do not stop it. It clears the dirty marks of the built plugins applied on
|
||||
every step and unchanged since the build; a plugin failed on any step stays dirty. Given a copy of a build (not the
|
||||
object `build` returned), it clears no marks.
|
||||
- `reload(host, built, ctx)`: applies `built` to the current set of the host's kind (its removed plugins and those
|
||||
rebuilt without a facet of the kind leave it, its facets replace theirs; applied and read in one step, so two
|
||||
reloads started together each pass their own) and reloads the host with that complete set.
|
||||
- `step(name, host)`: an in-process rollout step, one host named `name`. A step's `reload(built, ctx)` returns
|
||||
`{ hosts: [{ host, report } | { host, stopped: true } | { host, error }] }` (D12). `rollout(steps, built, ctx)`
|
||||
applies `built` to the current set of each step's kinds right before that step, runs the steps in order and stops
|
||||
at one that rejects (`stopped`); failed plugins and hosts do not stop it. It clears the dirty marks of the built
|
||||
plugins applied on every host of every step and unchanged since the build: a plugin failed on any host, or of a
|
||||
step's kinds on a host listed with an error, or with a built kind no step shipped, stays dirty. Given a copy of a
|
||||
build (not the object `build` returned), it clears no marks.
|
||||
- `installDependencies(name, ctx)` (the install mark clears, the plugin turns dirty), `close()` (the watcher stops).
|
||||
- **The watcher** (`fs.watch` on every directory of the roots, the base code and plugin directories linked from
|
||||
outside the roots, never on `node_modules` or `.git`; a directory appearing later under a watched one gets its watch,
|
||||
|
|
@ -85,11 +103,13 @@ Each a folder in `test/next/examples/` with its hosts in `main.ts` and real plug
|
|||
`35-plugins` (load, edit, mark, build, roll out), `36-plugin-failures` (per plugin failures at startup and reload, the
|
||||
closure rule, an abort after the commit), `37-plugin-imports` (what a facet may import, installs, the refusals),
|
||||
`38-watch-and-rollout` (every mark, added, removed and dropped facets, stopped rollouts) and `39-browser-build` (a
|
||||
browser build: stylesheet, assets by URL, plain data).
|
||||
browser build: stylesheet, assets by URL, plain data); with workers, `43-rollout` (a rollout over the server and its
|
||||
running workers).
|
||||
|
||||
## Inside the directory
|
||||
|
||||
`discover.ts` (discovery), `build.ts` (one build, its import rules and input capture), `evaluate.ts` (evaluation, the
|
||||
`require` a script gets, `pluginFile`), `stack.ts` (the source maps of evaluated scripts while their facets live,
|
||||
`mapStack`), `install.ts`, `watcher.ts` (the watcher and its marks, a JSON watchable), `plugins.ts` (the component:
|
||||
per-host loaded facets, `build`, `load`, `reload`, `step`, `rollout`).
|
||||
`mapStack`), `install.ts`, `watcher.ts` (the watcher and its marks, a JSON watchable), `runtime.ts` (per-host loaded
|
||||
facets, complete-set loads and reloads, no esbuild), `plugins.ts` (the component: kinds, current sets, `build`,
|
||||
`load`, `reload`, `step`, `rollout`).
|
||||
|
|
|
|||
|
|
@ -1,15 +1,17 @@
|
|||
// Building (slice 7c §2): one facet of one plugin as one script, by esbuild from cold. Its own files, files of other
|
||||
// discovered plugins (copies) and its npm `dependencies` are inlined; host packages, this component, `chord.external`
|
||||
// discovered plugins imported by name (`@app/notes/contract`: copies; decision "Plugins import each other by name") and
|
||||
// its npm `dependencies` are inlined; host packages, this component, `chord.external`
|
||||
// packages and Node builtins stay external, answered at evaluation by the `require` the script takes. The result is
|
||||
// plain data, the same value a worker or a page evaluates where it runs; a failed build is a result, never a throw.
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { isBuiltin } from "node:module";
|
||||
import { basename, dirname, extname, isAbsolute, relative, resolve, sep } from "node:path";
|
||||
import { basename, dirname, extname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||
import { type BuildFailure, build, type Plugin as EsbuildPlugin, type Loader, type Message } from "esbuild";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import type { Plugin } from "./discover.ts";
|
||||
import { PLUGINS_SPECIFIER, packageName } from "./evaluate.ts";
|
||||
import { inlineSourceMap } from "./stack.ts";
|
||||
|
||||
/** What `buildFacet` takes besides the plugin and the kind. */
|
||||
|
|
@ -48,9 +50,6 @@ export type BuiltFacet =
|
|||
}
|
||||
| { readonly kind: "failed"; readonly plugin: string; readonly facet: string; readonly errors: readonly string[] };
|
||||
|
||||
/** The one specifier of this component a script imports (`pluginFile`): external always, answered per facet. */
|
||||
export const PLUGINS_SPECIFIER = "@earendil-works/chord/next/plugins";
|
||||
|
||||
/** What a free `__dirname` and `__filename` become in a build: names no script has, found again by `freeGlobals`. */
|
||||
const GLOBAL_MARKERS = { __dirname: "__chord_plugin_dirname", __filename: "__chord_plugin_filename" };
|
||||
|
||||
|
|
@ -239,7 +238,7 @@ class ImportRules {
|
|||
if (options.platform === "node" && BROWSER_ONLY.has(extname(found.path))) {
|
||||
return { errors: [{ text: `"${args.path}": stylesheets and assets are for browser builds` }] };
|
||||
}
|
||||
const checked = this.#checkFile(found.path, args.importer, args.path);
|
||||
const checked = this.#checkFile(found.path, args.importer, args.path, resolved.named);
|
||||
if (checked !== undefined) return { errors: [{ text: checked }] };
|
||||
return { path: found.path, pluginData: resolved.asset ? ASSET : undefined };
|
||||
});
|
||||
|
|
@ -277,7 +276,13 @@ class ImportRules {
|
|||
): Promise<
|
||||
| { readonly kind: "external" }
|
||||
| { readonly kind: "default" }
|
||||
| { readonly kind: "path"; readonly path: string; readonly asset: boolean }
|
||||
| {
|
||||
readonly kind: "path";
|
||||
readonly path: string;
|
||||
readonly asset: boolean;
|
||||
/** The plugin an import by name named (`@app/notes/contract`). */
|
||||
readonly named?: Plugin;
|
||||
}
|
||||
| { readonly kind: "error"; readonly text: string }
|
||||
> {
|
||||
const browser = this.#options.platform === "browser";
|
||||
|
|
@ -299,10 +304,12 @@ class ImportRules {
|
|||
const from = this.#place(importer);
|
||||
if (from.kind === "package") return { kind: "default" };
|
||||
if (specifier.startsWith(".") || isAbsolute(specifier)) return { kind: "path", path: specifier, asset: false };
|
||||
const name = packageName(specifier);
|
||||
const named = name === this.#own.name ? this.#own : this.#others.find((each) => each.name === name);
|
||||
if (named !== undefined) return this.#byName(specifier, named, from);
|
||||
if (from.kind === "copied") {
|
||||
return { kind: "error", text: `"${specifier}": ${copiedRule(from.plugin)}` };
|
||||
}
|
||||
const name = packageName(specifier);
|
||||
if (this.#own.external.includes(name)) {
|
||||
if (!browser) return { kind: "external" };
|
||||
return { kind: "error", text: `"${specifier}": chord.external packages are for Node builds` };
|
||||
|
|
@ -310,24 +317,51 @@ class ImportRules {
|
|||
if (this.#own.dependencies.includes(name)) return { kind: "default" };
|
||||
return {
|
||||
kind: "error",
|
||||
text: `"${specifier}": neither a host package nor in the dependencies of plugin "${this.#own.name}"`,
|
||||
text: `"${specifier}": neither a host package, in the dependencies of plugin "${this.#own.name}" nor a discovered plugin "${name}"`,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Why the file `path` (resolved from a relative import in `importer`) may not be inlined: outside the plugin and
|
||||
* every other discovered plugin, or, from a copied file, outside that file's own plugin; `undefined` when it may (a
|
||||
* file of another plugin is recorded as copied).
|
||||
* An import by name of a file of plugin `named` (`@app/notes/contract`, decision "Plugins import each other by name"):
|
||||
* its path in that plugin's directory, wherever that is; from a copied file only its own plugin's.
|
||||
*/
|
||||
#checkFile(path: string, importer: string, specifier: string): string | undefined {
|
||||
#byName(
|
||||
specifier: string,
|
||||
named: Plugin,
|
||||
from: Place,
|
||||
):
|
||||
| { readonly kind: "path"; readonly path: string; readonly asset: false; readonly named: Plugin }
|
||||
| { readonly kind: "error"; readonly text: string } {
|
||||
const file = specifier.slice(named.name.length + 1);
|
||||
if (file === "") {
|
||||
return {
|
||||
kind: "error",
|
||||
text: `"${specifier}": names plugin "${named.name}", not a file of it ("${named.name}/contract")`,
|
||||
};
|
||||
}
|
||||
if (from.kind === "copied" && from.plugin !== named) {
|
||||
return { kind: "error", text: `"${specifier}": ${copiedRule(from.plugin)}` };
|
||||
}
|
||||
return { kind: "path", path: join(named.dir, file), asset: false, named };
|
||||
}
|
||||
|
||||
/**
|
||||
* Why the file `path` (resolved from `importer`'s import of `specifier`, by name of plugin `named` or a relative path)
|
||||
* may not be inlined: a relative path into another plugin's directory, outside every discovered plugin, or, from a
|
||||
* copied file, outside that file's own plugin; `undefined` when it may (another plugin's file is recorded as copied).
|
||||
*/
|
||||
#checkFile(path: string, importer: string, specifier: string, named: Plugin | undefined): string | undefined {
|
||||
const from = this.#place(importer);
|
||||
const to = this.#place(path);
|
||||
if (to.kind === "own" && from.kind === "own") return undefined;
|
||||
if (to.kind === "copied" && (from.kind === "own" || (from.kind === "copied" && from.plugin === to.plugin))) {
|
||||
if (to.kind === "copied" && (to.plugin === named || (from.kind === "copied" && from.plugin === to.plugin))) {
|
||||
this.#copied.add(path);
|
||||
return undefined;
|
||||
}
|
||||
if (from.kind === "copied") return `"${specifier}": ${copiedRule(from.plugin)}`;
|
||||
if (to.kind === "copied") {
|
||||
return `"${specifier}": a relative path into plugin "${to.plugin.name}" (import its files by name: "${to.plugin.name}/…")`;
|
||||
}
|
||||
return `"${specifier}": outside the plugin (base code is reached through a host package, packages through dependencies)`;
|
||||
}
|
||||
|
||||
|
|
@ -351,12 +385,6 @@ export function inside(dir: string, path: string): boolean {
|
|||
return rest !== "" && !rest.startsWith("..") && !isAbsolute(rest);
|
||||
}
|
||||
|
||||
/** The package a bare specifier names: `@scope/name` or `name`, without a subpath. */
|
||||
export function packageName(specifier: string): string {
|
||||
const parts = specifier.split("/");
|
||||
return specifier.startsWith("@") ? parts.slice(0, 2).join("/") : parts[0]!;
|
||||
}
|
||||
|
||||
/** One build error as `file:line:column: text`, its file relative to the plugin, with the hint where it applies. */
|
||||
function describe(plugin: Plugin, message: Message): string {
|
||||
let text = message.text;
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import { basename, isAbsolute, join, relative, resolve } from "node:path";
|
|||
import { Script } from "node:vm";
|
||||
import type { Principal } from "../../next/data/index.ts";
|
||||
import type { Facet } from "../../next/host/index.ts";
|
||||
import { type BuiltFacet, PLUGINS_SPECIFIER, packageName } from "./build.ts";
|
||||
import type { BuiltFacet } from "./build.ts";
|
||||
import type { Plugin } from "./discover.ts";
|
||||
import { registerScript } from "./stack.ts";
|
||||
|
||||
|
|
@ -66,6 +66,9 @@ export function pluginFile(path: string): string {
|
|||
throw new Error(`pluginFile("${path}"): only a plugin's facet script, evaluated by the plugins component, has one`);
|
||||
}
|
||||
|
||||
/** The one specifier of this component a script imports (`pluginFile`): external always, answered per facet. */
|
||||
export const PLUGINS_SPECIFIER = "@earendil-works/chord/next/plugins";
|
||||
|
||||
/** The `require` a facet script of `plugin` takes (`where` names the facet in its errors). */
|
||||
function requireFor(plugin: Plugin, hostPackages: HostPackages, where: string): (specifier: string) => unknown {
|
||||
let fromPlugin: NodeJS.Require | undefined;
|
||||
|
|
@ -86,6 +89,12 @@ function requireFor(plugin: Plugin, hostPackages: HostPackages, where: string):
|
|||
};
|
||||
}
|
||||
|
||||
/** The package a bare specifier names: `@scope/name` or `name`, without a subpath. */
|
||||
export function packageName(specifier: string): string {
|
||||
const parts = specifier.split("/");
|
||||
return specifier.startsWith("@") ? parts.slice(0, 2).join("/") : parts[0]!;
|
||||
}
|
||||
|
||||
/** The absolute path of `path` inside `plugin`'s directory; one out of it is refused. */
|
||||
function fileOf(plugin: Plugin, path: string): string {
|
||||
const file = resolve(plugin.dir, path);
|
||||
|
|
|
|||
|
|
@ -15,11 +15,14 @@ export { type InstallOptions, installDependencies } from "./install.ts";
|
|||
export {
|
||||
type Built,
|
||||
createPlugins,
|
||||
type KindDefinition,
|
||||
type Plugins,
|
||||
type PluginsError,
|
||||
type PluginsOptions,
|
||||
type RolloutHost,
|
||||
type RolloutResult,
|
||||
type RolloutStep,
|
||||
} from "./plugins.ts";
|
||||
export { createPluginRuntime, type PluginRuntime, type PluginScript, type RuntimeLoad } from "./runtime.ts";
|
||||
export { mapStack } from "./stack.ts";
|
||||
export type { PluginsState } from "./watcher.ts";
|
||||
|
|
|
|||
|
|
@ -1,39 +1,30 @@
|
|||
// The plugins component (slice 7c §4, §5; decision "Slice 7c's points"): discovery, builds, evaluation and the watcher
|
||||
// together, and per host the facet loaded for every plugin, so a reload passes the host its complete plugin set
|
||||
// (unchanged plugins as the same facet objects, removed ones left out) and never restarts an unchanged plugin; per
|
||||
// plugin, a failed build or evaluation keeps the plugin's loaded facet (left out at startup) and is reported with the
|
||||
// host's report. `rollout` runs the app's steps in order and clears the marks of plugins applied on every step.
|
||||
// The plugins component (slice 7c §4, §5; decision "Slice 7c's points"; slice 10 D11, D12): discovery, builds and the
|
||||
// watcher, and per kind the complete current set of scripts (built on first use, updated by every rollout that ships
|
||||
// the kind), which its plugin runtime loads and reloads hosts with: unchanged plugins as the same facet objects, removed
|
||||
// ones left out, a failed build or evaluation keeping the plugin's loaded facet. `rollout` runs the app's steps in order
|
||||
// (one result per host) and clears the marks of plugins applied on every host of every step.
|
||||
|
||||
import { setFlagsFromString } from "node:v8";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import { isChordError, type Principal } from "../../next/data/index.ts";
|
||||
import type {
|
||||
CommittedReload,
|
||||
Facet,
|
||||
Host,
|
||||
LoadReport,
|
||||
PluginFacet,
|
||||
PluginFailure,
|
||||
ReloadReport,
|
||||
} from "../../next/host/index.ts";
|
||||
import type { Principal } from "../../next/data/index.ts";
|
||||
import type { Host, LoadReport, PluginFacet, ReloadReport } from "../../next/host/index.ts";
|
||||
import { Batch, type BatchError } from "../../next/state/internal.ts";
|
||||
import type { Watchable } from "../../watch.ts";
|
||||
import { type BuiltFacet, buildFacet } from "./build.ts";
|
||||
import { type Discovery, discoverPlugins, type Plugin } from "./discover.ts";
|
||||
import { evaluateFacet, type HostPackages } from "./evaluate.ts";
|
||||
import type { HostPackages } from "./evaluate.ts";
|
||||
import { type InstallOptions, installDependencies } from "./install.ts";
|
||||
import { createPluginRuntime, type PluginRuntime, type PluginScript } from "./runtime.ts";
|
||||
import { type MarkSnapshot, type PluginsState, PluginWatcher } from "./watcher.ts";
|
||||
|
||||
/** What `createPlugins` takes. */
|
||||
export type PluginsOptions = {
|
||||
/** The directories plugins are listed in, in order: the first root's plugin wins a duplicate name. */
|
||||
readonly roots: readonly string[];
|
||||
/** The app's facet kinds (`["server", "worker", "web", "tui"]`): one file per kind in a plugin. */
|
||||
readonly kinds: readonly string[];
|
||||
/** The kinds built for a browser (`["web"]`); the others are built for Node. */
|
||||
readonly browserKinds?: readonly string[];
|
||||
/** The host packages by specifier: external in every build, given to every evaluation on Node. */
|
||||
readonly hostPackages: HostPackages;
|
||||
/**
|
||||
* The app's facet kinds (one file per kind in a plugin), each with where its builds run and the host packages they
|
||||
* keep external: `{ server: { platform: "node", hostPackages: ["@earendil-works/chord/next", "@app/core"] }, … }`.
|
||||
*/
|
||||
readonly kinds: Readonly<Record<string, KindDefinition>>;
|
||||
/** The app's base code: a change sets `base` (the app restarts its hosts; Chord restarts nothing). */
|
||||
readonly base?: readonly string[];
|
||||
/** Where the app serves browser builds' assets: the URL prefix their imports evaluate to. */
|
||||
|
|
@ -44,6 +35,9 @@ export type PluginsOptions = {
|
|||
readonly onError: (error: PluginsError) => void;
|
||||
};
|
||||
|
||||
/** One facet kind: Node or browser builds, and the host packages (by specifier) its builds keep external. */
|
||||
export type KindDefinition = { readonly platform: "node" | "browser"; readonly hostPackages: readonly string[] };
|
||||
|
||||
/** What the component reports. */
|
||||
export type PluginsError = BatchError | { readonly kind: "watch_failed"; readonly error: unknown };
|
||||
|
||||
|
|
@ -57,16 +51,26 @@ export type Built = {
|
|||
readonly removed: readonly string[];
|
||||
};
|
||||
|
||||
/** What a rollout step does: reloads its host (or hosts) with the facets of its kinds from a build. */
|
||||
/**
|
||||
* What a rollout step did on one of its hosts (`host` named by the step): its reload's report; `stopped` when the host
|
||||
* stopped cleanly during the step (it gets the current set at its next start); `error` when it crashed or its reload
|
||||
* failed (every plugin of the step's kinds in the build counts as failed there).
|
||||
*/
|
||||
export type RolloutHost =
|
||||
| { readonly host: string; readonly report: ReloadReport }
|
||||
| { readonly host: string; readonly stopped: true }
|
||||
| { readonly host: string; readonly error: unknown };
|
||||
|
||||
/** What a rollout step does: reloads its hosts with the facets of its kinds, one result per host (D12). */
|
||||
export type RolloutStep = {
|
||||
readonly name: string;
|
||||
readonly kinds: readonly string[];
|
||||
reload(built: Built, context: Context): Promise<ReloadReport>;
|
||||
reload(built: Built, context: Context): Promise<{ readonly hosts: readonly RolloutHost[] }>;
|
||||
};
|
||||
|
||||
/** What a rollout did: each step's report in order, and the step it stopped at (rejected), if it stopped. */
|
||||
/** What a rollout did: each step's hosts in order, and the step it stopped at (rejected), if it stopped. */
|
||||
export type RolloutResult = {
|
||||
readonly steps: readonly { readonly name: string; readonly report: ReloadReport }[];
|
||||
readonly steps: readonly { readonly name: string; readonly hosts: readonly RolloutHost[] }[];
|
||||
readonly stopped: { readonly step: string; readonly error: unknown } | undefined;
|
||||
};
|
||||
|
||||
|
|
@ -78,32 +82,41 @@ export interface Plugins {
|
|||
readonly discovery: Discovery;
|
||||
/**
|
||||
* Builds every facet of `names` (and of plugins discovered since the last build; discovery runs again first when
|
||||
* the plugin set changed): plain data, reused by later loads and reloads.
|
||||
* the plugin set changed): plain data. It changes no current set: a rollout ships it.
|
||||
*/
|
||||
build(names: readonly string[], context: Context): Promise<Built>;
|
||||
/**
|
||||
* Builds (if needed) and evaluates the `kind` facet of every discovered plugin and loads them with `core` (the app's
|
||||
* own facets): a plugin whose build or evaluation fails is left out and reported with the host's report.
|
||||
* The complete current set of `kind`: every discovered plugin's facet of it (built on first use), as the last
|
||||
* rollout or reload shipping `kind` left it; what a worker starts with.
|
||||
*/
|
||||
scripts(kind: string, context: Context): Promise<readonly PluginScript[]>;
|
||||
/**
|
||||
* `kind`'s current set now, synchronously (`undefined` before its first use): a rollout step reads it as the rollout
|
||||
* applied its build, before another rollout can change it.
|
||||
*/
|
||||
current(kind: string): readonly PluginScript[] | undefined;
|
||||
/**
|
||||
* Loads `host` with `kind`'s current set, evaluated with `modules` (the host packages, this process's instances, by
|
||||
* specifier; kept for the host's reloads), and `core` (the app's own facets): a plugin whose build or evaluation
|
||||
* fails is left out and reported with the host's report.
|
||||
*/
|
||||
load<P extends Principal>(
|
||||
host: Host<P>,
|
||||
options: { readonly kind: string; readonly core: readonly PluginFacet<P>[] },
|
||||
options: { readonly kind: string; readonly core: readonly PluginFacet<P>[]; readonly modules: HostPackages },
|
||||
context: Context,
|
||||
): Promise<LoadReport>;
|
||||
/**
|
||||
* Reloads `host` (loaded by `load`) with the facets of its kind in `built`, after its earlier reloads: the complete
|
||||
* set (the plugins it loaded and those `built` brings), its core facets and unchanged plugins as the same objects,
|
||||
* new scripts evaluated, a plugin whose build or evaluation failed passed as its loaded facet (an added one left out,
|
||||
* both given to the host's closure rule), the plugins `built` removed left out. The host's report, those failures
|
||||
* included; what the host committed is the loaded set also when the reload rejects `aborted` after its commit.
|
||||
* Applies `built` to the current set of `host`'s kind (loaded by `load`) and reloads `host` with that complete set,
|
||||
* after its earlier reloads (`PluginRuntime.reload`). The host's report.
|
||||
*/
|
||||
reload<P extends Principal>(host: Host<P>, built: Built, context: Context): Promise<ReloadReport>;
|
||||
/** An in-process rollout step named `name`: `reload(host, built)`, for the kind `host` was loaded with. */
|
||||
/** An in-process rollout step named `name`: `reload(host, built)`, one host named `name`. */
|
||||
step<P extends Principal>(name: string, host: Host<P>): RolloutStep;
|
||||
/**
|
||||
* Runs `steps` in order with `built` and stops at one that rejects (a host closed, an abort); failed plugins do not
|
||||
* stop it. Clears the dirty marks of the built plugins applied on every step (not failed on any) that did not
|
||||
* change since the build.
|
||||
* Runs `steps` in order with `built`, applying it to the current set of each step's kinds right before that step,
|
||||
* and stops at one that rejects; failed plugins and hosts do not stop it. Clears the dirty marks of the built
|
||||
* plugins applied on every host of every step (not failed on any, every built kind of theirs shipped by a step)
|
||||
* that did not change since the build.
|
||||
*/
|
||||
rollout(steps: readonly RolloutStep[], built: Built, context: Context): Promise<RolloutResult>;
|
||||
/** Installs plugin `name`'s dependencies (`installDependencies`): its install mark clears, and it is dirty. */
|
||||
|
|
@ -112,82 +125,35 @@ export interface Plugins {
|
|||
close(): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the plugins component: turns V8's compilation cache off (once per process: each distinct evaluated script
|
||||
* would stay cached) and starts the watcher. Discovery runs at the first `build` or `load`.
|
||||
*/
|
||||
/** Creates the plugins component (and its plugin runtime) and starts the watcher. Discovery runs at first use. */
|
||||
export function createPlugins(options: PluginsOptions, _context: Context): Plugins {
|
||||
if (!cacheOff) {
|
||||
setFlagsFromString("--no-compilation-cache");
|
||||
cacheOff = true;
|
||||
}
|
||||
return new PluginsComponent(options);
|
||||
}
|
||||
|
||||
/** Whether this process turned V8's compilation cache off already. */
|
||||
let cacheOff = false;
|
||||
|
||||
/** A plugin's facet as a host loaded it, with its script's digest. */
|
||||
type LoadedFacet = { readonly facet: Facet<Principal>; readonly digest: string };
|
||||
|
||||
/**
|
||||
* A host's plugins as the component loaded them: its kind, its core facets, each plugin's loaded facet, and its
|
||||
* reloads, one after another (each computes its complete set once the previous one adopted its result).
|
||||
*/
|
||||
type HostPlugins = {
|
||||
readonly kind: string;
|
||||
readonly core: readonly PluginFacet<Principal>[];
|
||||
readonly loaded: Map<string, LoadedFacet>;
|
||||
reloading: Promise<void>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Adopts as `state`'s loaded facets the set `next` a reload committed: a plugin the host kept old (`failed`) stays loaded
|
||||
* as it was, an added one it left out is not loaded.
|
||||
*/
|
||||
function adopt(state: HostPlugins, next: Map<string, LoadedFacet>, failed: readonly string[]): void {
|
||||
for (const plugin of failed) {
|
||||
const loaded = state.loaded.get(plugin);
|
||||
if (loaded === undefined) next.delete(plugin);
|
||||
else next.set(plugin, loaded);
|
||||
}
|
||||
state.loaded.clear();
|
||||
for (const [name, each] of next) state.loaded.set(name, each);
|
||||
}
|
||||
|
||||
/** What the commit of a reload that rejected applied: an `aborted` error after its commit says (`details.committed`). */
|
||||
function committedOf(error: unknown): CommittedReload | undefined {
|
||||
if (!isChordError(error, "aborted")) return undefined;
|
||||
const details = error.details as { readonly committed?: CommittedReload } | null | undefined;
|
||||
// The kernel's own error: its `committed` is a `CommittedReload`.
|
||||
return typeof details === "object" && details !== null ? details.committed : undefined;
|
||||
}
|
||||
|
||||
/** A plugin's facet passed to a host, or why it was not: a build or evaluation failure. */
|
||||
type Prepared =
|
||||
| { readonly kind: "facet"; readonly facet: Facet<Principal>; readonly digest: string }
|
||||
| { readonly kind: "failed"; readonly error: unknown };
|
||||
|
||||
class PluginsComponent implements Plugins {
|
||||
readonly #options: PluginsOptions;
|
||||
readonly #watcher: PluginWatcher;
|
||||
readonly #runtime: PluginRuntime;
|
||||
#discovery: Discovery = { plugins: [], problems: [] };
|
||||
#discovered = false;
|
||||
/** The latest build of every (plugin, kind) built so far, by `plugin\u0000kind`. */
|
||||
readonly #built = new Map<string, BuiltFacet>();
|
||||
/** Each loaded host's plugins, held as long as the host is. */
|
||||
readonly #hosts = new WeakMap<object, HostPlugins>();
|
||||
/** The current set of each kind used so far, by plugin name. */
|
||||
readonly #current = new Map<string, Map<string, PluginScript>>();
|
||||
/** The kind each loaded host was loaded with, held as long as the host is. */
|
||||
readonly #kinds = new WeakMap<object, string>();
|
||||
/** The marks each build started from, held as long as its result is (a rollout given a copy clears none). */
|
||||
readonly #snapshots = new WeakMap<Built, MarkSnapshot>();
|
||||
|
||||
constructor(options: PluginsOptions) {
|
||||
this.#options = options;
|
||||
this.#runtime = createPluginRuntime();
|
||||
const batch = new Batch((error) => options.onError(error), undefined);
|
||||
this.#watcher = new PluginWatcher(
|
||||
{
|
||||
roots: options.roots,
|
||||
base: options.base ?? [],
|
||||
kinds: options.kinds,
|
||||
kinds: Object.keys(options.kinds),
|
||||
failed: (error) => options.onError({ kind: "watch_failed", error }),
|
||||
},
|
||||
batch,
|
||||
|
|
@ -219,132 +185,68 @@ class PluginsComponent implements Plugins {
|
|||
return built;
|
||||
}
|
||||
|
||||
async scripts(kind: string, context: Context): Promise<readonly PluginScript[]> {
|
||||
return [...(await this.#set(kind, context)).values()];
|
||||
}
|
||||
|
||||
current(kind: string): readonly PluginScript[] | undefined {
|
||||
const set = this.#current.get(kind);
|
||||
return set === undefined ? undefined : [...set.values()];
|
||||
}
|
||||
|
||||
async load<P extends Principal>(
|
||||
host: Host<P>,
|
||||
options: { readonly kind: string; readonly core: readonly PluginFacet<P>[] },
|
||||
options: { readonly kind: string; readonly core: readonly PluginFacet<P>[]; readonly modules: HostPackages },
|
||||
context: Context,
|
||||
): Promise<LoadReport> {
|
||||
const { kind } = options;
|
||||
if (!this.#discovered) await this.#discover(context);
|
||||
const plugins = this.#discovery.plugins.filter((plugin) => plugin.facets[kind] !== undefined);
|
||||
const prepared = await Promise.all(
|
||||
plugins.map(async (plugin) => {
|
||||
const key = `${plugin.name}\u0000${kind}`;
|
||||
const built = this.#built.get(key) ?? (await this.#buildOne(plugin, kind, context));
|
||||
return [plugin.name, this.#prepare(plugin, built)] as const;
|
||||
}),
|
||||
);
|
||||
const failed: PluginFailure[] = [];
|
||||
const facets: PluginFacet<P>[] = [...options.core];
|
||||
for (const [plugin, each] of prepared) {
|
||||
if (each.kind === "failed") failed.push({ plugin, error: each.error });
|
||||
// Evaluated for this host's principal type: the facets are the app's (`Facet<P>` is invariant in `P`).
|
||||
else facets.push({ name: plugin, facet: each.facet as unknown as Facet<P> });
|
||||
}
|
||||
const report = await host.load(facets, context);
|
||||
const left = new Set(report.failed.map(({ plugin }) => plugin));
|
||||
const loaded = new Map<string, LoadedFacet>();
|
||||
for (const [plugin, each] of prepared) {
|
||||
if (each.kind === "facet" && !left.has(plugin)) loaded.set(plugin, each);
|
||||
}
|
||||
// The host's own facets, for its later reloads.
|
||||
const core = options.core as unknown as readonly PluginFacet<Principal>[];
|
||||
this.#hosts.set(host, { kind, core, loaded, reloading: Promise.resolve() });
|
||||
return { failed: [...failed, ...report.failed] };
|
||||
const { kind, core, modules } = options;
|
||||
const scripts = await this.scripts(kind, context);
|
||||
this.#kinds.set(host, kind);
|
||||
return this.#runtime.load(host, { kind, core, scripts, modules }, context);
|
||||
}
|
||||
|
||||
reload<P extends Principal>(host: Host<P>, built: Built, context: Context): Promise<ReloadReport> {
|
||||
const state = this.#hosts.get(host);
|
||||
if (state === undefined) throw new Error("plugins.reload: the host was not loaded by this component");
|
||||
// One after another per host: a reload computing its set from a loaded map an earlier one is about to change
|
||||
// would pass that plugin's old facet and roll it back.
|
||||
const reload = state.reloading.then(() => this.#reload(host, state, built, context));
|
||||
// The tail settles to nothing: a report kept as its value would keep a rejected facet's closure (its error).
|
||||
state.reloading = reload.then(
|
||||
() => {},
|
||||
() => {},
|
||||
);
|
||||
return reload;
|
||||
}
|
||||
|
||||
/**
|
||||
* One reload of `host` with `built`: the plugins it loaded and those `built` brings, without the ones `built`
|
||||
* removed or rebuilt without a facet of the host's kind (its own discovery's, never a later one's).
|
||||
*/
|
||||
async #reload<P extends Principal>(
|
||||
host: Host<P>,
|
||||
state: HostPlugins,
|
||||
built: Built,
|
||||
context: Context,
|
||||
): Promise<ReloadReport> {
|
||||
const { kind } = state;
|
||||
const fresh = new Map<string, BuiltFacet>();
|
||||
for (const facet of built.facets) if (facet.facet === kind) fresh.set(facet.plugin, facet);
|
||||
const names = [...new Set([...state.loaded.keys(), ...fresh.keys()])].sort();
|
||||
const failed: PluginFailure[] = [];
|
||||
const next = new Map<string, LoadedFacet>();
|
||||
for (const name of names) {
|
||||
// Own properties only: a plugin may be named like one of `Object.prototype`'s (`constructor`).
|
||||
const plugin = Object.hasOwn(built.plugins, name) ? built.plugins[name] : undefined;
|
||||
// Removed by this build's discovery, or rebuilt without a facet of this kind: left out.
|
||||
if (built.removed.includes(name) || (plugin !== undefined && plugin.facets[kind] === undefined)) continue;
|
||||
const loaded = state.loaded.get(name);
|
||||
const script = fresh.get(name);
|
||||
if (
|
||||
script === undefined ||
|
||||
plugin === undefined ||
|
||||
(script.kind === "script" && script.digest === loaded?.digest)
|
||||
) {
|
||||
if (loaded !== undefined) next.set(name, loaded);
|
||||
continue;
|
||||
}
|
||||
const prepared = this.#prepare(plugin, script);
|
||||
if (prepared.kind === "facet") {
|
||||
next.set(name, prepared);
|
||||
continue;
|
||||
}
|
||||
failed.push({ plugin: name, error: prepared.error });
|
||||
if (loaded !== undefined) next.set(name, loaded);
|
||||
}
|
||||
const facets = [...state.core, ...[...next].map(([name, { facet }]) => ({ name, facet }))];
|
||||
// The host takes facets of its own principal type: the component loaded them for it. The plugins whose build or
|
||||
// evaluation failed are the host's to report, and its closure rule's to consider.
|
||||
let report: ReloadReport;
|
||||
try {
|
||||
report = await host.reload(facets as unknown as readonly PluginFacet<P>[], context, { failed });
|
||||
} catch (error) {
|
||||
// Aborted after its commit: the commit stands, and is what the host runs now.
|
||||
const committed = committedOf(error);
|
||||
if (committed !== undefined) adopt(state, next, committed.failed);
|
||||
throw error;
|
||||
}
|
||||
adopt(
|
||||
state,
|
||||
next,
|
||||
report.failed.map(({ plugin }) => plugin),
|
||||
);
|
||||
return report;
|
||||
async reload<P extends Principal>(host: Host<P>, built: Built, context: Context): Promise<ReloadReport> {
|
||||
const kind = this.#kinds.get(host);
|
||||
if (kind === undefined) throw new Error("plugins.reload: the host was not loaded by this component");
|
||||
const set = await this.#set(kind, context);
|
||||
// Applied and read in one step: two reloads started together each pass the set their own build made.
|
||||
ship(set, kind, built);
|
||||
return this.#runtime.reload(host, [...set.values()], context);
|
||||
}
|
||||
|
||||
step<P extends Principal>(name: string, host: Host<P>): RolloutStep {
|
||||
const state = this.#hosts.get(host);
|
||||
if (state === undefined) throw new Error("plugins.step: the host was not loaded by this component");
|
||||
return { name, kinds: [state.kind], reload: (built, context) => this.reload(host, built, context) };
|
||||
const kind = this.#kinds.get(host);
|
||||
if (kind === undefined) throw new Error("plugins.step: the host was not loaded by this component");
|
||||
return {
|
||||
name,
|
||||
kinds: [kind],
|
||||
reload: async (built, context) => ({
|
||||
hosts: [{ host: name, report: await this.reload(host, built, context) }],
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
async rollout(steps: readonly RolloutStep[], built: Built, context: Context): Promise<RolloutResult> {
|
||||
const done: { readonly name: string; readonly report: ReloadReport }[] = [];
|
||||
const done: { readonly name: string; readonly hosts: readonly RolloutHost[] }[] = [];
|
||||
const failed = new Set<string>();
|
||||
const shipped = new Set<string>();
|
||||
for (const step of steps) {
|
||||
let report: ReloadReport;
|
||||
let hosts: readonly RolloutHost[];
|
||||
try {
|
||||
report = await step.reload(built, context);
|
||||
for (const kind of step.kinds) ship(await this.#set(kind, context), kind, built);
|
||||
hosts = (await step.reload(built, context)).hosts;
|
||||
} catch (error) {
|
||||
return { steps: done, stopped: { step: step.name, error } };
|
||||
}
|
||||
for (const { plugin } of report.failed) failed.add(plugin);
|
||||
done.push({ name: step.name, report });
|
||||
for (const kind of step.kinds) shipped.add(kind);
|
||||
for (const entry of hosts) {
|
||||
if ("report" in entry) for (const { plugin } of entry.report.failed) failed.add(plugin);
|
||||
else if ("error" in entry) for (const plugin of builtOf(built, step.kinds)) failed.add(plugin);
|
||||
}
|
||||
done.push({ name: step.name, hosts });
|
||||
}
|
||||
// A plugin with a built kind no step shipped is not applied everywhere it runs: it stays dirty.
|
||||
for (const { plugin, facet } of built.facets) if (!shipped.has(facet)) failed.add(plugin);
|
||||
const snapshot = this.#snapshots.get(built);
|
||||
if (snapshot !== undefined) {
|
||||
this.#watcher.applied(
|
||||
|
|
@ -370,17 +272,41 @@ class PluginsComponent implements Plugins {
|
|||
/** Discovers the plugins again; the changes of the plugin set and of `package.json`s seen until now clear. */
|
||||
async #discover(context: Context): Promise<void> {
|
||||
const seen = this.#watcher.discoverySnapshot();
|
||||
this.#discovery = await discoverPlugins(this.#options.roots, { kinds: this.#options.kinds }, context);
|
||||
this.#discovery = await discoverPlugins(
|
||||
this.#options.roots,
|
||||
{ kinds: Object.keys(this.#options.kinds) },
|
||||
context,
|
||||
);
|
||||
this.#discovered = true;
|
||||
this.#watcher.know(this.#discovery, seen);
|
||||
}
|
||||
|
||||
/** `kind`'s current set, made at its first use from every discovered plugin's facet of it (built if needed). */
|
||||
async #set(kind: string, context: Context): Promise<Map<string, PluginScript>> {
|
||||
let set = this.#current.get(kind);
|
||||
if (set !== undefined) return set;
|
||||
if (!this.#discovered) await this.#discover(context);
|
||||
const plugins = this.#discovery.plugins.filter((plugin) => plugin.facets[kind] !== undefined);
|
||||
const scripts = await Promise.all(
|
||||
plugins.map(async (plugin): Promise<PluginScript> => {
|
||||
const built =
|
||||
this.#built.get(`${plugin.name}\u0000${kind}`) ?? (await this.#buildOne(plugin, kind, context));
|
||||
return { plugin, built };
|
||||
}),
|
||||
);
|
||||
// Built concurrently: a set made meanwhile (by a concurrent first use) wins.
|
||||
set = this.#current.get(kind) ?? new Map(scripts.map((script) => [script.plugin.name, script]));
|
||||
this.#current.set(kind, set);
|
||||
return set;
|
||||
}
|
||||
|
||||
/** Builds `plugin`'s facet of `kind` and keeps it as the latest; the watcher learns what it read. */
|
||||
async #buildOne(plugin: Plugin, kind: string, context: Context): Promise<BuiltFacet> {
|
||||
const { hostPackages, browserKinds = [], publicPath } = this.#options;
|
||||
const platform = browserKinds.includes(kind) ? "browser" : "node";
|
||||
const { kinds, publicPath } = this.#options;
|
||||
// Discovery finds facet files of the app's kinds only.
|
||||
const { platform, hostPackages } = kinds[kind]!;
|
||||
const options = {
|
||||
hostPackages: Object.keys(hostPackages),
|
||||
hostPackages,
|
||||
platform,
|
||||
plugins: this.#discovery.plugins,
|
||||
...(publicPath === undefined ? {} : { publicPath }),
|
||||
|
|
@ -390,18 +316,24 @@ class PluginsComponent implements Plugins {
|
|||
this.#watcher.used([built]);
|
||||
return built;
|
||||
}
|
||||
}
|
||||
|
||||
/** `built` evaluated for `plugin`, or why not: its build failed, or its evaluation threw. */
|
||||
#prepare(plugin: Plugin, built: BuiltFacet): Prepared {
|
||||
if (built.kind === "failed") {
|
||||
const message = `plugin "${plugin.name}": its ${built.facet} facet failed to build:\n${built.errors.join("\n")}`;
|
||||
return { kind: "failed", error: new Error(message) };
|
||||
}
|
||||
try {
|
||||
const facet = evaluateFacet(built, { hostPackages: this.#options.hostPackages, plugin });
|
||||
return { kind: "facet", facet, digest: built.digest };
|
||||
} catch (error) {
|
||||
return { kind: "failed", error };
|
||||
/**
|
||||
* Applies `built` to `set`, `kind`'s current set: its removed plugins and those rebuilt without a facet of `kind` leave
|
||||
* it, its facets of `kind` replace theirs.
|
||||
*/
|
||||
function ship(set: Map<string, PluginScript>, kind: string, built: Built): void {
|
||||
for (const name of built.removed) set.delete(name);
|
||||
for (const [name, plugin] of Object.entries(built.plugins)) if (plugin.facets[kind] === undefined) set.delete(name);
|
||||
for (const facet of built.facets) {
|
||||
// Own properties only: a plugin may be named like one of `Object.prototype`'s (`constructor`).
|
||||
if (facet.facet === kind && Object.hasOwn(built.plugins, facet.plugin)) {
|
||||
set.set(facet.plugin, { plugin: built.plugins[facet.plugin]!, built: facet });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** The plugins of `built` with a facet of one of `kinds`. */
|
||||
function builtOf(built: Built, kinds: readonly string[]): string[] {
|
||||
return built.facets.filter(({ facet }) => kinds.includes(facet)).map(({ plugin }) => plugin);
|
||||
}
|
||||
|
|
|
|||
215
packages/chord/src/next-node/plugins/runtime.ts
Normal file
215
packages/chord/src/next-node/plugins/runtime.ts
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
// The plugin runtime (slice 10 §3, D11): loading and reloading hosts with shipped scripts, without building (no
|
||||
// esbuild, no watcher), so a worker evaluates what the server built. Per host it keeps the facet loaded for every plugin
|
||||
// and passes the host its complete set at each reload: unchanged plugins (the same digest) as the same facet objects,
|
||||
// removed ones (loaded, not shipped) left out; a plugin whose build or evaluation failed keeps its loaded facet (left
|
||||
// out at startup) and is reported with the host's report. The plugins component (`createPlugins`) uses one.
|
||||
|
||||
import { setFlagsFromString } from "node:v8";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import { isChordError, type Principal } from "../../next/data/index.ts";
|
||||
import type {
|
||||
CommittedReload,
|
||||
Facet,
|
||||
Host,
|
||||
LoadReport,
|
||||
PluginFacet,
|
||||
PluginFailure,
|
||||
ReloadReport,
|
||||
} from "../../next/host/index.ts";
|
||||
import type { BuiltFacet } from "./build.ts";
|
||||
import type { Plugin } from "./discover.ts";
|
||||
import { evaluateFacet, type HostPackages } from "./evaluate.ts";
|
||||
|
||||
/** One plugin's facet of a kind as shipped to a host (plain data): its build and its plugin record. */
|
||||
export type PluginScript = { readonly plugin: Plugin; readonly built: BuiltFacet };
|
||||
|
||||
/** What `PluginRuntime.load` takes besides the host. */
|
||||
export type RuntimeLoad<P extends Principal> = {
|
||||
readonly kind: string;
|
||||
readonly core: readonly PluginFacet<P>[];
|
||||
readonly scripts: readonly PluginScript[];
|
||||
readonly modules: HostPackages;
|
||||
};
|
||||
|
||||
/** The plugin runtime: loads and reloads hosts with complete sets of shipped scripts. */
|
||||
export interface PluginRuntime {
|
||||
/**
|
||||
* Evaluates `scripts` (the complete set of `kind`) with `modules` (the host packages their `require` answers: this
|
||||
* process's instances, kept for the host's reloads) and loads them on `host` with `core` (the app's own facets): a
|
||||
* plugin whose build or evaluation failed is left out and reported with the host's report.
|
||||
*/
|
||||
load<P extends Principal>(host: Host<P>, options: RuntimeLoad<P>, context: Context): Promise<LoadReport>;
|
||||
/**
|
||||
* Reloads `host` (loaded by `load`) with `scripts`, the complete set now, after its earlier reloads: its core facets
|
||||
* and unchanged plugins (the same digest) as the same objects, new scripts evaluated, a plugin whose build or
|
||||
* evaluation failed passed as its loaded facet (an added one left out; both given to the host's closure rule),
|
||||
* loaded plugins not in `scripts` left out. What the host committed is the loaded set, also when the reload rejects
|
||||
* `aborted` after its commit.
|
||||
*/
|
||||
reload<P extends Principal>(
|
||||
host: Host<P>,
|
||||
scripts: readonly PluginScript[],
|
||||
context: Context,
|
||||
): Promise<ReloadReport>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a plugin runtime: turns V8's compilation cache off (once per process: each distinct evaluated script would
|
||||
* stay cached).
|
||||
*/
|
||||
export function createPluginRuntime(): PluginRuntime {
|
||||
if (!cacheOff) {
|
||||
setFlagsFromString("--no-compilation-cache");
|
||||
cacheOff = true;
|
||||
}
|
||||
return new Runtime();
|
||||
}
|
||||
|
||||
/** Whether this process turned V8's compilation cache off already. */
|
||||
let cacheOff = false;
|
||||
|
||||
/** A plugin's facet as a host loaded it, with its script's digest. */
|
||||
type LoadedFacet = { readonly facet: Facet<Principal>; readonly digest: string };
|
||||
|
||||
/**
|
||||
* A host's plugins as the runtime loaded them: its core facets, its modules, each plugin's loaded facet, and its
|
||||
* reloads, one after another (each computes its complete set once the previous one adopted its result).
|
||||
*/
|
||||
type HostPlugins = {
|
||||
readonly core: readonly PluginFacet<Principal>[];
|
||||
readonly modules: HostPackages;
|
||||
readonly loaded: Map<string, LoadedFacet>;
|
||||
reloading: Promise<void>;
|
||||
};
|
||||
|
||||
/** A plugin's facet passed to a host, or why it was not: a build or evaluation failure. */
|
||||
type Prepared =
|
||||
| { readonly kind: "facet"; readonly facet: Facet<Principal>; readonly digest: string }
|
||||
| { readonly kind: "failed"; readonly error: unknown };
|
||||
|
||||
class Runtime implements PluginRuntime {
|
||||
/** Each loaded host's plugins, held as long as the host is. */
|
||||
readonly #hosts = new WeakMap<object, HostPlugins>();
|
||||
|
||||
async load<P extends Principal>(host: Host<P>, options: RuntimeLoad<P>, context: Context): Promise<LoadReport> {
|
||||
const { modules } = options;
|
||||
const prepared = options.scripts.map(
|
||||
({ plugin, built }) => [plugin.name, prepare(plugin, built, modules)] as const,
|
||||
);
|
||||
const failed: PluginFailure[] = [];
|
||||
const facets: PluginFacet<P>[] = [...options.core];
|
||||
for (const [plugin, each] of prepared) {
|
||||
if (each.kind === "failed") failed.push({ plugin, error: each.error });
|
||||
// Evaluated for this host's principal type: the facets are the app's (`Facet<P>` is invariant in `P`).
|
||||
else facets.push({ name: plugin, facet: each.facet as unknown as Facet<P> });
|
||||
}
|
||||
const report = await host.load(facets, context);
|
||||
const left = new Set(report.failed.map(({ plugin }) => plugin));
|
||||
const loaded = new Map<string, LoadedFacet>();
|
||||
for (const [plugin, each] of prepared) {
|
||||
if (each.kind === "facet" && !left.has(plugin)) loaded.set(plugin, each);
|
||||
}
|
||||
// The host's own facets, for its later reloads.
|
||||
const core = options.core as unknown as readonly PluginFacet<Principal>[];
|
||||
this.#hosts.set(host, { core, modules, loaded, reloading: Promise.resolve() });
|
||||
return { failed: [...failed, ...report.failed] };
|
||||
}
|
||||
|
||||
reload<P extends Principal>(
|
||||
host: Host<P>,
|
||||
scripts: readonly PluginScript[],
|
||||
context: Context,
|
||||
): Promise<ReloadReport> {
|
||||
const state = this.#hosts.get(host);
|
||||
if (state === undefined) throw new Error("plugins.reload: the host was not loaded by this runtime");
|
||||
// One after another per host: a reload computing its set from a loaded map an earlier one is about to change
|
||||
// would pass that plugin's old facet and roll it back.
|
||||
const reload = state.reloading.then(() => this.#reload(host, state, scripts, context));
|
||||
// The tail settles to nothing: a report kept as its value would keep a rejected facet's closure (its error).
|
||||
state.reloading = reload.then(
|
||||
() => {},
|
||||
() => {},
|
||||
);
|
||||
return reload;
|
||||
}
|
||||
|
||||
/** One reload of `host` with the complete set `scripts`. */
|
||||
async #reload<P extends Principal>(
|
||||
host: Host<P>,
|
||||
state: HostPlugins,
|
||||
scripts: readonly PluginScript[],
|
||||
context: Context,
|
||||
): Promise<ReloadReport> {
|
||||
const failed: PluginFailure[] = [];
|
||||
const next = new Map<string, LoadedFacet>();
|
||||
for (const { plugin, built } of scripts) {
|
||||
const { name } = plugin;
|
||||
const loaded = state.loaded.get(name);
|
||||
if (built.kind === "script" && built.digest === loaded?.digest) {
|
||||
next.set(name, loaded);
|
||||
continue;
|
||||
}
|
||||
const prepared = prepare(plugin, built, state.modules);
|
||||
if (prepared.kind === "facet") {
|
||||
next.set(name, prepared);
|
||||
continue;
|
||||
}
|
||||
failed.push({ plugin: name, error: prepared.error });
|
||||
if (loaded !== undefined) next.set(name, loaded);
|
||||
}
|
||||
const facets = [...state.core, ...[...next].map(([name, { facet }]) => ({ name, facet }))];
|
||||
// The host takes facets of its own principal type: the runtime loaded them for it. The plugins whose build or
|
||||
// evaluation failed are the host's to report, and its closure rule's to consider.
|
||||
let report: ReloadReport;
|
||||
try {
|
||||
report = await host.reload(facets as unknown as readonly PluginFacet<P>[], context, { failed });
|
||||
} catch (error) {
|
||||
// Aborted after its commit: the commit stands, and is what the host runs now.
|
||||
const committed = committedOf(error);
|
||||
if (committed !== undefined) adopt(state, next, committed.failed);
|
||||
throw error;
|
||||
}
|
||||
adopt(
|
||||
state,
|
||||
next,
|
||||
report.failed.map(({ plugin }) => plugin),
|
||||
);
|
||||
return report;
|
||||
}
|
||||
}
|
||||
|
||||
/** `built` evaluated for `plugin` with `modules`, or why not: its build failed, or its evaluation threw. */
|
||||
function prepare(plugin: Plugin, built: BuiltFacet, modules: HostPackages): Prepared {
|
||||
if (built.kind === "failed") {
|
||||
const message = `plugin "${plugin.name}": its ${built.facet} facet failed to build:\n${built.errors.join("\n")}`;
|
||||
return { kind: "failed", error: new Error(message) };
|
||||
}
|
||||
try {
|
||||
const facet = evaluateFacet(built, { hostPackages: modules, plugin });
|
||||
return { kind: "facet", facet, digest: built.digest };
|
||||
} catch (error) {
|
||||
return { kind: "failed", error };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Adopts as `state`'s loaded facets the set `next` a reload committed: a plugin the host kept old (`failed`) stays loaded
|
||||
* as it was, an added one it left out is not loaded.
|
||||
*/
|
||||
function adopt(state: HostPlugins, next: Map<string, LoadedFacet>, failed: readonly string[]): void {
|
||||
for (const plugin of failed) {
|
||||
const loaded = state.loaded.get(plugin);
|
||||
if (loaded === undefined) next.delete(plugin);
|
||||
else next.set(plugin, loaded);
|
||||
}
|
||||
state.loaded.clear();
|
||||
for (const [name, each] of next) state.loaded.set(name, each);
|
||||
}
|
||||
|
||||
/** What the commit of a reload that rejected applied: an `aborted` error after its commit says (`details.committed`). */
|
||||
function committedOf(error: unknown): CommittedReload | undefined {
|
||||
if (!isChordError(error, "aborted")) return undefined;
|
||||
const details = error.details as { readonly committed?: CommittedReload } | null | undefined;
|
||||
// The kernel's own error: its `committed` is a `CommittedReload`.
|
||||
return typeof details === "object" && details !== null ? details.committed : undefined;
|
||||
}
|
||||
95
packages/chord/src/next-node/workers/README.md
Normal file
95
packages/chord/src/next-node/workers/README.md
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
# workers
|
||||
|
||||
The workers component (slice 10 §3, §4, design §23, decision "Slice 10's points"): a forked worker process per address
|
||||
of one host kind, started on demand by the route it owns, stopped when idle, never restarted by itself; its plugins are
|
||||
built by the server and shipped. `workerMain` is the worker's side. Node-only. Exported as
|
||||
`@earendil-works/chord/next/workers` once v2 has its entry point.
|
||||
|
||||
## For apps (`index.ts`)
|
||||
|
||||
```ts
|
||||
// server.ts
|
||||
const workers = createWorkers(server, Session, {
|
||||
kind: "worker", entry: "./worker.ts", execArgv: ["--import", resolver], env: { … }, // bootstrap
|
||||
authorize: (principal, address) => app.mayReach(principal, address), // the route's (synchronous)
|
||||
principal: (address) => ({ kind: "worker", session: address.params.id }),
|
||||
start: (address, ctx) => app.startData(address, ctx), // JSON, before each start
|
||||
lock: (address) => `app:${dataDir}:worker:${address.key}`, // one live worker per address
|
||||
idleMs: () => app.idleMs(), // 5 min by default
|
||||
plugins,
|
||||
onOutput: (address, stream, chunk) => logs.append(address.key, stream, chunk),
|
||||
onError: (error) => log(error), // WorkersError
|
||||
}, ctx);
|
||||
workers.state // { [address key]: WorkerState }, a JSON watchable
|
||||
await workers.stop(Session({ id }), ctx); // suspended while held, then stopped
|
||||
await workers.retry(Session({ id }), ctx); // a failed or suspended address starts again (if held)
|
||||
await plugins.rollout([plugins.step("server", server), workers.step("workers")], built, ctx);
|
||||
await workers.close(ctx);
|
||||
|
||||
// worker.ts
|
||||
workerMain(async ({ address, start, parent, plugins, onClose }, ctx) => {
|
||||
const host = createHost({ address, principal: { kind: "worker" }, store, onError });
|
||||
onClose((ctx) => host.close(ctx)); // closers run in reverse order
|
||||
host.connect(parent, { principal: { kind: "server" }, trusted: true }, ctx); // first: `started` goes out now
|
||||
await plugins.load(host, { kind: "worker", core: [{ name: "@app/core", facet: core(start) }] }, ctx);
|
||||
return { busy: harness.activeTasks }; // extra keep-alive counts
|
||||
}, { hostPackages: { "@earendil-works/chord/next": chordNext }, redact: (error) => ({ message: redacted(error) }), onError });
|
||||
```
|
||||
|
||||
- **The route:** `createWorkers` calls `host.route(Kind, …)` itself and keeps its `RouteHandle`. A failed address's
|
||||
`resolve` rejects at once with its stored failure (its holders see `failed { waiting }`, nothing forks); the kernel
|
||||
reports that rejection (`resolve_failed`) once per new hold, so every tab opening a failed session adds one `onError`
|
||||
entry.
|
||||
- **Per address:** a child's events (`started`, `ready`, `idle`, `fatal`, its exit, its link closed by this side) are
|
||||
recorded on that child as they happen (a late event of an old child changes nothing); starts and stops run one after
|
||||
another. A start waits for the previous child's exit and rejects with a crash recorded meanwhile; a stop records its
|
||||
intent before `shutdown`, so the exit it causes is no crash.
|
||||
- **A start:** the app's `start(address, ctx)` (aborted by `close`), then the fork (`serialization: "advanced"`, a
|
||||
detached process group, the server's environment with `env` over it: a worker inherits every variable the server
|
||||
has, secrets included, unless the app starts the server with a trimmed one; stdout and stderr piped: the stderr tail
|
||||
kept, both mirrored to this process's own without `onOutput`), its IPC link made at once, the start message `{
|
||||
address, start, scripts, lock }` (`scripts`: the kind's current set, `plugins.scripts(kind)`). The child takes the
|
||||
lock (waiting for an old holder without polling), makes its parent link and sends `started` when the app's host
|
||||
connects; `resolve` returns the link then. The start timer (60 s) covers fork, lock and connect only. `ready` (after
|
||||
the app's function returned, with the idle state and the load report) makes the worker `running`.
|
||||
- **Failures stay** (R2): a crash (an exit nobody asked for), a `fatal`, a link closed by this side (the heartbeat, a
|
||||
protocol error: the child is stopped, grace 5 s, then killed) or a failed start leaves the address `failed`
|
||||
(`worker_failed` once, `worker_exited` for the exit) until `retry` or a rollout reaching workers.
|
||||
- **Keep-alive:** the child reports when its idleness flips: `host.activity`'s `references`, `jobs` and `lingering` and
|
||||
the app's `busy` (a JSON watchable of a record of numbers) all zero, no reload running; local calls send nothing.
|
||||
The server stops a worker idle since `idleMs()` (checked every second, from `ready`). A stop: `shutdown`; the child
|
||||
aborts its context, runs its closers in reverse order and exits (forced after 3 s); the server kills the tree after
|
||||
5 s.
|
||||
- **`state`:** per address key `{ status: "starting" | "running" | "stopping", pid, startedAt }` or `{ status:
|
||||
"failed", error: { message, code?, reason? }, at }`; no entry when stopped. The error is public (decision "worker
|
||||
errors in public"): a `fatal` as the child's `redact` made it, else a fixed message (`worker failed to start`,
|
||||
`worker exited (SIGSEGV)`, `worker exited (code 1)`); holders see the same. The full error and the stderr tail go
|
||||
only to `onError` (`worker_failed { address, error, stderrTail }`). Every app hook of a start (`start`, the fork,
|
||||
`lock`, `principal`) is inside that boundary; a throwing `authorize` refuses the hold (`forbidden`) and is reported
|
||||
`authorize_failed`, its error private too.
|
||||
- **`stop(address, ctx)`:** `RouteHandle.suspend`, then the stop: holders see `failed { waiting }` ("suspended") until
|
||||
their last release; a later hold starts it on demand; `retry` lifts it at once.
|
||||
- **`close(ctx)`:** starts waiting in `start(address)` are aborted, every child stopped, new starts reject
|
||||
`released/closed`; the host's close (the route's context) closes it too, and no exit after that is a failure.
|
||||
- **`step(name)`** (§4): clears the failures from before it and retries the held ones (listed `{ host, report }` with
|
||||
their load report once ready, or `{ host, error }`), then ships the kind's current set to every worker whose last
|
||||
set is older, in parallel, one reload outstanding per child: `{ host, report }`, `{ host, stopped: true }` (stopped
|
||||
cleanly meanwhile) or `{ host, error }` (a crash, a failed reload). The step's context bounds only its wait (an abort
|
||||
lists the waiting ones with the abort's error; a late reply still frees the child's slot).
|
||||
- **The child** (`workerMain`): reads the start message, takes the lock, makes its parent link (`started` on the app's
|
||||
`connect`), gives the app `plugins` (`WorkerPlugins.load(host, { kind, core })` over the shipped scripts, with the
|
||||
child's own `hostPackages`), applies `reload` messages once the app's function returned (none while shutting down).
|
||||
`shutdown`, the parent's disconnect, `SIGTERM` and `SIGINT` abort its context and run the closers registered so far,
|
||||
also during startup. An uncaught error or a throwing function sends `fatal` with `redact(error)` and exits 1.
|
||||
- `processLock(name, ctx)`: the lock (an abstract socket on Linux, a named pipe on Windows, a socket file elsewhere).
|
||||
|
||||
## Examples
|
||||
|
||||
`test/next/examples/`: `41-workers` (a worker started on the first hold, stopped when idle), `42-worker-crash` (a crash
|
||||
stays failed, Retry brings it back), `43-rollout` (an edit rolled out to the server and its workers).
|
||||
|
||||
## Inside the directory
|
||||
|
||||
`control.ts` (control messages and errors as they cross), `lock.ts` (process locks), `process.ts` (one forked worker
|
||||
as the server sees it), `workers.ts` (the component: the route, per-address records, idle stops, the rollout step,
|
||||
the state), `main.ts` (`workerMain`).
|
||||
121
packages/chord/src/next-node/workers/control.ts
Normal file
121
packages/chord/src/next-node/workers/control.ts
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
// The control channel between the server and a worker (slice 10 §3): JSON messages beside the link's `Uint8Array`
|
||||
// frames on one IPC channel (the IPC link skips them), and errors as they cross it (`{ message, code?, reason? }`,
|
||||
// never a stack or a cause).
|
||||
|
||||
import type { JsonValue } from "../../json.ts";
|
||||
import type { Address } from "../../next/contracts/index.ts";
|
||||
import { ChordError, type FailureCause, isChordError } from "../../next/data/index.ts";
|
||||
import { chordErrorFromJson } from "../../next/data/internal.ts";
|
||||
import type { ReloadReport } from "../../next/host/index.ts";
|
||||
import type { PluginScript } from "../plugins/runtime.ts";
|
||||
|
||||
/** An error as it crosses: its message, and a ChordError's code and reason. */
|
||||
export type CrossedError = { readonly message: string; readonly code?: string; readonly reason?: string };
|
||||
|
||||
/** A reload report as it crosses: its errors crossed. */
|
||||
export type CrossedReport = {
|
||||
readonly plugins: ReloadReport["plugins"];
|
||||
readonly failed: readonly { readonly plugin: string; readonly error: CrossedError }[];
|
||||
readonly objects: readonly { readonly object: string; readonly kind: string; readonly cause?: CrossedCause }[];
|
||||
};
|
||||
|
||||
/** A failure cause as it crosses. */
|
||||
type CrossedCause = { readonly kind: FailureCause["kind"]; readonly error: CrossedError };
|
||||
|
||||
/** What the server sends a worker. */
|
||||
export type ToWorker =
|
||||
| {
|
||||
readonly kind: "start";
|
||||
readonly address: Address;
|
||||
readonly start: JsonValue;
|
||||
readonly scripts: readonly PluginScript[];
|
||||
readonly lock: string;
|
||||
}
|
||||
| { readonly kind: "shutdown" }
|
||||
| { readonly kind: "reload"; readonly scripts: readonly PluginScript[] };
|
||||
|
||||
/** What a worker sends the server. */
|
||||
export type FromWorker =
|
||||
| { readonly kind: "started" }
|
||||
| { readonly kind: "ready"; readonly idle: boolean; readonly report: CrossedReport }
|
||||
| { readonly kind: "idle"; readonly idle: boolean }
|
||||
| { readonly kind: "fatal"; readonly error: CrossedError }
|
||||
| { readonly kind: "reloaded"; readonly report: CrossedReport }
|
||||
| { readonly kind: "reloaded"; readonly error: CrossedError };
|
||||
|
||||
/** `message` in its envelope, as it goes over the IPC channel. */
|
||||
export function envelope(message: ToWorker | FromWorker): { readonly [ENVELOPE]: ToWorker | FromWorker } {
|
||||
return { [ENVELOPE]: message };
|
||||
}
|
||||
|
||||
/** The envelope's key: a control message is `{ "chord.workers": message }`. */
|
||||
const ENVELOPE = "chord.workers";
|
||||
|
||||
/**
|
||||
* The control message in an IPC message, or `undefined` for anything else (a link frame, the app's own). A trust
|
||||
* boundary in name only: both ends are the app's processes, so only the envelope is checked.
|
||||
*/
|
||||
export function controlOf<M extends ToWorker | FromWorker>(message: unknown): M | undefined {
|
||||
if (typeof message !== "object" || message === null || !(ENVELOPE in message)) return undefined;
|
||||
// Sent by the other end of this channel through `envelope`.
|
||||
return (message as { readonly [ENVELOPE]: M })[ENVELOPE];
|
||||
}
|
||||
|
||||
/** `error` as it crosses. */
|
||||
export function crossed(error: unknown): CrossedError {
|
||||
if (isChordError(error)) {
|
||||
return {
|
||||
message: error.message,
|
||||
code: error.code,
|
||||
...(error.reason === undefined ? {} : { reason: error.reason }),
|
||||
};
|
||||
}
|
||||
return { message: error instanceof Error ? error.message : String(error) };
|
||||
}
|
||||
|
||||
/** A crossed error as an error again: a ChordError when its code is one, an `Error` otherwise. */
|
||||
export function uncrossed(error: CrossedError): Error {
|
||||
if (error.code === undefined) return new Error(error.message);
|
||||
try {
|
||||
return chordErrorFromJson({
|
||||
code: error.code,
|
||||
message: error.message,
|
||||
...(error.reason === undefined ? {} : { reason: error.reason }),
|
||||
});
|
||||
} catch {
|
||||
return new Error(error.message);
|
||||
}
|
||||
}
|
||||
|
||||
/** `report` as it crosses. */
|
||||
export function crossedReport(report: ReloadReport): CrossedReport {
|
||||
return {
|
||||
plugins: report.plugins,
|
||||
failed: report.failed.map(({ plugin, error }) => ({ plugin, error: crossed(error) })),
|
||||
objects: report.objects.map((each) =>
|
||||
each.kind === "parked"
|
||||
? {
|
||||
object: each.object,
|
||||
kind: each.kind,
|
||||
cause: { kind: each.cause.kind, error: crossed(each.cause.error) },
|
||||
}
|
||||
: each,
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/** A crossed report as a report again. */
|
||||
export function uncrossedReport(report: CrossedReport): ReloadReport {
|
||||
return {
|
||||
plugins: report.plugins,
|
||||
failed: report.failed.map(({ plugin, error }) => ({ plugin, error: uncrossed(error) })),
|
||||
objects: report.objects.map(({ object, kind, cause }) => {
|
||||
if (cause === undefined)
|
||||
return { object, kind: kind as Exclude<ReloadReport["objects"][number]["kind"], "parked"> };
|
||||
const error = uncrossed(cause.error);
|
||||
// A parked object's cause is a ChordError on the worker; one that crossed without a code is wrapped as one.
|
||||
const chord = isChordError(error) ? error : new ChordError({ code: "unavailable", message: error.message });
|
||||
return { object, kind: "parked", cause: { kind: cause.kind, error: chord } as FailureCause };
|
||||
}),
|
||||
};
|
||||
}
|
||||
20
packages/chord/src/next-node/workers/index.ts
Normal file
20
packages/chord/src/next-node/workers/index.ts
Normal file
|
|
@ -0,0 +1,20 @@
|
|||
// Chord v2's workers component (`@earendil-works/chord/next/workers`, slice 10 §3, §4): a forked worker process per
|
||||
// address of a host kind, started on demand, stopped when idle, never restarted by itself, its plugins shipped from the
|
||||
// server; `workerMain` is the worker's side. Node-only.
|
||||
|
||||
export type { CrossedError } from "./control.ts";
|
||||
export { type ProcessLock, processLock } from "./lock.ts";
|
||||
export {
|
||||
type WorkerMainOptions,
|
||||
type WorkerPlugins,
|
||||
type WorkerResult,
|
||||
type WorkerStart,
|
||||
workerMain,
|
||||
} from "./main.ts";
|
||||
export {
|
||||
createWorkers,
|
||||
type WorkerState,
|
||||
type Workers,
|
||||
type WorkersError,
|
||||
type WorkersOptions,
|
||||
} from "./workers.ts";
|
||||
142
packages/chord/src/next-node/workers/lock.ts
Normal file
142
packages/chord/src/next-node/workers/lock.ts
Normal file
|
|
@ -0,0 +1,142 @@
|
|||
// Process locks (slice 10 §3, Chord v1's `processLock`): one live worker per address. A lock is a listening socket the
|
||||
// kernel owns for the holder's lifetime (an abstract socket on Linux and Android, a named pipe on Windows, a socket file
|
||||
// elsewhere), so there is no race between two would-be holders; a socket file a crashed holder left fails closed (the
|
||||
// error names it: a takeover would race another taker). A child finding the lock held waits
|
||||
// without polling: it connects to the holder's socket, which the holder never answers, and tries again when that
|
||||
// connection closes (the holder exited).
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { unlinkSync } from "node:fs";
|
||||
import { lstat, mkdir } from "node:fs/promises";
|
||||
import { connect, createServer, type Server, type Socket } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import type { Context } from "../../context/index.ts";
|
||||
|
||||
/** A held lock: `release` frees it (the process's exit does too). */
|
||||
export type ProcessLock = { release(): Promise<void> };
|
||||
|
||||
/** Where a lock lives: an abstract socket (Linux, Android), a named pipe (Windows) or a socket file (elsewhere). */
|
||||
export type LockAddress = { readonly kind: "abstract" | "pipe" | "file"; readonly address: string };
|
||||
|
||||
/**
|
||||
* Takes the lock `name`, waiting while another process holds it (until `context` aborts: rejects with its reason).
|
||||
* Throws what listening throws besides "in use", and for a stale socket file (macOS, the BSDs) an error naming it.
|
||||
*/
|
||||
export async function processLock(name: string, context: Context): Promise<ProcessLock> {
|
||||
const key = createHash("sha256").update(name).digest("hex").slice(0, 32);
|
||||
if (process.platform === "win32")
|
||||
return lockAt({ kind: "pipe", address: `\\\\.\\pipe\\chord-lock-${key}` }, context);
|
||||
if (process.platform === "linux" || process.platform === "android") {
|
||||
return lockAt({ kind: "abstract", address: `\0chord-lock-${key}` }, context);
|
||||
}
|
||||
return lockAt({ kind: "file", address: await socketPath(`chord-lock-${key}.sock`) }, context);
|
||||
}
|
||||
|
||||
/**
|
||||
* Takes the lock at `lock`, waiting while it is held. A socket file nobody accepts on (a crashed holder's) fails closed:
|
||||
* taking it over races another taker (two could both unlink and listen), so the error names the file to remove.
|
||||
*/
|
||||
export async function lockAt(lock: LockAddress, context: Context): Promise<ProcessLock> {
|
||||
const { address } = lock;
|
||||
const file = lock.kind === "file";
|
||||
// Connections are kept open, never answered: a waiter learns of the release when its connection closes.
|
||||
const waiters = new Set<Socket>();
|
||||
const server = createServer((socket) => {
|
||||
waiters.add(socket);
|
||||
socket.on("error", () => {});
|
||||
socket.once("close", () => waiters.delete(socket));
|
||||
});
|
||||
for (;;) {
|
||||
context.abortSignal?.throwIfAborted();
|
||||
if (await listened(server, address)) break;
|
||||
const held = await waitWhileHeld(address, context);
|
||||
if (!held && file) {
|
||||
throw new Error(`lock: ${address} is a stale lock file (no process holds it); remove it if no worker runs`);
|
||||
}
|
||||
}
|
||||
const removeFile = (): void => {
|
||||
try {
|
||||
unlinkSync(address);
|
||||
} catch {
|
||||
// Gone already.
|
||||
}
|
||||
};
|
||||
if (file) process.once("exit", removeFile);
|
||||
return {
|
||||
release: () =>
|
||||
new Promise<void>((resolve, reject) => {
|
||||
if (file) process.removeListener("exit", removeFile);
|
||||
server.close((error) => (error ? reject(error) : resolve()));
|
||||
// Waiters see their connections close (and `close` waits for them).
|
||||
for (const socket of waiters) socket.destroy();
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** Whether `server` now listens on `address`; `false` when it is in use. */
|
||||
function listened(server: Server, address: string): Promise<boolean> {
|
||||
return new Promise<boolean>((resolve, reject) => {
|
||||
const failed = (error: NodeJS.ErrnoException): void => {
|
||||
if (error.code === "EADDRINUSE") resolve(false);
|
||||
else reject(error);
|
||||
};
|
||||
server.once("error", failed);
|
||||
server.listen(address, () => {
|
||||
server.removeListener("error", failed);
|
||||
resolve(true);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Connects to the holder of `address` and resolves when that connection closes or the address is gone (`true`: try
|
||||
* again), or with `false` when nobody accepts on it (a leftover socket file); `context`'s abort rejects.
|
||||
*/
|
||||
function waitWhileHeld(address: string, context: Context): Promise<boolean> {
|
||||
return new Promise<boolean>((resolve, reject) => {
|
||||
const signal = context.abortSignal;
|
||||
if (signal?.aborted) {
|
||||
reject(signal.reason);
|
||||
return;
|
||||
}
|
||||
const socket = connect(address);
|
||||
const abort = (): void => {
|
||||
socket.destroy();
|
||||
reject(signal?.reason);
|
||||
};
|
||||
signal?.addEventListener("abort", abort, { once: true });
|
||||
const settle = (held: boolean): void => {
|
||||
signal?.removeEventListener("abort", abort);
|
||||
resolve(held);
|
||||
};
|
||||
let connected = false;
|
||||
socket.once("connect", () => {
|
||||
connected = true;
|
||||
});
|
||||
socket.on("error", (error: NodeJS.ErrnoException) => {
|
||||
if (!connected && error.code === "ECONNREFUSED") settle(false);
|
||||
});
|
||||
socket.once("close", () => settle(true));
|
||||
});
|
||||
}
|
||||
|
||||
/** sun_path holds 104 bytes on macOS and the BSDs, the terminating NUL included. */
|
||||
const MAX_SOCKET_PATH_BYTES = 103;
|
||||
|
||||
/**
|
||||
* A lock socket's path: in the temporary directory, or, when that does not fit in sun_path (a deep TMPDIR), in a
|
||||
* private per-user directory under /tmp that only its owner can reach.
|
||||
*/
|
||||
async function socketPath(file: string): Promise<string> {
|
||||
const preferred = join(tmpdir(), file);
|
||||
if (Buffer.byteLength(preferred) <= MAX_SOCKET_PATH_BYTES) return preferred;
|
||||
const uid = process.getuid?.() ?? 0;
|
||||
const directory = `/tmp/chord-lock-${uid}`;
|
||||
await mkdir(directory, { mode: 0o700, recursive: true });
|
||||
const stats = await lstat(directory);
|
||||
if (!stats.isDirectory() || stats.uid !== uid || (stats.mode & 0o077) !== 0) {
|
||||
throw new Error(`lock directory ${directory} must be a directory only its owner can access`);
|
||||
}
|
||||
return join(directory, file);
|
||||
}
|
||||
293
packages/chord/src/next-node/workers/main.ts
Normal file
293
packages/chord/src/next-node/workers/main.ts
Normal file
|
|
@ -0,0 +1,293 @@
|
|||
// The worker's side (slice 10 §3): `workerMain` reads the start message, takes the address's lock, makes the link to
|
||||
// the server (`started` goes out when the app's host listens on it), runs the app's function with a plugin runtime over
|
||||
// the shipped scripts, reports `ready` and each idle flip, applies reloads, and on `shutdown`, the parent's disconnect,
|
||||
// `SIGTERM` or `SIGINT` aborts its context, runs the app's closers in reverse order and exits.
|
||||
|
||||
import { BACKGROUND_CONTEXT, type Context, withAbortSignal } from "../../context/index.ts";
|
||||
import type { JsonValue } from "../../json.ts";
|
||||
import type { Address } from "../../next/contracts/index.ts";
|
||||
import type { Principal } from "../../next/data/index.ts";
|
||||
import type { Host, LoadReport, PluginFacet, ReloadReport } from "../../next/host/index.ts";
|
||||
import { type IpcPort, ipcLink, type Link } from "../../next/link/index.ts";
|
||||
import type { Watchable } from "../../watch.ts";
|
||||
import type { HostPackages } from "../plugins/evaluate.ts";
|
||||
import { createPluginRuntime, type PluginScript } from "../plugins/runtime.ts";
|
||||
import {
|
||||
type CrossedError,
|
||||
controlOf,
|
||||
crossed,
|
||||
crossedReport,
|
||||
envelope,
|
||||
type FromWorker,
|
||||
type ToWorker,
|
||||
} from "./control.ts";
|
||||
import { processLock } from "./lock.ts";
|
||||
|
||||
/** What the app's function gets. */
|
||||
export type WorkerStart = {
|
||||
readonly address: Address;
|
||||
/** The start data the server's `start(address)` gave. */
|
||||
readonly start: JsonValue;
|
||||
/** The link to the server: `connect` it before anything slow (`started` goes out then). */
|
||||
readonly parent: Link;
|
||||
readonly plugins: WorkerPlugins;
|
||||
/** Registers a closer: closers run in reverse order at shutdown. */
|
||||
onClose(closer: (context: Context) => Promise<void> | void): void;
|
||||
};
|
||||
|
||||
/** The worker's plugins: the shipped scripts of its kind, loaded on its host; the server's reloads applied to it. */
|
||||
export interface WorkerPlugins {
|
||||
/** Loads `host` with the start message's scripts of `kind` and `core` (the app's own facets). */
|
||||
load<P extends Principal>(
|
||||
host: Host<P>,
|
||||
options: { readonly kind: string; readonly core: readonly PluginFacet<P>[] },
|
||||
context: Context,
|
||||
): Promise<LoadReport>;
|
||||
}
|
||||
|
||||
/** What the app's function returns: extra keep-alive counts (busy while any is nonzero). */
|
||||
export type WorkerResult = { readonly busy?: Watchable<Readonly<Record<string, number>>> };
|
||||
|
||||
/** What `workerMain` takes besides the app's function. */
|
||||
export type WorkerMainOptions = {
|
||||
/** The host packages by specifier: this process's own instances (functions cannot cross IPC). */
|
||||
readonly hostPackages: HostPackages;
|
||||
/** What crosses as `fatal` for an error (never the original message unless this returns it). */
|
||||
readonly redact: (error: unknown) => CrossedError;
|
||||
readonly onError: (error: unknown) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* Runs this process as a worker: `fn` once the start message arrived and the lock is taken. An uncaught error or a
|
||||
* throwing `fn` sends `fatal` (redacted) and exits.
|
||||
*/
|
||||
export function workerMain(
|
||||
fn: (start: WorkerStart, context: Context) => Promise<WorkerResult | undefined>,
|
||||
options: WorkerMainOptions,
|
||||
): void {
|
||||
// `process` is the IPC port of a forked child.
|
||||
new Worker(process as unknown as IpcPort, options).run(fn);
|
||||
}
|
||||
|
||||
/** How long a shutdown's closers get before the worker exits anyway: 3 s. */
|
||||
const FORCED_EXIT = 3_000;
|
||||
|
||||
class Worker {
|
||||
readonly #port: IpcPort;
|
||||
readonly #options: WorkerMainOptions;
|
||||
readonly #controller = new AbortController();
|
||||
readonly #context: Context;
|
||||
readonly #closers: ((context: Context) => Promise<void> | void)[] = [];
|
||||
readonly #start = Promise.withResolvers<Extract<ToWorker, { readonly kind: "start" }>>();
|
||||
/** Reloads that arrived, applied one after another once `fn` returned. */
|
||||
readonly #reloads: (readonly PluginScript[])[] = [];
|
||||
#loaded: { readonly host: Host<Principal>; readonly runtime: ReturnType<typeof createPluginRuntime> } | undefined;
|
||||
#ready = false;
|
||||
#reloading = false;
|
||||
#busy: Watchable<Readonly<Record<string, number>>> | undefined;
|
||||
/** The last idle state sent. */
|
||||
#idle: boolean | undefined;
|
||||
#stopping = false;
|
||||
|
||||
constructor(port: IpcPort, options: WorkerMainOptions) {
|
||||
this.#port = port;
|
||||
this.#options = options;
|
||||
this.#context = withAbortSignal(this.#controller.signal, BACKGROUND_CONTEXT);
|
||||
port.on("message", (message) => this.#message(controlOf<ToWorker>(message)));
|
||||
port.on("disconnect", () => void this.#shutdown());
|
||||
for (const signal of ["SIGTERM", "SIGINT"] as const) process.once(signal, () => void this.#shutdown());
|
||||
process.once("uncaughtException", (error) => this.#fatal(error));
|
||||
process.once("unhandledRejection", (error) => this.#fatal(error));
|
||||
}
|
||||
|
||||
/** Takes the lock, makes the parent link, runs `fn`, then reports `ready` and applies reloads. */
|
||||
run(fn: (start: WorkerStart, context: Context) => Promise<WorkerResult | undefined>): void {
|
||||
// A throw while shutting down is the shutdown's own cancellation (its `ctx` aborted): no fatal, the closers run.
|
||||
void this.#run(fn).catch((error: unknown) => {
|
||||
if (!this.#stopping) this.#fatal(error);
|
||||
});
|
||||
}
|
||||
|
||||
async #run(fn: (start: WorkerStart, context: Context) => Promise<WorkerResult | undefined>): Promise<void> {
|
||||
const message = await this.#start.promise;
|
||||
await processLock(message.lock, this.#context);
|
||||
const parent = new ParentLink(ipcLink(this.#port), () => this.#send({ kind: "started" }));
|
||||
const runtime = createPluginRuntime();
|
||||
const plugins: WorkerPlugins = {
|
||||
load: async (host, { kind, core }, context) => {
|
||||
this.#loaded = { host: host as unknown as Host<Principal>, runtime };
|
||||
return runtime.load(
|
||||
host,
|
||||
{ kind, core, scripts: message.scripts, modules: this.#options.hostPackages },
|
||||
context,
|
||||
);
|
||||
},
|
||||
};
|
||||
let report: LoadReport = { failed: [] };
|
||||
const loading: WorkerPlugins = {
|
||||
load: async (host, options, context) => {
|
||||
report = await plugins.load(host, options, context);
|
||||
return report;
|
||||
},
|
||||
};
|
||||
const start: WorkerStart = {
|
||||
address: message.address,
|
||||
start: message.start,
|
||||
parent,
|
||||
plugins: loading,
|
||||
onClose: (closer) => void this.#closers.push(closer),
|
||||
};
|
||||
const result = await fn(start, this.#context);
|
||||
if (this.#stopping) return;
|
||||
this.#busy = result?.busy;
|
||||
await this.#follow();
|
||||
this.#ready = true;
|
||||
const failed = new Set(report.failed.map(({ plugin }) => plugin));
|
||||
const added = message.scripts.map(({ plugin }) => plugin.name).filter((name) => !failed.has(name));
|
||||
const loaded: ReloadReport = { plugins: { added, changed: [], removed: [] }, failed: report.failed, objects: [] };
|
||||
this.#idle = this.#isIdle();
|
||||
this.#send({ kind: "ready", idle: this.#idle, report: crossedReport(loaded) });
|
||||
void this.#applyReloads();
|
||||
}
|
||||
|
||||
/** Follows the host's activity and the app's `busy`: each idle flip is reported. */
|
||||
async #follow(): Promise<void> {
|
||||
const sources: Watchable<unknown>[] = [];
|
||||
if (this.#loaded !== undefined) sources.push(this.#loaded.host.activity);
|
||||
if (this.#busy !== undefined) sources.push(this.#busy);
|
||||
for (const source of sources) {
|
||||
const watch = await source.watch(this.#context);
|
||||
watch.start(async () => this.#flip());
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether every count is zero: the host's references, jobs and lingering objects, the app's `busy`, no reload. */
|
||||
#isIdle(): boolean {
|
||||
if (this.#reloading) return false;
|
||||
const activity = this.#loaded?.host.activity.value;
|
||||
if (activity !== undefined && activity.references + activity.jobs + activity.lingering > 0) return false;
|
||||
return Object.values(this.#busy?.value ?? {}).every((count) => count === 0);
|
||||
}
|
||||
|
||||
/** Reports the idle state when it flipped (after `ready`). */
|
||||
#flip(): void {
|
||||
if (!this.#ready) return;
|
||||
const idle = this.#isIdle();
|
||||
if (idle === this.#idle) return;
|
||||
this.#idle = idle;
|
||||
this.#send({ kind: "idle", idle });
|
||||
}
|
||||
|
||||
/** A control message from the server. */
|
||||
#message(message: ToWorker | undefined): void {
|
||||
if (message === undefined) return;
|
||||
switch (message.kind) {
|
||||
case "start":
|
||||
this.#start.resolve(message);
|
||||
return;
|
||||
case "shutdown":
|
||||
void this.#shutdown();
|
||||
return;
|
||||
case "reload":
|
||||
// A worker shutting down applies no reload (its exit answers it).
|
||||
if (this.#stopping) return;
|
||||
this.#reloads.push(message.scripts);
|
||||
if (this.#ready) void this.#applyReloads();
|
||||
}
|
||||
}
|
||||
|
||||
/** Applies the reloads that arrived, one after another, each answered `reloaded`. */
|
||||
async #applyReloads(): Promise<void> {
|
||||
if (this.#reloading || this.#reloads.length === 0) return;
|
||||
const loaded = this.#loaded;
|
||||
this.#reloading = true;
|
||||
this.#flip();
|
||||
for (
|
||||
let scripts = this.#reloads.shift();
|
||||
scripts !== undefined && !this.#stopping;
|
||||
scripts = this.#reloads.shift()
|
||||
) {
|
||||
if (loaded === undefined) {
|
||||
this.#send({ kind: "reloaded", error: { message: "worker: plugins.load was never called" } });
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
const report = await loaded.runtime.reload(loaded.host, scripts, this.#context);
|
||||
this.#send({ kind: "reloaded", report: crossedReport(report) });
|
||||
} catch (error) {
|
||||
this.#send({ kind: "reloaded", error: crossed(error) });
|
||||
}
|
||||
}
|
||||
this.#reloading = false;
|
||||
this.#flip();
|
||||
}
|
||||
|
||||
/** Aborts the context, runs the closers in reverse order and exits (forced after 3 s). */
|
||||
async #shutdown(): Promise<void> {
|
||||
if (this.#stopping) return;
|
||||
this.#stopping = true;
|
||||
this.#controller.abort(new Error("worker: shutting down"));
|
||||
setTimeout(() => process.exit(0), FORCED_EXIT).unref();
|
||||
const context = BACKGROUND_CONTEXT;
|
||||
for (const closer of this.#closers.reverse()) {
|
||||
try {
|
||||
await closer(context);
|
||||
} catch (error) {
|
||||
this.#options.onError(error);
|
||||
}
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
/** Sends `fatal` with the redacted error, then exits. */
|
||||
#fatal(error: unknown): void {
|
||||
this.#options.onError(error);
|
||||
let redacted: CrossedError;
|
||||
try {
|
||||
redacted = this.#options.redact(error);
|
||||
} catch {
|
||||
redacted = { message: "worker: fatal error" };
|
||||
}
|
||||
if (!this.#port.connected) process.exit(1);
|
||||
// `process.send` takes a callback once the message is out.
|
||||
(process.send as (message: unknown, callback: () => void) => void)(
|
||||
envelope({ kind: "fatal", error: redacted }),
|
||||
() => process.exit(1),
|
||||
);
|
||||
}
|
||||
|
||||
/** Sends a control message; dropped when the server is gone. */
|
||||
#send(message: FromWorker): void {
|
||||
if (this.#port.connected) process.send?.(envelope(message), undefined, undefined, () => {});
|
||||
}
|
||||
}
|
||||
|
||||
/** The link to the server as the app gets it: `started` goes out when the app's host sets its listener (`connect`). */
|
||||
class ParentLink implements Link {
|
||||
readonly #inner: Link;
|
||||
#started: (() => void) | undefined;
|
||||
|
||||
constructor(inner: Link, started: () => void) {
|
||||
this.#inner = inner;
|
||||
this.#started = started;
|
||||
}
|
||||
|
||||
send(message: Uint8Array): void {
|
||||
this.#inner.send(message);
|
||||
}
|
||||
|
||||
onMessage(listener: (message: Uint8Array) => void): void {
|
||||
this.#inner.onMessage(listener);
|
||||
const started = this.#started;
|
||||
this.#started = undefined;
|
||||
started?.();
|
||||
}
|
||||
|
||||
close(): void {
|
||||
this.#inner.close();
|
||||
}
|
||||
|
||||
onClose(listener: () => void): void {
|
||||
this.#inner.onClose(listener);
|
||||
}
|
||||
}
|
||||
150
packages/chord/src/next-node/workers/process.ts
Normal file
150
packages/chord/src/next-node/workers/process.ts
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
// A worker process as the server sees it (slice 10 §3, Chord v1's `forkDriver`): forked with V8 serialization as a
|
||||
// detached process group (the whole tree can be killed), stdout and stderr piped (the stderr tail kept, both mirrored to
|
||||
// the server's own when the app takes no output), its IPC link made at the fork so the child's `hello` and early holds
|
||||
// are queued, never lost; control messages, the link's close and the exit are told as they happen.
|
||||
|
||||
import { type ChildProcess, fork } from "node:child_process";
|
||||
import type { Link } from "../../next/link/index.ts";
|
||||
import { ipcLink } from "../../next/link/index.ts";
|
||||
import { controlOf, envelope, type FromWorker, type ToWorker } from "./control.ts";
|
||||
|
||||
/** How a worker process is started: the bootstrap options of the workers component. */
|
||||
export type ForkOptions = {
|
||||
readonly entry: string;
|
||||
readonly execArgv?: readonly string[];
|
||||
readonly env?: Readonly<Record<string, string>>;
|
||||
/** Its output; without it, mirrored to this process's own. */
|
||||
readonly onOutput?: (stream: "stdout" | "stderr", chunk: Uint8Array) => void;
|
||||
};
|
||||
|
||||
/** How a worker process ended. */
|
||||
export type Exit = { readonly code: number | null; readonly signal: NodeJS.Signals | null };
|
||||
|
||||
/** What a worker process tells as it happens. */
|
||||
export type ProcessEvents = {
|
||||
readonly control: (message: FromWorker) => void;
|
||||
/** Its link was closed by this side (the heartbeat, a protocol error) while the child may live. */
|
||||
readonly linkLost: () => void;
|
||||
/** Its link was closed by the child's side (it exits, or it closed its IPC channel and may live on). */
|
||||
readonly linkClosed: () => void;
|
||||
readonly exited: (exit: Exit) => void;
|
||||
};
|
||||
|
||||
/** How much of a worker's stderr is kept for its failure: 4 KiB. */
|
||||
const STDERR_TAIL = 4096;
|
||||
|
||||
/** A control message to a child that is gone: dropped (its exit or link close tells). */
|
||||
const ignoreSendError = (): void => {};
|
||||
|
||||
/** One forked worker process. */
|
||||
export class WorkerProcess {
|
||||
readonly pid: number;
|
||||
readonly startedAt = Date.now();
|
||||
/** The link to the child's host, made at the fork. */
|
||||
readonly link: Link;
|
||||
/** Resolves with its exit; every process it spawned is killed then. */
|
||||
readonly exited: Promise<Exit>;
|
||||
readonly #child: ChildProcess;
|
||||
#stderr = "";
|
||||
|
||||
constructor(options: ForkOptions, events: ProcessEvents) {
|
||||
const child = fork(options.entry, [], {
|
||||
env: { ...process.env, ...options.env },
|
||||
execArgv: [...(options.execArgv ?? [])],
|
||||
detached: true,
|
||||
serialization: "advanced",
|
||||
stdio: ["ignore", "pipe", "pipe", "ipc"],
|
||||
});
|
||||
this.#child = child;
|
||||
this.pid = child.pid ?? -1;
|
||||
for (const stream of ["stdout", "stderr"] as const) {
|
||||
child[stream]?.on("data", (chunk: Buffer) => {
|
||||
if (stream === "stderr") this.#stderr = (this.#stderr + chunk.toString("utf8")).slice(-STDERR_TAIL);
|
||||
if (options.onOutput !== undefined) options.onOutput(stream, chunk);
|
||||
else process[stream].write(chunk);
|
||||
});
|
||||
}
|
||||
this.link = new TellingLink(ipcLink(child), events);
|
||||
child.on("message", (message: unknown) => {
|
||||
const control = controlOf<FromWorker>(message);
|
||||
if (control !== undefined) events.control(control);
|
||||
});
|
||||
this.exited = new Promise<Exit>((resolve) => {
|
||||
const ended = (exit: Exit): void => {
|
||||
// However the leader ended, nothing it spawned outlives it.
|
||||
this.#killGroup();
|
||||
resolve(exit);
|
||||
events.exited(exit);
|
||||
};
|
||||
child.once("exit", (code, signal) => ended({ code, signal }));
|
||||
// A fork that failed (no `exit` follows).
|
||||
child.once("error", () => {
|
||||
if (child.pid === undefined) ended({ code: null, signal: null });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/** The last bytes it wrote to stderr, if any. */
|
||||
get stderrTail(): string | undefined {
|
||||
return this.#stderr === "" ? undefined : this.#stderr;
|
||||
}
|
||||
|
||||
/** Sends a control message; dropped when the child is gone (its exit tells). */
|
||||
send(message: ToWorker): void {
|
||||
if (this.#child.connected) this.#child.send(envelope(message), ignoreSendError);
|
||||
}
|
||||
|
||||
/** Kills the child and every process it spawned. */
|
||||
kill(): void {
|
||||
if (!this.#killGroup()) this.#child.kill("SIGKILL");
|
||||
}
|
||||
|
||||
/** Kills the process group; `false` when there is none. */
|
||||
#killGroup(): boolean {
|
||||
try {
|
||||
process.kill(-this.pid, "SIGKILL");
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** A link that also tells who closed it (`linkLost`: this side, `linkClosed`: the child's), whoever listens to it. */
|
||||
class TellingLink implements Link {
|
||||
readonly #inner: Link;
|
||||
readonly #events: Pick<ProcessEvents, "linkLost" | "linkClosed">;
|
||||
#listener: (() => void) | undefined;
|
||||
#closed = false;
|
||||
|
||||
constructor(inner: Link, events: Pick<ProcessEvents, "linkLost" | "linkClosed">) {
|
||||
this.#inner = inner;
|
||||
this.#events = events;
|
||||
inner.onClose(() => {
|
||||
const here = this.#closed;
|
||||
this.#closed = true;
|
||||
this.#listener?.();
|
||||
if (!here) events.linkClosed();
|
||||
});
|
||||
}
|
||||
|
||||
send(message: Uint8Array): void {
|
||||
this.#inner.send(message);
|
||||
}
|
||||
|
||||
onMessage(listener: (message: Uint8Array) => void): void {
|
||||
this.#inner.onMessage(listener);
|
||||
}
|
||||
|
||||
close(): void {
|
||||
if (this.#closed) return;
|
||||
this.#closed = true;
|
||||
this.#events.linkLost();
|
||||
this.#inner.close();
|
||||
}
|
||||
|
||||
onClose(listener: () => void): void {
|
||||
if (this.#closed) listener();
|
||||
else this.#listener = listener;
|
||||
}
|
||||
}
|
||||
742
packages/chord/src/next-node/workers/workers.ts
Normal file
742
packages/chord/src/next-node/workers/workers.ts
Normal file
|
|
@ -0,0 +1,742 @@
|
|||
// The workers component (slice 10 §3, §4; decision "Slice 10's points"): a forked worker process per address of one
|
||||
// host kind, started on demand by the route it owns, stopped when idle, never restarted by itself (R2): a crash, a
|
||||
// `fatal`, a link lost while its child lives or a failed start leaves the address failed until `retry` or a rollout.
|
||||
// Per address, a child's events are recorded on that child as they happen; starts and stops run one after another.
|
||||
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import { awaitWithContext, BACKGROUND_CONTEXT, withAbortSignal } from "../../context/index.ts";
|
||||
import type { JsonValue } from "../../json.ts";
|
||||
import type { Address, HostKind, KindShape } from "../../next/contracts/index.ts";
|
||||
import type { Params } from "../../next/contracts/internal.ts";
|
||||
import { ChordError, type Principal } from "../../next/data/index.ts";
|
||||
import type { Host, RouteHandle } from "../../next/host/index.ts";
|
||||
import { Batch, type BatchError, Source } from "../../next/state/internal.ts";
|
||||
import type { Watchable } from "../../watch.ts";
|
||||
import type { Plugins, RolloutHost, RolloutStep } from "../plugins/plugins.ts";
|
||||
import type { PluginScript } from "../plugins/runtime.ts";
|
||||
import { type CrossedError, type CrossedReport, type FromWorker, uncrossed, uncrossedReport } from "./control.ts";
|
||||
import { type Exit, WorkerProcess } from "./process.ts";
|
||||
|
||||
/** What `createWorkers` takes. `kind`, `entry`, `execArgv` and `env` are bootstrap; the functions are called per use. */
|
||||
export type WorkersOptions<P extends Principal, K extends KindShape> = {
|
||||
/** The facet kind its workers load (`"worker"`), shipped by the plugins component. */
|
||||
readonly kind: string;
|
||||
/** The worker's entry module (it calls `workerMain`), its Node flags and its environment. */
|
||||
readonly entry: string;
|
||||
readonly execArgv?: readonly string[];
|
||||
readonly env?: Readonly<Record<string, string>>;
|
||||
/** The route's `authorize`: whether `principal` may reach `address` (synchronous). */
|
||||
readonly authorize: (principal: P, address: Address<K>) => boolean;
|
||||
/** Who the worker at `address` is (stamped on its link). */
|
||||
readonly principal: (address: Address<K>) => P;
|
||||
/** The worker's start data (JSON), before each start; abortable (`close`), a throw fails the start. */
|
||||
readonly start: (address: Address<K>, context: Context) => Promise<JsonValue> | JsonValue;
|
||||
/** The lock one live worker of `address` holds (across processes). */
|
||||
readonly lock: (address: Address<K>) => string;
|
||||
/** How long a worker idles before it is stopped: 5 minutes by default. */
|
||||
readonly idleMs?: () => number;
|
||||
readonly plugins: Plugins;
|
||||
/** A worker's output; without it, mirrored to this process's own. */
|
||||
readonly onOutput?: (address: Address<K>, stream: "stdout" | "stderr", chunk: Uint8Array) => void;
|
||||
readonly onError: (error: WorkersError) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* One address's worker (R5): running or on its way, or failed (until `retry` or a rollout) with a public error: a fixed
|
||||
* message (`worker failed to start`, `worker exited (SIGSEGV)`) or the child's redacted `fatal`.
|
||||
*/
|
||||
export type WorkerState =
|
||||
| { readonly status: "starting" | "running" | "stopping"; readonly pid: number; readonly startedAt: number }
|
||||
| { readonly status: "failed"; readonly error: CrossedError; readonly at: number };
|
||||
|
||||
/** What the component reports (and the state's watch listeners that threw). */
|
||||
export type WorkersError =
|
||||
| {
|
||||
readonly kind: "worker_failed";
|
||||
readonly address: string;
|
||||
/** The full error (a start's throw, a crash, the child's `fatal`): for the app's logs, never public. */
|
||||
readonly error: unknown;
|
||||
readonly stderrTail?: string;
|
||||
}
|
||||
| {
|
||||
/** The app's `authorize` threw: the hold is refused (`forbidden`), the error stays private. */
|
||||
readonly kind: "authorize_failed";
|
||||
readonly address: string;
|
||||
readonly error: unknown;
|
||||
}
|
||||
| {
|
||||
readonly kind: "worker_exited";
|
||||
readonly address: string;
|
||||
readonly code: number | null;
|
||||
readonly signal: string | null;
|
||||
}
|
||||
| BatchError;
|
||||
|
||||
/** The workers component. */
|
||||
export interface Workers<K extends KindShape> {
|
||||
/** Per address key its `WorkerState`; no entry when stopped or never started. A JSON watchable. */
|
||||
readonly state: Watchable<Readonly<Record<string, WorkerState>>>;
|
||||
/** Suspends `address` while it has holders (`RouteHandle.suspend`), then stops its worker. */
|
||||
stop(address: Address<K>, context: Context): Promise<void>;
|
||||
/** Clears `address`'s failure and suspension: it starts again now if held (`RouteHandle.retry`). */
|
||||
retry(address: Address<K>, context: Context): Promise<void>;
|
||||
/** A rollout step shipping the kind's current set to every worker, one result per worker (§4). */
|
||||
step(name: string): RolloutStep;
|
||||
/** Cancels starts, stops every worker; new starts reject `released/closed`. */
|
||||
close(context: Context): Promise<void>;
|
||||
}
|
||||
|
||||
/** Creates the workers component on `host`: it routes `Kind` itself (`host.route`, before `load`). */
|
||||
export function createWorkers<P extends Principal, N extends string, Q extends Params>(
|
||||
host: Host<P>,
|
||||
Kind: HostKind<N, Q>,
|
||||
options: WorkersOptions<P, HostKind<N, Q>>,
|
||||
context: Context,
|
||||
): Workers<HostKind<N, Q>> {
|
||||
// Its addresses are `Address<HostKind<N, Q>>`, as the route hands them over.
|
||||
const component = new WorkersComponent(options as unknown as WorkersOptions<Principal, HostKind>, context);
|
||||
const route = host.route(
|
||||
Kind,
|
||||
{
|
||||
authorize: (principal, address) => {
|
||||
// A throw would reach the holder as the hold's failure: the app's error stays private.
|
||||
try {
|
||||
return options.authorize(principal, address);
|
||||
} catch (error) {
|
||||
options.onError({ kind: "authorize_failed", address: address.key, error });
|
||||
return false;
|
||||
}
|
||||
},
|
||||
// The principal is `options.principal`'s, a `P`.
|
||||
resolve: (address, routing) =>
|
||||
component.resolve(address, routing) as Promise<{ link: Link; principal: P; trusted: true }>,
|
||||
},
|
||||
context,
|
||||
);
|
||||
component.route = route as unknown as RouteHandle<HostKind>;
|
||||
return component as unknown as Workers<HostKind<N, Q>>;
|
||||
}
|
||||
|
||||
/**
|
||||
* A failure an address keeps until `retry` or a rollout: what the public sees (a fixed message, or the child's redacted
|
||||
* `fatal`; decision "Slice 10 fix round, worker errors in public") and when.
|
||||
*/
|
||||
type Failure = { readonly error: CrossedError; readonly at: number };
|
||||
|
||||
/**
|
||||
* Why a child fails when it exits, recorded as it happens: a start that failed, a `fatal`, a lost link. `error` is the
|
||||
* full error (`onError` only); `public` what holders and `state` get (a fixed one from the exit when absent).
|
||||
*/
|
||||
type Cause = {
|
||||
readonly error: unknown;
|
||||
readonly public?: CrossedError;
|
||||
/** The component ended the child itself (a failed start): its exit is no `worker_exited`. */
|
||||
readonly killed?: true;
|
||||
};
|
||||
|
||||
/** How a child's reloads end once it is gone: stopped cleanly (a stop's exit) or with an error. */
|
||||
type Ended = { readonly kind: "stopped" } | { readonly kind: "failed"; readonly error: unknown };
|
||||
|
||||
/** One worker process of an address, with what its events recorded. */
|
||||
type Child = {
|
||||
readonly process: WorkerProcess;
|
||||
status: "starting" | "running" | "stopping";
|
||||
/** A stop was asked for: its exit is no crash. */
|
||||
stopping: boolean;
|
||||
cause: Cause | undefined;
|
||||
/** `started`: the app's host listens on its link. */
|
||||
readonly started: PromiseWithResolvers<void>;
|
||||
/** `ready`: its load report; rejected when it exits before. */
|
||||
readonly ready: PromiseWithResolvers<CrossedReport>;
|
||||
/** Since when it is idle (from `ready`); `undefined` while busy or before `ready`. */
|
||||
idleSince: number | undefined;
|
||||
/** An idle stop is queued (checked again when it runs). */
|
||||
idleStop: boolean;
|
||||
/** The set its start message or its last reload that applied without failures carried. */
|
||||
shipped: string;
|
||||
/** Its reloads, one outstanding at a time; settles to nothing (a report kept as its value would stay alive). */
|
||||
reloading: Promise<void>;
|
||||
/** The waiter of its outstanding reload. */
|
||||
reloaded: ((reply: Extract<FromWorker, { readonly kind: "reloaded" }>) => void) | undefined;
|
||||
/** Set once its link is gone or it exited: every reload queued behind settles with it. */
|
||||
ended: Ended | undefined;
|
||||
/** The grace timer of a stop or of a lost link. */
|
||||
grace: ReturnType<typeof setTimeout> | undefined;
|
||||
};
|
||||
|
||||
/** One address: its operations in a queue, its child, its failure. Kept while it has any of them. */
|
||||
type Routed = {
|
||||
readonly address: Address<HostKind>;
|
||||
queue: Promise<void>;
|
||||
pending: number;
|
||||
child: Child | undefined;
|
||||
failure: Failure | undefined;
|
||||
/** A start was queued and has not settled. */
|
||||
starting: Promise<unknown> | undefined;
|
||||
};
|
||||
|
||||
/** Starts a worker may take until `started` (fork, lock, connect): 60 s. */
|
||||
const START_TIMEOUT = 60_000;
|
||||
|
||||
/** How long a stopping worker (or one whose link is gone) gets before its tree is killed: 5 s. */
|
||||
const GRACE = 5_000;
|
||||
|
||||
/** How often idle workers are looked for: every second. */
|
||||
const IDLE_CHECK = 1_000;
|
||||
|
||||
/** How long a worker idles before it is stopped unless `idleMs` says otherwise: 5 minutes. */
|
||||
const IDLE_MS = 5 * 60_000;
|
||||
|
||||
/** The public error of a start that failed. */
|
||||
const START_FAILED: CrossedError = { message: "worker failed to start", code: "unavailable" };
|
||||
|
||||
/** The link type a route's `resolve` returns. */
|
||||
type Link = WorkerProcess["link"];
|
||||
|
||||
class WorkersComponent implements Workers<HostKind> {
|
||||
/** Set by `createWorkers` once the route exists. */
|
||||
route!: RouteHandle<HostKind>;
|
||||
readonly state: Watchable<Readonly<Record<string, WorkerState>>>;
|
||||
readonly #state: WorkersState;
|
||||
readonly #options: WorkersOptions<Principal, HostKind>;
|
||||
readonly #addresses = new Map<string, Routed>();
|
||||
/** Aborted at `close`: starts waiting in the app's `start` or for `started` end. */
|
||||
readonly #closing = new AbortController();
|
||||
/** `#closing`'s signal as a context: what a start's waits are bounded by. */
|
||||
readonly #closingContext = withAbortSignal(this.#closing.signal, BACKGROUND_CONTEXT);
|
||||
/** Whether the route's context closes it already (its first `resolve` attached that). */
|
||||
#routing = false;
|
||||
#idleTimer: ReturnType<typeof setInterval> | undefined;
|
||||
|
||||
constructor(options: WorkersOptions<Principal, HostKind>, context: Context) {
|
||||
this.#options = options;
|
||||
this.#state = new WorkersState(this.#addresses, new Batch((error) => options.onError(error), undefined));
|
||||
// Its value is a `Record<string, WorkerState>`.
|
||||
this.state = this.#state as unknown as Watchable<Readonly<Record<string, WorkerState>>>;
|
||||
this.#closeWith(context);
|
||||
}
|
||||
|
||||
/**
|
||||
* The route's `resolve`: a failed address rejects at once with its (public) failure, nothing forks; otherwise a
|
||||
* start, after the previous child's exit.
|
||||
*/
|
||||
resolve(address: Address<HostKind>, routing: Context): Promise<{ link: Link; principal: Principal; trusted: true }> {
|
||||
if (!this.#routing) {
|
||||
this.#routing = true;
|
||||
this.#closeWith(routing);
|
||||
}
|
||||
const routed = this.#routed(address);
|
||||
if (routed.failure !== undefined) return Promise.reject(uncrossed(routed.failure.error));
|
||||
if (this.#closing.signal.aborted) return Promise.reject(closed());
|
||||
const started = this.#enqueue(routed, () => this.#start(routed, routing));
|
||||
routed.starting = started;
|
||||
void started.then(
|
||||
() => this.#settled(routed, started),
|
||||
() => this.#settled(routed, started),
|
||||
);
|
||||
return started;
|
||||
}
|
||||
|
||||
async stop(address: Address<HostKind>, context: Context): Promise<void> {
|
||||
this.route.suspend(address);
|
||||
const routed = this.#addresses.get(address.key);
|
||||
if (routed === undefined) return;
|
||||
await awaitWithContext(
|
||||
this.#enqueue(routed, () => this.#stop(routed)),
|
||||
context,
|
||||
);
|
||||
}
|
||||
|
||||
async retry(address: Address<HostKind>, _context: Context): Promise<void> {
|
||||
const routed = this.#addresses.get(address.key);
|
||||
if (routed !== undefined) {
|
||||
routed.failure = undefined;
|
||||
this.#state.refresh();
|
||||
this.#forget(routed);
|
||||
}
|
||||
this.route.retry(address);
|
||||
}
|
||||
|
||||
step(name: string): RolloutStep {
|
||||
return { name, kinds: [this.#options.kind], reload: (_built, context) => this.#rollout(context) };
|
||||
}
|
||||
|
||||
async close(context: Context): Promise<void> {
|
||||
if (!this.#closing.signal.aborted) this.#closing.abort(closed());
|
||||
clearInterval(this.#idleTimer);
|
||||
this.#idleTimer = undefined;
|
||||
const stops = [...this.#addresses.values()].map((routed) => this.#enqueue(routed, () => this.#stop(routed)));
|
||||
await awaitWithContext(Promise.all(stops), context);
|
||||
}
|
||||
|
||||
/** `context`'s abort closes the component (the route's is aborted when the host closes). */
|
||||
#closeWith(context: Context): void {
|
||||
const signal = context.abortSignal;
|
||||
if (signal === undefined || signal.aborted) return;
|
||||
signal.addEventListener("abort", () => void this.close(BACKGROUND_CONTEXT).catch(() => {}), { once: true });
|
||||
}
|
||||
|
||||
/** The record of `address`, made by its first use. */
|
||||
#routed(address: Address<HostKind>): Routed {
|
||||
let routed = this.#addresses.get(address.key);
|
||||
if (routed === undefined) {
|
||||
routed = {
|
||||
address,
|
||||
queue: Promise.resolve(),
|
||||
pending: 0,
|
||||
child: undefined,
|
||||
failure: undefined,
|
||||
starting: undefined,
|
||||
};
|
||||
this.#addresses.set(address.key, routed);
|
||||
}
|
||||
return routed;
|
||||
}
|
||||
|
||||
/** Runs `operation` after the address's earlier ones. */
|
||||
#enqueue<T>(routed: Routed, operation: () => Promise<T>): Promise<T> {
|
||||
routed.pending++;
|
||||
const run = routed.queue.then(operation);
|
||||
routed.queue = run.then(
|
||||
() => this.#done(routed),
|
||||
() => this.#done(routed),
|
||||
);
|
||||
return run;
|
||||
}
|
||||
|
||||
/** An operation ended. */
|
||||
#done(routed: Routed): void {
|
||||
routed.pending--;
|
||||
this.#forget(routed);
|
||||
}
|
||||
|
||||
/** A start settled: no longer starting. */
|
||||
#settled(routed: Routed, started: Promise<unknown>): void {
|
||||
if (routed.starting === started) routed.starting = undefined;
|
||||
}
|
||||
|
||||
/** Forgets an address with no operation, no child and no failure. */
|
||||
#forget(routed: Routed): void {
|
||||
if (routed.pending === 0 && routed.child === undefined && routed.failure === undefined) {
|
||||
this.#addresses.delete(routed.address.key);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A start: the app's start data, the fork, the start message with the kind's current set, then `started` within the
|
||||
* start timer (fork, lock and connect only). Its failure fails the address; `close` ends it at any of its waits (an
|
||||
* app's `start` that ignores its signal too), a forked child is stopped.
|
||||
*/
|
||||
async #start(routed: Routed, routing: Context): Promise<{ link: Link; principal: Principal; trusted: true }> {
|
||||
const { address } = routed;
|
||||
// After the previous child's exit (its link may close first): a crash recorded meanwhile rejects this start.
|
||||
await routed.child?.process.exited;
|
||||
const crashed = routed.failure;
|
||||
if (crashed !== undefined) throw uncrossed(crashed.error);
|
||||
if (this.#closing.signal.aborted) throw closed();
|
||||
const context = withAbortSignal(this.#closing.signal, routing);
|
||||
let data: JsonValue;
|
||||
try {
|
||||
data = await awaitWithContext(Promise.resolve(this.#options.start(address, context)), this.#closingContext);
|
||||
} catch (error) {
|
||||
if (this.#closing.signal.aborted) throw closed();
|
||||
throw this.#fail(routed, { error, public: START_FAILED }, undefined);
|
||||
}
|
||||
if (this.#closing.signal.aborted) throw closed();
|
||||
let child: Child;
|
||||
try {
|
||||
child = this.#fork(routed);
|
||||
} catch (error) {
|
||||
throw this.#fail(routed, { error, public: START_FAILED }, undefined);
|
||||
}
|
||||
const timer = setTimeout(() => {
|
||||
const error = new Error(`worker ${address.key}: did not start within ${START_TIMEOUT / 1000} s`);
|
||||
child.cause ??= { error, killed: true };
|
||||
child.process.kill();
|
||||
}, START_TIMEOUT);
|
||||
// Every app hook of the start runs in here: what one throws stays private (`worker_failed`), its holders and
|
||||
// `state` get the fixed public message (Astra's recheck R3: `principal` threw through to a guest).
|
||||
try {
|
||||
const scripts = await this.#options.plugins.scripts(this.#options.kind, context);
|
||||
child.shipped = keyOf(scripts);
|
||||
const lock = this.#options.lock(address);
|
||||
child.process.send({ kind: "start", address, start: data, scripts, lock });
|
||||
await awaitWithContext(child.started.promise, this.#closingContext);
|
||||
return { link: child.process.link, principal: this.#options.principal(address), trusted: true };
|
||||
} catch (error) {
|
||||
// Closing is a stop (no failure); anything else fails the start.
|
||||
if (this.#closing.signal.aborted) child.stopping = true;
|
||||
else child.cause ??= { error, public: START_FAILED, killed: true };
|
||||
child.process.kill();
|
||||
await child.process.exited;
|
||||
if (this.#closing.signal.aborted) throw closed();
|
||||
throw uncrossed(routed.failure?.error ?? START_FAILED);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/** Forks the address's worker and records its events on it as they happen. */
|
||||
#fork(routed: Routed): Child {
|
||||
const { address } = routed;
|
||||
const onOutput = this.#options.onOutput;
|
||||
let child!: Child;
|
||||
const process = new WorkerProcess(
|
||||
{
|
||||
entry: this.#options.entry,
|
||||
...(this.#options.execArgv === undefined ? {} : { execArgv: this.#options.execArgv }),
|
||||
...(this.#options.env === undefined ? {} : { env: this.#options.env }),
|
||||
...(onOutput === undefined ? {} : { onOutput: (stream, chunk) => onOutput(address, stream, chunk) }),
|
||||
},
|
||||
{
|
||||
control: (message) => this.#control(routed, child, message),
|
||||
linkLost: () => this.#linkLost(child),
|
||||
linkClosed: () => this.#linkClosed(child),
|
||||
exited: (exit) => this.#exited(routed, child, exit),
|
||||
},
|
||||
);
|
||||
child = {
|
||||
process,
|
||||
status: "starting",
|
||||
stopping: false,
|
||||
cause: undefined,
|
||||
started: Promise.withResolvers(),
|
||||
ready: Promise.withResolvers(),
|
||||
idleSince: undefined,
|
||||
idleStop: false,
|
||||
shipped: "",
|
||||
reloading: Promise.resolve(),
|
||||
reloaded: undefined,
|
||||
ended: undefined,
|
||||
grace: undefined,
|
||||
};
|
||||
// Nobody may wait for them.
|
||||
child.started.promise.catch(() => {});
|
||||
child.ready.promise.catch(() => {});
|
||||
routed.child = child;
|
||||
this.#state.refresh();
|
||||
this.#watchIdle();
|
||||
return child;
|
||||
}
|
||||
|
||||
/** A control message of `child`. */
|
||||
#control(routed: Routed, child: Child, message: FromWorker): void {
|
||||
switch (message.kind) {
|
||||
case "started":
|
||||
child.started.resolve();
|
||||
return;
|
||||
case "ready":
|
||||
child.status = "running";
|
||||
child.idleSince = message.idle ? Date.now() : undefined;
|
||||
child.ready.resolve(message.report);
|
||||
if (routed.child === child) this.#state.refresh();
|
||||
return;
|
||||
case "idle":
|
||||
child.idleSince = message.idle ? (child.idleSince ?? Date.now()) : undefined;
|
||||
return;
|
||||
case "fatal":
|
||||
// Redacted by the child's `redact`: public as it is.
|
||||
child.cause ??= { error: uncrossed(message.error), public: message.error };
|
||||
child.started.reject(uncrossed(message.error));
|
||||
child.process.kill();
|
||||
return;
|
||||
case "reloaded":
|
||||
child.reloaded?.(message);
|
||||
}
|
||||
}
|
||||
|
||||
/** This side closed `child`'s link (the heartbeat, a protocol error): its child is stopped, its exit a crash. */
|
||||
#linkLost(child: Child): void {
|
||||
this.#ended(child);
|
||||
if (child.stopping) return;
|
||||
child.cause ??= { error: new Error("worker: link lost") };
|
||||
this.#shutdown(child);
|
||||
this.#state.refresh();
|
||||
}
|
||||
|
||||
/**
|
||||
* The child's side closed its link (it is exiting, or it closed its IPC and lives on): its reloads end; one that does
|
||||
* not exit within the grace is killed, its exit a crash (link lost).
|
||||
*/
|
||||
#linkClosed(child: Child): void {
|
||||
this.#ended(child);
|
||||
if (child.stopping || child.grace !== undefined) return;
|
||||
child.grace = setTimeout(() => {
|
||||
child.cause ??= { error: new Error("worker: link lost") };
|
||||
child.process.kill();
|
||||
}, GRACE);
|
||||
}
|
||||
|
||||
/**
|
||||
* `child` exited: a stop's exit forgets it; any other is a crash that fails the address (`worker_exited` and
|
||||
* `worker_failed` once, the full error and stderr tail for the app's logs only), except once the component closes.
|
||||
*/
|
||||
#exited(routed: Routed, child: Child, exit: Exit): void {
|
||||
clearTimeout(child.grace);
|
||||
const clean = child.stopping && child.cause === undefined;
|
||||
const cause: Cause = child.cause ?? { error: exitError(routed.address, exit, child) };
|
||||
const error = cause.public ?? publicExit(exit, child);
|
||||
child.started.reject(uncrossed(error));
|
||||
child.ready.reject(uncrossed(error));
|
||||
this.#ended(child);
|
||||
if (routed.child !== child) return;
|
||||
routed.child = undefined;
|
||||
if (!clean && !this.#closing.signal.aborted) {
|
||||
const { code, signal } = exit;
|
||||
// A failed start's kill is the component's own act (Opus's second recheck): only `worker_failed`.
|
||||
if (cause.killed !== true) {
|
||||
this.#options.onError({ kind: "worker_exited", address: routed.address.key, code, signal });
|
||||
}
|
||||
this.#fail(routed, { ...cause, public: error }, child.process.stderrTail);
|
||||
}
|
||||
this.#state.refresh();
|
||||
this.#forget(routed);
|
||||
}
|
||||
|
||||
/**
|
||||
* `child`'s link is gone or it exited (the first of these): its outstanding reload and every one queued behind it
|
||||
* settle, `stopped` when a stop asked for it, else failed.
|
||||
*/
|
||||
#ended(child: Child): void {
|
||||
if (child.ended !== undefined) return;
|
||||
child.ended =
|
||||
child.stopping && child.cause === undefined
|
||||
? { kind: "stopped" }
|
||||
: { kind: "failed", error: new ChordError({ code: "unavailable", message: "worker: its link is gone" }) };
|
||||
child.reloaded?.({ kind: "reloaded", error: { message: "worker: its link is gone" } });
|
||||
}
|
||||
|
||||
/**
|
||||
* Fails `routed` with `cause` (`worker_failed` once, with the full error and `stderrTail`); returns the public error
|
||||
* a start rejects with.
|
||||
*/
|
||||
#fail(routed: Routed, cause: Cause & { readonly public: CrossedError }, stderrTail: string | undefined): Error {
|
||||
routed.failure = { error: cause.public, at: Date.now() };
|
||||
this.#options.onError({
|
||||
kind: "worker_failed",
|
||||
address: routed.address.key,
|
||||
error: cause.error,
|
||||
...(stderrTail === undefined ? {} : { stderrTail }),
|
||||
});
|
||||
this.#state.refresh();
|
||||
return uncrossed(cause.public);
|
||||
}
|
||||
|
||||
/** A stop: `shutdown`, the grace, then the tree killed; resolves at its exit. */
|
||||
async #stop(routed: Routed): Promise<void> {
|
||||
const child = routed.child;
|
||||
if (child === undefined) return;
|
||||
child.stopping = true;
|
||||
this.#shutdown(child);
|
||||
this.#state.refresh();
|
||||
await child.process.exited;
|
||||
}
|
||||
|
||||
/** Asks `child` to shut down and kills its tree after the grace. */
|
||||
#shutdown(child: Child): void {
|
||||
child.status = "stopping";
|
||||
child.process.send({ kind: "shutdown" });
|
||||
clearTimeout(child.grace);
|
||||
child.grace = setTimeout(() => child.process.kill(), GRACE);
|
||||
}
|
||||
|
||||
/** Starts the idle check (every second) while any worker exists. */
|
||||
#watchIdle(): void {
|
||||
if (this.#idleTimer !== undefined) return;
|
||||
this.#idleTimer = setInterval(() => this.#checkIdle(), IDLE_CHECK);
|
||||
this.#idleTimer.unref();
|
||||
}
|
||||
|
||||
/** Queues a stop for every worker idle since `idleMs()`; stops checking once no worker is left. */
|
||||
#checkIdle(): void {
|
||||
const idleMs = this.#idleMs();
|
||||
const now = Date.now();
|
||||
let workers = 0;
|
||||
for (const routed of this.#addresses.values()) {
|
||||
const child = routed.child;
|
||||
if (child === undefined) continue;
|
||||
workers++;
|
||||
if (child.idleStop || child.status !== "running") continue;
|
||||
if (child.idleSince === undefined || now - child.idleSince < idleMs) continue;
|
||||
child.idleStop = true;
|
||||
void this.#enqueue(routed, () => this.#idleStop(routed, child));
|
||||
}
|
||||
if (workers === 0) {
|
||||
clearInterval(this.#idleTimer);
|
||||
this.#idleTimer = undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/** An idle stop when it runs: only if `child` is still the address's and still idle (a hold may have come since). */
|
||||
async #idleStop(routed: Routed, child: Child): Promise<void> {
|
||||
child.idleStop = false;
|
||||
if (routed.child !== child || child.idleSince === undefined) return;
|
||||
if (Date.now() - child.idleSince < this.#idleMs()) return;
|
||||
await this.#stop(routed);
|
||||
}
|
||||
|
||||
/** How long a worker idles before it is stopped. */
|
||||
#idleMs(): number {
|
||||
return this.#options.idleMs?.() ?? IDLE_MS;
|
||||
}
|
||||
|
||||
/**
|
||||
* The rollout step (§4): the set as this rollout applied it, read now (before another rollout changes it); the
|
||||
* failures recorded before it cleared and the held ones retried (listed once ready or failed); the set shipped to
|
||||
* every worker whose last applied set differs, in parallel. `context`'s abort ends only this wait: the entries
|
||||
* still waiting are listed with its error.
|
||||
*/
|
||||
async #rollout(context: Context): Promise<{ readonly hosts: readonly RolloutHost[] }> {
|
||||
const { kind, plugins } = this.#options;
|
||||
const scripts = plugins.current(kind) ?? (await plugins.scripts(kind, context));
|
||||
const key = keyOf(scripts);
|
||||
const entries: { readonly host: string; readonly entry: Promise<RolloutHost> }[] = [];
|
||||
for (const routed of [...this.#addresses.values()]) {
|
||||
if (routed.failure === undefined) continue;
|
||||
routed.failure = undefined;
|
||||
this.#state.refresh();
|
||||
this.route.retry(routed.address);
|
||||
const started = routed.starting;
|
||||
if (started === undefined) this.#forget(routed);
|
||||
else entries.push({ host: routed.address.key, entry: this.#retried(routed, started) });
|
||||
}
|
||||
for (const routed of this.#addresses.values()) {
|
||||
const child = routed.child;
|
||||
if (child === undefined || child.status === "stopping" || child.shipped === key) continue;
|
||||
entries.push({ host: routed.address.key, entry: this.#reload(routed.address.key, child, scripts, key) });
|
||||
}
|
||||
return { hosts: await Promise.all(entries.map(({ host, entry }) => bounded(host, entry, context))) };
|
||||
}
|
||||
|
||||
/** A retried address's entry: its start's load report once ready, or its error. */
|
||||
async #retried(routed: Routed, started: Promise<unknown>): Promise<RolloutHost> {
|
||||
const host = routed.address.key;
|
||||
try {
|
||||
await started;
|
||||
const child = routed.child;
|
||||
if (child === undefined) throw new Error("worker: exited");
|
||||
return { host, report: uncrossedReport(await child.ready.promise) };
|
||||
} catch (error) {
|
||||
return { host, error };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Ships `scripts` to `child` after its outstanding reload; settles on `reloaded`, its link's loss or its exit. Only a
|
||||
* reload that applied without failures marks the set shipped (a failed plugin is shipped again next time).
|
||||
*/
|
||||
#reload(host: string, child: Child, scripts: readonly PluginScript[], key: string): Promise<RolloutHost> {
|
||||
const reload = child.reloading.then(
|
||||
() =>
|
||||
new Promise<RolloutHost>((resolve) => {
|
||||
const settle = (ended: Ended): void => {
|
||||
resolve(ended.kind === "stopped" ? { host, stopped: true } : { host, error: ended.error });
|
||||
};
|
||||
if (child.ended !== undefined) {
|
||||
settle(child.ended);
|
||||
return;
|
||||
}
|
||||
child.reloaded = (reply) => {
|
||||
child.reloaded = undefined;
|
||||
if ("report" in reply) {
|
||||
if (reply.report.failed.length === 0) child.shipped = key;
|
||||
resolve({ host, report: uncrossedReport(reply.report) });
|
||||
} else if (child.ended !== undefined) {
|
||||
settle(child.ended);
|
||||
} else {
|
||||
resolve({ host, error: uncrossed(reply.error) });
|
||||
}
|
||||
};
|
||||
child.process.send({ kind: "reload", scripts });
|
||||
}),
|
||||
);
|
||||
child.reloading = reload.then(
|
||||
() => {},
|
||||
() => {},
|
||||
);
|
||||
return reload;
|
||||
}
|
||||
}
|
||||
|
||||
/** What new starts reject with once the component closes. */
|
||||
function closed(): ChordError {
|
||||
return new ChordError({ code: "released", reason: "closed", message: "workers: closed" });
|
||||
}
|
||||
|
||||
/** The full error of an exit nobody asked for (the app's logs). */
|
||||
function exitError(address: Address, { code, signal }: Exit, child: Child): Error {
|
||||
const how = signal === null ? `code ${code}` : `signal ${signal}`;
|
||||
const when = child.status === "starting" ? "exited before it was ready" : "exited";
|
||||
return new ChordError({ code: "unavailable", message: `worker ${address.key}: ${when} (${how})` });
|
||||
}
|
||||
|
||||
/** The public error of an exit: a start that failed, or the exit's code or signal. */
|
||||
function publicExit({ code, signal }: Exit, child: Child): CrossedError {
|
||||
if (child.status === "starting") return START_FAILED;
|
||||
return { message: `worker exited (${signal ?? `code ${code}`})`, code: "unavailable" };
|
||||
}
|
||||
|
||||
/** A set's identity: its plugins and digests. */
|
||||
function keyOf(scripts: readonly PluginScript[]): string {
|
||||
return scripts
|
||||
.map(({ plugin, built }) => `${plugin.name}@${built.kind === "script" ? built.digest : "failed"}`)
|
||||
.sort()
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
/** `entry`, or `host` listed with the abort's error once `context` aborts first (the entry goes on). */
|
||||
async function bounded(host: string, entry: Promise<RolloutHost>, context: Context): Promise<RolloutHost> {
|
||||
try {
|
||||
return await awaitWithContext(entry, context);
|
||||
} catch (error) {
|
||||
return { host, error };
|
||||
}
|
||||
}
|
||||
|
||||
/** The workers' state: a JSON watchable of every address's `WorkerState`, published once per microtask turn. */
|
||||
class WorkersState extends Source {
|
||||
readonly #addresses: ReadonlyMap<string, Routed>;
|
||||
#value: Readonly<Record<string, WorkerState>> | undefined;
|
||||
#scheduled = false;
|
||||
|
||||
constructor(addresses: ReadonlyMap<string, Routed>, batch: Batch) {
|
||||
super(batch);
|
||||
this.#addresses = addresses;
|
||||
}
|
||||
|
||||
get value(): JsonValue {
|
||||
this.#value ??= stateOf(this.#addresses);
|
||||
// Plain JSON: statuses, numbers and crossed errors.
|
||||
return this.#value as unknown as JsonValue;
|
||||
}
|
||||
|
||||
deliver(context: Context): void {
|
||||
const value = this.value;
|
||||
this.publish(value, [["r", value]], false, context);
|
||||
}
|
||||
|
||||
/** Something changed: published with the turn's other changes while someone watches or derives from it. */
|
||||
refresh(): void {
|
||||
this.#value = undefined;
|
||||
if (this.#scheduled || (!this.watched && this.dependents.size === 0)) return;
|
||||
this.#scheduled = true;
|
||||
queueMicrotask(() => {
|
||||
this.#scheduled = false;
|
||||
this.batch.within(() => this.changed(), BACKGROUND_CONTEXT);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** Every address's state: its child's, or its failure; none for an address with neither. */
|
||||
function stateOf(addresses: ReadonlyMap<string, Routed>): Record<string, WorkerState> {
|
||||
const state: Record<string, WorkerState> = {};
|
||||
for (const [key, { child, failure }] of addresses) {
|
||||
if (failure !== undefined) {
|
||||
state[key] = { status: "failed", ...failure };
|
||||
} else if (child !== undefined) {
|
||||
const { pid, startedAt } = child.process;
|
||||
state[key] = { status: child.status, pid, startedAt };
|
||||
}
|
||||
}
|
||||
return state;
|
||||
}
|
||||
|
|
@ -196,7 +196,7 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
|
|||
waited), `close_timeout` (`object`, `native`: a close still running at its `closeTimeout`, a possible leak), `drain_timeout` (`object`: calls still running at `closeTimeout`; their natives close anyway),
|
||||
`listener_error` (a throwing watch listener), `delivery_failed` (a frame delivery or a link's flush that threw) and
|
||||
`protocol_error` (`error`: what closed a link, see `link/`) and `resolve_failed` (`address`: its key, `error`: a
|
||||
route's `resolve` rejected; the address waits its backoff), `routine_failed` (`object`, `routine`: its position,
|
||||
route's `resolve` rejected; the address's holders fail `waiting` with it), `routine_failed` (`object`, `routine`: its position,
|
||||
`error`: a routine call threw, or its watch could not be taken; a source's failure calls never reach a routine), `job_failed` (`object`, `job`,
|
||||
`id`, `error`: a run threw, not after an abort), `job_finish_failed` (the same: a `finish` threw; it is retried) and
|
||||
`job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end),
|
||||
|
|
@ -233,8 +233,8 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
|
|||
live is ignored; failure calls wake nothing; a composition made by the object's own contributions leaving (it
|
||||
failed live) does not wake it. There is no backoff between wakes: wake an object on sources that change rarely
|
||||
(models, a harness), not on every event (a writable set per message retries a failing object per message).
|
||||
- `host.root(fn, ctx, { principal? })` (slice 8 D3): `fn(ctx)` with a handler's `Ctx<P>`: the host's principal or the
|
||||
given one, this host's incarnation, `ctx`'s telemetry, a signal linked to `ctx`'s, `ctx.get`. When `fn` settles,
|
||||
- `host.root(fn, ctx, { principal? })` (slice 8 D3): `fn(ctx)` with a handler's `Ctx<P>`: the host's principal (an
|
||||
omitted `principal`) or the given one (an explicit `undefined`: anonymous, slice 10), this host's incarnation, `ctx`'s telemetry, a signal linked to `ctx`'s, `ctx.get`. When `fn` settles,
|
||||
`ctx` aborts or the host closes, the scope is sealed: its references released (their calls reject `released`), a
|
||||
waiting `ctx.get` gives up, a later one is refused `released/settled`, its signal aborted. An abort rejects
|
||||
`host.root` at once (an uncooperative `fn` runs on and fails at its next use of Chord); an aborted `ctx` runs no
|
||||
|
|
@ -327,12 +327,15 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
|
|||
starts (`false`: broken `forbidden`; a throw: broken with that error). `resolve(address, context)` returns `{ link,
|
||||
principal, trusted?, codec? }`, the address's link (role `route`); it runs at most once at a time per address, later
|
||||
holds wait for it, and `context` is aborted only when the host closes (a link it returns later is closed at once). A
|
||||
`resolve` whose holds were all released meanwhile finishes and its link is kept. A rejection (`resolve_failed`), a
|
||||
link closed already, or the address's link closing while holds over it exist starts the address's backoff (1 s
|
||||
doubling to 60 s, separate from objects' identities); the holds stay `loading` and resolve again after it while they
|
||||
exist. Only a link that completed its handshake resets the backoff. A `resolve` returns the link before the worker
|
||||
loaded (the worker connects first): waiting for its `load` could deadlock. Chord never closes a route link except
|
||||
at `host.close`. When this host holds over a route link it sends whom each hold acts for (the table in `remote/`):
|
||||
`resolve` whose holds were all released meanwhile finishes and its link is kept. A route has no timer (slice 10
|
||||
R2): a rejection (`resolve_failed`), a link closed already, or a link closing before its handshake fails the
|
||||
address's holders `failed { waiting }` with that error (they stay on its path: its next link binds them); the
|
||||
address resolves again only for a new hold, on `retry`, or once after a link that completed its handshake dropped
|
||||
while held. It returns a `RouteHandle`: `retry(address)` ends the address's failure and suspension and resolves it
|
||||
now if held; `suspend(address)` resolves nothing for it while it has holders (they fail `waiting`,
|
||||
"<key>: suspended", once its link is gone), forgotten with its last holder. A `resolve` returns the link before the
|
||||
worker loaded (the worker connects first): waiting for its `load` could deadlock. Chord closes a route link itself
|
||||
only at `host.close`, on the heartbeat and on a protocol error. When this host holds over a route link it sends whom each hold acts for (the table in `remote/`):
|
||||
a guest's handler that `ctx.get`s a worker type reaches the worker as the guest.
|
||||
- `host.close(ctx)`: aborts every route's `resolve`, closes its links first, children, routes, then the parent (no
|
||||
`gone` goes out), ends every remote reference (`released/closed`, parked calls rejected with it), every request
|
||||
|
|
@ -568,7 +571,7 @@ path or leaves it waiting, `acquire` gets one for a caller), `UnboundIndex` (req
|
|||
`checkDependencies` and `ParkedLong` (what a parked attempt's dependencies say; the `parked_long` report),
|
||||
`Destination` (where a reference or an arriving hold goes, by address: here, a route's path or the parent's; whether
|
||||
a hold served here is placed here), `Route` (one per routed kind: per address its `Path`, at most one running
|
||||
`resolve`, the backoff per address; an address is kept while it has holders, a link or a running `resolve`),
|
||||
`resolve`, its failure and suspension; an address is kept while it has holders, a link or a running `resolve`),
|
||||
`ChordObject` (one per identity: its state across attempts (what a parked attempt shows), its first outcome, holders,
|
||||
retries passed on while parked, cycle checks with its instance id and the chain it was created for, lingering, ending and revocation, the count of its running and finishing job runs,
|
||||
which keeps it like a reference: `countRuns`, its contributions while live and its wake signals), `RetryWakes` (an
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ import { type Presence, References } from "./presence.ts";
|
|||
import { Binding, type CheckedParams, type Held, type Reference, type Target } from "./reference.ts";
|
||||
import { RevocationKeys, type RevokedBy } from "./revocation.ts";
|
||||
import { type RootOptions, RootScopes } from "./root.ts";
|
||||
import { Route, type RouteOptions } from "./route.ts";
|
||||
import { Route, type RouteHandle, type RouteOptions } from "./route.ts";
|
||||
import {
|
||||
addStaged,
|
||||
emptyStaged,
|
||||
|
|
@ -263,15 +263,16 @@ export interface Host<P extends Principal = Principal, R extends JsonValue = Jso
|
|||
connect(link: Link, options: LinkOptions<P>, context: Context): void;
|
||||
/**
|
||||
* How this host reaches hosts of `Kind` (called before `load`, once per kind): `authorize` admits every hold routed
|
||||
* over it, `resolve` makes the link of an address. Holds for an address wait while it resolves and again after its
|
||||
* link drops (a backoff per address, 1 s doubling to 60 s), while they exist. `context` is the parent of the route's
|
||||
* work (its `resolve`s, the holds and calls on its links), aborted for it when this host closes.
|
||||
* over it, `resolve` makes the link of an address, for a new hold, on `retry`, or once after a link that completed
|
||||
* its handshake dropped while held (no timer). A rejected `resolve`, or a link closed before its handshake, fails the
|
||||
* address's holders `failed { waiting }` (they stay on its path). `context` is the parent of the route's work (its
|
||||
* `resolve`s, the holds and calls on its links), aborted for it when this host closes. Returns the address controls.
|
||||
*/
|
||||
route<N extends string, Q extends Params>(
|
||||
Kind: HostKind<N, Q>,
|
||||
options: RouteOptions<P, HostKind<N, Q>>,
|
||||
context: Context,
|
||||
): void;
|
||||
): RouteHandle<HostKind<N, Q>>;
|
||||
/**
|
||||
* Revokes `revocation` (§4, R10, R11), synchronously: every remote object whose principal it revokes (`revokedBy`)
|
||||
* ends at once, whatever references remain (holders get `forbidden`, running calls are aborted, an activation is
|
||||
|
|
@ -300,7 +301,7 @@ export interface Host<P extends Principal = Principal, R extends JsonValue = Jso
|
|||
registry<C, V>(R: Registry<C, V>, context: Context, options?: GetOptions): Promise<RegistryValue<V>>;
|
||||
/**
|
||||
* Runs `fn` with a call context for code outside objects (a Durable tool, a CLI): the host's principal (or
|
||||
* `options.principal`), this host's incarnation, `context`'s telemetry and signal, `ctx.get`. When `fn` settles or
|
||||
* `options.principal`; an explicit `undefined` runs anonymous), this host's incarnation, `context`'s telemetry and signal, `ctx.get`. When `fn` settles or
|
||||
* `context` aborts, the context is sealed: its signal aborted, `ctx.get` refused `released`, every reference it took
|
||||
* released; an abort rejects at once. `host.close` seals the open ones; after it, `root` rejects `released`.
|
||||
*/
|
||||
|
|
@ -862,7 +863,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
Kind: HostKind<N, Q>,
|
||||
options: RouteOptions<P, HostKind<N, Q>>,
|
||||
context: Context,
|
||||
): void {
|
||||
): RouteHandle<HostKind<N, Q>> {
|
||||
this.#assertOpen();
|
||||
if (this.#state !== "new") throw new Error("route: routes are declared before load");
|
||||
if (this.#routes.has(Kind.kind)) throw new Error(`route: ${Kind.kind} is routed already`);
|
||||
|
|
@ -876,6 +877,10 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
routing,
|
||||
);
|
||||
this.#routes.set(Kind.kind, route);
|
||||
return {
|
||||
retry: (address) => route.retry(address.key),
|
||||
suspend: (address) => route.suspend(address.key),
|
||||
};
|
||||
}
|
||||
|
||||
revoke(revocation: R, context: Context): void {
|
||||
|
|
@ -958,8 +963,10 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
async root<T>(fn: (ctx: Ctx<P>) => Promise<T>, context: Context, options: RootOptions<P> = {}): Promise<T> {
|
||||
this.batch.assertOutside("root");
|
||||
this.#assertOpen();
|
||||
// The scope's ctx acts as `principal`, a `P`.
|
||||
return this.#roots.run(fn as (ctx: Ctx) => Promise<T>, options.principal ?? this.principal, context);
|
||||
// The scope's ctx acts as `principal`, a `P`: an explicit `undefined` runs anonymous (an HTTP request nobody
|
||||
// authenticated), only an omitted one takes the host's.
|
||||
const principal = "principal" in options ? options.principal : this.principal;
|
||||
return this.#roots.run(fn as (ctx: Ctx) => Promise<T>, principal, context);
|
||||
}
|
||||
|
||||
/** The record of registry `name` on this host, made by its first use. */
|
||||
|
|
@ -1098,6 +1105,14 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
this.#wait(binding, Type.name, unbound, () => this.#place(binding, Type, params, context));
|
||||
return undefined;
|
||||
}
|
||||
// An anonymous scope (an HTTP request nobody authenticated) is no principal a remote type's factory can serve
|
||||
// (Opus's build review F5): refused, never bound as the host.
|
||||
if (served!.kind !== "local" && principal === undefined) {
|
||||
throw new ChordError({
|
||||
code: "forbidden",
|
||||
message: `${Type.name}: an anonymous caller gets no remote object`,
|
||||
});
|
||||
}
|
||||
// Checked above: served here at the caller's version.
|
||||
const object = this.#object(served!, params, principal, context, NO_CHAIN, binding.dependent);
|
||||
binding.bind(object);
|
||||
|
|
@ -1169,7 +1184,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
return this.#link(link, "route", options, context);
|
||||
}
|
||||
|
||||
/** A link completed its handshake: a route's resets its address's backoff. */
|
||||
/** A link completed its handshake: a route's address resolves again once if it drops. */
|
||||
linkOpened(link: HostLink): void {
|
||||
if (link.role !== "route") return;
|
||||
for (const route of this.#routes.values()) route.opened(link);
|
||||
|
|
@ -1353,7 +1368,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
|
|||
const closed = this.#closedError();
|
||||
// Code outside objects stops at its next use of Chord: an `fn` waiting in `sleep(ms, ctx)` ends.
|
||||
this.#roots.close(closed);
|
||||
for (const route of this.#routes.values()) route.close();
|
||||
for (const holder of this.#pathHolders()) if (holder.kind === "remote") holder.lost(closed);
|
||||
// At the fence too: requests still waiting for a publication or a provider end as the others did, and gets
|
||||
// waiting for a publication that will not come fail now, not after the slow closes below (fix round 1).
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ export type { RefState } from "./ref-state.ts";
|
|||
export type { Held, Reference, RefIn } from "./reference.ts";
|
||||
export type { RevokedBy } from "./revocation.ts";
|
||||
export type { RootOptions } from "./root.ts";
|
||||
export type { RouteOptions } from "./route.ts";
|
||||
export type { RouteHandle, RouteOptions } from "./route.ts";
|
||||
export type { OnHandler, OnLiveHandler, OnNativeHandler, RoutineDeclarations } from "./routines.ts";
|
||||
export { sleep, untilAborted } from "./sleep.ts";
|
||||
export type { PluginFacet } from "./staging.ts";
|
||||
|
|
|
|||
|
|
@ -2,19 +2,22 @@
|
|||
// (a Durable tool, a CLI), sealed when `fn` settles, when `context` aborts and when the host closes: its signal aborted,
|
||||
// its `ctx.get` refused `released`, every reference it took released.
|
||||
|
||||
import { awaitWithContext, type Context } from "../../context/index.ts";
|
||||
import { awaitWithContext, type Context, withAbortSignal, withoutAbortSignal } from "../../context/index.ts";
|
||||
import { ChordError, type Principal } from "../data/index.ts";
|
||||
import { CallContext, type Ctx, withLinkedAbortSignal } from "./ctx.ts";
|
||||
import { CallContext, type Ctx } from "./ctx.ts";
|
||||
import type { AnyHost } from "./host.ts";
|
||||
|
||||
/** What `host.root` takes besides `fn` and the context. */
|
||||
export type RootOptions<P extends Principal = Principal> = {
|
||||
/** Whom the scope acts as: the host's principal by default; a trusted same-host caller's choice (base code). */
|
||||
readonly principal?: P;
|
||||
/**
|
||||
* Whom the scope acts as: the host's principal when omitted; a trusted same-host caller's choice (base code), an
|
||||
* explicit `undefined` anonymous (`ctx.principal` undefined).
|
||||
*/
|
||||
readonly principal?: P | undefined;
|
||||
};
|
||||
|
||||
/** An open root scope: its seal, and the rejection its caller gets when the host closes. */
|
||||
type OpenScope = { readonly seal: (reason: unknown) => void; readonly closed: PromiseWithResolvers<never> };
|
||||
/** An open root scope: its seal, and how its caller is rejected when the host closes. */
|
||||
type OpenScope = { readonly seal: (reason: unknown) => void; readonly reject: (reason: unknown) => void };
|
||||
|
||||
/** The open root scopes of a host, each sealed once; `close` seals the rest and rejects their callers. */
|
||||
export class RootScopes {
|
||||
|
|
@ -33,41 +36,50 @@ export class RootScopes {
|
|||
* next use of Chord), or when `context` was aborted already (`fn` does not run).
|
||||
*/
|
||||
run<T>(fn: (ctx: Ctx) => Promise<T>, principal: Principal | undefined, context: Context): Promise<T> {
|
||||
context.abortSignal?.throwIfAborted();
|
||||
const caller = context.abortSignal;
|
||||
caller?.throwIfAborted();
|
||||
const host = this.#host;
|
||||
// The scope's own signal, aborted by its seal; the caller's abort seals it (one listener, no linked signal: a
|
||||
// request's scope is made per HTTP request).
|
||||
const controller = new AbortController();
|
||||
const linked = withLinkedAbortSignal(controller.signal, context);
|
||||
const scope = linked.context;
|
||||
const scope = withAbortSignal(controller.signal, caller === undefined ? context : withoutAbortSignal(context));
|
||||
const ctx = new CallContext(scope, { principal, incarnation: host.incarnation }, host);
|
||||
const result = Promise.withResolvers<T>();
|
||||
const open: OpenScope = {
|
||||
seal: (reason) => {
|
||||
if (!this.#open.delete(open)) return;
|
||||
scope.abortSignal?.removeEventListener("abort", aborted);
|
||||
caller?.removeEventListener("abort", aborted);
|
||||
// References first: a `get` still waiting gives up with `released`, not with the abort's reason.
|
||||
ctx.settle();
|
||||
// While still linked, so the scope's own signal (a caller's context has one) aborts too.
|
||||
controller.abort(reason);
|
||||
linked.release();
|
||||
},
|
||||
closed: Promise.withResolvers<never>(),
|
||||
reject: result.reject,
|
||||
};
|
||||
const aborted = (): void => open.seal(scope.abortSignal?.reason);
|
||||
const aborted = (): void => open.seal(caller?.reason);
|
||||
this.#open.add(open);
|
||||
scope.abortSignal?.addEventListener("abort", aborted, { once: true });
|
||||
caller?.addEventListener("abort", aborted, { once: true });
|
||||
// Plugin code: a synchronous throw becomes the rejection.
|
||||
const running = (async () => fn(ctx))();
|
||||
const settled = (): void => open.seal(SETTLED);
|
||||
running.then(settled, settled);
|
||||
// A method's caller gets `released` when its object ends, whatever the handler still does: so does this one. One
|
||||
// wait on the caller's signal around both, so the close removes its listener too.
|
||||
return awaitWithContext(Promise.race([running, open.closed.promise]), context);
|
||||
running.then(
|
||||
(value) => {
|
||||
open.seal(SETTLED);
|
||||
result.resolve(value);
|
||||
},
|
||||
(error: unknown) => {
|
||||
open.seal(SETTLED);
|
||||
result.reject(error);
|
||||
},
|
||||
);
|
||||
// A method's caller gets `released` when its object ends, whatever the handler still does: so does this one
|
||||
// (`close` rejects `result`), and at once when its context aborts.
|
||||
return caller === undefined ? result.promise : awaitWithContext(result.promise, context);
|
||||
}
|
||||
|
||||
/** The host closes: every open scope is sealed with `reason`, its caller rejected with it. */
|
||||
close(reason: unknown): void {
|
||||
for (const open of [...this.#open]) {
|
||||
open.seal(reason);
|
||||
open.closed.reject(reason);
|
||||
open.reject(reason);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,15 +1,15 @@
|
|||
// Routes (§3): how a host reaches hosts of one kind. `authorize` admits each hold routed over the route; `resolve`
|
||||
// makes the link of one address, at most once at a time, again after the link drops while holds exist, with a
|
||||
// backoff per address.
|
||||
// makes the link of one address, at most once at a time. No timer (slice 10 R2): an address resolves for a new hold,
|
||||
// on `RouteHandle.retry`, or once after its link dropped while held; a rejected `resolve` (or a link closed before its
|
||||
// handshake) fails its holders `failed { waiting }` until then, and a suspended address resolves nothing while held.
|
||||
|
||||
import type { Context } from "../../context/index.ts";
|
||||
import type { Address, HostKind, KindShape } from "../contracts/index.ts";
|
||||
import { kindParams, type ParamsData } from "../contracts/internal.ts";
|
||||
import { ChordError, type Principal } from "../data/index.ts";
|
||||
import { checkAddress } from "../data/internal.ts";
|
||||
import { asChordError, checkAddress } from "../data/internal.ts";
|
||||
import type { Codec, Link, PartLimits } from "../link/index.ts";
|
||||
import { type HostLink, Path, type Upstream } from "../remote/internal.ts";
|
||||
import { Backoff } from "./backoff.ts";
|
||||
import { type HostLink, Path, type PathEvent, type Upstream } from "../remote/internal.ts";
|
||||
import type { AnyHost } from "./host.ts";
|
||||
|
||||
/** What `host.route` takes for a kind. */
|
||||
|
|
@ -33,8 +33,29 @@ export type RouteOptions<P extends Principal, K extends KindShape> = {
|
|||
}>;
|
||||
};
|
||||
|
||||
/** One address of a route: its path (link and holders) and whether its `resolve` runs. */
|
||||
type Routed = { readonly address: Address; readonly path: Path; resolving: boolean };
|
||||
/** What `host.route` returns: the app's controls of its addresses (D10b, R1). */
|
||||
export type RouteHandle<K extends KindShape> = {
|
||||
/**
|
||||
* Resolves `address` again now if it has holders and no link: its failure and its suspension end (a Retry button);
|
||||
* an address nobody holds resolves at its next hold.
|
||||
*/
|
||||
retry(address: Address<K>): void;
|
||||
/**
|
||||
* Resolves nothing for `address` while it has holders: they wait `failed { waiting }` ("suspended") once its link is
|
||||
* gone; the suspension ends with its last holder or `retry`.
|
||||
*/
|
||||
suspend(address: Address<K>): void;
|
||||
};
|
||||
|
||||
/** One address of a route: its path, whether its `resolve` runs, why it fails its holders, whether it is suspended. */
|
||||
type Routed = {
|
||||
readonly address: Address;
|
||||
readonly path: Path;
|
||||
resolving: boolean;
|
||||
/** A rejected `resolve`, or a link closed before its handshake: its holders fail with it until the next resolve. */
|
||||
failure: ChordError | undefined;
|
||||
suspended: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* A host's route for one kind. An address is kept while it has holders, a link or a running `resolve`: a resolve whose
|
||||
|
|
@ -48,10 +69,8 @@ export class Route {
|
|||
/** The route's context, aborted when the host closes: every `resolve` and its links' work run under it. */
|
||||
readonly #context: Context;
|
||||
readonly #addresses = new Map<string, Routed>();
|
||||
/** The addresses' links, which tell their address when they open or close. */
|
||||
readonly #links = new Map<HostLink, Routed>();
|
||||
/** Per address key, separate from objects' identities. */
|
||||
readonly #backoff = new Backoff((key) => this.#due(key));
|
||||
/** The addresses' links, which tell their address when they close, and whether each completed its handshake. */
|
||||
readonly #links = new Map<HostLink, { readonly routed: Routed; greeted: boolean }>();
|
||||
|
||||
constructor(kind: HostKind, options: RouteOptions<Principal, KindShape>, host: AnyHost, context: Context) {
|
||||
this.kind = kind.kind;
|
||||
|
|
@ -72,64 +91,100 @@ export class Route {
|
|||
}
|
||||
let routed = this.#addresses.get(address.key);
|
||||
if (routed === undefined) {
|
||||
const made: Routed = { address, path: new Path(() => this.#check(made)), resolving: false };
|
||||
const made: Routed = {
|
||||
address,
|
||||
path: new Path((event) => this.#changed(made, event)),
|
||||
resolving: false,
|
||||
failure: undefined,
|
||||
suspended: false,
|
||||
};
|
||||
routed = made;
|
||||
this.#addresses.set(address.key, made);
|
||||
}
|
||||
return routed.path;
|
||||
}
|
||||
|
||||
/** One of this route's links completed its handshake: its address's backoff is over. */
|
||||
opened(link: HostLink): void {
|
||||
const routed = this.#links.get(link);
|
||||
if (routed !== undefined) this.#backoff.delete(routed.address.key);
|
||||
/** `RouteHandle.retry`: the address's failure and suspension end; it resolves now if held and unlinked. */
|
||||
retry(key: string): void {
|
||||
const routed = this.#addresses.get(key);
|
||||
if (routed === undefined) return;
|
||||
routed.failure = undefined;
|
||||
routed.suspended = false;
|
||||
this.#resolveIfHeld(routed);
|
||||
}
|
||||
|
||||
/** One of this route's links closed: the address waits its backoff, then resolves again while it has holders. */
|
||||
/** `RouteHandle.suspend`: a held address resolves nothing until `retry` or its last holder; unheld, nothing is kept. */
|
||||
suspend(key: string): void {
|
||||
const routed = this.#addresses.get(key);
|
||||
if (routed === undefined || !routed.path.held) return;
|
||||
routed.suspended = true;
|
||||
if (routed.path.link === undefined && !routed.resolving) routed.path.unreachable(suspended(routed));
|
||||
}
|
||||
|
||||
/** One of this route's links completed its handshake: a later drop of it resolves its address again. */
|
||||
opened(link: HostLink): void {
|
||||
const entry = this.#links.get(link);
|
||||
if (entry !== undefined) entry.greeted = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* One of this route's links closed: its address resolves again once while held if the link had completed its
|
||||
* handshake, else its holders fail (no loop over a worker that dies at start).
|
||||
*/
|
||||
closed(link: HostLink): boolean {
|
||||
const routed = this.#links.get(link);
|
||||
if (routed === undefined) return false;
|
||||
const entry = this.#links.get(link);
|
||||
if (entry === undefined) return false;
|
||||
this.#links.delete(link);
|
||||
if (!this.#host.closing) this.#backoff.failed(routed.address.key);
|
||||
const { routed } = entry;
|
||||
if (!entry.greeted) {
|
||||
routed.failure = new ChordError({
|
||||
code: "unavailable",
|
||||
message: `${routed.address.key}: the link closed before its handshake`,
|
||||
});
|
||||
}
|
||||
routed.path.disconnected(link);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** The host closes: no address resolves again. Returns every holder of every path, for the host to end. */
|
||||
close(): Upstream[] {
|
||||
this.#backoff.clear();
|
||||
return this.holders();
|
||||
}
|
||||
|
||||
/** Every holder of every path of this route, linked or not (revocation ends a revoked principal's). */
|
||||
holders(): Upstream[] {
|
||||
return [...this.#addresses.values()].flatMap(({ path }) => path.holders());
|
||||
}
|
||||
|
||||
/**
|
||||
* An address's holders or link changed: it resolves when it has holders, no link, no `resolve` running and no
|
||||
* backoff window; it is forgotten when it has neither holders, a link nor a running `resolve`.
|
||||
* An address's path changed without a link: a new holder resolves it (a failed address too: a new hold is a new
|
||||
* request), a dropped link resolves it once (or fails it, closed before its handshake), the last holder's release
|
||||
* forgets it (its failure and suspension too) unless its link or `resolve` keeps it.
|
||||
*/
|
||||
#check(routed: Routed): void {
|
||||
const { path, address } = routed;
|
||||
if (path.link !== undefined || routed.resolving || this.#host.closing) return;
|
||||
if (!path.held) {
|
||||
this.#addresses.delete(address.key);
|
||||
return;
|
||||
#changed(routed: Routed, event: PathEvent): void {
|
||||
if (this.#host.closing) return;
|
||||
switch (event.kind) {
|
||||
case "released":
|
||||
routed.suspended = false;
|
||||
if (routed.path.link === undefined && !routed.resolving) this.#addresses.delete(routed.address.key);
|
||||
return;
|
||||
case "attached":
|
||||
if (routed.suspended) event.holder.unreachable(suspended(routed));
|
||||
else if (!routed.resolving) void this.#resolve(routed);
|
||||
return;
|
||||
case "dropped":
|
||||
if (!routed.path.held) this.#addresses.delete(routed.address.key);
|
||||
else if (routed.suspended) routed.path.unreachable(suspended(routed));
|
||||
else if (routed.failure !== undefined) routed.path.unreachable(routed.failure);
|
||||
else void this.#resolve(routed);
|
||||
}
|
||||
if (!this.#backoff.waiting(address.key)) void this.#resolve(routed);
|
||||
}
|
||||
|
||||
/** A backoff window ended. */
|
||||
#due(key: string): void {
|
||||
const routed = this.#addresses.get(key);
|
||||
if (routed === undefined) this.#backoff.delete(key);
|
||||
else this.#check(routed);
|
||||
/** Resolves the address if it has holders, no link, no running `resolve` and no suspension. */
|
||||
#resolveIfHeld(routed: Routed): void {
|
||||
const { path } = routed;
|
||||
if (!path.held || path.link !== undefined || routed.resolving || this.#host.closing) return;
|
||||
void this.#resolve(routed);
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs `resolve` once for the address. A rejection, or a link closed already, starts the address's backoff; a link
|
||||
* returned after the host closed is closed at once.
|
||||
* Runs `resolve` once for the address. A rejection, or a link closed already, fails its holders with the error until
|
||||
* the next resolve; a link returned after the host closed is closed at once.
|
||||
*/
|
||||
async #resolve(routed: Routed): Promise<void> {
|
||||
const key = routed.address.key;
|
||||
|
|
@ -141,8 +196,7 @@ export class Route {
|
|||
routed.resolving = false;
|
||||
if (this.#host.closing) return;
|
||||
this.#host.report({ kind: "resolve_failed", address: key, error });
|
||||
this.#backoff.failed(key);
|
||||
this.#check(routed);
|
||||
this.#settle(routed, asChordError(error, key));
|
||||
return;
|
||||
}
|
||||
routed.resolving = false;
|
||||
|
|
@ -153,13 +207,33 @@ export class Route {
|
|||
const { link: made, ...options } = resolved;
|
||||
const link = this.#host.routeLink(made, options, this.#context);
|
||||
if (!link.open) {
|
||||
this.#backoff.failed(key);
|
||||
this.#check(routed);
|
||||
this.#settle(
|
||||
routed,
|
||||
new ChordError({ code: "unavailable", message: `${key}: the link closed before its handshake` }),
|
||||
);
|
||||
return;
|
||||
}
|
||||
this.#links.set(link, routed);
|
||||
routed.failure = undefined;
|
||||
// A peer that greeted first had its `hello` handled while the link was made, before it was registered here.
|
||||
if (link.greeted) this.#backoff.delete(key);
|
||||
this.#links.set(link, { routed, greeted: link.greeted });
|
||||
routed.path.connected(link);
|
||||
}
|
||||
|
||||
/**
|
||||
* A resolve failed with `error`: its holders fail `failed { waiting }` with it (they stay on its path for its next
|
||||
* link), or the address goes when nobody holds it any more.
|
||||
*/
|
||||
#settle(routed: Routed, error: ChordError): void {
|
||||
if (!routed.path.held) {
|
||||
this.#addresses.delete(routed.address.key);
|
||||
return;
|
||||
}
|
||||
routed.failure = error;
|
||||
routed.path.unreachable(routed.suspended ? suspended(routed) : error);
|
||||
}
|
||||
}
|
||||
|
||||
/** The failure holders of a suspended address see. */
|
||||
function suspended(routed: Routed): ChordError {
|
||||
return new ChordError({ code: "unavailable", message: `${routed.address.key}: suspended` });
|
||||
}
|
||||
|
|
|
|||
|
|
@ -14,7 +14,9 @@ within each link's limits, compression at the origin, activation chains and revo
|
|||
`call`, `abort`, `retry` and `ack` to the provider side of their `origin` (0 when absent; a `hold` makes it, other
|
||||
messages for an origin without one are ignored), `closed { origin }` ends that provider side; `bound`, `state`,
|
||||
`failed`, `loading`, `result`, `error` and `gone` to the consumer side; `revoke` to the host, from a parent link only
|
||||
(`host.revoke` with this host's own `revokedBy`; anywhere else it is ignored). `origin` is honoured on every link.
|
||||
(`host.revoke` with this host's own `revokedBy`; anywhere else it is ignored). `origin` is honoured only on parent and route links, where a relay's messages arrive; on a child link a message
|
||||
carrying one is a protocol error (decision "Origins only from relays"). A provider side (one per origin) goes once it
|
||||
has no holds, no calls and nothing in flight; its origin's next hold makes a new one (origins are never reused).
|
||||
Closing it, from either end, ends both sides: the provider sides release the peer's holds and abort their calls, the
|
||||
consumer side's calls reject and its path tells the holders. It makes the parts this host originates (`part`:
|
||||
deflated above 16 KiB with the host's compressor) and opens the parts it owns (`openPart`: within the limits its
|
||||
|
|
@ -139,7 +141,9 @@ within each link's limits, compression at the origin, activation chains and revo
|
|||
waiting when its hold gets `loading` is rejected `unavailable` (it may name a method of the old variant).
|
||||
- `Path` (the parent, or one route address): its current link and its holders (`RemoteObject`s and `RelayedHold`s),
|
||||
held on that link and again on every next one; when the link drops, every holder is told (`dropped`). `changed`
|
||||
tells its owner (a route) when it needs a link or lost its last holder.
|
||||
tells its owner (a route) what happened without a link: a holder `attached`, the link `dropped`, the last holder
|
||||
`released`. `unreachable(error)` fails every holder `failed { waiting }` (a route's `resolve` failed, a suspension):
|
||||
a `RemoteObject` moves there, a `RelayedHold` sends `failed` downstream; the next link binds them again.
|
||||
- `RemoteObject` (one per remote reference, owned by its host across the links of its path): its `state` (`loading`
|
||||
until a link's hold is bound and again on every drop or `loading`, `failed` from `failed` with `retryAt = now +
|
||||
retryIn` for a transient cause, `retryAt` undefined for a broken or waiting one, `live` on `bound`, `released`;
|
||||
|
|
|
|||
|
|
@ -60,8 +60,9 @@ export class HostLink implements ChannelHandler {
|
|||
readonly context: Context;
|
||||
readonly consumer: ConsumerSide;
|
||||
/**
|
||||
* The provider side of each origin the peer sent holds for, made by its first; `origin` is honoured on every link
|
||||
* (decision "Relay flow control per origin").
|
||||
* The provider side of each origin the peer sent holds for, made by its first and dropped once it holds nothing;
|
||||
* `origin` is honoured only on parent and route links, where a relay's messages arrive (decision "Origins only from
|
||||
* relays").
|
||||
*/
|
||||
readonly #providers = new Map<number, ProviderSide>();
|
||||
readonly #channel: Channel;
|
||||
|
|
@ -159,6 +160,10 @@ export class HostLink implements ChannelHandler {
|
|||
* the channel holds later messages until it settled.
|
||||
*/
|
||||
message(message: Message): void | Promise<void> {
|
||||
// A child (a tab, a TUI) relays nothing: its origins would each get their own call window and record.
|
||||
if (this.role === "child" && "origin" in message && message.origin !== undefined) {
|
||||
throw new Error(`${message.kind}: an origin on a child link`);
|
||||
}
|
||||
switch (message.kind) {
|
||||
case "hold":
|
||||
this.#hold(message);
|
||||
|
|
@ -193,6 +198,13 @@ export class HostLink implements ChannelHandler {
|
|||
}
|
||||
}
|
||||
|
||||
/** `provider` holds nothing any more: it goes (its relay's upstream links are told `closed`). */
|
||||
drop(provider: ProviderSide): void {
|
||||
if (this.#providers.get(provider.origin) !== provider) return;
|
||||
this.#providers.delete(provider.origin);
|
||||
provider.closed();
|
||||
}
|
||||
|
||||
/** Ends the holds on this link that hold nothing and act for a principal `revoked` names (§4). */
|
||||
revokeUnbound(revoked: (principal: Principal) => boolean, error: ChordError): void {
|
||||
for (const provider of this.#providers.values()) provider.revokeUnbound(revoked, error);
|
||||
|
|
|
|||
|
|
@ -3,6 +3,6 @@
|
|||
|
||||
export type { HoldRequest } from "./hold.ts";
|
||||
export { HostLink, type LinkRole } from "./host-link.ts";
|
||||
export { Path, type Upstream } from "./path.ts";
|
||||
export { Path, type PathEvent, type Upstream } from "./path.ts";
|
||||
export type { ProviderSide } from "./provider.ts";
|
||||
export { RemoteObject } from "./remote-object.ts";
|
||||
|
|
|
|||
|
|
@ -9,19 +9,32 @@ import type { RemoteObject } from "./remote-object.ts";
|
|||
/** What a path holds on its links: one of this host's remote references, or a hold it relays. */
|
||||
export type Upstream = RemoteObject | RelayedHold;
|
||||
|
||||
/**
|
||||
* What changed on a path without a link, for its route: a holder attached, its link dropped (its holders went
|
||||
* loading), its last holder left.
|
||||
*/
|
||||
export type PathEvent =
|
||||
| { readonly kind: "attached"; readonly holder: Upstream }
|
||||
| { readonly kind: "dropped" }
|
||||
| { readonly kind: "released" };
|
||||
|
||||
const RELEASED: PathEvent = { kind: "released" };
|
||||
|
||||
const DROPPED: PathEvent = { kind: "dropped" };
|
||||
|
||||
/**
|
||||
* The parent link or one route address: its current link and its holders, held on that link and again on every next
|
||||
* one. `changed` runs when it gets a holder without a link, loses its link or its last holder (a route decides whether
|
||||
* to resolve).
|
||||
* to resolve), saying which.
|
||||
*/
|
||||
export class Path {
|
||||
readonly #holders = new Set<Upstream>();
|
||||
readonly #changed: () => void;
|
||||
readonly #changed: (event: PathEvent) => void;
|
||||
#link: HostLink | undefined;
|
||||
/** Whether it ever had a link: a host never connected to a parent has no parent path to wait on. */
|
||||
#linked = false;
|
||||
|
||||
constructor(changed: () => void) {
|
||||
constructor(changed: (event: PathEvent) => void) {
|
||||
this.#changed = changed;
|
||||
}
|
||||
|
||||
|
|
@ -43,14 +56,14 @@ export class Path {
|
|||
/** Holds `holder` over this path: on the current link now, and on every next one. */
|
||||
attach(holder: Upstream): void {
|
||||
this.#holders.add(holder);
|
||||
if (this.#link === undefined) this.#changed();
|
||||
if (this.#link === undefined) this.#changed({ kind: "attached", holder });
|
||||
else holder.attach(this.#link);
|
||||
}
|
||||
|
||||
/** `holder` was released or ended; no later link holds it. */
|
||||
detach(holder: Upstream): void {
|
||||
this.#holders.delete(holder);
|
||||
if (this.#holders.size === 0) this.#changed();
|
||||
if (this.#holders.size === 0) this.#changed(RELEASED);
|
||||
}
|
||||
|
||||
/** `link` is the path's link now: every holder is held on it. */
|
||||
|
|
@ -66,7 +79,12 @@ export class Path {
|
|||
this.#link = undefined;
|
||||
const error = new ChordError({ code: "unavailable", message: "the link closed" });
|
||||
for (const holder of this.#holders) holder.dropped(error);
|
||||
this.#changed();
|
||||
this.#changed(DROPPED);
|
||||
}
|
||||
|
||||
/** Its address cannot be reached for now (a route's `resolve` failed, a suspension): every holder fails waiting. */
|
||||
unreachable(error: ChordError): void {
|
||||
for (const holder of this.#holders) holder.unreachable(error);
|
||||
}
|
||||
|
||||
/** Every holder, for the host's close. */
|
||||
|
|
|
|||
|
|
@ -83,17 +83,20 @@ export class ProviderSide {
|
|||
break;
|
||||
case "release":
|
||||
this.#release(message.hold);
|
||||
this.#settle();
|
||||
break;
|
||||
case "call":
|
||||
return this.#call(message);
|
||||
case "abort":
|
||||
this.#abort(message.call);
|
||||
this.#settle();
|
||||
break;
|
||||
case "retry":
|
||||
this.#entries.get(message.hold)?.retry(message);
|
||||
break;
|
||||
case "ack":
|
||||
this.#ack();
|
||||
this.#settle();
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -228,12 +231,22 @@ export class ProviderSide {
|
|||
if (call.kind === "served") call.abort.abort(reason);
|
||||
}
|
||||
if (hold.kind === "served") this.unbind(hold);
|
||||
this.#settle();
|
||||
}
|
||||
|
||||
/** A call ended with `reply` (a result or an error): its slot is free; nothing is sent for one already gone. */
|
||||
reply(id: number, reply: Extract<OriginMessage, { readonly kind: "result" | "error" }>): void {
|
||||
if (!this.#calls.delete(id)) return;
|
||||
this.send(reply);
|
||||
this.#settle();
|
||||
}
|
||||
|
||||
/**
|
||||
* Drops this side from its link once it holds nothing: no holds, no calls, nothing in flight. Its origin's next hold
|
||||
* makes a new one (origins are never reused), so a long-lived link keeps records only for live work.
|
||||
*/
|
||||
#settle(): void {
|
||||
if (this.#entries.size === 0 && this.#calls.size === 0 && this.#sent.length === 0) this.link.drop(this);
|
||||
}
|
||||
|
||||
/** A hold's object is live: its `bound` goes now when a slot is free, else with the next free slot's bindings. */
|
||||
|
|
|
|||
|
|
@ -68,6 +68,11 @@ export class RelayedHold {
|
|||
if (upstream !== undefined) this.#downstream.upstreamDropped(this, error);
|
||||
}
|
||||
|
||||
/** Its path's address cannot be reached for now: downstream it fails `waiting` with `error` until the next link. */
|
||||
unreachable(error: ChordError): void {
|
||||
this.#downstream.send({ kind: "failed", hold: this.id, cause: { kind: "waiting", error: wireError(error) } });
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether it waits for its path's link (a route still resolving, or a dropped link's next): a revocation ends it then,
|
||||
* since the link it is held on next gets no past `revoke` (§4).
|
||||
|
|
|
|||
|
|
@ -153,6 +153,16 @@ export class RemoteObject implements Target {
|
|||
this.#hold = { side: link.consumer, id: link.consumer.attach(this, chain) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Its path's address cannot be reached for now (a route's `resolve` failed, a suspension): `failed { waiting }` with
|
||||
* `error`, still on the path, bound by its next link.
|
||||
*/
|
||||
unreachable(error: ChordError): void {
|
||||
this.#lastFailure = error;
|
||||
if (this.#state.kind !== "ended")
|
||||
this.#move({ kind: "failed", cause: { kind: "waiting", error }, retryAt: undefined });
|
||||
}
|
||||
|
||||
/** The link dropped: loading until a new link's hold is bound (`#startOver`). */
|
||||
dropped(_error: ChordError): void {
|
||||
this.#hold = undefined;
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
import { readFileSync } from "node:fs";
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { pluginFile } from "@earendil-works/chord/next/plugins";
|
||||
import { Files } from "../contracts/contract.ts";
|
||||
import { Files } from "@test/contracts/contract";
|
||||
|
||||
export default defineFacet((host) =>
|
||||
host.serve(Files, () => ({ text: constant(readFileSync(pluginFile("assets/data.txt"), "utf8").trim()) })),
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// Fixture plugin (slice 7c): the server facet, serving `Index` with the same label.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { Index } from "../contracts/contract.ts";
|
||||
import { Index } from "@test/contracts/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
export default defineFacet((host) => host.serve(Index, () => ({ label: constant(label) })));
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// Fixture plugin (slice 7c): the worker facet, serving `Notes` with a count document.
|
||||
|
||||
import { constant, defineFacet, defineState, int, obj } from "@earendil-works/chord/next";
|
||||
import { Notes } from "../contracts/contract.ts";
|
||||
import { Notes } from "@test/contracts/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
const Count = defineState("count", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 0 }) });
|
||||
|
|
|
|||
12
packages/chord/test/next-node/fixtures/workers/child.ts
Normal file
12
packages/chord/test/next-node/fixtures/workers/child.ts
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
// Fixture child process (slice 10): writes to stdout and stderr, spawns a grandchild, sends a link frame and a control
|
||||
// message at once, then lives until killed.
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { envelope } from "../../../../src/next-node/workers/control.ts";
|
||||
|
||||
const grandchild = spawn(process.execPath, ["-e", "setTimeout(() => {}, 60000)"], { stdio: "ignore" });
|
||||
process.stdout.write(`grandchild ${grandchild.pid}\n`);
|
||||
process.stderr.write("an error line\n");
|
||||
process.send?.(new Uint8Array([1, 2, 3]));
|
||||
process.send?.(envelope({ kind: "started" }));
|
||||
setInterval(() => {}, 1000);
|
||||
47
packages/chord/test/next-node/fixtures/workers/control.ts
Normal file
47
packages/chord/test/next-node/fixtures/workers/control.ts
Normal file
|
|
@ -0,0 +1,47 @@
|
|||
// Fixture contracts (slice 10): the session kind the workers serve, the fixture worker's `Control` and its start data.
|
||||
|
||||
import { hostKind, int, method, remote, str } from "../../../../src/next/contracts/index.ts";
|
||||
|
||||
/** The host kind of the fixture workers. */
|
||||
export const Session = hostKind("session", { id: str() });
|
||||
|
||||
/** What the fixture worker does on request, at its own address. */
|
||||
export const Control = remote("test.control", {
|
||||
version: 1,
|
||||
params: { id: str() },
|
||||
hostAddress: ({ id }) => Session({ id }),
|
||||
variants: {
|
||||
main: {
|
||||
methods: {
|
||||
crash: method(int()),
|
||||
fatal: method(str()),
|
||||
block: method(int()),
|
||||
blockThenCrash: method(int()),
|
||||
busy: method(int()),
|
||||
localCalls: method(int(), int()),
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
/** The fixture worker's start data: how it behaves. */
|
||||
export type WorkerBehaviour = {
|
||||
/** Exit with this code before connecting. */
|
||||
readonly exit?: number;
|
||||
/** Throw an error with this message before connecting (`fatal`, redacted: `secret…` words removed). */
|
||||
readonly fatal?: string;
|
||||
/** `false`: never connect (the start timer). */
|
||||
readonly connect?: boolean;
|
||||
/** Wait this long after connecting, before the load. */
|
||||
readonly loadMs?: number;
|
||||
/** Never load. */
|
||||
readonly hang?: boolean;
|
||||
/** A file the closers append their names to, in the order they run. */
|
||||
readonly log?: string;
|
||||
/** How long the second closer takes. */
|
||||
readonly closeMs?: number;
|
||||
/** After its load: close its IPC channel and block, alive (the server must reap it). */
|
||||
readonly disconnectAndBlock?: boolean;
|
||||
/** 300 ms after its load: this many sequential local calls, then `local calls sent <n>` on stdout. */
|
||||
readonly localCalls?: number;
|
||||
};
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
// Fixture plugin (slice 10): the label the worker facet shows; the rollout tests edit it.
|
||||
|
||||
export const label = "v1";
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
{
|
||||
"name": "@test/notes",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
// Fixture plugin (slice 10): the worker facet, serving `Notes` at the worker's address with the label of `label.ts`.
|
||||
|
||||
import { constant, defineFacet, hostKind, remote, str } from "@earendil-works/chord/next";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
const Session = hostKind("session", { id: str() });
|
||||
|
||||
const Notes = remote("test.notes", {
|
||||
version: 1,
|
||||
params: { id: str() },
|
||||
hostAddress: ({ id }) => Session({ id }),
|
||||
variants: { main: { state: { label: str() } } },
|
||||
});
|
||||
|
||||
export default defineFacet((host) =>
|
||||
host.serve(Notes, ({ id }) => ({ variant: "main", members: { label: constant(`${id} ${label}`) } })),
|
||||
);
|
||||
146
packages/chord/test/next-node/fixtures/workers/worker.ts
Normal file
146
packages/chord/test/next-node/fixtures/workers/worker.ts
Normal file
|
|
@ -0,0 +1,146 @@
|
|||
// Fixture worker (slice 10): `workerMain` with a host at its address, the shipped plugins, and a core facet serving
|
||||
// `Control` (crash, fatal, block, busy, local calls) and `Busy` (the app's keep-alive counts). Its start data picks a
|
||||
// behaviour for the tests: exit or throw before connecting, never connect, a slow or endless load, slow closers that
|
||||
// log their order to a file.
|
||||
|
||||
import { appendFileSync } from "node:fs";
|
||||
import { setTimeout as sleep } from "node:timers/promises";
|
||||
import { BACKGROUND_CONTEXT } from "../../../../src/context/index.ts";
|
||||
import { int, local, map, method } from "../../../../src/next/contracts/index.ts";
|
||||
import { defineState } from "../../../../src/next/documents/index.ts";
|
||||
import { createHost, defineFacet } from "../../../../src/next/host/index.ts";
|
||||
import * as chordNext from "../../../../src/next/index.ts";
|
||||
import { workerMain } from "../../../../src/next-node/workers/index.ts";
|
||||
import { Control, type WorkerBehaviour } from "./control.ts";
|
||||
|
||||
// Every IPC message this process sends, counted (a local call must send none).
|
||||
let sent = 0;
|
||||
const send = process.send!.bind(process) as (...args: unknown[]) => boolean;
|
||||
process.send = ((...args: unknown[]) => {
|
||||
sent++;
|
||||
return send(...args);
|
||||
}) as typeof process.send;
|
||||
|
||||
const Busy = local("test.busy", { version: 1, params: {}, members: { counts: map(int()), set: method(int()) } });
|
||||
const Echo = local("test.echo", { version: 1, params: {}, members: { echo: method(int(), int()) } });
|
||||
const Counts = defineState("counts", { schema: map(int()), version: 1, initial: () => ({ tasks: 0 }) });
|
||||
|
||||
/** The worker's host, set once made: a timer reaches its objects through it. */
|
||||
let host: ReturnType<typeof createHost>;
|
||||
|
||||
const core = defineFacet((facet) => {
|
||||
facet.serve(Busy, (_, self) => {
|
||||
const counts = self.state(Counts);
|
||||
return {
|
||||
counts: self.derive([counts], ([value]) => value),
|
||||
set: async (n, ctx) => {
|
||||
await counts.change((draft) => {
|
||||
draft.tasks = n;
|
||||
}, ctx);
|
||||
return null;
|
||||
},
|
||||
};
|
||||
});
|
||||
facet.serve(Echo, () => ({ echo: async (n) => n }));
|
||||
facet.serve(Control, (_params, _principal, self) => {
|
||||
const busy = self.get(Busy, {});
|
||||
const echo = self.get(Echo, {});
|
||||
return {
|
||||
variant: "main",
|
||||
members: {
|
||||
crash: async (code) => {
|
||||
setTimeout(() => process.exit(code), 10);
|
||||
return null;
|
||||
},
|
||||
fatal: async (message) => {
|
||||
setTimeout(() => {
|
||||
throw new Error(message);
|
||||
}, 10);
|
||||
return null;
|
||||
},
|
||||
block: async (ms) => {
|
||||
setTimeout(() => {
|
||||
const until = Date.now() + ms;
|
||||
while (Date.now() < until) {}
|
||||
}, 10);
|
||||
return null;
|
||||
},
|
||||
blockThenCrash: async (ms) => {
|
||||
setTimeout(() => {
|
||||
const until = Date.now() + ms;
|
||||
while (Date.now() < until) {}
|
||||
process.exit(9);
|
||||
}, 10);
|
||||
return null;
|
||||
},
|
||||
busy: async (ms, ctx) => {
|
||||
await busy.set(1, ctx);
|
||||
// After the call settled (its ctx sealed, maybe this object ended): through the host.
|
||||
setTimeout(async () => {
|
||||
const counts = await host.get(Busy, {}, BACKGROUND_CONTEXT);
|
||||
await counts.set(0, BACKGROUND_CONTEXT);
|
||||
counts.release();
|
||||
}, ms);
|
||||
return null;
|
||||
},
|
||||
localCalls: async (n, ctx) => {
|
||||
const before = sent;
|
||||
for (let i = 0; i < n; i++) await echo.echo(i, ctx);
|
||||
return sent - before;
|
||||
},
|
||||
},
|
||||
};
|
||||
});
|
||||
});
|
||||
|
||||
workerMain(
|
||||
async ({ address, start, parent, plugins, onClose }, ctx) => {
|
||||
const behaviour = start as WorkerBehaviour;
|
||||
if (behaviour.exit !== undefined) process.exit(behaviour.exit);
|
||||
if (behaviour.fatal !== undefined) throw new Error(behaviour.fatal);
|
||||
const worker = createHost({ address, principal: { kind: "worker" }, onError: () => {} });
|
||||
host = worker;
|
||||
onClose(async () => {
|
||||
if (behaviour.log !== undefined) appendFileSync(behaviour.log, "host\n");
|
||||
await worker.close(BACKGROUND_CONTEXT);
|
||||
});
|
||||
onClose(async () => {
|
||||
if (behaviour.closeMs !== undefined) await sleep(behaviour.closeMs);
|
||||
if (behaviour.log !== undefined) appendFileSync(behaviour.log, "store\n");
|
||||
});
|
||||
if (behaviour.connect === false) await sleep(1e9, undefined, { signal: ctx.abortSignal });
|
||||
worker.connect(parent, { principal: { kind: "server" }, trusted: true }, ctx);
|
||||
if (behaviour.loadMs !== undefined) await sleep(behaviour.loadMs, undefined, { signal: ctx.abortSignal });
|
||||
if (behaviour.hang === true) await sleep(1e9, undefined, { signal: ctx.abortSignal });
|
||||
await plugins.load(worker, { kind: "worker", core: [{ name: "@test/core", facet: core }] }, ctx);
|
||||
if (behaviour.disconnectAndBlock === true) {
|
||||
setTimeout(() => {
|
||||
// Blocks before `workerMain`'s own disconnect listener runs: the child lives on without its channel.
|
||||
process.prependOnceListener("disconnect", () => {
|
||||
const until = Date.now() + 20_000;
|
||||
while (Date.now() < until) {}
|
||||
});
|
||||
process.disconnect?.();
|
||||
}, 300);
|
||||
}
|
||||
const calls = behaviour.localCalls;
|
||||
if (calls !== undefined) {
|
||||
setTimeout(async () => {
|
||||
const echo = await worker.get(Echo, {}, BACKGROUND_CONTEXT);
|
||||
const before = sent;
|
||||
for (let i = 0; i < calls; i++) await echo.echo(i, BACKGROUND_CONTEXT);
|
||||
process.stdout.write(`local calls sent ${sent - before}\n`);
|
||||
echo.release();
|
||||
}, 300);
|
||||
}
|
||||
return { busy: (await worker.get(Busy, {}, ctx)).counts };
|
||||
},
|
||||
{
|
||||
hostPackages: { "@earendil-works/chord/next": chordNext },
|
||||
redact: (error) => ({
|
||||
message: String(error instanceof Error ? error.message : error).replace(/secret\S*/g, "[redacted]"),
|
||||
}),
|
||||
// Its errors on stderr, as an app logs them: the server keeps the tail private.
|
||||
onError: (error) => console.error(error),
|
||||
},
|
||||
);
|
||||
154
packages/chord/test/next-node/http/admission.test.ts
Normal file
154
packages/chord/test/next-node/http/admission.test.ts
Normal file
|
|
@ -0,0 +1,154 @@
|
|||
// Tests of admission (slice 10 §2, R4; security baseline (1)-(3), "Relays"): local only on a loopback connection with
|
||||
// a loopback `Host` and its port and no forwarded headers; allowed hosts as https; duplicate `Host`s refused; relays
|
||||
// verifying forwarded requests; foreign requests and `Origin` rules.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
admit,
|
||||
type Connection,
|
||||
isForeign,
|
||||
isNavigation,
|
||||
originAccepted,
|
||||
} from "../../../src/next-node/http/admission.ts";
|
||||
import { type Incoming, webIncoming } from "../../../src/next-node/http/incoming.ts";
|
||||
import type { HttpRelay } from "../../../src/next-node/http/router.ts";
|
||||
|
||||
const LOOPBACK: Connection = { localAddress: "127.0.0.1", localPort: 8788, remoteAddress: "127.0.0.1" };
|
||||
const allowed = new Set(["notes.example.com"]);
|
||||
|
||||
/** A request with `headers` (a list: duplicates kept), as the component reads it. */
|
||||
function request(headers: [string, string][] | Record<string, string>): Incoming {
|
||||
return webIncoming(new Request("http://x/", { headers }));
|
||||
}
|
||||
|
||||
/** What admission gives a request with `headers` on `connection`, without relays. */
|
||||
function admitted(headers: [string, string][], connection = LOOPBACK, relays: readonly HttpRelay[] = []) {
|
||||
return admit(request(headers), connection, allowed, relays);
|
||||
}
|
||||
|
||||
describe("provenance", () => {
|
||||
it("is local on a loopback connection with a loopback Host and its port (without it on port 80)", () => {
|
||||
for (const host of ["127.0.0.1:8788", "LOCALHOST:8788", "[::1]:8788"]) {
|
||||
expect(admitted([["host", host]])).toEqual({
|
||||
provenance: { kind: "local" },
|
||||
origin: `http://${host.toLowerCase()}`,
|
||||
framer: undefined,
|
||||
allowed: undefined,
|
||||
relay: undefined,
|
||||
});
|
||||
}
|
||||
expect(admitted([["host", "localhost"]], { localAddress: "::ffff:127.0.0.1", localPort: 80 })).toMatchObject({
|
||||
provenance: { kind: "local" },
|
||||
});
|
||||
});
|
||||
|
||||
// Opus's security audit H0: a loopback `Host` from a LAN address, or forwarded by a local proxy, was local.
|
||||
it("refuses a loopback Host on another connection, without a connection, or with forwarded headers", () => {
|
||||
const host: [string, string] = ["host", "127.0.0.1:8788"];
|
||||
expect(admitted([host], { localAddress: "192.168.1.5", localPort: 8788 })).toEqual({ kind: "refused" });
|
||||
expect(admitted([host], {})).toEqual({ kind: "refused" });
|
||||
expect(admitted([host], { localPort: 8788 })).toEqual({ kind: "refused" });
|
||||
for (const forwarded of ["x-forwarded-for", "x-forwarded-proto", "forwarded"]) {
|
||||
expect(admitted([host, [forwarded, "1"]])).toEqual({ kind: "refused" });
|
||||
}
|
||||
});
|
||||
|
||||
it("admits an allowed host as remote https; refuses another, another port, two Hosts, none", () => {
|
||||
expect(admitted([["host", "Notes.Example.com"]])).toEqual({
|
||||
provenance: { kind: "remote", host: "notes.example.com" },
|
||||
origin: "https://notes.example.com",
|
||||
framer: undefined,
|
||||
allowed: "notes.example.com",
|
||||
relay: undefined,
|
||||
});
|
||||
for (const headers of [
|
||||
[["host", "evil.com"]],
|
||||
[["host", "127.0.0.1:1"]],
|
||||
[["host", "localhost"]],
|
||||
[["host", "localhost.:8788"]],
|
||||
[["host", "notes.example.com:443"]],
|
||||
[
|
||||
["host", "127.0.0.1:8788"],
|
||||
["host", "notes.example.com"],
|
||||
],
|
||||
[
|
||||
["host", "notes.example.com"],
|
||||
["host", "127.0.0.1:8788"],
|
||||
],
|
||||
[],
|
||||
] as [string, string][][]) {
|
||||
expect(admitted(headers)).toEqual({ kind: "refused" });
|
||||
}
|
||||
});
|
||||
|
||||
it("admits a forwarded request on loopback that a relay verifies, with its origin and framer", () => {
|
||||
const radius: HttpRelay = {
|
||||
name: "radius",
|
||||
verify: (request) =>
|
||||
request.headers.get("x-relay-secret") === "s"
|
||||
? { origin: "https://me.radius.example", framer: "https://radius.example" }
|
||||
: undefined,
|
||||
};
|
||||
const broken: HttpRelay = {
|
||||
name: "broken",
|
||||
verify: () => {
|
||||
throw new Error("verify");
|
||||
},
|
||||
};
|
||||
const host: [string, string] = ["host", "127.0.0.1:8788"];
|
||||
const forwarded: [string, string] = ["x-forwarded-for", "203.0.113.9"];
|
||||
expect(admitted([host, forwarded, ["x-relay-secret", "s"]], LOOPBACK, [radius])).toEqual({
|
||||
provenance: { kind: "relayed", relay: "radius", origin: "https://me.radius.example" },
|
||||
origin: "https://me.radius.example",
|
||||
framer: "https://radius.example",
|
||||
allowed: undefined,
|
||||
relay: radius,
|
||||
});
|
||||
expect(admitted([host, forwarded, ["x-relay-secret", "wrong"]], LOOPBACK, [radius])).toEqual({ kind: "refused" });
|
||||
// Fable's recheck N2: a relay's origin reached the CSP unchecked (`; script-src *`).
|
||||
for (const bad of [
|
||||
{ origin: "https://a.example; script-src *" },
|
||||
{ origin: "https://a.example", framer: "javascript:x" },
|
||||
{ origin: "https://a.example/path" },
|
||||
]) {
|
||||
const lying: HttpRelay = { name: "lying", verify: () => bad };
|
||||
expect(admitted([host, forwarded], LOOPBACK, [lying])).toMatchObject({
|
||||
kind: "refused",
|
||||
failed: { relay: "lying" },
|
||||
});
|
||||
}
|
||||
const failed = admitted([host, forwarded], LOOPBACK, [broken]);
|
||||
expect(failed).toMatchObject({ kind: "refused", failed: { relay: "broken", error: { message: "verify" } } });
|
||||
});
|
||||
});
|
||||
|
||||
describe("foreign requests and Origin", () => {
|
||||
it("is foreign for another site by Fetch Metadata, or a frame not from this origin; not without the headers", () => {
|
||||
const headers = request;
|
||||
expect(isForeign(headers({ "sec-fetch-site": "cross-site" }))).toBe(true);
|
||||
expect(isForeign(headers({ "sec-fetch-site": "same-site" }))).toBe(true);
|
||||
expect(isForeign(headers({ "sec-fetch-site": "same-origin", "sec-fetch-dest": "iframe" }))).toBe(false);
|
||||
expect(isForeign(headers({ "sec-fetch-site": "none", "sec-fetch-dest": "iframe" }))).toBe(true);
|
||||
expect(isForeign(headers({ "sec-fetch-dest": "embed" }))).toBe(true);
|
||||
expect(isForeign(headers({ "sec-fetch-site": "same-origin" }))).toBe(false);
|
||||
expect(isForeign(headers({ "sec-fetch-site": "none" }))).toBe(false);
|
||||
expect(isForeign(headers({}))).toBe(false);
|
||||
expect(isNavigation("GET", headers({ "sec-fetch-mode": "navigate", "sec-fetch-dest": "document" }))).toBe(true);
|
||||
expect(isNavigation("POST", headers({ "sec-fetch-mode": "navigate", "sec-fetch-dest": "document" }))).toBe(false);
|
||||
expect(isNavigation("GET", headers({ "sec-fetch-mode": "navigate", "sec-fetch-dest": "empty" }))).toBe(true);
|
||||
expect(isNavigation("GET", headers({ "sec-fetch-mode": "navigate", "sec-fetch-dest": "iframe" }))).toBe(false);
|
||||
});
|
||||
|
||||
it("accepts an absent or equal Origin; an unsafe method from a browser needs it", () => {
|
||||
const origin = "http://127.0.0.1:8788";
|
||||
const accepted = (method: string, entries: Record<string, string>) =>
|
||||
originAccepted(method, request(entries), origin);
|
||||
expect(accepted("GET", {})).toBe(true);
|
||||
expect(accepted("POST", {})).toBe(true);
|
||||
expect(accepted("POST", { origin })).toBe(true);
|
||||
expect(accepted("GET", { origin: "http://evil.com" })).toBe(false);
|
||||
expect(accepted("POST", { origin: "null" })).toBe(false);
|
||||
expect(accepted("POST", { "sec-fetch-site": "same-origin" })).toBe(false);
|
||||
expect(accepted("GET", { "sec-fetch-site": "same-origin" })).toBe(true);
|
||||
});
|
||||
});
|
||||
58
packages/chord/test/next-node/http/authenticate.test.ts
Normal file
58
packages/chord/test/next-node/http/authenticate.test.ts
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
// Tests of the authenticators: asked in contributor order, the first principal wins, a throwing one reported and
|
||||
// skipped, none: anonymous.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { BACKGROUND_CONTEXT as context, withAbortSignal } from "../../../src/context/index.ts";
|
||||
import type { RegistryTable } from "../../../src/next/registries/index.ts";
|
||||
import { type Authenticator, authenticate, composeAuthenticators } from "../../../src/next-node/http/authenticate.ts";
|
||||
import type { HttpRequest } from "../../../src/next-node/http/index.ts";
|
||||
|
||||
const request = { headers: new Headers() } as HttpRequest;
|
||||
|
||||
describe("authenticators", () => {
|
||||
it("are asked in contributor order; the first principal wins; a throwing one is reported and skipped", async () => {
|
||||
const asked: string[] = [];
|
||||
const answer = (name: string, principal: { kind: string }) => {
|
||||
asked.push(name);
|
||||
return principal;
|
||||
};
|
||||
const table: RegistryTable<Authenticator> = [
|
||||
{ plugin: "@app/a", version: 1, value: () => void asked.push("a") },
|
||||
{
|
||||
plugin: "@app/b",
|
||||
version: 1,
|
||||
value: () => {
|
||||
asked.push("b");
|
||||
throw new Error("broken");
|
||||
},
|
||||
},
|
||||
{ plugin: "@app/c", version: 1, value: async () => answer("c", { kind: "owner" }) },
|
||||
{ plugin: "@app/d", version: 1, value: () => answer("d", { kind: "guest" }) },
|
||||
];
|
||||
const failed: [string, unknown][] = [];
|
||||
const principal = await authenticate(
|
||||
composeAuthenticators(table),
|
||||
request,
|
||||
(plugin, error) => failed.push([plugin, error]),
|
||||
context,
|
||||
);
|
||||
expect([principal, asked, failed]).toEqual([
|
||||
{ kind: "owner" },
|
||||
["a", "b", "c"],
|
||||
[["@app/b", new Error("broken")]],
|
||||
]);
|
||||
});
|
||||
|
||||
// Sol's second recheck C: a fresh authenticator started after the request was cancelled.
|
||||
it("start no authenticator once the context is aborted", async () => {
|
||||
let calls = 0;
|
||||
const table: RegistryTable<Authenticator> = [{ plugin: "@app/a", version: 1, value: () => void calls++ }];
|
||||
const aborted = withAbortSignal(AbortSignal.abort(), context);
|
||||
expect(await authenticate(composeAuthenticators(table), request, () => {}, aborted)).toBeUndefined();
|
||||
expect(calls).toBe(0);
|
||||
});
|
||||
|
||||
it("give no principal when none knows the request", async () => {
|
||||
expect(await authenticate([], request, () => {}, context)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
56
packages/chord/test/next-node/http/cookies.test.ts
Normal file
56
packages/chord/test/next-node/http/cookies.test.ts
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
// Tests of cookies (security baseline (5)): the safe defaults, `Secure` on a remote origin, a signature checked when
|
||||
// read back (a changed value, another name or secret reads nothing), and an expiry signed with it.
|
||||
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { Provenance } from "../../../src/next-node/http/admission.ts";
|
||||
import { cookie, type HttpRequest, readCookie } from "../../../src/next-node/http/index.ts";
|
||||
|
||||
/** A request carrying `header` as its `Cookie`, from `provenance`. */
|
||||
function requestWith(header: string, provenance: Provenance = { kind: "local" }): HttpRequest {
|
||||
return { headers: new Headers({ cookie: header }), provenance } as HttpRequest;
|
||||
}
|
||||
|
||||
describe("cookies", () => {
|
||||
it("are HttpOnly, Path=/, SameSite=Strict by default; Secure on a remote origin; Lax and Max-Age when asked", () => {
|
||||
const local = requestWith("");
|
||||
expect(cookie("session", "a b", { secret: "s", request: local })).toMatch(
|
||||
/^session=a%20b\.\.[A-Za-z0-9_-]{43}; HttpOnly; Path=\/; SameSite=Strict$/,
|
||||
);
|
||||
const remote = requestWith("", { kind: "remote", host: "notes.example.com" });
|
||||
expect(cookie("session", "x", { secret: "s", request: remote, sameSite: "Lax", maxAge: 60.5 })).toMatch(
|
||||
/; HttpOnly; Path=\/; SameSite=Lax; Secure; Max-Age=60$/,
|
||||
);
|
||||
});
|
||||
|
||||
it("read back only with their signature, name and secret", () => {
|
||||
const set = cookie("session", "owner:1", { secret: "s", request: requestWith("") });
|
||||
const value = set.split(";")[0]!;
|
||||
expect(readCookie(requestWith(`other=1; ${value}`), "session", "s")).toBe("owner:1");
|
||||
expect(readCookie(requestWith(value), "session", "t")).toBeUndefined();
|
||||
expect(readCookie(requestWith(value.replace("owner", "admin")), "session", "s")).toBeUndefined();
|
||||
expect(readCookie(requestWith(value.replace("session", "sess")), "sess", "s")).toBeUndefined();
|
||||
expect(readCookie(requestWith("session=unsigned"), "session", "s")).toBeUndefined();
|
||||
expect(readCookie(requestWith(""), "session", "s")).toBeUndefined();
|
||||
});
|
||||
|
||||
// Opus's recheck R3: a copied cookie stayed valid past its Max-Age.
|
||||
it("refuse a cookie past its signed expiry, and one whose expiry was changed", () => {
|
||||
vi.useFakeTimers({ toFake: ["Date"] });
|
||||
try {
|
||||
const set = cookie("session", "a.b", { secret: "s", request: requestWith(""), maxAge: 60 });
|
||||
const value = set.split(";")[0]!;
|
||||
expect(readCookie(requestWith(value), "session", "s")).toBe("a.b");
|
||||
const [payload = "", expires = "", signature = ""] = [
|
||||
value.slice(0, value.lastIndexOf(".", value.lastIndexOf(".") - 1)),
|
||||
value.split(".").at(-2),
|
||||
value.split(".").at(-1),
|
||||
];
|
||||
const extended = `${payload}.${Number(expires) + 3600}.${signature}`;
|
||||
expect(readCookie(requestWith(extended), "session", "s")).toBeUndefined();
|
||||
vi.advanceTimersByTime(61_000);
|
||||
expect(readCookie(requestWith(value), "session", "s")).toBeUndefined();
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
135
packages/chord/test/next-node/http/file.test.ts
Normal file
135
packages/chord/test/next-node/http/file.test.ts
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
// Tests of file responses (slice 10 §2): the type by extension (no active types), a `root` refusing a file whose real
|
||||
// path lies outside it, and the file handle opened by the first read and closed when the request's scope seals, on
|
||||
// every path (a full read, `HEAD`, a 416, a wrap replacing the response, a disconnect).
|
||||
|
||||
import { readdirSync, readlinkSync } from "node:fs";
|
||||
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
|
||||
import { request as httpRequest } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { fileResponse, HttpFailure } from "../../../src/next-node/http/index.ts";
|
||||
import { contributing, send, served } from "./support.ts";
|
||||
|
||||
const dirs: string[] = [];
|
||||
|
||||
afterEach(async () => {
|
||||
for (const dir of dirs.splice(0)) await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
/** A file of `size` bytes named `name` in a fresh directory. */
|
||||
async function file(name: string, size: number): Promise<string> {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-http-"));
|
||||
dirs.push(dir);
|
||||
const path = join(dir, name);
|
||||
await writeFile(path, Buffer.alloc(size, 97));
|
||||
return path;
|
||||
}
|
||||
|
||||
/** How many descriptors of this process are open on `path` (Linux). */
|
||||
function opened(path: string): number {
|
||||
let count = 0;
|
||||
for (const fd of readdirSync("/proc/self/fd")) {
|
||||
try {
|
||||
if (readlinkSync(`/proc/self/fd/${fd}`) === path) count++;
|
||||
} catch {
|
||||
// Closed meanwhile.
|
||||
}
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
describe("fileResponse", () => {
|
||||
// Fable's recheck N1: a missing file was a reported 500, one report per request from anyone.
|
||||
it("answers a missing file 404 (HttpFailure), with or without root", async () => {
|
||||
const dir = join(await file("a.txt", 1), "..");
|
||||
for (const options of [{}, { root: dir }]) {
|
||||
for (const name of ["missing.txt", "a.txt/below"]) {
|
||||
const refused = await fileResponse(join(dir, name), options).catch((error: unknown) => error);
|
||||
expect(refused).toBeInstanceOf(HttpFailure);
|
||||
expect((refused as HttpFailure).status).toBe(404);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Fable's security audit M1: a symbolic link inside a served directory reached any file.
|
||||
it("with a root, refuses a file whose real path lies outside it (403); serves one inside", async () => {
|
||||
const secret = await file("secret.txt", 3);
|
||||
const root = join(secret, "..", "pub");
|
||||
await mkdir(root);
|
||||
await writeFile(join(root, "a.txt"), "a");
|
||||
await symlink(secret, join(root, "link.txt"));
|
||||
await expect(fileResponse(join(root, "a.txt"), { root })).resolves.toMatchObject({ size: 1 });
|
||||
for (const name of ["link.txt", "../secret.txt"]) {
|
||||
const refused = await fileResponse(join(root, name), { root }).catch((error: unknown) => error);
|
||||
expect(refused).toBeInstanceOf(HttpFailure);
|
||||
expect((refused as HttpFailure).status).toBe(403);
|
||||
}
|
||||
await expect(fileResponse(join(root, "link.txt"))).resolves.toMatchObject({ size: 3 });
|
||||
});
|
||||
|
||||
it("types by extension: text, images, audio, video; never HTML or SVG unless named", async () => {
|
||||
const types = await Promise.all(
|
||||
["a.txt", "a.png", "a.mp3", "a.webm", "a.html", "a.svg", "a.bin"].map(async (name) => {
|
||||
const response = await fileResponse(await file(name, 1));
|
||||
return response.kind === "bytes" ? response.mime : response.kind;
|
||||
}),
|
||||
);
|
||||
expect(types).toEqual([
|
||||
"text/plain; charset=utf-8",
|
||||
"image/png",
|
||||
"audio/mpeg",
|
||||
"video/webm",
|
||||
"application/octet-stream",
|
||||
"application/octet-stream",
|
||||
"application/octet-stream",
|
||||
]);
|
||||
expect(await fileResponse(await file("b.svg", 1), { mime: "image/svg+xml" })).toMatchObject({
|
||||
mime: "image/svg+xml",
|
||||
});
|
||||
await expect(fileResponse(join(tmpdir(), "chord-http-missing"))).rejects.toThrow();
|
||||
});
|
||||
|
||||
it.skipIf(process.platform !== "linux")(
|
||||
"closes its handle on every path; opens none when nothing is written",
|
||||
async () => {
|
||||
const path = await file("big.bin", 50 * 1024 * 1024);
|
||||
const server = await served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"GET /f": async () => fileResponse(path, { ranges: true, download: "big.bin" }),
|
||||
"GET /wrapped": async () => fileResponse(path),
|
||||
},
|
||||
wraps: [
|
||||
{
|
||||
name: "replace",
|
||||
phase: "transform",
|
||||
route: "GET /wrapped",
|
||||
wrap: async () => ({ kind: "error", status: 403 }),
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
const { port } = server;
|
||||
const full = await send(port, "GET", "/f");
|
||||
expect([full.status, full.body.length, opened(path)]).toEqual([200, 50 * 1024 * 1024, 0]);
|
||||
expect([(await send(port, "HEAD", "/f")).status, opened(path)]).toEqual([200, 0]);
|
||||
const past = await send(port, "GET", "/f", { headers: { range: "bytes=999999999-" } });
|
||||
expect([past.status, opened(path)]).toEqual([416, 0]);
|
||||
expect([(await send(port, "GET", "/wrapped")).status, opened(path)]).toEqual([403, 0]);
|
||||
await new Promise<void>((resolve) => {
|
||||
const request = httpRequest({ host: "127.0.0.1", port, path: "/f", agent: false }, (response) => {
|
||||
response.once("data", () => {
|
||||
expect(opened(path)).toBe(1);
|
||||
request.destroy();
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
request.end();
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
expect(opened(path)).toBe(0);
|
||||
await server.close();
|
||||
},
|
||||
);
|
||||
});
|
||||
106
packages/chord/test/next-node/http/headers.test.ts
Normal file
106
packages/chord/test/next-node/http/headers.test.ts
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
// Tests of the default headers (security baseline (4), "A CSP nonce per response"): the fixed ones, `X-Frame-Options`
|
||||
// only without framers, the CSP with the response's nonce, the WebSocket origin, the framers (a relay's too) and
|
||||
// contributed sources; a fresh nonce each time; and headers checked as they are set, several lines for a list.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { RegistryTable } from "../../../src/next/registries/index.ts";
|
||||
import type { Admitted } from "../../../src/next-node/http/admission.ts";
|
||||
import { AnswerHeaders, DefaultHeaders, Nonces } from "../../../src/next-node/http/headers.ts";
|
||||
import { composeRouter, type HttpContribution } from "../../../src/next-node/http/router.ts";
|
||||
import { expectCollected } from "../../next/support.ts";
|
||||
|
||||
/** The router of one contribution. */
|
||||
function routerOf(value: HttpContribution) {
|
||||
const table: RegistryTable<HttpContribution> = [{ plugin: "@app/a", version: 1, value }];
|
||||
return composeRouter(table, { report: () => {} });
|
||||
}
|
||||
|
||||
/** The default headers of one component. */
|
||||
const defaults = new DefaultHeaders();
|
||||
|
||||
/** `defaults.of`. */
|
||||
function defaultHeaders(...args: Parameters<DefaultHeaders["of"]>) {
|
||||
return defaults.of(...args);
|
||||
}
|
||||
|
||||
const LOCAL: Admitted = {
|
||||
provenance: { kind: "local" },
|
||||
origin: "http://127.0.0.1:8788",
|
||||
framer: undefined,
|
||||
allowed: undefined,
|
||||
relay: undefined,
|
||||
};
|
||||
|
||||
describe("default headers", () => {
|
||||
it("refuse caching, sniffing, referrers and framing; the CSP allows scripts by nonce and the WebSocket origin", () => {
|
||||
const headers = defaultHeaders(LOCAL, routerOf({}), "N");
|
||||
expect([...headers.lines()]).toEqual([
|
||||
["Cache-Control", "no-store"],
|
||||
["X-Content-Type-Options", "nosniff"],
|
||||
["Referrer-Policy", "no-referrer"],
|
||||
["X-Frame-Options", "DENY"],
|
||||
[
|
||||
"Content-Security-Policy",
|
||||
"default-src 'none'; script-src 'self' 'nonce-N'; connect-src 'self' ws://127.0.0.1:8788; base-uri 'none'; form-action 'self'; frame-ancestors 'none'",
|
||||
],
|
||||
]);
|
||||
});
|
||||
|
||||
it("name the framers (a relay's included) without X-Frame-Options, and add contributed sources", () => {
|
||||
const router = routerOf({
|
||||
framers: ["https://a.example"],
|
||||
csp: { "img-src": ["data:"], "script-src": ["https://cdn.example"] },
|
||||
});
|
||||
const relayed = { ...LOCAL, origin: "https://me.relay.example", framer: "https://relay.example" };
|
||||
const headers = defaultHeaders(relayed, router, "N");
|
||||
expect(headers.has("x-frame-options")).toBe(false);
|
||||
expect(headers.get("content-security-policy")).toBe(
|
||||
"default-src 'none'; script-src 'self' 'nonce-N' https://cdn.example; connect-src 'self' wss://me.relay.example; base-uri 'none'; form-action 'self'; frame-ancestors https://a.example https://relay.example; img-src data:",
|
||||
);
|
||||
});
|
||||
|
||||
it("before admission: no WebSocket origin; a nonce is 16 fresh random bytes each time", () => {
|
||||
expect(defaultHeaders(undefined, undefined, "N").get("content-security-policy")).toContain("connect-src 'self';");
|
||||
const nonces = new Nonces();
|
||||
const made = new Set(Array.from({ length: 1_000 }, () => nonces.next()));
|
||||
expect(made.size).toBe(1_000);
|
||||
for (const nonce of made) expect(Buffer.from(nonce, "base64").length).toBe(16);
|
||||
});
|
||||
|
||||
it("keep the CSP of local and allowed-host requests per router; a relayed one's never", () => {
|
||||
const router = routerOf({});
|
||||
const first = defaultHeaders(LOCAL, router, "A").get("content-security-policy");
|
||||
expect(defaultHeaders(LOCAL, router, "B").get("content-security-policy")).toBe(
|
||||
first?.replace("'nonce-A'", "'nonce-B'"),
|
||||
);
|
||||
const relayed = { ...LOCAL, origin: "https://u1.relay.example", relay: { name: "r", verify: () => undefined } };
|
||||
expect(defaultHeaders(relayed, router, "C").get("content-security-policy")).toContain("wss://u1.relay.example");
|
||||
});
|
||||
|
||||
it("refuse a header name or value HTTP cannot carry; replace same-named ones; a list is one line per value", () => {
|
||||
const headers = new AnswerHeaders();
|
||||
expect(() => headers.set("X-Split", "a\r\nSet-Cookie: x=1")).toThrow(TypeError);
|
||||
expect(() => headers.set("X Bad", "1")).toThrow(TypeError);
|
||||
expect(() => headers.set("X-Nul", "a\0b")).toThrow(TypeError);
|
||||
expect(() => headers.set("Set-Cookie", ["a=1", "b=2\n"])).toThrow(TypeError);
|
||||
headers.set("content-security-policy", "a");
|
||||
headers.set("Content-Security-Policy", "b");
|
||||
headers.set("Set-Cookie", ["a=1", "b=2"]);
|
||||
expect([...headers.lines()]).toEqual([
|
||||
["Content-Security-Policy", "b"],
|
||||
["Set-Cookie", "a=1"],
|
||||
["Set-Cookie", "b=2"],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("GC", () => {
|
||||
it("keeps no policy of a router gone", async () => {
|
||||
const own = new DefaultHeaders();
|
||||
let router: ReturnType<typeof routerOf> | undefined = routerOf({ framers: ["https://a.example"] });
|
||||
own.of(LOCAL, router, "N");
|
||||
const ref = new WeakRef(router);
|
||||
router = undefined;
|
||||
await expectCollected(ref);
|
||||
});
|
||||
});
|
||||
818
packages/chord/test/next-node/http/http.test.ts
Normal file
818
packages/chord/test/next-node/http/http.test.ts
Normal file
|
|
@ -0,0 +1,818 @@
|
|||
// Tests of the HTTP component (slice 10 §2; decisions "The HTTP core is a fetch handler", "The HTTP server is a native
|
||||
// object", "The HTTP component's security baseline"): `fetch` waiting for the publication, the registries resolved
|
||||
// once, the listener opening after the publication (none without `listen`, a later change reported), allowed hosts from
|
||||
// contributions, authenticators, foreign requests and `Origin`, pages and cross-site routes, the default headers on the
|
||||
// component's own answers, throwing routes and wraps, a reload swapping a route while a request runs, `close` (also
|
||||
// while authenticating), a mounted handler, and requests' scopes collected.
|
||||
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { withAbortSignal } from "../../../src/context/index.ts";
|
||||
import { arr, local, method, str } from "../../../src/next/contracts/index.ts";
|
||||
import { ChordError } from "../../../src/next/data/index.ts";
|
||||
import { type Ctx, defineFacet } from "../../../src/next/host/index.ts";
|
||||
import { Authenticators, Http, type HttpContribution, HttpFailure } from "../../../src/next-node/http/index.ts";
|
||||
import { expectCollected } from "../../next/support.ts";
|
||||
import { context, contributing, send, served, type TestPrincipal } from "./support.ts";
|
||||
|
||||
/** A route answering who it runs as. */
|
||||
const whoami = contributing({
|
||||
routes: { "GET /whoami": async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }) },
|
||||
});
|
||||
|
||||
/** A loopback request to `fetch` directly, as a mounted server would make it. */
|
||||
function loopback(path: string, init: RequestInit & { headers?: Record<string, string> } = {}): Request {
|
||||
return new Request(`http://127.0.0.1:8788${path}`, {
|
||||
...init,
|
||||
headers: { host: "127.0.0.1:8788", ...init.headers },
|
||||
});
|
||||
}
|
||||
|
||||
/** Resolves with `value` after `ms`. */
|
||||
function sleepThen<T>(ms: number, value: T): Promise<T> {
|
||||
return new Promise((resolve) => setTimeout(() => resolve(value), ms));
|
||||
}
|
||||
|
||||
/** The connection of `loopback`'s requests. */
|
||||
const LOOPBACK = { localAddress: "127.0.0.1", localPort: 8788 };
|
||||
|
||||
describe("startup", () => {
|
||||
it("fetch waits for the publication, then serves; the registries are resolved once", async () => {
|
||||
const server = await served({ "@app/who": whoami }, { load: false });
|
||||
const registry = vi.spyOn(server.host, "registry");
|
||||
const early = server.http.fetch(loopback("/whoami"), LOOPBACK);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await server.load();
|
||||
expect(await (await early).text()).toBe("null");
|
||||
const calls = registry.mock.calls.length;
|
||||
for (let i = 0; i < 3; i++) await server.http.fetch(loopback("/whoami"), LOOPBACK);
|
||||
expect(registry.mock.calls.length).toBe(calls);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("answers 503 to a request waiting past the host's readyTimeout", async () => {
|
||||
const server = await served({}, { load: false, readyTimeout: 50 });
|
||||
expect((await server.http.fetch(loopback("/x"), LOOPBACK)).status).toBe(503);
|
||||
await server.load();
|
||||
expect((await server.http.fetch(loopback("/x"), LOOPBACK)).status).toBe(404);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("listens once published where its listen setting says; without it no listener", async () => {
|
||||
const server = await served({ "@app/who": whoami });
|
||||
expect(server.port).toBeGreaterThan(0);
|
||||
expect((await send(server.port, "GET", "/whoami")).status).toBe(200);
|
||||
await server.close();
|
||||
const mounted = await served({ "@app/who": whoami }, { config: { listen: undefined } });
|
||||
const listening = mounted.http.listening(context).then(
|
||||
() => "listening",
|
||||
() => "closed",
|
||||
);
|
||||
expect((await mounted.http.fetch(loopback("/whoami"), LOOPBACK)).status).toBe(200);
|
||||
await mounted.close();
|
||||
expect(await listening).toBe("closed");
|
||||
});
|
||||
});
|
||||
|
||||
describe("Host and principals", () => {
|
||||
it("refuses a Host neither loopback nor contributed (403, with the default headers); admits a contributed one", async () => {
|
||||
const server = await served({ "@app/who": whoami, "@app/hosts": contributing({ hosts: ["Notes.Example.com"] }) });
|
||||
const refused = await send(server.port, "GET", "/whoami", { headers: { host: "evil.com" } });
|
||||
expect([refused.status, refused.headers["x-content-type-options"], refused.headers["cache-control"]]).toEqual([
|
||||
403,
|
||||
"nosniff",
|
||||
"no-store",
|
||||
]);
|
||||
const through = await send(server.port, "GET", "/whoami", { headers: { host: "notes.example.com" } });
|
||||
expect(through.status).toBe(200);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("admits hosts a writable contribution adds and removes at runtime", async () => {
|
||||
const Tunnel = local("app.tunnel", { version: 1, params: {}, members: { name: method(arr(str())) } });
|
||||
const tunnel = defineFacet<TestPrincipal>((host) =>
|
||||
host.serve(Tunnel, (_, self) => {
|
||||
const contribution = self.writable<HttpContribution>({ hosts: [] });
|
||||
self.contribute(Http, contribution);
|
||||
return {
|
||||
name: async (hosts) => {
|
||||
contribution.set({ hosts });
|
||||
return null;
|
||||
},
|
||||
};
|
||||
}),
|
||||
);
|
||||
const server = await served({ "@app/tunnel": tunnel, "@app/who": whoami });
|
||||
const status = async () =>
|
||||
(await send(server.port, "GET", "/whoami", { headers: { host: "t.example.com" } })).status;
|
||||
expect(await status()).toBe(403);
|
||||
const handle = await server.host.get(Tunnel, {}, context);
|
||||
await handle.name(["t.example.com"], context);
|
||||
expect(await status()).toBe(200);
|
||||
await handle.name([], context);
|
||||
expect(await status()).toBe(403);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("runs a request without a principal anonymous; asks the authenticators in order, skipping a throwing one", async () => {
|
||||
const server = await served({
|
||||
"@app/who": whoami,
|
||||
"@app/a-broken": defineFacet<TestPrincipal>((_host, self) =>
|
||||
self.contribute(Authenticators, () => {
|
||||
throw new Error("broken");
|
||||
}),
|
||||
),
|
||||
});
|
||||
expect((await send(server.port, "GET", "/whoami")).body.toString()).toBe("null");
|
||||
const owner = await send(server.port, "GET", "/whoami", { headers: { "x-principal": "owner" } });
|
||||
expect(owner.body.toString()).toBe('{"kind":"owner"}');
|
||||
expect(server.errors).toMatchObject([
|
||||
{ kind: "hook_failed", hook: "authenticate", plugin: "@app/a-broken" },
|
||||
{ kind: "hook_failed", hook: "authenticate", plugin: "@app/a-broken" },
|
||||
]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Opus's security audit H1: a cross-site form posted with the owner's credentials.
|
||||
it("never authenticates a foreign request; refuses a foreign Origin and an unsafe browser request without one", async () => {
|
||||
let deleted = 0;
|
||||
const server = await served({
|
||||
"@app/notes": contributing({
|
||||
routes: {
|
||||
"POST /delete": async (_request, ctx) => {
|
||||
if (ctx.principal?.kind === "owner") deleted++;
|
||||
return { kind: "json", value: deleted };
|
||||
},
|
||||
"GET /whoami": async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }),
|
||||
},
|
||||
}),
|
||||
});
|
||||
const owner = { "x-principal": "owner" };
|
||||
const own = `http://127.0.0.1:${server.port}`;
|
||||
const crossSite = { ...owner, "sec-fetch-site": "cross-site" };
|
||||
expect(
|
||||
(await send(server.port, "POST", "/delete", { headers: { ...crossSite, origin: "https://evil.example" } }))
|
||||
.status,
|
||||
).toBe(403);
|
||||
expect(
|
||||
(await send(server.port, "POST", "/delete", { headers: { ...owner, "sec-fetch-site": "same-origin" } }))
|
||||
.status,
|
||||
).toBe(403);
|
||||
expect((await send(server.port, "GET", "/whoami", { headers: crossSite })).body.toString()).toBe("null");
|
||||
const same = await send(server.port, "POST", "/delete", {
|
||||
headers: { ...owner, origin: own, "sec-fetch-site": "same-origin" },
|
||||
});
|
||||
expect([same.status, same.body.toString()]).toEqual([200, "1"]);
|
||||
// Without Fetch Metadata (a script, the TUI): not a browser's request.
|
||||
expect((await send(server.port, "POST", "/delete", { headers: owner })).body.toString()).toBe("2");
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("authenticates a top-level navigation to a page from another site; a cross-site route runs anonymous", async () => {
|
||||
const seen: unknown[] = [];
|
||||
const server = await served({
|
||||
"@app/pages": contributing({
|
||||
routes: {
|
||||
"GET /app": {
|
||||
page: true,
|
||||
handler: async (_request, ctx) => {
|
||||
seen.push(ctx.principal ?? null);
|
||||
return { kind: "html", body: "app" };
|
||||
},
|
||||
},
|
||||
"POST /hook": {
|
||||
crossSite: true,
|
||||
handler: async (_request, ctx) => {
|
||||
seen.push(ctx.principal ?? null);
|
||||
return { kind: "json", value: 1 };
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
const navigate = {
|
||||
"x-principal": "owner",
|
||||
"sec-fetch-site": "cross-site",
|
||||
"sec-fetch-mode": "navigate",
|
||||
"sec-fetch-dest": "document",
|
||||
};
|
||||
await send(server.port, "GET", "/app", { headers: navigate });
|
||||
await send(server.port, "GET", "/app", { headers: { ...navigate, "sec-fetch-dest": "iframe" } });
|
||||
const hook = await send(server.port, "POST", "/hook", {
|
||||
headers: { "x-principal": "owner", origin: "https://github.com", "sec-fetch-site": "cross-site" },
|
||||
});
|
||||
expect([hook.status, seen]).toEqual([200, [{ kind: "owner" }, null, null]]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("default headers", () => {
|
||||
it("sends the baseline on every answer, a CSP with the response's own nonce and the contributed framers and sources", async () => {
|
||||
let nonce = "";
|
||||
const server = await served({
|
||||
"@app/page": contributing({
|
||||
routes: {
|
||||
"GET /page": async (request) => {
|
||||
nonce = request.nonce;
|
||||
return { kind: "html", body: `<script nonce="${request.nonce}"></script>` };
|
||||
},
|
||||
},
|
||||
framers: ["https://shell.example.com"],
|
||||
csp: { "img-src": ["https://cdn.example.com"] },
|
||||
}),
|
||||
});
|
||||
const page = await send(server.port, "GET", "/page");
|
||||
expect(page.headers).toMatchObject({
|
||||
"cache-control": "no-store",
|
||||
"x-content-type-options": "nosniff",
|
||||
"referrer-policy": "no-referrer",
|
||||
});
|
||||
expect(page.headers["x-frame-options"]).toBeUndefined();
|
||||
expect(page.headers["content-security-policy"]).toBe(
|
||||
`default-src 'none'; script-src 'self' 'nonce-${nonce}'; connect-src 'self' ws://127.0.0.1:${server.port}; base-uri 'none'; form-action 'self'; frame-ancestors https://shell.example.com; img-src https://cdn.example.com`,
|
||||
);
|
||||
// Fable's recheck: a nonce read from one page must not run a script injected into another.
|
||||
const first = nonce;
|
||||
await send(server.port, "GET", "/page");
|
||||
expect(nonce).not.toBe(first);
|
||||
const missing = await send(server.port, "GET", "/nope");
|
||||
expect([missing.status, missing.headers["x-content-type-options"]]).toEqual([404, "nosniff"]);
|
||||
expect(missing.headers["content-security-policy"]).toMatch(/'nonce-[A-Za-z0-9+/]{22}=='/);
|
||||
expect(missing.headers["content-security-policy"]).not.toContain(nonce);
|
||||
await server.close();
|
||||
const plain = await served({});
|
||||
expect((await send(plain.port, "GET", "/")).headers["x-frame-options"]).toBe("DENY");
|
||||
await plain.close();
|
||||
});
|
||||
|
||||
it("lets a relay's verified request through with its framer", async () => {
|
||||
const server = await served({
|
||||
"@app/who": whoami,
|
||||
"@app/relay": contributing({
|
||||
relays: [
|
||||
{
|
||||
name: "radius",
|
||||
verify: (request) =>
|
||||
request.headers.get("x-relay") === "secret"
|
||||
? { origin: "https://me.relay.example", framer: "https://relay.example" }
|
||||
: undefined,
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
const forwarded = { "x-forwarded-for": "203.0.113.9" };
|
||||
expect((await send(server.port, "GET", "/whoami", { headers: forwarded })).status).toBe(403);
|
||||
const relayed = await send(server.port, "GET", "/whoami", { headers: { ...forwarded, "x-relay": "secret" } });
|
||||
expect(relayed.status).toBe(200);
|
||||
expect(relayed.headers["content-security-policy"]).toContain("frame-ancestors https://relay.example");
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("routes and wraps failing", () => {
|
||||
it("answers a throwing route 500 without its message, a throwing wrap 500 naming its plugin, an HttpFailure its status", async () => {
|
||||
const server = await served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"GET /throws": async () => {
|
||||
throw new Error("secret");
|
||||
},
|
||||
"GET /failure": async () => {
|
||||
throw new HttpFailure(422, "no");
|
||||
},
|
||||
},
|
||||
}),
|
||||
"@app/b": contributing({
|
||||
routes: { "GET /wrapped": async () => ({ kind: "json", value: 1 }) },
|
||||
wraps: [
|
||||
{
|
||||
name: "broken",
|
||||
phase: "headers",
|
||||
wrap: async () => {
|
||||
throw new Error("wrap");
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
const thrown = await send(server.port, "GET", "/throws");
|
||||
expect([thrown.status, thrown.body.toString()]).toEqual([500, "Internal Server Error"]);
|
||||
expect((await send(server.port, "GET", "/failure")).status).toBe(422);
|
||||
expect((await send(server.port, "GET", "/wrapped")).status).toBe(500);
|
||||
expect(server.errors).toMatchObject([
|
||||
{ kind: "route_failed", pattern: "GET /throws", plugin: "@app/a", error: { message: "secret" } },
|
||||
{ kind: "wrap_failed", name: "broken", plugin: "@app/b", error: { message: "wrap" } },
|
||||
]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("runs wraps outermost first around the route, in its request's scope", async () => {
|
||||
const order: string[] = [];
|
||||
const server = await served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"GET /r": async (_request, ctx) => {
|
||||
order.push(`route as ${ctx.principal?.kind ?? "anonymous"}`);
|
||||
return { kind: "json", value: 1 };
|
||||
},
|
||||
},
|
||||
wraps: [
|
||||
{
|
||||
name: "inner",
|
||||
phase: "transform",
|
||||
wrap: async (request, next) => {
|
||||
order.push("transform");
|
||||
return next(request);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "outer",
|
||||
phase: "observe",
|
||||
wrap: async (request, next) => {
|
||||
order.push("observe");
|
||||
const response = await next(request);
|
||||
return { ...response, headers: { "X-Observed": "1" } };
|
||||
},
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
const answer = await send(server.port, "GET", "/r", { headers: { "x-principal": "owner" } });
|
||||
expect([order, answer.headers["x-observed"]]).toEqual([["observe", "transform", "route as owner"], "1"]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("reloads and close", () => {
|
||||
it("a reload swapping a route leaves a running request on the old router; the next uses the new one", async () => {
|
||||
const gate = Promise.withResolvers<void>();
|
||||
const old = contributing({
|
||||
routes: {
|
||||
"GET /v": async () => {
|
||||
await gate.promise;
|
||||
return { kind: "html", body: "old" };
|
||||
},
|
||||
},
|
||||
});
|
||||
const server = await served({ "@app/v": old });
|
||||
const running = send(server.port, "GET", "/v");
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const next = contributing({ routes: { "GET /v": async () => ({ kind: "html", body: "new" }) } });
|
||||
await server.host.reload(server.facets({ "@app/v": next }), context);
|
||||
expect((await send(server.port, "GET", "/v")).body.toString()).toBe("new");
|
||||
gate.resolve();
|
||||
expect((await running).body.toString()).toBe("old");
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("close refuses new requests (503), waits for running ones up to closeTimeout, then seals them (503)", async () => {
|
||||
let sealed: AbortSignal | undefined;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/slow": contributing({
|
||||
routes: {
|
||||
"GET /quick": async () => {
|
||||
await new Promise((resolve) => setTimeout(resolve, 30));
|
||||
return { kind: "html", body: "done" };
|
||||
},
|
||||
"GET /stuck": async (_request, ctx: Ctx) => {
|
||||
sealed = ctx.abortSignal;
|
||||
return new Promise(() => {});
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: { host: "127.0.0.1", port: 0 }, closeTimeout: 100 } },
|
||||
);
|
||||
const quick = send(server.port, "GET", "/quick");
|
||||
const stuck = send(server.port, "GET", "/stuck");
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
const closing = server.http.close(context);
|
||||
expect((await server.http.fetch(loopback("/quick"), LOOPBACK)).status).toBe(503);
|
||||
expect((await quick).body.toString()).toBe("done");
|
||||
await closing;
|
||||
expect(sealed?.aborted).toBe(true);
|
||||
await server.host.close(context);
|
||||
expect((await stuck).status).toBe(503);
|
||||
await expect(send(server.port, "GET", "/quick")).rejects.toThrow();
|
||||
});
|
||||
|
||||
// Astra's build review: a request authenticating when close began ran its route afterwards.
|
||||
it("never runs a route for a request still authenticating when close sealed it", async () => {
|
||||
const entered = Promise.withResolvers<void>();
|
||||
const gate = Promise.withResolvers<void>();
|
||||
let calls = 0;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/r": contributing({
|
||||
routes: {
|
||||
"GET /": async () => {
|
||||
calls++;
|
||||
return { kind: "json", value: 1 };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{
|
||||
config: { listen: { host: "127.0.0.1", port: 0 }, closeTimeout: 20 },
|
||||
authenticate: async () => {
|
||||
entered.resolve();
|
||||
await gate.promise;
|
||||
return { kind: "owner" };
|
||||
},
|
||||
},
|
||||
);
|
||||
const answer = server.http.fetch(loopback("/"), LOOPBACK);
|
||||
await entered.promise;
|
||||
await server.http.close(context);
|
||||
gate.resolve();
|
||||
expect([(await answer).status, calls]).toEqual([503, 0]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("mounting", () => {
|
||||
it("without a connection a request is never local", async () => {
|
||||
const server = await served({ "@app/who": whoami }, { config: { listen: undefined } });
|
||||
expect((await server.http.fetch(loopback("/whoami"))).status).toBe(403);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("cancellation", () => {
|
||||
// Sol's second recheck C: after cancellation the chain still started the next authenticator.
|
||||
it("starts no further authenticator once close sealed the request", async () => {
|
||||
const entered = Promise.withResolvers<void>();
|
||||
const gate = Promise.withResolvers<void>();
|
||||
let later = 0;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }),
|
||||
"@app/z-later": defineFacet<TestPrincipal>((_host, self) =>
|
||||
self.contribute(Authenticators, () => {
|
||||
later++;
|
||||
return { kind: "owner" };
|
||||
}),
|
||||
),
|
||||
},
|
||||
{
|
||||
config: { listen: undefined, closeTimeout: 10 },
|
||||
authenticate: async () => {
|
||||
entered.resolve();
|
||||
await gate.promise;
|
||||
return undefined;
|
||||
},
|
||||
},
|
||||
);
|
||||
const answer = server.http.fetch(loopback("/"), LOOPBACK);
|
||||
await entered.promise;
|
||||
await server.http.close(context);
|
||||
expect((await answer).status).toBe(503);
|
||||
gate.resolve();
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(later).toBe(0);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("asks no authenticator for a request aborted before it arrived", async () => {
|
||||
let calls = 0;
|
||||
const server = await served(
|
||||
{ "@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }) },
|
||||
{
|
||||
config: { listen: undefined },
|
||||
authenticate: () => {
|
||||
calls++;
|
||||
return undefined;
|
||||
},
|
||||
},
|
||||
);
|
||||
await server.http.fetch(loopback("/", { signal: AbortSignal.abort() }), LOOPBACK);
|
||||
expect(calls).toBe(0);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Sol's third recheck: a request aborted while its route returned the `bytes` description got a stream born sealed,
|
||||
// which never errored; the request never finished.
|
||||
it.each(["inside the route", "after the route returned"] as const)(
|
||||
"ends a request whose bytes Response was born sealed (aborted %s)",
|
||||
async (when) => {
|
||||
const abort = new AbortController();
|
||||
const entered = Promise.withResolvers<void>();
|
||||
let reads = 0;
|
||||
let bornSealed: boolean | undefined;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/b": contributing({
|
||||
routes: {
|
||||
"GET /": async (_request, ctx) => {
|
||||
entered.resolve();
|
||||
if (when === "inside the route") abort.abort(new Error("client gone"));
|
||||
bornSealed = ctx.abortSignal?.aborted;
|
||||
return {
|
||||
kind: "bytes",
|
||||
mime: "application/octet-stream",
|
||||
read: async () => {
|
||||
reads++;
|
||||
return new Uint8Array([1]);
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined, closeTimeout: 1_000 } },
|
||||
);
|
||||
const answer = server.http.fetch(loopback("/", { signal: abort.signal }), LOOPBACK);
|
||||
await entered.promise;
|
||||
if (when === "after the route returned") abort.abort(new Error("client gone"));
|
||||
await answer;
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
const started = Date.now();
|
||||
await server.http.close(context);
|
||||
expect([Date.now() - started < 500, reads, bornSealed]).toEqual([true, 0, when === "inside the route"]);
|
||||
await server.close();
|
||||
},
|
||||
);
|
||||
|
||||
// Sol's second recheck B: an aborted `Response` nobody touched again kept its request: close waited its bound.
|
||||
it("ends a request whose bytes Response was aborted and dropped without another read", async () => {
|
||||
const server = await served(
|
||||
{
|
||||
"@app/b": contributing({
|
||||
routes: {
|
||||
"GET /": async () => ({
|
||||
kind: "bytes",
|
||||
mime: "application/octet-stream",
|
||||
read: async () => new Uint8Array([1]),
|
||||
}),
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined, closeTimeout: 1_000 } },
|
||||
);
|
||||
const abort = new AbortController();
|
||||
await server.http.fetch(loopback("/", { signal: abort.signal }), LOOPBACK);
|
||||
abort.abort(new Error("client gone"));
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
const started = Date.now();
|
||||
await server.http.close(context);
|
||||
expect(Date.now() - started).toBeLessThan(500);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Astra's recheck R4, Sol's N1: a close before the first resolution started a stopped watch; its throw ended the process.
|
||||
it("a close before the publication leaves nothing running: the waiting request answers 503", async () => {
|
||||
const server = await served({}, { load: false, config: { listen: undefined } });
|
||||
const waiting = server.http.fetch(loopback("/x"), LOOPBACK);
|
||||
await server.http.close(withAbortSignal(AbortSignal.abort(), context));
|
||||
await server.load();
|
||||
expect((await waiting).status).toBe(503);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("a close while the first request resolves the registries starts no follow", async () => {
|
||||
const server = await served({ "@app/who": whoami }, { config: { listen: undefined, closeTimeout: 10 } });
|
||||
const pending = server.http.fetch(loopback("/whoami"), LOOPBACK);
|
||||
await server.http.close(withAbortSignal(AbortSignal.timeout(20), context));
|
||||
expect([200, 503]).toContain((await pending).status);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Sol's recheck N9.
|
||||
it("a close whose context is aborted already does not wait", async () => {
|
||||
const entered = Promise.withResolvers<void>();
|
||||
const gate = Promise.withResolvers<void>();
|
||||
const server = await served(
|
||||
{
|
||||
"@app/wait": contributing({
|
||||
routes: {
|
||||
"GET /wait": async () => {
|
||||
entered.resolve();
|
||||
await gate.promise;
|
||||
return { kind: "html", body: "done" };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined, closeTimeout: 1000 } },
|
||||
);
|
||||
const answer = server.http.fetch(loopback("/wait"), LOOPBACK);
|
||||
await entered.promise;
|
||||
const started = Date.now();
|
||||
await server.http.close(withAbortSignal(AbortSignal.abort(), context));
|
||||
expect(Date.now() - started).toBeLessThan(500);
|
||||
gate.resolve();
|
||||
await answer;
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Sol's recheck N5: an authenticator ignoring its signal kept fetch pending after close returned.
|
||||
it("close settles a request whose authenticator ignores its signal (503)", async () => {
|
||||
const entered = Promise.withResolvers<void>();
|
||||
const gate = Promise.withResolvers<void>();
|
||||
const server = await served(
|
||||
{ "@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }) },
|
||||
{
|
||||
config: { listen: undefined, closeTimeout: 10 },
|
||||
authenticate: async () => {
|
||||
entered.resolve();
|
||||
await gate.promise;
|
||||
return { kind: "owner" };
|
||||
},
|
||||
},
|
||||
);
|
||||
const answer = server.http.fetch(loopback("/"), LOOPBACK);
|
||||
await entered.promise;
|
||||
await server.http.close(context);
|
||||
const settled = await Promise.race([answer.then((response) => response.status), sleepThen(30, "pending")]);
|
||||
expect(settled).toBe(503);
|
||||
gate.resolve();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Astra's recheck R2, Sol's N3: a held `Response` went on calling `read` after its scope was sealed.
|
||||
it.each(["the request aborted", "the component closed"] as const)("reads no bytes after %s", async (kind) => {
|
||||
let reads = 0;
|
||||
let readAfterSeal = false;
|
||||
const abort = new AbortController();
|
||||
const server = await served(
|
||||
{
|
||||
"@app/bytes": contributing({
|
||||
routes: {
|
||||
"GET /stream": async () => ({
|
||||
kind: "bytes",
|
||||
mime: "application/octet-stream",
|
||||
read: async (_offset, _length, ctx) => {
|
||||
reads++;
|
||||
readAfterSeal ||= ctx.abortSignal?.aborted ?? false;
|
||||
return new Uint8Array([reads]);
|
||||
},
|
||||
}),
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined, closeTimeout: 10 } },
|
||||
);
|
||||
const response = await server.http.fetch(loopback("/stream", { signal: abort.signal }), LOOPBACK);
|
||||
const reader = response.body!.getReader();
|
||||
await reader.read();
|
||||
if (kind === "the request aborted") abort.abort(new Error("client gone"));
|
||||
else await server.http.close(context);
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
await expect(reader.read()).rejects.toThrow();
|
||||
expect({ reads, readAfterSeal }).toEqual({ reads: 1, readAfterSeal: false });
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Sol's recheck N4.
|
||||
it("a body that errors after valid JSON fails the request (400); the route's side effect never runs", async () => {
|
||||
let writes = 0;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/body": contributing({
|
||||
routes: {
|
||||
"POST /body": async (request) => {
|
||||
const value = await request.json();
|
||||
writes++;
|
||||
return { kind: "json", value };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined } },
|
||||
);
|
||||
let pulls = 0;
|
||||
const body = new ReadableStream<Uint8Array>({
|
||||
pull(controller) {
|
||||
if (pulls++ === 0) controller.enqueue(new TextEncoder().encode("{}"));
|
||||
else controller.error(new Error("incoming body checksum failed"));
|
||||
},
|
||||
});
|
||||
const init = { method: "POST", body, duplex: "half" as const };
|
||||
const response = await server.http.fetch(loopback("/body", init), LOOPBACK);
|
||||
expect([response.status, writes]).toEqual([400, 0]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("answers", () => {
|
||||
// Sol's recheck N8: `Response` threw on a body for these statuses.
|
||||
it.each([204, 205, 304])("answers %s without a body", async (status) => {
|
||||
const server = await served(
|
||||
{ "@app/s": contributing({ routes: { "GET /": async () => ({ kind: "json", value: null, status }) } }) },
|
||||
{ config: { listen: undefined } },
|
||||
);
|
||||
const response = await server.http.fetch(loopback("/"), LOOPBACK);
|
||||
expect([response.status, response.body, response.headers.has("content-length")]).toEqual([status, null, false]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Fable's recheck N3: the kernel's `forbidden` reached the client as a reported 500.
|
||||
it("answers a forbidden escaping a route 403, unreported", async () => {
|
||||
const server = await served(
|
||||
{
|
||||
"@app/r": contributing({
|
||||
routes: {
|
||||
"GET /": async () => {
|
||||
throw new ChordError({ code: "forbidden", message: "no" });
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined } },
|
||||
);
|
||||
expect((await server.http.fetch(loopback("/"), LOOPBACK)).status).toBe(403);
|
||||
expect(server.errors).toEqual([]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Opus's recheck R2: one value per header name allowed one cookie per response.
|
||||
it("sends several Set-Cookie headers", async () => {
|
||||
const server = await served({
|
||||
"@app/r": contributing({
|
||||
routes: { "GET /": async () => ({ kind: "json", value: 1, headers: { "Set-Cookie": ["a=1", "b=2"] } }) },
|
||||
}),
|
||||
});
|
||||
expect((await send(server.port, "GET", "/")).headers["set-cookie"]).toEqual(["a=1", "b=2"]);
|
||||
const fetched = await server.http.fetch(
|
||||
new Request(`http://127.0.0.1:${server.port}/`, { headers: { host: `127.0.0.1:${server.port}` } }),
|
||||
{ localAddress: "127.0.0.1", localPort: server.port },
|
||||
);
|
||||
expect(fetched.headers.getSetCookie()).toEqual(["a=1", "b=2"]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("GC", () => {
|
||||
// Sol's second recheck B, her GC probe: 8 of 8 such requests kept their scopes.
|
||||
it("collects the scopes of aborted bytes Responses dropped without another read", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const server = await served(
|
||||
{
|
||||
"@app/b": contributing({
|
||||
routes: {
|
||||
"GET /": async (_request, ctx) => {
|
||||
refs.push(new WeakRef(ctx));
|
||||
return { kind: "bytes", mime: "application/octet-stream", read: async () => new Uint8Array([1]) };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined } },
|
||||
);
|
||||
for (let i = 0; i < 8; i++) {
|
||||
const abort = new AbortController();
|
||||
await server.http.fetch(loopback("/", { signal: abort.signal }), LOOPBACK);
|
||||
abort.abort(new Error("client gone"));
|
||||
}
|
||||
await expectCollected(...refs);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("collects the scopes of bytes Responses born sealed (aborted while the route returned)", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const aborts: AbortController[] = [];
|
||||
const server = await served(
|
||||
{
|
||||
"@app/b": contributing({
|
||||
routes: {
|
||||
"GET /": async (_request, ctx) => {
|
||||
refs.push(new WeakRef(ctx));
|
||||
aborts.at(-1)?.abort(new Error("client gone"));
|
||||
return { kind: "bytes", mime: "application/octet-stream", read: async () => new Uint8Array([1]) };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: undefined } },
|
||||
);
|
||||
// In a function of its own: the test's frame keeps no last request.
|
||||
const fetchAborted = async (): Promise<void> => {
|
||||
const abort = new AbortController();
|
||||
aborts.push(abort);
|
||||
await server.http.fetch(loopback("/", { signal: abort.signal }), LOOPBACK);
|
||||
};
|
||||
for (let i = 0; i < 8; i++) await fetchAborted();
|
||||
aborts.length = 0;
|
||||
await expectCollected(...refs);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("collects requests' scopes and requests once answered", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const server = await served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"GET /r": async (request, ctx) => {
|
||||
refs.push(new WeakRef(ctx), new WeakRef(request));
|
||||
return { kind: "json", value: 1 };
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
for (let i = 0; i < 5; i++) await send(server.port, "GET", "/r");
|
||||
await expectCollected(...refs);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
215
packages/chord/test/next-node/http/node.test.ts
Normal file
215
packages/chord/test/next-node/http/node.test.ts
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
// Tests of Chord's own Node server (the listener's native): request targets (origin-form only; a malformed or
|
||||
// double-slash path never crashes or reroutes), raw `Host` headers, Web requests made only when read, the request's scope lasting until its response is
|
||||
// flushed (and `close` waiting for it), and a client's disconnect aborting the scope.
|
||||
|
||||
import { once } from "node:events";
|
||||
import { createServer, request as httpRequest, type IncomingMessage } from "node:http";
|
||||
import { connect } from "node:net";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { nodeRequest } from "../../../src/next-node/http/index.ts";
|
||||
import { context, contributing, served } from "./support.ts";
|
||||
|
||||
/** What `port` answers a raw request `text` with: its status line and body. */
|
||||
async function raw(port: number, text: string): Promise<{ readonly status: string; readonly body: string }> {
|
||||
const socket = connect(port, "127.0.0.1");
|
||||
let out = "";
|
||||
socket.on("data", (chunk) => {
|
||||
out += chunk.toString("latin1");
|
||||
});
|
||||
socket.on("error", () => {});
|
||||
await once(socket, "connect");
|
||||
socket.write(text);
|
||||
await once(socket, "close");
|
||||
const [head = "", body = ""] = out.split("\r\n\r\n");
|
||||
return { status: head.split("\r\n")[0] ?? "", body };
|
||||
}
|
||||
|
||||
/** A route answering its path and params, under `GET /who` and `GET /n/:name`. */
|
||||
const echo = contributing({
|
||||
routes: {
|
||||
"GET /who": async (request) => ({ kind: "json", value: request.path }),
|
||||
"GET /n/:name": async (request) => ({ kind: "json", value: request.params }),
|
||||
},
|
||||
});
|
||||
|
||||
describe("targets", () => {
|
||||
// Astra's build review, Fable's security audit C1: `GET //[` threw out of the request handler and ended the process.
|
||||
it("answers a malformed or double-slash path without crashing or rerouting it (Opus's audit H5)", async () => {
|
||||
const server = await served({ "@app/echo": echo });
|
||||
const host = `Host: 127.0.0.1:${server.port}`;
|
||||
const get = (target: string) =>
|
||||
raw(server.port, `GET ${target} HTTP/1.1\r\n${host}\r\nConnection: close\r\n\r\n`);
|
||||
expect((await get("//[")).status).toBe("HTTP/1.1 404 Not Found");
|
||||
expect((await get("//evil.example/who")).status).toBe("HTTP/1.1 404 Not Found");
|
||||
expect(await get("/a/../who")).toEqual({ status: "HTTP/1.1 200 OK", body: '"/who"' });
|
||||
// The URL parser resolves `%2e%2e` as a dot segment: `/n/%2e%2e` is `/`.
|
||||
expect((await get("/n/%2e%2e")).status).toBe("HTTP/1.1 404 Not Found");
|
||||
expect((await get("/n/a%5Cb")).status).toBe("HTTP/1.1 400 Bad Request");
|
||||
expect((await get("/n/..%2F..%2Fsecret")).status).toBe("HTTP/1.1 400 Bad Request");
|
||||
expect((await get("/who")).status).toBe("HTTP/1.1 200 OK");
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("refuses an absolute-form target, two Host headers and none (403)", async () => {
|
||||
const server = await served({ "@app/echo": echo });
|
||||
const host = `Host: 127.0.0.1:${server.port}`;
|
||||
const answers = await Promise.all([
|
||||
raw(server.port, `GET http://evil.example/who HTTP/1.1\r\n${host}\r\nConnection: close\r\n\r\n`),
|
||||
raw(server.port, `GET /who HTTP/1.1\r\n${host}\r\nHost: evil.example\r\nConnection: close\r\n\r\n`),
|
||||
raw(server.port, "GET /who HTTP/1.0\r\n\r\n"),
|
||||
]);
|
||||
expect(answers.map(({ status }) => status)).toEqual([
|
||||
"HTTP/1.1 403 Forbidden",
|
||||
"HTTP/1.1 403 Forbidden",
|
||||
"HTTP/1.1 403 Forbidden",
|
||||
]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("refuses a Host with userinfo (403) and a method a Web Request cannot carry (501)", async () => {
|
||||
const server = await served({ "@app/echo": echo });
|
||||
const answers = await Promise.all([
|
||||
raw(server.port, `GET /who HTTP/1.1\r\nHost: attacker@127.0.0.1:${server.port}\r\nConnection: close\r\n\r\n`),
|
||||
raw(server.port, `TRACE /who HTTP/1.1\r\nHost: 127.0.0.1:${server.port}\r\nConnection: close\r\n\r\n`),
|
||||
]);
|
||||
// `TRACK` never reaches Chord: Node's parser refuses it (400); `CONNECT` goes to Node's `connect` event.
|
||||
expect(answers.map(({ status }) => status)).toEqual(["HTTP/1.1 403 Forbidden", "HTTP/1.1 501 Not Implemented"]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("mounting on an app's own Node server", () => {
|
||||
// Sol's recheck N6: a hand-made `Request` threw unhandled on `Host: [` or an absolute-form target (the process
|
||||
// ended) and kept only the first of two `Host`s.
|
||||
it("converts with the listener's checks, never throwing: bad targets, userinfo, TRACE refused; duplicate Hosts kept", async () => {
|
||||
const server = await served({ "@app/echo": echo }, { config: { listen: undefined } });
|
||||
const own = createServer(async (message, res) => {
|
||||
const converted = nodeRequest(message, res);
|
||||
if (converted.kind === "refused") return void res.writeHead(converted.status).end();
|
||||
const response = await server.http.fetch(converted.request, converted.connection);
|
||||
for (const [name, value] of response.headers) res.appendHeader(name, value);
|
||||
res.writeHead(response.status).end(Buffer.from(await response.arrayBuffer()));
|
||||
});
|
||||
await new Promise<void>((resolve) => own.listen(0, "127.0.0.1", resolve));
|
||||
const address = own.address();
|
||||
const port = typeof address === "object" && address !== null ? address.port : 0;
|
||||
const host = `Host: 127.0.0.1:${port}`;
|
||||
const get = (text: string) => raw(port, `${text}Connection: close\r\n\r\n`);
|
||||
expect(await get(`GET /who HTTP/1.1\r\n${host}\r\n`)).toEqual({ status: "HTTP/1.1 200 OK", body: '"/who"' });
|
||||
expect((await get("GET /who HTTP/1.1\r\nHost: [\r\n")).status).toBe("HTTP/1.1 400 Bad Request");
|
||||
expect((await get(`GET http://evil.example/who HTTP/1.1\r\n${host}\r\n`)).status).toBe("HTTP/1.1 403 Forbidden");
|
||||
expect((await get(`GET /who HTTP/1.1\r\n${host}\r\nHost: evil.example\r\n`)).status).toBe(
|
||||
"HTTP/1.1 403 Forbidden",
|
||||
);
|
||||
// Sol's second recheck A: both threw out of the conversion; the app's callback ended the process.
|
||||
expect((await get(`TRACE /who HTTP/1.1\r\n${host}\r\n`)).status).toBe("HTTP/1.1 501 Not Implemented");
|
||||
expect((await get(`GET /who HTTP/1.1\r\nHost: attacker@127.0.0.1:${port}\r\n`)).status).toBe(
|
||||
"HTTP/1.1 403 Forbidden",
|
||||
);
|
||||
expect((await get(`GET /who HTTP/1.1\r\n${host}\r\n`)).status).toBe("HTTP/1.1 200 OK");
|
||||
await new Promise((resolve) => own.close(resolve));
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("Web requests", () => {
|
||||
it("makes a Web Request only when something reads the headers or the body (here: no authenticator does)", async () => {
|
||||
const server = await served(
|
||||
{
|
||||
"@app/r": contributing({
|
||||
routes: {
|
||||
"GET /plain": async (request) => ({ kind: "json", value: request.path }),
|
||||
"GET /headers": async (request) => ({ kind: "json", value: request.headers.get("x-thing") }),
|
||||
"POST /body": async (request) => ({ kind: "json", value: await request.json() }),
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ authenticate: () => undefined },
|
||||
);
|
||||
const original = globalThis.Request;
|
||||
let made = 0;
|
||||
globalThis.Request = class extends original {
|
||||
constructor(...args: ConstructorParameters<typeof Request>) {
|
||||
super(...args);
|
||||
made++;
|
||||
}
|
||||
};
|
||||
try {
|
||||
const host = `Host: 127.0.0.1:${server.port}\r\nConnection: close`;
|
||||
await raw(server.port, `GET /plain HTTP/1.1\r\n${host}\r\n\r\n`);
|
||||
expect(made).toBe(0);
|
||||
const read = await raw(server.port, `GET /headers HTTP/1.1\r\n${host}\r\nX-Thing: t\r\n\r\n`);
|
||||
const posted = await raw(server.port, `POST /body HTTP/1.1\r\n${host}\r\nContent-Length: 3\r\n\r\n"a"`);
|
||||
expect([read.body, posted.body, made]).toEqual(['"t"', '"a"', 2]);
|
||||
} finally {
|
||||
globalThis.Request = original;
|
||||
}
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("the request's scope", () => {
|
||||
// Sol's build review: the scope sealed, and `close` stopped waiting, before a large response was flushed.
|
||||
it("lasts until the response is flushed; close waits for it", async () => {
|
||||
const size = 32 * 1024 * 1024;
|
||||
let signal: AbortSignal | undefined;
|
||||
const server = await served(
|
||||
{
|
||||
"@app/large": contributing({
|
||||
routes: {
|
||||
"GET /large": async (_request, ctx) => {
|
||||
signal = ctx.abortSignal;
|
||||
return { kind: "html", body: "x".repeat(size) };
|
||||
},
|
||||
},
|
||||
}),
|
||||
},
|
||||
{ config: { listen: { host: "127.0.0.1", port: 0 }, closeTimeout: 2000 } },
|
||||
);
|
||||
const request = httpRequest({ host: "127.0.0.1", port: server.port, path: "/large", agent: false });
|
||||
const response = await new Promise<IncomingMessage>((resolve) => {
|
||||
request.on("response", (arrived: IncomingMessage) => {
|
||||
arrived.pause();
|
||||
resolve(arrived);
|
||||
});
|
||||
request.end();
|
||||
});
|
||||
let length = 0;
|
||||
response.on("data", (chunk: Buffer) => {
|
||||
length += chunk.length;
|
||||
});
|
||||
response.pause();
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
const sealedBeforeFlush = signal?.aborted;
|
||||
const closing = server.http.close(context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
response.resume();
|
||||
await once(response, "end");
|
||||
await closing;
|
||||
expect({ sealedBeforeFlush, length }).toEqual({ sealedBeforeFlush: false, length: size });
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("is aborted when the client disconnects", async () => {
|
||||
const entered = Promise.withResolvers<AbortSignal>();
|
||||
const server = await served({
|
||||
"@app/slow": contributing({
|
||||
routes: {
|
||||
"GET /slow": async (_request, ctx) => {
|
||||
entered.resolve(ctx.abortSignal!);
|
||||
return new Promise(() => {});
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
const request = httpRequest({ host: "127.0.0.1", port: server.port, path: "/slow", agent: false });
|
||||
request.on("error", () => {});
|
||||
request.end();
|
||||
const signal = await entered.promise;
|
||||
const aborted = new Promise((resolve) => signal.addEventListener("abort", resolve, { once: true }));
|
||||
request.destroy();
|
||||
await aborted;
|
||||
expect(signal.aborted).toBe(true);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
119
packages/chord/test/next-node/http/request.test.ts
Normal file
119
packages/chord/test/next-node/http/request.test.ts
Normal file
|
|
@ -0,0 +1,119 @@
|
|||
// Tests of requests (slice 10 §2, D4): the summary a route gets, `json` and `body` with their limits and statuses
|
||||
// (400, 413, an `HttpFailure` escaping a route answering its status), and a body nobody read (drained or closed).
|
||||
|
||||
import { Agent, request as httpRequest } from "node:http";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { HttpRequest } from "../../../src/next-node/http/index.ts";
|
||||
import { contributing, send, served } from "./support.ts";
|
||||
|
||||
describe("the request a route gets", () => {
|
||||
it("has its method, path, params, query, headers, origin, provenance and the response's nonce", async () => {
|
||||
let seen: HttpRequest | undefined;
|
||||
const server = await served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"GET /notes/:id": async (request) => {
|
||||
seen = request;
|
||||
return { kind: "json", value: null };
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
await send(server.port, "GET", "/notes/a%20b?x=1&x=2", { headers: { "X-Thing": "t" } });
|
||||
expect(seen).toMatchObject({
|
||||
method: "GET",
|
||||
path: "/notes/a%20b",
|
||||
params: { id: "a b" },
|
||||
origin: `http://127.0.0.1:${server.port}`,
|
||||
provenance: { kind: "local" },
|
||||
});
|
||||
expect([seen?.query.getAll("x"), seen?.headers.get("x-thing")]).toEqual([["1", "2"], "t"]);
|
||||
expect(seen?.nonce).toMatch(/^[A-Za-z0-9+/]{22}==$/);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("bodies", () => {
|
||||
/** A server whose `POST /json` answers the JSON it read (limit 16), `POST /body` the bytes it read (limit 16). */
|
||||
const reading = () =>
|
||||
served({
|
||||
"@app/a": contributing({
|
||||
routes: {
|
||||
"POST /json": async (request) => ({ kind: "json", value: await request.json(16) }),
|
||||
"POST /body": async (request) => {
|
||||
let bytes = 0;
|
||||
for await (const chunk of request.body(16)) bytes += chunk.byteLength;
|
||||
return { kind: "json", value: bytes };
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
|
||||
it("parses JSON within its limit; malformed is 400, larger is 413 (declared or streamed)", async () => {
|
||||
const server = await reading();
|
||||
const { port } = server;
|
||||
expect((await send(port, "POST", "/json", { body: '{"a":1}' })).body.toString()).toBe('{"a":1}');
|
||||
expect((await send(port, "POST", "/json", { body: "{nope" })).status).toBe(400);
|
||||
expect((await send(port, "POST", "/json", { body: JSON.stringify({ long: "x".repeat(40) }) })).status).toBe(413);
|
||||
const chunked = await send(port, "POST", "/body", {
|
||||
headers: { "transfer-encoding": "chunked" },
|
||||
body: "y".repeat(40),
|
||||
});
|
||||
expect(chunked.status).toBe(413);
|
||||
expect((await send(port, "POST", "/body", { body: "y".repeat(10) })).body.toString()).toBe("10");
|
||||
expect(server.errors).toEqual([]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Sol's and Astra's build reviews: an unfinished chunked upload past the limit got no answer.
|
||||
it("answers 413 before a chunked upload past the limit has ended", async () => {
|
||||
const server = await reading();
|
||||
const request = httpRequest({
|
||||
host: "127.0.0.1",
|
||||
port: server.port,
|
||||
path: "/json",
|
||||
method: "POST",
|
||||
headers: { "transfer-encoding": "chunked" },
|
||||
agent: false,
|
||||
});
|
||||
request.on("error", () => {});
|
||||
const status = new Promise<number>((resolve) => {
|
||||
request.on("response", (response) => {
|
||||
response.resume();
|
||||
resolve(response.statusCode ?? 0);
|
||||
});
|
||||
});
|
||||
request.write(`"${"x".repeat(40)}`);
|
||||
expect(await status).toBe(413);
|
||||
request.destroy();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("drains a small body nobody read and keeps the connection; a large one closes it", async () => {
|
||||
const server = await served({
|
||||
"@app/a": contributing({ routes: { "POST /ignore": async () => ({ kind: "json", value: 1 }) } }),
|
||||
});
|
||||
const agent = new Agent({ keepAlive: true, maxSockets: 1 });
|
||||
const post = (body: string) =>
|
||||
new Promise<{ connection: string | undefined; socket: unknown }>((resolve, reject) => {
|
||||
const request = httpRequest(
|
||||
{ host: "127.0.0.1", port: server.port, method: "POST", path: "/ignore", agent },
|
||||
(response) => {
|
||||
response.resume();
|
||||
response.on("end", () =>
|
||||
resolve({ connection: response.headers.connection, socket: request.socket }),
|
||||
);
|
||||
},
|
||||
);
|
||||
request.on("error", reject);
|
||||
request.end(body);
|
||||
});
|
||||
const first = await post("x".repeat(1000));
|
||||
const second = await post("x".repeat(1000));
|
||||
expect(first.connection).toBe("keep-alive");
|
||||
expect(second.socket).toBe(first.socket);
|
||||
expect((await post("x".repeat(100_000))).connection).toBe("close");
|
||||
agent.destroy();
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
245
packages/chord/test/next-node/http/response.test.ts
Normal file
245
packages/chord/test/next-node/http/response.test.ts
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
// Tests of responses (slice 10 §2, D5, R6; security baseline (4), (6)): every kind written by a running server, the
|
||||
// response's headers over the default ones (an invalid one reported, 500), ranges (206, 416, multi-range), `HEAD`,
|
||||
// attachments and inline bodies, safe download names, bodies without `size`, a short or throwing read, a disconnect
|
||||
// stopping the reads.
|
||||
|
||||
import { request as httpRequest } from "node:http";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { Ctx } from "../../../src/next/host/index.ts";
|
||||
import type { HttpResponse } from "../../../src/next-node/http/index.ts";
|
||||
import { contributing, send, served } from "./support.ts";
|
||||
|
||||
/** A server answering `GET /r` with `response`. */
|
||||
async function answering(response: HttpResponse) {
|
||||
return served({ "@app/r": contributing({ routes: { "GET /r": async () => response } }) });
|
||||
}
|
||||
|
||||
/** A `bytes` response over `data` that records its reads. */
|
||||
function bytesOf(data: Buffer, extra: Partial<Extract<HttpResponse, { kind: "bytes" }>> = {}) {
|
||||
const reads: [number, number][] = [];
|
||||
const response: HttpResponse = {
|
||||
kind: "bytes",
|
||||
mime: "application/octet-stream",
|
||||
size: data.length,
|
||||
read: async (offset, length) => {
|
||||
reads.push([offset, length]);
|
||||
return data.subarray(offset, offset + length);
|
||||
},
|
||||
...extra,
|
||||
};
|
||||
return { response, reads };
|
||||
}
|
||||
|
||||
describe("descriptions", () => {
|
||||
it("writes json, html, redirect and error, the default headers first, the response's replacing same-named ones", async () => {
|
||||
const cases: [HttpResponse, number, string, string | undefined][] = [
|
||||
[{ kind: "json", value: { a: 1 } }, 200, '{"a":1}', "application/json; charset=utf-8"],
|
||||
[
|
||||
{ kind: "html", body: "<p>x</p>", status: 201, headers: { "content-security-policy": "own" } },
|
||||
201,
|
||||
"<p>x</p>",
|
||||
"text/html; charset=utf-8",
|
||||
],
|
||||
[{ kind: "redirect", to: "/next", status: 303 }, 303, "", undefined],
|
||||
[{ kind: "error", status: 418 }, 418, "I'm a Teapot", "text/plain; charset=utf-8"],
|
||||
[{ kind: "error", status: 409, message: "taken" }, 409, "taken", "text/plain; charset=utf-8"],
|
||||
];
|
||||
for (const [response, status, body, type] of cases) {
|
||||
const server = await answering(response);
|
||||
const answer = await send(server.port, "GET", "/r");
|
||||
expect([answer.status, answer.body.toString(), answer.headers["content-type"]]).toEqual([status, body, type]);
|
||||
expect([answer.headers["cache-control"], answer.headers["x-content-type-options"]]).toEqual([
|
||||
"no-store",
|
||||
"nosniff",
|
||||
]);
|
||||
expect(answer.headers["content-security-policy"]).toMatch(
|
||||
response.headers === undefined ? /^default-src 'none'/ : /^own$/,
|
||||
);
|
||||
if (response.kind === "redirect") expect(answer.headers.location).toBe("/next");
|
||||
await server.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("answers 500 for a header with a CR or LF and a value that is no JSON, reported response_failed", async () => {
|
||||
const server = await served({
|
||||
"@app/r": contributing({
|
||||
routes: {
|
||||
"GET /header": async () => ({ kind: "json", value: 1, headers: { "X-Split": "a\r\nSet-Cookie: x=1" } }),
|
||||
"GET /bigint": async () => ({ kind: "json", value: { n: 1n } as never }),
|
||||
},
|
||||
}),
|
||||
});
|
||||
for (const path of ["/header", "/bigint"]) {
|
||||
const answer = await send(server.port, "GET", path);
|
||||
expect([answer.status, answer.headers["set-cookie"], answer.headers["x-content-type-options"]]).toEqual([
|
||||
500,
|
||||
undefined,
|
||||
"nosniff",
|
||||
]);
|
||||
}
|
||||
expect(server.errors).toMatchObject([
|
||||
{ kind: "response_failed", pattern: "GET /header" },
|
||||
{ kind: "response_failed", pattern: "GET /bigint" },
|
||||
]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("bytes", () => {
|
||||
const data = Buffer.from("0123456789".repeat(60_000));
|
||||
|
||||
it("sends Content-Length, reads 256 KiB pieces, answers a single range 206 and an unsatisfiable one 416", async () => {
|
||||
const { response, reads } = bytesOf(data, { ranges: true });
|
||||
const server = await answering(response);
|
||||
const whole = await send(server.port, "GET", "/r");
|
||||
expect([whole.status, whole.headers["content-length"], whole.headers["accept-ranges"]]).toEqual([
|
||||
200,
|
||||
"600000",
|
||||
"bytes",
|
||||
]);
|
||||
expect(whole.body.equals(data)).toBe(true);
|
||||
expect(reads).toEqual([
|
||||
[0, 262144],
|
||||
[262144, 262144],
|
||||
[524288, 75712],
|
||||
]);
|
||||
const range = await send(server.port, "GET", "/r", { headers: { range: "bytes=10-19" } });
|
||||
expect([range.status, range.body.toString(), range.headers["content-range"]]).toEqual([
|
||||
206,
|
||||
"0123456789",
|
||||
"bytes 10-19/600000",
|
||||
]);
|
||||
const suffix = await send(server.port, "GET", "/r", { headers: { range: "bytes=-3" } });
|
||||
expect([suffix.status, suffix.body.toString()]).toEqual([206, "789"]);
|
||||
const past = await send(server.port, "GET", "/r", { headers: { range: "bytes=600000-" } });
|
||||
expect([past.status, past.headers["content-range"]]).toEqual([416, "bytes */600000"]);
|
||||
const multi = await send(server.port, "GET", "/r", { headers: { range: "bytes=0-1,5-6" } });
|
||||
expect([multi.status, multi.body.length]).toEqual([200, 600000]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("ignores ranges unless asked; HEAD sends headers only and reads nothing", async () => {
|
||||
const { response, reads } = bytesOf(data);
|
||||
const server = await answering(response);
|
||||
const ranged = await send(server.port, "GET", "/r", { headers: { range: "bytes=0-1" } });
|
||||
expect([ranged.status, ranged.headers["accept-ranges"]]).toEqual([200, undefined]);
|
||||
reads.length = 0;
|
||||
const head = await send(server.port, "HEAD", "/r");
|
||||
expect([head.status, head.headers["content-length"], head.body.length, reads]).toEqual([200, "600000", 0, []]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("names a download with an ASCII fallback and the UTF-8 name, control and bidi characters dropped", async () => {
|
||||
const server = await answering(bytesOf(Buffer.from("x"), { download: 'Grüße "1"\r\n\u202etxt.exe' }).response);
|
||||
const answer = await send(server.port, "GET", "/r");
|
||||
expect(answer.headers["content-disposition"]).toBe(
|
||||
`attachment; filename="Gr__e _1_txt.exe"; filename*=UTF-8''Gr%C3%BC%C3%9Fe%20%221%22txt.exe`,
|
||||
);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("is an attachment unless inline; an inline one gets a sandbox CSP", async () => {
|
||||
const attached = await answering(bytesOf(Buffer.from("x")).response);
|
||||
expect((await send(attached.port, "GET", "/r")).headers["content-disposition"]).toBe("attachment");
|
||||
await attached.close();
|
||||
const shown = await answering(bytesOf(Buffer.from("x"), { inline: true }).response);
|
||||
const answer = await send(shown.port, "GET", "/r");
|
||||
expect(answer.headers["content-disposition"]).toBe("inline");
|
||||
expect(answer.headers["content-security-policy"]).toMatch(/; sandbox$/);
|
||||
await shown.close();
|
||||
});
|
||||
|
||||
it("refuses a foreign request (another site's) 403, reading nothing", async () => {
|
||||
const { response, reads } = bytesOf(Buffer.from("secret"));
|
||||
const server = await answering(response);
|
||||
const foreign = await send(server.port, "GET", "/r", { headers: { "sec-fetch-site": "cross-site" } });
|
||||
expect([foreign.status, reads]).toEqual([403, []]);
|
||||
expect((await send(server.port, "GET", "/r", { headers: { "sec-fetch-site": "same-origin" } })).status).toBe(200);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("without a size: chunked, read with growing offsets until an empty chunk, no ranges (R6)", async () => {
|
||||
const pieces = [Buffer.from("ab"), Buffer.from("cde"), Buffer.alloc(0)];
|
||||
const offsets: number[] = [];
|
||||
const server = await answering({
|
||||
kind: "bytes",
|
||||
mime: "text/plain",
|
||||
ranges: true,
|
||||
read: async (offset) => {
|
||||
offsets.push(offset);
|
||||
return pieces.shift()!;
|
||||
},
|
||||
});
|
||||
const answer = await send(server.port, "GET", "/r", { headers: { range: "bytes=0-1" } });
|
||||
expect([
|
||||
answer.status,
|
||||
answer.body.toString(),
|
||||
answer.headers["transfer-encoding"],
|
||||
answer.headers["accept-ranges"],
|
||||
]).toEqual([200, "abcde", "chunked", undefined]);
|
||||
expect(offsets).toEqual([0, 2, 5]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("destroys the connection on a short or a throwing read, reported response_failed", async () => {
|
||||
const server = await served({
|
||||
"@app/r": contributing({
|
||||
routes: {
|
||||
"GET /short": async () => ({
|
||||
kind: "bytes",
|
||||
mime: "text/plain",
|
||||
size: 10,
|
||||
read: async () => Buffer.from("abc"),
|
||||
}),
|
||||
// Astra's build review: a throwing read hung the response, unreported.
|
||||
"GET /throws": async () => ({
|
||||
kind: "bytes",
|
||||
mime: "text/plain",
|
||||
size: 10,
|
||||
read: async () => {
|
||||
throw new Error("read failed");
|
||||
},
|
||||
}),
|
||||
},
|
||||
}),
|
||||
});
|
||||
await expect(send(server.port, "GET", "/short")).rejects.toThrow();
|
||||
await expect(send(server.port, "GET", "/throws")).rejects.toThrow();
|
||||
expect(server.errors).toMatchObject([
|
||||
{ kind: "response_failed", pattern: "GET /short" },
|
||||
{ kind: "response_failed", pattern: "GET /throws", error: { message: "read failed" } },
|
||||
]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("stops reading when the client disconnects: the read's context is aborted", async () => {
|
||||
let reads = 0;
|
||||
let signal: AbortSignal | undefined;
|
||||
const server = await answering({
|
||||
kind: "bytes",
|
||||
mime: "application/octet-stream",
|
||||
size: 1024 * 1024 * 1024,
|
||||
read: async (_offset, length, ctx: Ctx) => {
|
||||
reads++;
|
||||
signal = ctx.abortSignal;
|
||||
return Buffer.alloc(length);
|
||||
},
|
||||
});
|
||||
await new Promise<void>((resolve) => {
|
||||
const request = httpRequest({ host: "127.0.0.1", port: server.port, path: "/r", agent: false }, (response) => {
|
||||
response.once("data", () => {
|
||||
request.destroy();
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
request.end();
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
const after = reads;
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
expect(signal?.aborted).toBe(true);
|
||||
expect(reads).toBe(after);
|
||||
expect(after).toBeLessThan(1000);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
228
packages/chord/test/next-node/http/router.test.ts
Normal file
228
packages/chord/test/next-node/http/router.test.ts
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
// Tests of the router (slice 10 §2, D2, D8): patterns, decoding (one rule for `:name` and rest segments), precedence,
|
||||
// `HEAD` per path, each route's own param names, route options, conflicts, wraps resolved by phase, contributor order,
|
||||
// scope and route, allowed hosts, framers, relays, CSP sources, sockets.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { RegistryConflict, RegistryTable } from "../../../src/next/registries/index.ts";
|
||||
import {
|
||||
composeRouter,
|
||||
type HttpContribution,
|
||||
type HttpRoute,
|
||||
type HttpWrap,
|
||||
type RouteMatch,
|
||||
type Router,
|
||||
} from "../../../src/next-node/http/router.ts";
|
||||
|
||||
/** A route named `name` (its handler answers its name). */
|
||||
const route =
|
||||
(name: string): HttpRoute =>
|
||||
async () => ({ kind: "html", body: name });
|
||||
|
||||
/** The router of `entries` (plugin, contribution), with the conflicts it reported. */
|
||||
function compose(...entries: [string, HttpContribution][]): { router: Router; conflicts: RegistryConflict[] } {
|
||||
const conflicts: RegistryConflict[] = [];
|
||||
const table: RegistryTable<HttpContribution> = entries.map(([plugin, value]) => ({ plugin, version: 1, value }));
|
||||
return { router: composeRouter(table, { report: (conflict) => conflicts.push(conflict) }), conflicts };
|
||||
}
|
||||
|
||||
/** The matched route's pattern and params, or what `route` answered. */
|
||||
function matched(router: Router, method: string, path: string) {
|
||||
const match = router.route(method, path);
|
||||
if (match === undefined || match === "bad_path") return match;
|
||||
return { pattern: match.pattern, params: match.params };
|
||||
}
|
||||
|
||||
describe("patterns and precedence (D8)", () => {
|
||||
const { router } = compose([
|
||||
"@app/a",
|
||||
{
|
||||
routes: {
|
||||
"GET /notes": route("list"),
|
||||
"GET /notes/:id": route("one"),
|
||||
"GET /notes/new": route("new"),
|
||||
"GET /notes/:id/export": route("export"),
|
||||
"GET /files/*": route("files"),
|
||||
"GET /files/special/:name": route("special"),
|
||||
"GET /": route("root"),
|
||||
},
|
||||
},
|
||||
]);
|
||||
|
||||
it("matches exact segments, :name and a trailing *, the most specific from the left", () => {
|
||||
expect(matched(router, "GET", "/notes")).toEqual({ pattern: "GET /notes", params: {} });
|
||||
expect(matched(router, "GET", "/notes/new")).toEqual({ pattern: "GET /notes/new", params: {} });
|
||||
expect(matched(router, "GET", "/notes/7")).toEqual({ pattern: "GET /notes/:id", params: { id: "7" } });
|
||||
expect(matched(router, "GET", "/notes/7/export")).toMatchObject({ params: { id: "7" } });
|
||||
expect(matched(router, "GET", "/files/special/x")).toEqual({
|
||||
pattern: "GET /files/special/:name",
|
||||
params: { name: "x" },
|
||||
});
|
||||
// The exact branch fails deeper: the rest takes it.
|
||||
expect(matched(router, "GET", "/files/special/x/y")).toEqual({
|
||||
pattern: "GET /files/*",
|
||||
params: { "*": "special/x/y" },
|
||||
});
|
||||
expect(matched(router, "GET", "/")).toEqual({ pattern: "GET /", params: {} });
|
||||
});
|
||||
|
||||
it("a /* matches its prefix with an empty rest; :name needs a non-empty segment", () => {
|
||||
expect(matched(router, "GET", "/files")).toEqual({ pattern: "GET /files/*", params: { "*": "" } });
|
||||
expect(matched(router, "GET", "/notes/")).toBeUndefined();
|
||||
});
|
||||
|
||||
// Opus's build review F4, security audit H5: `%2F` in `:name` reached a file path.
|
||||
it("percent-decodes :name and rest segments; one decoding to /, \\, NUL, . or .. is a bad path", () => {
|
||||
expect(matched(router, "GET", "/notes/a%20b")).toMatchObject({ params: { id: "a b" } });
|
||||
expect(matched(router, "GET", "/files/a%20b/c")).toMatchObject({ params: { "*": "a b/c" } });
|
||||
for (const bad of ["a%2Fb", "..%2F..%2Fsecret", "a%5Cb", "a%00b", "%2e%2e", "%2E", ".", ".."]) {
|
||||
expect(router.route("GET", `/notes/${bad}`)).toBe("bad_path");
|
||||
expect(router.route("GET", `/files/x/${bad}`)).toBe("bad_path");
|
||||
}
|
||||
expect(router.route("GET", "/notes/%E0%A4%A")).toBe("bad_path");
|
||||
});
|
||||
|
||||
it("falls back to the GET route for HEAD; another method or path has none", () => {
|
||||
expect(matched(router, "HEAD", "/notes")).toEqual({ pattern: "GET /notes", params: {} });
|
||||
expect(router.route("POST", "/notes")).toBeUndefined();
|
||||
expect(router.route("GET", "/nope")).toBeUndefined();
|
||||
});
|
||||
|
||||
// Opus's build review F2, Astra's: one explicit HEAD route anywhere hid every GET fallback.
|
||||
it("falls back to GET per path when another path has an explicit HEAD route", () => {
|
||||
const { router } = compose([
|
||||
"@app/a",
|
||||
{ routes: { "HEAD /download": route("head"), "GET /download": route("get"), "GET /notes": route("notes") } },
|
||||
]);
|
||||
expect(matched(router, "HEAD", "/download")).toMatchObject({ pattern: "HEAD /download" });
|
||||
expect(matched(router, "HEAD", "/notes")).toMatchObject({ pattern: "GET /notes" });
|
||||
});
|
||||
|
||||
// Opus's build review F3, Astra's: the first route's param name was used for every route at that position.
|
||||
it("names each route's params by its own pattern", () => {
|
||||
const { router } = compose(["@app/a", { routes: { "GET /:id/a": route("a"), "GET /:name/b": route("b") } }]);
|
||||
expect(matched(router, "GET", "/v/a")).toEqual({ pattern: "GET /:id/a", params: { id: "v" } });
|
||||
expect(matched(router, "GET", "/v/b")).toEqual({ pattern: "GET /:name/b", params: { name: "v" } });
|
||||
});
|
||||
|
||||
it("takes a route with options: page and crossSite", () => {
|
||||
const { router } = compose([
|
||||
"@app/a",
|
||||
{
|
||||
routes: {
|
||||
"GET /p": { handler: route("p"), page: true },
|
||||
"POST /hook": { handler: route("h"), crossSite: true },
|
||||
},
|
||||
},
|
||||
]);
|
||||
expect(router.route("GET", "/p")).toMatchObject({ page: true, crossSite: false });
|
||||
expect(router.route("POST", "/hook")).toMatchObject({ page: false, crossSite: true });
|
||||
expect(router.route("GET", "/notes")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("conflicts", () => {
|
||||
it("reports one pattern from two contributions (also with other param names); the first by plugin name wins", () => {
|
||||
const { router, conflicts } = compose(
|
||||
["@app/a", { routes: { "GET /x/:id": route("a") } }],
|
||||
["@app/b", { routes: { "GET /x/:key": route("b"), BAD: route("bad") }, sockets: { nope: () => {} } }],
|
||||
);
|
||||
expect((router.route("GET", "/x/1") as RouteMatch).plugin).toBe("@app/a");
|
||||
expect(conflicts).toEqual([
|
||||
{ message: 'route "GET /x/:key" is contributed twice', plugins: ["@app/a", "@app/b"] },
|
||||
{ message: 'route "BAD" is no "METHOD /pattern"', plugins: ["@app/b"] },
|
||||
{ message: 'socket "nope" is no pattern', plugins: ["@app/b"] },
|
||||
]);
|
||||
});
|
||||
|
||||
it("refuses a route for a method a Web Request cannot carry, reported", () => {
|
||||
const { router, conflicts } = compose([
|
||||
"@app/a",
|
||||
{ routes: { "TRACE /x": route("t"), "CONNECT /y": route("c") } },
|
||||
]);
|
||||
expect(router.route("TRACE", "/x")).toBeUndefined();
|
||||
expect(conflicts).toEqual([
|
||||
{ message: 'route "TRACE /x": TRACE is served for no resource (501)', plugins: ["@app/a"] },
|
||||
{ message: 'route "CONNECT /y": CONNECT is served for no resource (501)', plugins: ["@app/a"] },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("wraps (D2)", () => {
|
||||
const wrap = (name: string, phase: HttpWrap["phase"], extra: Partial<HttpWrap> = {}): HttpWrap => ({
|
||||
name,
|
||||
phase,
|
||||
wrap: (request, next) => next(request),
|
||||
...extra,
|
||||
});
|
||||
|
||||
it("orders by phase (outermost first), then contributor order; own wraps around their plugin's routes only", () => {
|
||||
const { router } = compose(
|
||||
[
|
||||
"@app/a",
|
||||
{
|
||||
routes: { "GET /a": route("a") },
|
||||
wraps: [wrap("a-transform", "transform"), wrap("a-observe", "observe")],
|
||||
},
|
||||
],
|
||||
[
|
||||
"@app/b",
|
||||
{
|
||||
routes: { "GET /b": route("b") },
|
||||
wraps: [wrap("b-all", "headers", { scope: "all" }), wrap("b-own", "observe")],
|
||||
},
|
||||
],
|
||||
);
|
||||
const names = (path: string) => (router.route("GET", path) as RouteMatch).wraps.map(({ wrap }) => wrap.name);
|
||||
expect(names("/a")).toEqual(["a-observe", "b-all", "a-transform"]);
|
||||
expect(names("/b")).toEqual(["b-own", "b-all"]);
|
||||
});
|
||||
|
||||
it("narrows to one route; a route nobody contributes is a reported conflict", () => {
|
||||
const { router, conflicts } = compose([
|
||||
"@app/a",
|
||||
{
|
||||
routes: { "GET /a": route("a"), "GET /b": route("b") },
|
||||
wraps: [wrap("only-a", "observe", { route: "GET /a" }), wrap("ghost", "observe", { route: "GET /c" })],
|
||||
},
|
||||
]);
|
||||
expect((router.route("GET", "/a") as RouteMatch).wraps.map(({ wrap }) => wrap.name)).toEqual(["only-a"]);
|
||||
expect((router.route("GET", "/b") as RouteMatch).wraps).toEqual([]);
|
||||
expect(conflicts).toEqual([
|
||||
{ message: 'wrap "ghost" names route "GET /c", which nobody contributes', plugins: ["@app/a"] },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("hosts and sockets", () => {
|
||||
it("collects hosts lowercased, framers, relays and CSP sources; matches sockets with their largest frame", () => {
|
||||
const terminal = () => {};
|
||||
const relay = { name: "radius", verify: () => undefined };
|
||||
const contribution: HttpContribution = {
|
||||
hosts: ["Notes.Example.com"],
|
||||
sockets: { "/terminal/:id": terminal },
|
||||
maxPayload: 10,
|
||||
framers: ["https://frame.example.com"],
|
||||
relays: [relay],
|
||||
csp: { "img-src": ["https://cdn.example.com"] },
|
||||
};
|
||||
const { router } = compose(
|
||||
["@app/a", contribution],
|
||||
["@app/b", { hosts: ["b.example.com"], csp: { "img-src": ["data:"] } }],
|
||||
);
|
||||
expect([...router.hosts]).toEqual(["notes.example.com", "b.example.com"]);
|
||||
expect([[...router.framers], router.relays, [...router.csp]]).toEqual([
|
||||
["https://frame.example.com"],
|
||||
[relay],
|
||||
[["img-src", ["https://cdn.example.com", "data:"]]],
|
||||
]);
|
||||
expect(router.socket("/terminal/7")).toMatchObject({
|
||||
pattern: "/terminal/:id",
|
||||
plugin: "@app/a",
|
||||
params: { id: "7" },
|
||||
maxPayload: 10,
|
||||
});
|
||||
expect(router.socket("/terminal/..")).toBe("bad_path");
|
||||
expect(router.socket("/nope")).toBeUndefined();
|
||||
expect(router.socketHandlers.has(terminal)).toBe(true);
|
||||
});
|
||||
});
|
||||
204
packages/chord/test/next-node/http/support.ts
Normal file
204
packages/chord/test/next-node/http/support.ts
Normal file
|
|
@ -0,0 +1,204 @@
|
|||
// Test support for the HTTP component (slice 10 §2): a server host with the component, the app's configuration and
|
||||
// authenticator, and plugin facets contributing to `Http`; raw requests (any `Host`, any body) and WebSocket clients
|
||||
// against its listener.
|
||||
|
||||
import { request as httpRequest, type IncomingHttpHeaders } from "node:http";
|
||||
import { WebSocket } from "ws";
|
||||
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
|
||||
import type { Address } from "../../../src/next/contracts/index.ts";
|
||||
import { createHost, defineFacet, type Facet, type Host, type HostError } from "../../../src/next/host/index.ts";
|
||||
import {
|
||||
type Authenticator,
|
||||
Authenticators,
|
||||
createHttp,
|
||||
Http,
|
||||
type HttpComponent,
|
||||
type HttpContribution,
|
||||
type HttpError,
|
||||
type HttpOptions,
|
||||
} from "../../../src/next-node/http/index.ts";
|
||||
|
||||
/** The principals of these tests. */
|
||||
export type TestPrincipal =
|
||||
| { readonly kind: "server" }
|
||||
| { readonly kind: "owner" }
|
||||
| { readonly kind: "guest"; readonly id: string };
|
||||
|
||||
/** A running server: its host, its component, its port once loaded, what both reported. */
|
||||
export type Served = {
|
||||
readonly host: Host<TestPrincipal>;
|
||||
readonly http: HttpComponent<TestPrincipal>;
|
||||
/** The listener's port (after `load`). */
|
||||
readonly port: number;
|
||||
readonly errors: HttpError[];
|
||||
readonly hostErrors: HostError[];
|
||||
/** The core facets and the plugins, as loaded. */
|
||||
facets(plugins?: Readonly<Record<string, Facet<TestPrincipal>>>): { name: string; facet: Facet<TestPrincipal> }[];
|
||||
close(): Promise<void>;
|
||||
};
|
||||
|
||||
/** A plugin facet contributing `contribution` to `Http`. */
|
||||
export function contributing(contribution: HttpContribution): Facet<TestPrincipal> {
|
||||
return defineFacet<TestPrincipal>((_host, self) => self.contribute(Http, contribution));
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the component on a new server host (listening on loopback, a free port), then loads it with the app's
|
||||
* authenticator (`@app/auth`: the `x-principal` header names one, else anonymous) and `plugins` (by name) unless
|
||||
* `load` is false; `config` overrides the settings (`listen: undefined`: no listener).
|
||||
*/
|
||||
export async function served(
|
||||
plugins: Readonly<Record<string, Facet<TestPrincipal>>> = {},
|
||||
options: {
|
||||
readonly config?: Omit<HttpOptions, "onError">;
|
||||
readonly authenticate?: Authenticator;
|
||||
readonly load?: boolean;
|
||||
readonly readyTimeout?: number;
|
||||
readonly address?: Address;
|
||||
} = {},
|
||||
): Promise<Served & { load(): Promise<void> }> {
|
||||
const errors: HttpError[] = [];
|
||||
const hostErrors: HostError[] = [];
|
||||
const host = createHost<TestPrincipal>({
|
||||
principal: { kind: "server" },
|
||||
onError: (error) => hostErrors.push(error),
|
||||
...(options.readyTimeout === undefined ? {} : { readyTimeout: options.readyTimeout }),
|
||||
...(options.address === undefined ? {} : { address: options.address }),
|
||||
});
|
||||
const config: Omit<HttpOptions, "onError"> = { listen: { host: "127.0.0.1", port: 0 }, ...options.config };
|
||||
const http = createHttp(host, { ...config, onError: (error) => errors.push(error) }, context);
|
||||
const authenticate = options.authenticate ?? ((request) => principalOf(request.headers));
|
||||
const core = [
|
||||
{ name: "chord.http", facet: http.facet },
|
||||
{
|
||||
name: "@app/auth",
|
||||
facet: defineFacet<TestPrincipal>((_host, self) => self.contribute(Authenticators, authenticate)),
|
||||
},
|
||||
];
|
||||
const facets = (more = plugins) => [...core, ...Object.entries(more).map(([name, facet]) => ({ name, facet }))];
|
||||
let port = 0;
|
||||
const load = async (): Promise<void> => {
|
||||
await host.load(facets(), context);
|
||||
if (config.listen !== undefined) port = (await http.listening(context)).port;
|
||||
};
|
||||
if (options.load !== false) await load();
|
||||
return {
|
||||
host,
|
||||
http,
|
||||
get port() {
|
||||
return port;
|
||||
},
|
||||
errors,
|
||||
hostErrors,
|
||||
facets,
|
||||
load,
|
||||
close: async () => {
|
||||
await http.close(context);
|
||||
await host.close(context);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The principal an `x-principal` header names: `owner`, `guest:<id>`; none without it. */
|
||||
function principalOf(headers: Headers): TestPrincipal | undefined {
|
||||
const named = headers.get("x-principal");
|
||||
if (named === "owner") return { kind: "owner" };
|
||||
if (named?.startsWith("guest:")) return { kind: "guest", id: named.slice(6) };
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/** A response as a test reads it. */
|
||||
export type Answer = { readonly status: number; readonly headers: IncomingHttpHeaders; readonly body: Buffer };
|
||||
|
||||
/**
|
||||
* A request to `port` with raw control: `Host` defaults to `127.0.0.1:<port>`; `body` is sent as is; `signal` aborts it.
|
||||
*/
|
||||
export function send(
|
||||
port: number,
|
||||
method: string,
|
||||
path: string,
|
||||
options: {
|
||||
readonly headers?: Readonly<Record<string, string>>;
|
||||
readonly body?: string | Buffer;
|
||||
readonly agent?: false;
|
||||
} = {},
|
||||
): Promise<Answer> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const request = httpRequest(
|
||||
{
|
||||
host: "127.0.0.1",
|
||||
port,
|
||||
method,
|
||||
path,
|
||||
agent: false,
|
||||
headers: { host: `127.0.0.1:${port}`, ...options.headers },
|
||||
},
|
||||
(response) => {
|
||||
const chunks: Buffer[] = [];
|
||||
response.on("data", (chunk: Buffer) => chunks.push(chunk));
|
||||
response.on("end", () =>
|
||||
resolve({ status: response.statusCode ?? 0, headers: response.headers, body: Buffer.concat(chunks) }),
|
||||
);
|
||||
response.on("error", reject);
|
||||
},
|
||||
);
|
||||
request.on("error", reject);
|
||||
request.end(options.body);
|
||||
});
|
||||
}
|
||||
|
||||
/** A WebSocket client to `path` on `port` (`Host` loopback), with `headers`; resolves once open, rejects when refused. */
|
||||
export function socketTo(
|
||||
port: number,
|
||||
path: string,
|
||||
headers: Readonly<Record<string, string>> = {},
|
||||
options: { readonly maxPayload?: number } = {},
|
||||
): Promise<WebSocket> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const socket = new WebSocket(`ws://127.0.0.1:${port}${path}`, { headers, ...options });
|
||||
// Frames sent at once by the server arrive with the upgrade: queued from the start.
|
||||
inboxes.set(socket, messagesOf(socket));
|
||||
socket.once("open", () => resolve(socket));
|
||||
socket.once("unexpected-response", (_request, response) => {
|
||||
reject(new Error(`refused ${response.statusCode}`));
|
||||
socket.terminate();
|
||||
});
|
||||
socket.once("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
/** The messages of every socket `socketTo` opened, queued from its start. */
|
||||
const inboxes = new WeakMap<WebSocket, { next(): Promise<string> }>();
|
||||
|
||||
/** The next text message of a socket `socketTo` opened. */
|
||||
export function nextMessage(socket: WebSocket): Promise<string> {
|
||||
return inboxes.get(socket)!.next();
|
||||
}
|
||||
|
||||
/** The text messages `socket` receives, from now on, in order: `next()` waits for the next one. */
|
||||
export function messagesOf(socket: WebSocket): { next(): Promise<string> } {
|
||||
const queued: string[] = [];
|
||||
const waiting: ((text: string) => void)[] = [];
|
||||
socket.on("message", (data) => {
|
||||
const text = String(data);
|
||||
const waiter = waiting.shift();
|
||||
if (waiter === undefined) queued.push(text);
|
||||
else waiter(text);
|
||||
});
|
||||
return {
|
||||
next: () => {
|
||||
const text = queued.shift();
|
||||
return text === undefined ? new Promise((resolve) => waiting.push(resolve)) : Promise.resolve(text);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** The close code and reason of `socket`'s close. */
|
||||
export function closeOf(socket: WebSocket): Promise<{ readonly code: number; readonly reason: string }> {
|
||||
return new Promise((resolve) => {
|
||||
if (socket.readyState === socket.CLOSED) resolve({ code: 1006, reason: "" });
|
||||
socket.once("close", (code, reason) => resolve({ code, reason: reason.toString() }));
|
||||
});
|
||||
}
|
||||
|
||||
export { context };
|
||||
458
packages/chord/test/next-node/http/upgrades.test.ts
Normal file
458
packages/chord/test/next-node/http/upgrades.test.ts
Normal file
|
|
@ -0,0 +1,458 @@
|
|||
// Tests of upgraded connections (slice 10 §2, D6, Q6; decisions "allowed hosts only through contributions", "plugin
|
||||
// sockets get a heartbeat"): the links endpoint (`host.accept`, options per socket, refused sockets, the 1 MiB frame
|
||||
// default, the output cap), plugin sockets (handed to their handler, an oversized frame closing only that socket, a
|
||||
// throwing handler), connections terminated at once when their socket handler or host leaves (a raw peer can act no
|
||||
// more), an upgrade checked again after authenticating, the heartbeat, close bounded for a peer that never answers,
|
||||
// and nothing kept of a closed socket or its `ws` server.
|
||||
|
||||
import { once } from "node:events";
|
||||
import { connect, type Socket } from "node:net";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { WebSocket, WebSocketServer } from "ws";
|
||||
import { arr, hostKind, local, method, remote, str } from "../../../src/next/contracts/index.ts";
|
||||
import { createHost, defineFacet } from "../../../src/next/host/index.ts";
|
||||
import { webSocketLink } from "../../../src/next/link/index.ts";
|
||||
import { Http, type HttpContribution } from "../../../src/next-node/http/index.ts";
|
||||
import { limitedSocket } from "../../../src/next-node/http/upgrades.ts";
|
||||
import { expectCollected, settledHeap } from "../../next/support.ts";
|
||||
import { closeOf, context, contributing, nextMessage, send, served, socketTo, type TestPrincipal } from "./support.ts";
|
||||
|
||||
const Server = hostKind("server");
|
||||
|
||||
/** Who the server sees calling: served at the server's address. */
|
||||
const Who = remote("test.who", {
|
||||
version: 1,
|
||||
params: {},
|
||||
hostAddress: () => Server(),
|
||||
variants: { main: { methods: { who: method(undefined, str()) } } },
|
||||
});
|
||||
|
||||
const serveWho = defineFacet<TestPrincipal>((host) =>
|
||||
host.serve(Who, () => ({
|
||||
variant: "main",
|
||||
members: { who: async (ctx) => JSON.stringify(ctx.principal ?? null) },
|
||||
})),
|
||||
);
|
||||
|
||||
/** A plugin socket that echoes text, with its handler's request params and principal sent first. */
|
||||
const echo = contributing({
|
||||
sockets: {
|
||||
"/echo/:room": (socket, request, ctx) => {
|
||||
socket.send(`${request.params.room} as ${ctx.principal?.kind ?? "nobody"}`);
|
||||
socket.on("message", (data) => socket.send(String(data)));
|
||||
},
|
||||
},
|
||||
maxPayload: 64,
|
||||
});
|
||||
|
||||
describe("the links endpoint", () => {
|
||||
it("accepts a link as the authenticated principal, with the options links.options gave that socket", async () => {
|
||||
const options: unknown[] = [];
|
||||
const server = await served(
|
||||
{ "@app/who": serveWho },
|
||||
{
|
||||
address: Server(),
|
||||
config: {
|
||||
links: {
|
||||
path: "/chord",
|
||||
options: (request, principal) => {
|
||||
options.push([request.path, principal]);
|
||||
return { maxPayload: 1024 * 1024 };
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
const tab = createHost<TestPrincipal>({ onError: () => {} });
|
||||
await tab.load([], context);
|
||||
// The link wraps the socket at once: the server's `hello` comes with the upgrade.
|
||||
const socket = new WebSocket(`ws://127.0.0.1:${server.port}/chord`, { headers: { "x-principal": "owner" } });
|
||||
tab.connect(webSocketLink(socket as never), { principal: { kind: "server" } }, context);
|
||||
const who = await tab.get(Who, {}, context);
|
||||
expect(await who.who(context)).toBe('{"kind":"owner"}');
|
||||
expect(options).toEqual([["/chord", { kind: "owner" }]]);
|
||||
await tab.close(context);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("refuses a socket without a principal, from another Origin or Host, foreign, or whose links.options throws (403)", async () => {
|
||||
const server = await served(
|
||||
{ "@app/hosts": contributing({ hosts: ["a.example.com"] }) },
|
||||
{
|
||||
config: {
|
||||
links: {
|
||||
path: "/chord",
|
||||
options: (request) => {
|
||||
if (request.headers.has("x-fail")) throw new Error("options");
|
||||
return {};
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
);
|
||||
const { port } = server;
|
||||
const owner = { "x-principal": "owner" };
|
||||
const refusals = [
|
||||
socketTo(port, "/chord"),
|
||||
socketTo(port, "/chord", { ...owner, origin: "http://evil.com" }),
|
||||
socketTo(port, "/chord", { ...owner, origin: "null" }),
|
||||
socketTo(port, "/chord", { ...owner, origin: `HTTP://127.0.0.1:${port}` }),
|
||||
socketTo(port, "/chord", { ...owner, host: "evil.com" }),
|
||||
socketTo(port, "/chord", { ...owner, "sec-fetch-site": "cross-site" }),
|
||||
socketTo(port, "/chord", { ...owner, "x-fail": "1" }),
|
||||
];
|
||||
for (const refusal of refusals) await expect(refusal).rejects.toThrow("refused 403");
|
||||
const same = await socketTo(port, "/chord", { ...owner, origin: `http://127.0.0.1:${port}` });
|
||||
same.close();
|
||||
expect(server.errors).toEqual([{ kind: "hook_failed", hook: "links.options", error: new Error("options") }]);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("takes frames of at most 1 MiB unless links.options says otherwise (Opus's audit H2)", async () => {
|
||||
const server = await served({}, { config: { links: { path: "/chord" } } });
|
||||
const socket = await socketTo(server.port, "/chord", { "x-principal": "owner" });
|
||||
const closed = closeOf(socket);
|
||||
socket.send(Buffer.alloc(1024 * 1024 + 1));
|
||||
expect((await closed).code).toBe(1009);
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("cuts off a client that stopped reading: past 4 MiB unsent beyond the frame being sent", () => {
|
||||
const sent: number[] = [];
|
||||
let terminated = 0;
|
||||
const fake = {
|
||||
OPEN: 1,
|
||||
readyState: 1,
|
||||
bufferedAmount: 0,
|
||||
binaryType: "nodebuffer",
|
||||
send: (data: Uint8Array) => void sent.push(data.byteLength),
|
||||
terminate: () => void terminated++,
|
||||
};
|
||||
const limited = limitedSocket(fake as never);
|
||||
limited.send(new Uint8Array(10));
|
||||
fake.bufferedAmount = 5 * 1024 * 1024;
|
||||
limited.send(new Uint8Array(10));
|
||||
expect([sent, terminated]).toEqual([[10], 1]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("plugin sockets", () => {
|
||||
it("hands the socket and its request to the handler in a scope with its principal; no principal is refused", async () => {
|
||||
const server = await served({ "@app/echo": echo });
|
||||
await expect(socketTo(server.port, "/echo/lobby")).rejects.toThrow("refused 403");
|
||||
await expect(socketTo(server.port, "/nope", { "x-principal": "owner" })).rejects.toThrow("refused 404");
|
||||
const socket = await socketTo(server.port, "/echo/lobby", { "x-principal": "owner" });
|
||||
expect(await nextMessage(socket)).toBe("lobby as owner");
|
||||
socket.send("hi");
|
||||
expect(await nextMessage(socket)).toBe("hi");
|
||||
socket.close();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("an oversized frame closes only that socket (1009), never the server", async () => {
|
||||
const server = await served({ "@app/echo": echo });
|
||||
const [big, other] = await Promise.all([
|
||||
socketTo(server.port, "/echo/a", { "x-principal": "owner" }),
|
||||
socketTo(server.port, "/echo/b", { "x-principal": "owner" }),
|
||||
]);
|
||||
await Promise.all([nextMessage(big), nextMessage(other)]);
|
||||
const closed = closeOf(big);
|
||||
big.send("x".repeat(100));
|
||||
expect((await closed).code).toBe(1009);
|
||||
other.send("still here");
|
||||
expect(await nextMessage(other)).toBe("still here");
|
||||
other.close();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("a throwing handler is reported socket_failed and closes its socket (1011)", async () => {
|
||||
const server = await served({
|
||||
"@app/bad": contributing({
|
||||
sockets: {
|
||||
"/bad": () => {
|
||||
throw new Error("handler");
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
const socket = await socketTo(server.port, "/bad", { "x-principal": "owner" });
|
||||
expect((await closeOf(socket)).code).toBe(1011);
|
||||
expect(server.errors).toMatchObject([{ kind: "socket_failed", pattern: "/bad", plugin: "@app/bad" }]);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("revocation", () => {
|
||||
it("terminates a plugin's sockets at once when a reload removes its contribution, sealing their scopes; others stay (Q6)", async () => {
|
||||
let signal: AbortSignal | undefined;
|
||||
const removed = contributing({
|
||||
sockets: {
|
||||
"/removed": (_socket, _request, ctx) => {
|
||||
signal = ctx.abortSignal;
|
||||
},
|
||||
},
|
||||
});
|
||||
const other = contributing({ sockets: { "/other": () => {} } });
|
||||
const server = await served({ "@app/removed": removed, "@app/other": other });
|
||||
const doomed = await socketTo(server.port, "/removed", { "x-principal": "owner" });
|
||||
const kept = await socketTo(server.port, "/other", { "x-principal": "owner" });
|
||||
const closed = closeOf(doomed);
|
||||
await server.host.reload(server.facets({ "@app/other": other }), context);
|
||||
expect(signal?.aborted).toBe(true);
|
||||
expect((await closed).code).toBe(1006);
|
||||
expect(kept.readyState).toBe(kept.OPEN);
|
||||
kept.close();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("terminates links and sockets through a host no longer contributed; loopback ones stay", async () => {
|
||||
const server = await served(
|
||||
{ "@app/echo": echo, "@app/hosts": contributing({ hosts: ["a.example.com"] }) },
|
||||
{ config: { links: { path: "/chord" } } },
|
||||
);
|
||||
const through = { "x-principal": "owner", host: "a.example.com" };
|
||||
const link = await socketTo(server.port, "/chord", through);
|
||||
const socket = await socketTo(server.port, "/echo/a", through);
|
||||
const loopback = await socketTo(server.port, "/echo/b", { "x-principal": "owner" });
|
||||
const closes = [closeOf(link), closeOf(socket)];
|
||||
await server.host.reload(server.facets({ "@app/echo": echo }), context);
|
||||
expect((await Promise.all(closes)).map(({ code }) => code)).toEqual([1006, 1006]);
|
||||
expect(loopback.readyState).toBe(loopback.OPEN);
|
||||
expect((await send(server.port, "GET", "/", { headers: { host: "a.example.com" } })).status).toBe(403);
|
||||
loopback.close();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
it("keeps a writable contribution's own sockets when only its hosts change", async () => {
|
||||
const Tunnel = local("app.tunnel", { version: 1, params: {}, members: { name: method(arr(str())) } });
|
||||
const handler = (): void => {};
|
||||
const tunnel = defineFacet<TestPrincipal>((host) =>
|
||||
host.serve(Tunnel, (_, self) => {
|
||||
const contribution = self.writable<HttpContribution>({ hosts: [], sockets: { "/own": handler } });
|
||||
self.contribute(Http, contribution);
|
||||
return {
|
||||
name: async (hosts) => {
|
||||
contribution.set({ hosts, sockets: { "/own": handler } });
|
||||
return null;
|
||||
},
|
||||
};
|
||||
}),
|
||||
);
|
||||
const server = await served({ "@app/tunnel": tunnel });
|
||||
const handle = await server.host.get(Tunnel, {}, context);
|
||||
const socket = await socketTo(server.port, "/own", { "x-principal": "owner" });
|
||||
await handle.name(["t.example.com"], context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
expect(socket.readyState).toBe(socket.OPEN);
|
||||
socket.close();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Astra's security audit A1: a policy close left the peer able to invoke the plugin's handlers.
|
||||
it("a raw peer whose host or contribution left can send nothing more to the handler", async () => {
|
||||
const calls: string[] = [];
|
||||
const sockets = contributing({
|
||||
sockets: { "/echo": (socket) => void socket.on("message", (data) => calls.push(String(data))) },
|
||||
});
|
||||
const tunnel = contributing({ hosts: ["t.example"] });
|
||||
const server = await served({ "@app/sockets": sockets, "@app/tunnel": tunnel });
|
||||
const throughHost = await rawSocket(server.port, "t.example", "/echo");
|
||||
const local = await rawSocket(server.port, `localhost:${server.port}`, "/echo");
|
||||
await server.host.reload(server.facets({ "@app/sockets": sockets }), context);
|
||||
throughHost.write(frame("after the host left"));
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await server.host.reload(server.facets({}), context);
|
||||
local.write(frame("after the contribution left"));
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
expect(calls).toEqual([]);
|
||||
throughHost.destroy();
|
||||
local.destroy();
|
||||
await server.close();
|
||||
});
|
||||
|
||||
// Astra's security audit A2: an upgrade authenticating across the removal of its host was still handed out.
|
||||
it("checks an upgrade again after authenticating: one whose host left meanwhile is refused", async () => {
|
||||
const gate = Promise.withResolvers<void>();
|
||||
const started = Promise.withResolvers<void>();
|
||||
const sockets = contributing({ sockets: { "/echo": () => {} } });
|
||||
const server = await served(
|
||||
{ "@app/sockets": sockets, "@app/tunnel": contributing({ hosts: ["t.example"] }) },
|
||||
{
|
||||
authenticate: async () => {
|
||||
started.resolve();
|
||||
await gate.promise;
|
||||
return { kind: "owner" };
|
||||
},
|
||||
},
|
||||
);
|
||||
const opening = socketTo(server.port, "/echo", { host: "t.example" });
|
||||
await started.promise;
|
||||
await server.host.reload(server.facets({ "@app/sockets": sockets }), context);
|
||||
gate.resolve();
|
||||
await expect(opening).rejects.toThrow("refused 403");
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("cancellation", () => {
|
||||
// Astra's recheck R1, Sol's N5: a client gone while its upgrade authenticated left that work running.
|
||||
// Fable's second recheck: only a reset (RST) was seen; a clean disconnect (FIN, a tab closing) was not.
|
||||
it.each(["ends its connection", "resets its connection"] as const)(
|
||||
"aborts an upgrade's authentication when its client %s",
|
||||
async (how) => {
|
||||
const entered = Promise.withResolvers<void>();
|
||||
let signal: AbortSignal | undefined;
|
||||
const server = await served(
|
||||
{ "@app/s": contributing({ sockets: { "/s": () => {} } }) },
|
||||
{
|
||||
authenticate: async (_request, ctx) => {
|
||||
signal = ctx.abortSignal;
|
||||
entered.resolve();
|
||||
return new Promise(() => {});
|
||||
},
|
||||
},
|
||||
);
|
||||
const socket = connect(server.port, "127.0.0.1");
|
||||
socket.on("error", () => {});
|
||||
await once(socket, "connect");
|
||||
socket.write(
|
||||
`GET /s HTTP/1.1\r\nHost: 127.0.0.1:${server.port}\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n\r\n`,
|
||||
);
|
||||
await entered.promise;
|
||||
const aborted = new Promise((resolve) => signal?.addEventListener("abort", resolve, { once: true }));
|
||||
if (how === "ends its connection") socket.end();
|
||||
else socket.resetAndDestroy();
|
||||
await aborted;
|
||||
expect(signal?.aborted).toBe(true);
|
||||
socket.destroy();
|
||||
await server.close();
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
describe("heartbeat", () => {
|
||||
it("pings a plugin socket after 15 s of silence and terminates it 15 s later without an answer", async () => {
|
||||
const server = await served({
|
||||
"@app/quiet": contributing({
|
||||
sockets: { "/quiet": (socket) => void socket.on("message", (data) => socket.send(data)) },
|
||||
}),
|
||||
});
|
||||
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
|
||||
try {
|
||||
const silent = new WebSocket(`ws://127.0.0.1:${server.port}/quiet`, {
|
||||
headers: { "x-principal": "owner" },
|
||||
autoPong: false,
|
||||
});
|
||||
const answering = new WebSocket(`ws://127.0.0.1:${server.port}/quiet`, {
|
||||
headers: { "x-principal": "owner" },
|
||||
});
|
||||
await Promise.all([once(silent, "open"), once(answering, "open")]);
|
||||
const pinged = Promise.all([once(silent, "ping"), once(answering, "ping")]);
|
||||
await vi.advanceTimersByTimeAsync(15_000);
|
||||
await pinged;
|
||||
// The answering client's pong travels for real (only timers are fake): an echo after it shows it arrived.
|
||||
const echoed = once(answering, "message");
|
||||
answering.send("after the pong");
|
||||
await echoed;
|
||||
const closed = closeOf(silent);
|
||||
await vi.advanceTimersByTimeAsync(15_000);
|
||||
expect((await closed).code).toBe(1006);
|
||||
expect(answering.readyState).toBe(answering.OPEN);
|
||||
answering.close();
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("close", () => {
|
||||
// Sol's build review: close waited on a peer that never acknowledged its close frame.
|
||||
it("terminates a socket that never acknowledges its close at closeTimeout", async () => {
|
||||
const server = await served(
|
||||
{ "@app/s": contributing({ sockets: { "/s": () => {} } }) },
|
||||
{ config: { closeTimeout: 20 } },
|
||||
);
|
||||
const raw = await rawSocket(server.port, `127.0.0.1:${server.port}`, "/s");
|
||||
raw.pause();
|
||||
const started = Date.now();
|
||||
await server.http.close(context);
|
||||
expect(Date.now() - started).toBeLessThan(500);
|
||||
raw.destroy();
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("GC", () => {
|
||||
// Sol's build review: a `ws` server was kept per frame size for the component's life.
|
||||
it("keeps no ws server once its socket closed", async () => {
|
||||
const refs: WeakRef<WebSocketServer>[] = [];
|
||||
const original = WebSocketServer.prototype.handleUpgrade;
|
||||
WebSocketServer.prototype.handleUpgrade = function (this: WebSocketServer, ...args: Parameters<typeof original>) {
|
||||
refs.push(new WeakRef(this));
|
||||
return original.apply(this, args);
|
||||
};
|
||||
let maxPayload = 1024;
|
||||
const server = await served(
|
||||
{},
|
||||
{ config: { links: { path: "/chord", options: () => ({ maxPayload: maxPayload++ }) } } },
|
||||
);
|
||||
try {
|
||||
for (let i = 0; i < 8; i++) {
|
||||
const socket = await socketTo(server.port, "/chord", { "x-principal": "owner" });
|
||||
const closed = closeOf(socket);
|
||||
socket.close();
|
||||
await closed;
|
||||
}
|
||||
await settledHeap();
|
||||
await expectCollected(...refs);
|
||||
} finally {
|
||||
WebSocketServer.prototype.handleUpgrade = original;
|
||||
await server.close();
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps nothing of a closed plugin socket", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const server = await served({
|
||||
"@app/kept": contributing({
|
||||
sockets: {
|
||||
"/kept": (socket, request, ctx) => {
|
||||
refs.push(new WeakRef(socket), new WeakRef(request), new WeakRef(ctx));
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
const socket = await socketTo(server.port, "/kept", { "x-principal": "owner" });
|
||||
const closed = closeOf(socket);
|
||||
socket.close();
|
||||
await closed;
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await expectCollected(...refs);
|
||||
await server.close();
|
||||
});
|
||||
});
|
||||
|
||||
/** A raw TCP client upgraded to the plugin socket `path` through `host` as the owner. */
|
||||
async function rawSocket(port: number, host: string, path: string): Promise<Socket> {
|
||||
const socket = connect(port, "127.0.0.1");
|
||||
await once(socket, "connect");
|
||||
socket.on("error", () => {});
|
||||
socket.write(
|
||||
`GET ${path} HTTP/1.1\r\nHost: ${host}\r\nConnection: Upgrade\r\nUpgrade: websocket\r\nSec-WebSocket-Key: AAAAAAAAAAAAAAAAAAAAAA==\r\nSec-WebSocket-Version: 13\r\nx-principal: owner\r\n\r\n`,
|
||||
);
|
||||
const [answer] = await once(socket, "data");
|
||||
expect(String(answer)).toContain("101 Switching Protocols");
|
||||
return socket;
|
||||
}
|
||||
|
||||
/** A masked client text frame. */
|
||||
function frame(text: string): Buffer {
|
||||
const bytes = Buffer.from(text);
|
||||
const mask = Buffer.from([1, 2, 3, 4]);
|
||||
return Buffer.concat([
|
||||
Buffer.from([0x81, 0x80 | bytes.length]),
|
||||
mask,
|
||||
Buffer.from(bytes.map((byte, index) => byte ^ mask[index % 4]!)),
|
||||
]);
|
||||
}
|
||||
|
|
@ -222,7 +222,7 @@ describe("building a facet: the import table", () => {
|
|||
});
|
||||
await writeFiles(root, { "base.ts": "export const base = 1;" });
|
||||
await writePlugin(root, "copier", "@test/copier", {
|
||||
"worker.ts": 'import { y } from "../shared/shared.ts";\nexport default { setup: () => y };',
|
||||
"worker.ts": 'import { y } from "@test/shared/shared";\nexport default { setup: () => y };',
|
||||
});
|
||||
await writePlugin(root, "shared", "@test/shared", {
|
||||
"shared.ts": 'import y from "some-package";\nexport { y };',
|
||||
|
|
@ -231,7 +231,7 @@ describe("building a facet: the import table", () => {
|
|||
const out = plugins.find(({ name }) => name === "@test/out")!;
|
||||
expect(errors(await buildFacet(out, "worker", { ...NODE, plugins }, context))).toEqual([
|
||||
'worker.ts:1:22: "../base.ts": outside the plugin (base code is reached through a host package, packages through dependencies)',
|
||||
'worker.ts:2:15: "left-out": neither a host package nor in the dependencies of plugin "@test/out"',
|
||||
'worker.ts:2:15: "left-out": neither a host package, in the dependencies of plugin "@test/out" nor a discovered plugin "left-out"',
|
||||
]);
|
||||
const copier = plugins.find(({ name }) => name === "@test/copier")!;
|
||||
expect(errors(await buildFacet(copier, "worker", { ...NODE, plugins }, context))).toEqual([
|
||||
|
|
@ -239,6 +239,43 @@ describe("building a facet: the import table", () => {
|
|||
]);
|
||||
});
|
||||
|
||||
it("imports another plugin's file by name from any root; refuses relative paths into it, names a missing plugin", async () => {
|
||||
const first = await tempRoot();
|
||||
const second = await tempRoot();
|
||||
await writePlugin(second, "elsewhere", "@test/contracts", {
|
||||
"contract.ts": 'export const name = "contract";',
|
||||
"other.ts": 'import { name } from "./contract.ts";\nexport const other = name;',
|
||||
});
|
||||
await writePlugin(first, "user", "@test/user", {
|
||||
"worker.ts": [
|
||||
'import { name } from "@test/contracts/contract";',
|
||||
'import { other } from "@test/contracts/other";',
|
||||
"export default { setup: () => [name, other] };",
|
||||
].join("\n"),
|
||||
});
|
||||
await writePlugin(first, "relative", "@test/relative", {
|
||||
"worker.ts": 'import { name } from "../user/worker.ts";\nexport default { setup: () => name };',
|
||||
});
|
||||
await writePlugin(first, "missing", "@test/missing", {
|
||||
"worker.ts": 'import { name } from "@test/absent/contract";\nexport default { setup: () => name };',
|
||||
});
|
||||
const { plugins } = await discoverPlugins([first, second], { kinds: KINDS }, context);
|
||||
const build = (name: string) =>
|
||||
buildFacet(plugins.find((each) => each.name === name)!, "worker", { ...NODE, plugins }, context);
|
||||
const user = await build("@test/user");
|
||||
if (user.kind === "failed") throw new Error(user.errors.join("\n"));
|
||||
expect(user.copied.map((file) => file.slice(second.length + 1)).sort()).toEqual([
|
||||
"elsewhere/contract.ts",
|
||||
"elsewhere/other.ts",
|
||||
]);
|
||||
expect(errors(await build("@test/relative"))).toEqual([
|
||||
'worker.ts:1:22: "../user/worker.ts": a relative path into plugin "@test/user" (import its files by name: "@test/user/…")',
|
||||
]);
|
||||
expect(errors(await build("@test/missing"))).toEqual([
|
||||
'worker.ts:1:22: "@test/absent/contract": neither a host package, in the dependencies of plugin "@test/missing" nor a discovered plugin "@test/absent"',
|
||||
]);
|
||||
});
|
||||
|
||||
it("returns a syntax error and a missing facet as failed results, never a throw", async () => {
|
||||
const root = await tempRoot();
|
||||
await writePlugin(root, "syntax", "@test/syntax", { "worker.ts": "export default {" });
|
||||
|
|
|
|||
|
|
@ -12,7 +12,12 @@ import { createHost, defineFacet, type Host, type ReloadReport } from "../../../
|
|||
import type { ChordHost } from "../../../src/next/host/internal.ts";
|
||||
import { memoryLinkPair } from "../../../src/next/link/index.ts";
|
||||
import { constant } from "../../../src/next/state/index.ts";
|
||||
import { createPlugins, type Plugins, type RolloutStep } from "../../../src/next-node/plugins/index.ts";
|
||||
import {
|
||||
createPlugins,
|
||||
type Plugins,
|
||||
type RolloutHost,
|
||||
type RolloutStep,
|
||||
} from "../../../src/next-node/plugins/index.ts";
|
||||
import {
|
||||
changesOf,
|
||||
delivered,
|
||||
|
|
@ -22,7 +27,7 @@ import {
|
|||
transported,
|
||||
valuesOf,
|
||||
} from "../../next/support.ts";
|
||||
import { HOST_PACKAGES, KINDS, tempRoot, writeFiles, writePackage, writePlugin } from "./support.ts";
|
||||
import { HOST_PACKAGES, kindsOf, tempRoot, writeFiles, writePackage, writePlugin } from "./support.ts";
|
||||
|
||||
/** The fixtures' contracts, as the `contracts` fixture declares them. */
|
||||
const Worker = hostKind("test.worker");
|
||||
|
|
@ -46,15 +51,20 @@ async function itemPlugin(root: string, dir: string, label: string): Promise<voi
|
|||
});
|
||||
}
|
||||
|
||||
/** The report of an in-process step's one host. */
|
||||
function reportOf(hosts: readonly RolloutHost[]): ReloadReport {
|
||||
const [entry] = hosts;
|
||||
if (entry === undefined || !("report" in entry)) throw new Error("no report");
|
||||
return entry.report;
|
||||
}
|
||||
|
||||
/** The component over `root`, its reports collected. */
|
||||
function component(root: string): { plugins: Plugins; reported: unknown[] } {
|
||||
const reported: unknown[] = [];
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: KINDS,
|
||||
browserKinds: ["web"],
|
||||
hostPackages: HOST_PACKAGES,
|
||||
kinds: kindsOf(HOST_PACKAGES),
|
||||
onError: (e) => reported.push(e),
|
||||
},
|
||||
context,
|
||||
|
|
@ -81,7 +91,11 @@ describe("per plugin at startup", () => {
|
|||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
const core = defineFacet((h) => h.serve(Item, () => ({ label: constant("core") })));
|
||||
const report = await plugins.load(host, { kind: "worker", core: [{ name: "@app/core", facet: core }] }, context);
|
||||
const report = await plugins.load(
|
||||
host,
|
||||
{ kind: "worker", core: [{ name: "@app/core", facet: core }], modules: HOST_PACKAGES },
|
||||
context,
|
||||
);
|
||||
expect(report.failed.map(({ plugin, error }) => [plugin, (error as Error).message])).toEqual([
|
||||
[
|
||||
"@test/syntax",
|
||||
|
|
@ -105,7 +119,7 @@ describe("per plugin at reload", () => {
|
|||
for (const dir of ["a", "b", "c"]) await itemPlugin(root, dir, `${dir}1`);
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
const label = async (dir: string) =>
|
||||
(await host.get(local(`test.${dir}`, { version: 1, params: {}, members: { label: str() } }), {}, context))
|
||||
.label.value;
|
||||
|
|
@ -180,7 +194,7 @@ describe("per plugin at reload", () => {
|
|||
await writePlugin(root, "consumer", "@test/consumer", { "worker.ts": consumer(1) });
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
const C = local("test.consumer", { version: 1, params: {}, members: { label: str() } });
|
||||
const shown = await host.get(C, {}, context);
|
||||
await writeFiles(join(root, "provider"), { "worker.ts": `${provider(2)}\nexport const broken = ;` });
|
||||
|
|
@ -201,14 +215,19 @@ describe("per plugin at reload", () => {
|
|||
});
|
||||
|
||||
describe("rollout", () => {
|
||||
/** A step recording its name into `order`, answering `report`, or rejecting with `error`. */
|
||||
const step = (name: string, order: string[], outcome: { report?: ReloadReport; error?: Error }): RolloutStep => ({
|
||||
/** A step recording its name into `order`, answering `report` for one host (or `hosts`), or rejecting with `error`. */
|
||||
const step = (
|
||||
name: string,
|
||||
order: string[],
|
||||
outcome: { report?: ReloadReport; hosts?: readonly RolloutHost[]; error?: Error },
|
||||
kinds: readonly string[] = ["worker"],
|
||||
): RolloutStep => ({
|
||||
name,
|
||||
kinds: ["worker"],
|
||||
kinds,
|
||||
reload: async () => {
|
||||
order.push(name);
|
||||
if (outcome.error !== undefined) throw outcome.error;
|
||||
return outcome.report!;
|
||||
return { hosts: outcome.hosts ?? [{ host: name, report: outcome.report! }] };
|
||||
},
|
||||
});
|
||||
const report = (failed: readonly string[]): ReloadReport => ({
|
||||
|
|
@ -244,6 +263,56 @@ describe("rollout", () => {
|
|||
plugins.close();
|
||||
});
|
||||
|
||||
it("counts a host's error as the step's built plugins failing there; a host stopped cleanly dirties nothing (D12)", async () => {
|
||||
const root = await tempRoot();
|
||||
await itemPlugin(root, "a", "a1");
|
||||
await itemPlugin(root, "b", "b1");
|
||||
const { plugins } = component(root);
|
||||
await plugins.build([], context);
|
||||
await writeFile(join(root, "a", "worker.ts"), "// a\n");
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] }), { timeout: 5_000 });
|
||||
const stopped: RolloutHost[] = [{ host: "session|S1", stopped: true }];
|
||||
await plugins.rollout(
|
||||
[step("workers", [], { hosts: stopped })],
|
||||
await plugins.build(["@test/a"], context),
|
||||
context,
|
||||
);
|
||||
expect(plugins.state.value).toMatchObject({ dirty: [] });
|
||||
await writeFile(join(root, "a", "worker.ts"), "// a again\n");
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] }), { timeout: 5_000 });
|
||||
const crashed: RolloutHost[] = [
|
||||
{ host: "session|S1", report: report([]) },
|
||||
{ host: "session|S2", error: new Error("crashed") },
|
||||
];
|
||||
const result = await plugins.rollout(
|
||||
[step("workers", [], { hosts: crashed })],
|
||||
await plugins.build(["@test/a"], context),
|
||||
context,
|
||||
);
|
||||
expect(result.steps).toEqual([{ name: "workers", hosts: crashed }]);
|
||||
expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] });
|
||||
plugins.close();
|
||||
});
|
||||
|
||||
it("keeps dirty a plugin whose built kind no step shipped", async () => {
|
||||
const root = await tempRoot();
|
||||
await writePlugin(root, "a", "@test/a", { "worker.ts": "export default { setup: () => {} };", "server.ts": "" });
|
||||
const { plugins } = component(root);
|
||||
await plugins.build([], context);
|
||||
await writeFile(join(root, "a", "worker.ts"), "export default { setup: () => {} }; // edited\n");
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] }), { timeout: 5_000 });
|
||||
const built = await plugins.build(["@test/a"], context);
|
||||
await plugins.rollout([step("worker", [], { report: report([]) })], built, context);
|
||||
expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] });
|
||||
await plugins.rollout(
|
||||
[step("worker", [], { report: report([]) }), step("server", [], { report: report([]) }, ["server"])],
|
||||
built,
|
||||
context,
|
||||
);
|
||||
expect(plugins.state.value).toMatchObject({ dirty: [] });
|
||||
plugins.close();
|
||||
});
|
||||
|
||||
it("stops at a step that rejects, clearing no mark; a change during the build keeps its mark", async () => {
|
||||
const root = await tempRoot();
|
||||
await itemPlugin(root, "a", "a1");
|
||||
|
|
@ -275,6 +344,38 @@ describe("rollout", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("current sets (slice 10 D11)", () => {
|
||||
it("scripts(kind): built on first use; a build changes no set; a rollout applies it right before a step of its kind", async () => {
|
||||
const root = await tempRoot();
|
||||
await itemPlugin(root, "a", "a1");
|
||||
await writePlugin(root, "b", "@test/b", { "server.ts": "export default { setup: () => {} };" });
|
||||
const { plugins } = component(root);
|
||||
const digests = async () =>
|
||||
(await plugins.scripts("worker", context)).map(({ plugin, built }) => [
|
||||
plugin.name,
|
||||
built.kind === "script" ? built.digest.slice(0, 8) : built.kind,
|
||||
]);
|
||||
const first = await digests();
|
||||
expect(first.map(([name]) => name)).toEqual(["@test/a"]);
|
||||
await itemPlugin(root, "a", "a2");
|
||||
const built = await plugins.build(["@test/a"], context);
|
||||
expect(await digests()).toEqual(first);
|
||||
const rejecting: RolloutStep = {
|
||||
name: "server",
|
||||
kinds: ["server"],
|
||||
reload: async () => {
|
||||
throw new Error("closed");
|
||||
},
|
||||
};
|
||||
const shipped: RolloutStep = { name: "workers", kinds: ["worker"], reload: async () => ({ hosts: [] }) };
|
||||
await plugins.rollout([rejecting, shipped], built, context);
|
||||
expect(await digests()).toEqual(first);
|
||||
await plugins.rollout([shipped], built, context);
|
||||
expect(await digests()).not.toEqual(first);
|
||||
plugins.close();
|
||||
});
|
||||
});
|
||||
|
||||
describe("installs through the component", () => {
|
||||
it("clears the plugin's install mark and dirties it", async () => {
|
||||
const root = await tempRoot();
|
||||
|
|
@ -285,8 +386,7 @@ describe("installs through the component", () => {
|
|||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: KINDS,
|
||||
hostPackages: HOST_PACKAGES,
|
||||
kinds: kindsOf(HOST_PACKAGES),
|
||||
install: { npmCommand: [process.execPath, join(tools, "npm.mjs")] },
|
||||
onError: (error) => reported.push(error),
|
||||
},
|
||||
|
|
@ -321,12 +421,10 @@ describe("GC", () => {
|
|||
"export default defineFacet((host) => host.serve(Item, () => ({ label: constant(state.label) })));",
|
||||
].join("\n");
|
||||
await writePlugin(root, "gc", "@test/gc", { "worker.ts": source("a") });
|
||||
const plugins = createPlugins(
|
||||
{ roots: [root], kinds: KINDS, hostPackages: { ...HOST_PACKAGES, "@test/probe": probe }, onError: () => {} },
|
||||
context,
|
||||
);
|
||||
const modules = { ...HOST_PACKAGES, "@test/probe": probe };
|
||||
const plugins = createPlugins({ roots: [root], kinds: kindsOf(modules), onError: () => {} }, context);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules }, context);
|
||||
const first = await plugins.build(["@test/gc"], context);
|
||||
await writeFiles(join(root, "gc"), { "worker.ts": source("b") });
|
||||
const second = await plugins.build(["@test/gc"], context);
|
||||
|
|
@ -358,8 +456,8 @@ describe("end to end: a server and a worker", () => {
|
|||
{ authorize: () => true, resolve: async () => ({ link: down, principal: { kind: "worker" } }) },
|
||||
context,
|
||||
);
|
||||
await plugins.load(worker, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(server, { kind: "server", core: [] }, context);
|
||||
await plugins.load(worker, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
await plugins.load(server, { kind: "server", core: [], modules: HOST_PACKAGES }, context);
|
||||
const notes = await server.get(Notes, {}, context);
|
||||
const index = await server.get(Index, {}, context);
|
||||
expect([notes.label.value, index.label.value]).toEqual(["v1", "v1"]);
|
||||
|
|
@ -370,7 +468,9 @@ describe("end to end: a server and a worker", () => {
|
|||
await writeFile(join(root, "notes", "label.ts"), 'export const label = "v2";\n');
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/notes"] }), { timeout: 5_000 });
|
||||
const rolled = await plugins.rollout(steps, await plugins.build(plugins.state.value.dirty, context), context);
|
||||
expect(rolled.steps.map(({ name, report }) => [name, report.plugins.changed, report.objects])).toEqual([
|
||||
expect(
|
||||
rolled.steps.map(({ name, hosts }) => [name, reportOf(hosts).plugins.changed, reportOf(hosts).objects]),
|
||||
).toEqual([
|
||||
["server", ["@test/notes"], [{ object: "test.index", kind: "swapped" }]],
|
||||
["worker", ["@test/notes"], [{ object: "test.notes", kind: "swapped" }]],
|
||||
]);
|
||||
|
|
@ -383,7 +483,7 @@ describe("end to end: a server and a worker", () => {
|
|||
await writeFile(join(root, "notes", "label.ts"), "export const label = ;\n");
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/notes"] }), { timeout: 5_000 });
|
||||
const broken = await plugins.rollout(steps, await plugins.build(plugins.state.value.dirty, context), context);
|
||||
expect(broken.steps.map(({ name, report }) => [name, failedNames(report)])).toEqual([
|
||||
expect(broken.steps.map(({ name, hosts }) => [name, failedNames(reportOf(hosts))])).toEqual([
|
||||
["server", ["@test/notes"]],
|
||||
["worker", ["@test/notes"]],
|
||||
]);
|
||||
|
|
@ -403,7 +503,7 @@ describe("fix round 1", () => {
|
|||
await itemPlugin(root, "b", "b1");
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
await itemPlugin(root, "a", "a2");
|
||||
const builtA = await plugins.build(["@test/a"], context);
|
||||
await itemPlugin(root, "b", "b2");
|
||||
|
|
@ -424,7 +524,7 @@ describe("fix round 1", () => {
|
|||
await itemPlugin(root, "b", "b1");
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
await itemPlugin(root, "a", "a2");
|
||||
const first = await plugins.build(["@test/a"], context);
|
||||
await rm(join(root, "b"), { recursive: true });
|
||||
|
|
@ -444,7 +544,7 @@ describe("fix round 1", () => {
|
|||
await itemPlugin(root, "b", "b1");
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
const Item = local("test.a", { version: 1, params: {}, members: { label: str() } });
|
||||
const item = await host.get(Item, {}, context);
|
||||
await itemPlugin(root, "a", "a2");
|
||||
|
|
@ -476,7 +576,7 @@ describe("fix round 1", () => {
|
|||
await itemPlugin(root, "a", "a1");
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
// The plugin keeps a server facet only.
|
||||
await rm(join(root, "a", "worker.ts"));
|
||||
await writeFiles(join(root, "a"), {
|
||||
|
|
@ -500,7 +600,7 @@ describe("fix round 1", () => {
|
|||
await itemPlugin(root, "a", "a1");
|
||||
const { plugins } = component(root);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
const item = await host.get(local("test.a", { version: 1, params: {}, members: { label: str() } }), {}, context);
|
||||
await itemPlugin(root, "a", "a2");
|
||||
await vi.waitFor(() => expect(plugins.state.value).toMatchObject({ dirty: ["@test/a"] }), { timeout: 5_000 });
|
||||
|
|
@ -526,8 +626,7 @@ describe("fix round 1", () => {
|
|||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: KINDS,
|
||||
hostPackages: HOST_PACKAGES,
|
||||
kinds: kindsOf(HOST_PACKAGES),
|
||||
install: { npmCommand: [process.execPath, join(tools, "npm.mjs")] },
|
||||
onError: () => {},
|
||||
},
|
||||
|
|
@ -567,12 +666,10 @@ describe("the reload queues and garbage collection", () => {
|
|||
"});",
|
||||
].join("\n");
|
||||
const dir = await writePlugin(root, "failed", "@test/failed", { "worker.ts": source(false) });
|
||||
const plugins = createPlugins(
|
||||
{ roots: [root], kinds: KINDS, hostPackages: { ...HOST_PACKAGES, "@test/probe": probe }, onError: () => {} },
|
||||
context,
|
||||
);
|
||||
const modules = { ...HOST_PACKAGES, "@test/probe": probe };
|
||||
const plugins = createPlugins({ roots: [root], kinds: kindsOf(modules), onError: () => {} }, context);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await plugins.load(host, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(host, { kind: "worker", core: [], modules }, context);
|
||||
await writeFiles(dir, { "worker.ts": source(true) });
|
||||
const built = await plugins.build(["@test/failed"], context);
|
||||
// The report (its failure's error holds the new facet's closure) is dropped here; the queues must not keep it.
|
||||
|
|
|
|||
126
packages/chord/test/next-node/plugins/runtime.test.ts
Normal file
126
packages/chord/test/next-node/plugins/runtime.test.ts
Normal file
|
|
@ -0,0 +1,126 @@
|
|||
// Tests of the plugin runtime (slice 10 §3, D11): loading and reloading hosts with complete sets of shipped scripts
|
||||
// (unchanged digests kept as the same facet, removed = loaded minus shipped, failed builds and evaluations kept old),
|
||||
// and that importing it loads no esbuild.
|
||||
|
||||
import { execFileSync } from "node:child_process";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
|
||||
import { local, str } from "../../../src/next/contracts/index.ts";
|
||||
import { createHost } from "../../../src/next/host/index.ts";
|
||||
import {
|
||||
buildFacet,
|
||||
createPluginRuntime,
|
||||
discoverPlugins,
|
||||
type PluginScript,
|
||||
} from "../../../src/next-node/plugins/index.ts";
|
||||
import { HOST_PACKAGES, tempRoot, writePlugin } from "./support.ts";
|
||||
|
||||
/** Writes plugin `@test/<dir>` serving `test.<dir>` labelled `label` from its worker facet (or `source` as it is). */
|
||||
async function plugin(root: string, dir: string, label: string, source?: string): Promise<void> {
|
||||
await writePlugin(root, dir, `@test/${dir}`, {
|
||||
"worker.ts":
|
||||
source ??
|
||||
[
|
||||
'import { constant, defineFacet, local, str } from "@earendil-works/chord/next";',
|
||||
`const Item = local("test.${dir}", { version: 1, params: {}, members: { label: str() } });`,
|
||||
`export default defineFacet((host) => host.serve(Item, () => ({ label: constant(${JSON.stringify(label)}) })));`,
|
||||
].join("\n"),
|
||||
});
|
||||
}
|
||||
|
||||
/** The worker scripts of every plugin under `root`, built now: what the server would ship. */
|
||||
async function shipped(root: string): Promise<PluginScript[]> {
|
||||
const { plugins } = await discoverPlugins([root], { kinds: ["worker"] }, context);
|
||||
const options = { hostPackages: Object.keys(HOST_PACKAGES), platform: "node", plugins } as const;
|
||||
return Promise.all(
|
||||
plugins.map(async (each) => ({ plugin: each, built: await buildFacet(each, "worker", options, context) })),
|
||||
);
|
||||
}
|
||||
|
||||
/** The label `test.<dir>` shows on `host`. */
|
||||
async function label(host: ReturnType<typeof createHost>, dir: string): Promise<string> {
|
||||
const Item = local(`test.${dir}`, { version: 1, params: {}, members: { label: str() } });
|
||||
return (await host.get(Item, {}, context)).label.value;
|
||||
}
|
||||
|
||||
describe("loading and reloading with complete sets", () => {
|
||||
it("loads a set and leaves out a plugin whose build or evaluation failed, reported with the host's report", async () => {
|
||||
const root = await tempRoot();
|
||||
await plugin(root, "a", "a1");
|
||||
await plugin(root, "broken", "", "export const = ;");
|
||||
await plugin(root, "throws", "", 'throw new Error("at import");');
|
||||
const runtime = createPluginRuntime();
|
||||
const host = createHost({ onError: () => {} });
|
||||
const report = await runtime.load(
|
||||
host,
|
||||
{ kind: "worker", core: [], scripts: await shipped(root), modules: HOST_PACKAGES },
|
||||
context,
|
||||
);
|
||||
expect(report.failed.map(({ plugin }) => plugin).sort()).toEqual(["@test/broken", "@test/throws"]);
|
||||
expect(await label(host, "a")).toBe("a1");
|
||||
await host.close(context);
|
||||
});
|
||||
|
||||
it("reloads a complete set: same digests unchanged, a changed one applied, a loaded one not shipped removed", async () => {
|
||||
const root = await tempRoot();
|
||||
await plugin(root, "a", "a1");
|
||||
await plugin(root, "b", "b1");
|
||||
await plugin(root, "c", "c1");
|
||||
const runtime = createPluginRuntime();
|
||||
const host = createHost({ onError: () => {} });
|
||||
await runtime.load(
|
||||
host,
|
||||
{ kind: "worker", core: [], scripts: await shipped(root), modules: HOST_PACKAGES },
|
||||
context,
|
||||
);
|
||||
await plugin(root, "b", "b2");
|
||||
const next = (await shipped(root)).filter(({ plugin }) => plugin.name !== "@test/c");
|
||||
const report = await runtime.reload(host, next, context);
|
||||
expect(report.plugins).toEqual({ added: [], changed: ["@test/b"], removed: ["@test/c"] });
|
||||
expect([await label(host, "a"), await label(host, "b")]).toEqual(["a1", "b2"]);
|
||||
await host.close(context);
|
||||
});
|
||||
|
||||
it("keeps the loaded facet of a plugin whose new build or evaluation failed; an added failing one is left out", async () => {
|
||||
const root = await tempRoot();
|
||||
await plugin(root, "a", "a1");
|
||||
const runtime = createPluginRuntime();
|
||||
const host = createHost({ onError: () => {} });
|
||||
await runtime.load(
|
||||
host,
|
||||
{ kind: "worker", core: [], scripts: await shipped(root), modules: HOST_PACKAGES },
|
||||
context,
|
||||
);
|
||||
await plugin(root, "a", "", "export const = ;");
|
||||
await plugin(root, "new", "", 'throw new Error("at import");');
|
||||
const report = await runtime.reload(host, await shipped(root), context);
|
||||
expect(report.failed.map(({ plugin }) => plugin).sort()).toEqual(["@test/a", "@test/new"]);
|
||||
expect(report.plugins).toEqual({ added: [], changed: [], removed: [] });
|
||||
expect(await label(host, "a")).toBe("a1");
|
||||
await host.close(context);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the runtime's imports", () => {
|
||||
it("loads no esbuild: a worker that only evaluates never imports it", () => {
|
||||
const runtime = fileURLToPath(new URL("../../../src/next-node/plugins/runtime.ts", import.meta.url));
|
||||
// A hook refusing esbuild, then the runtime imported.
|
||||
const script = [
|
||||
'import { registerHooks } from "node:module";',
|
||||
"registerHooks({ resolve(specifier, context, next) {",
|
||||
' if (specifier === "esbuild") throw new Error("esbuild imported");',
|
||||
" return next(specifier, context);",
|
||||
"} });",
|
||||
`const { createPluginRuntime } = await import(${JSON.stringify(runtime)});`,
|
||||
"createPluginRuntime();",
|
||||
'console.log("ok");',
|
||||
].join("\n");
|
||||
const output = execFileSync(
|
||||
process.execPath,
|
||||
["--experimental-strip-types", "--no-warnings", "--input-type=module", "--eval", script],
|
||||
{ encoding: "utf8" },
|
||||
);
|
||||
expect(output.trim()).toBe("ok");
|
||||
});
|
||||
});
|
||||
|
|
@ -7,7 +7,7 @@ import { dirname, join } from "node:path";
|
|||
import { fileURLToPath } from "node:url";
|
||||
import { afterEach } from "vitest";
|
||||
import * as chordNext from "../../../src/next/index.ts";
|
||||
import type { HostPackages } from "../../../src/next-node/plugins/index.ts";
|
||||
import type { HostPackages, KindDefinition } from "../../../src/next-node/plugins/index.ts";
|
||||
|
||||
/** The fixture plugins' directory. */
|
||||
export const FIXTURES = fileURLToPath(new URL("../fixtures/plugins/", import.meta.url));
|
||||
|
|
@ -18,6 +18,16 @@ export const HOST_PACKAGES: HostPackages = { "@earendil-works/chord/next": chord
|
|||
/** The facet kinds of the fixtures' app. */
|
||||
export const KINDS = ["server", "worker", "web"] as const;
|
||||
|
||||
/** The fixtures' app's kinds as the component takes them: Node servers and workers, a browser web, `modules`' names. */
|
||||
export function kindsOf(modules: HostPackages): Readonly<Record<string, KindDefinition>> {
|
||||
const hostPackages = Object.keys(modules);
|
||||
return {
|
||||
server: { platform: "node", hostPackages },
|
||||
worker: { platform: "node", hostPackages },
|
||||
web: { platform: "browser", hostPackages },
|
||||
};
|
||||
}
|
||||
|
||||
const made: string[] = [];
|
||||
|
||||
afterEach(async () => {
|
||||
|
|
|
|||
55
packages/chord/test/next-node/workers/control.test.ts
Normal file
55
packages/chord/test/next-node/workers/control.test.ts
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
// Tests of the control channel's crossings (slice 10 §3): errors cross as `{ message, code?, reason? }` and back, a
|
||||
// reload report with its errors and parked causes.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { ChordError, isChordError } from "../../../src/next/data/index.ts";
|
||||
import {
|
||||
controlOf,
|
||||
crossed,
|
||||
crossedReport,
|
||||
envelope,
|
||||
uncrossed,
|
||||
uncrossedReport,
|
||||
} from "../../../src/next-node/workers/control.ts";
|
||||
|
||||
describe("errors", () => {
|
||||
it("cross as message, code and reason, never a stack or cause; come back as ChordErrors when they had a code", () => {
|
||||
const chord = new ChordError({ code: "unavailable", message: "gone" }, { cause: new Error("inner") });
|
||||
expect(crossed(chord)).toEqual({ message: "gone", code: "unavailable" });
|
||||
expect(crossed(new Error("plain"))).toEqual({ message: "plain" });
|
||||
expect(crossed("text")).toEqual({ message: "text" });
|
||||
const back = uncrossed({ message: "gone", code: "released", reason: "closed" });
|
||||
expect(isChordError(back, "released") && back.reason).toBe("closed");
|
||||
expect(uncrossed({ message: "odd", code: "no-such-code" })).toMatchObject({ message: "odd" });
|
||||
});
|
||||
|
||||
it("crosses a report's failures and parked causes and back", () => {
|
||||
const report = crossedReport({
|
||||
plugins: { added: ["@a/x"], changed: [], removed: [] },
|
||||
failed: [{ plugin: "@a/y", error: new Error("broken") }],
|
||||
objects: [
|
||||
{ object: "a.one", kind: "swapped" },
|
||||
{
|
||||
object: "a.two",
|
||||
kind: "parked",
|
||||
cause: { kind: "waiting", error: new ChordError({ code: "unavailable", message: "w" }) },
|
||||
},
|
||||
],
|
||||
});
|
||||
expect(JSON.parse(JSON.stringify(report))).toEqual(report);
|
||||
const back = uncrossedReport(report);
|
||||
expect(back.failed[0]!.error).toBeInstanceOf(Error);
|
||||
expect(back.objects[1]).toMatchObject({
|
||||
kind: "parked",
|
||||
cause: { kind: "waiting", error: { code: "unavailable" } },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("envelopes", () => {
|
||||
it("tells control messages from anything else on the channel", () => {
|
||||
expect(controlOf(envelope({ kind: "started" }))).toEqual({ kind: "started" });
|
||||
expect(controlOf(new Uint8Array(1))).toBeUndefined();
|
||||
expect(controlOf({ kind: "started" })).toBeUndefined();
|
||||
});
|
||||
});
|
||||
62
packages/chord/test/next-node/workers/lock.test.ts
Normal file
62
packages/chord/test/next-node/workers/lock.test.ts
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
// Tests of process locks (slice 10 §3): one holder at a time, a waiter taking the lock when the holder releases it
|
||||
// (told by its connection's close, no polling), a waiter's abort.
|
||||
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { BACKGROUND_CONTEXT as context, withAbortSignal } from "../../../src/context/index.ts";
|
||||
import { processLock } from "../../../src/next-node/workers/index.ts";
|
||||
import { lockAt } from "../../../src/next-node/workers/lock.ts";
|
||||
|
||||
describe("processLock", () => {
|
||||
it("makes a second taker wait until the holder releases, then takes it", async () => {
|
||||
const name = `chord-test-lock-${process.pid}-${Date.now()}`;
|
||||
const first = await processLock(name, context);
|
||||
let taken = false;
|
||||
const second = processLock(name, context).then((lock) => {
|
||||
taken = true;
|
||||
return lock;
|
||||
});
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
expect(taken).toBe(false);
|
||||
const released = Date.now();
|
||||
await first.release();
|
||||
await (await second).release();
|
||||
expect(taken).toBe(true);
|
||||
expect(Date.now() - released).toBeLessThan(100);
|
||||
});
|
||||
|
||||
it("stops waiting when the waiter's context aborts", async () => {
|
||||
const name = `chord-test-lock-abort-${process.pid}-${Date.now()}`;
|
||||
const held = await processLock(name, context);
|
||||
const abort = new AbortController();
|
||||
const waiting = processLock(name, withAbortSignal(abort.signal, context));
|
||||
abort.abort(new Error("gave up"));
|
||||
await expect(waiting).rejects.toThrow("gave up");
|
||||
await held.release();
|
||||
await (await processLock(name, context)).release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("socket file locks (macOS, the BSDs)", () => {
|
||||
it("fail closed on a stale lock file, naming it: two takers never both take it over (Sol)", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-lock-file-"));
|
||||
const address = join(dir, "lock.sock");
|
||||
// What a crashed holder leaves: a path nobody accepts on.
|
||||
await writeFile(address, "");
|
||||
const takers = [lockAt({ kind: "file", address }, context), lockAt({ kind: "file", address }, context)];
|
||||
for (const taker of takers) await expect(taker).rejects.toThrow(`lock: ${address} is a stale lock file`);
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("are taken, waited for and released like the others", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-lock-file-"));
|
||||
const address = join(dir, "lock.sock");
|
||||
const first = await lockAt({ kind: "file", address }, context);
|
||||
const second = lockAt({ kind: "file", address }, context);
|
||||
await first.release();
|
||||
await (await second).release();
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
87
packages/chord/test/next-node/workers/main.test.ts
Normal file
87
packages/chord/test/next-node/workers/main.test.ts
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
// Tests of the worker's side (slice 10 §3): the start message, the lock, `started` on `connect`, `ready`; shutdown
|
||||
// during startup, the parent's disconnect and SIGTERM each run the closers and exit; `fatal` redacted.
|
||||
|
||||
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { hostKind, str } from "../../../src/next/contracts/index.ts";
|
||||
import type { FromWorker } from "../../../src/next-node/workers/control.ts";
|
||||
import { type Exit, WorkerProcess } from "../../../src/next-node/workers/process.ts";
|
||||
import type { WorkerBehaviour } from "../fixtures/workers/control.ts";
|
||||
import { until } from "./support.ts";
|
||||
|
||||
const ENTRY = fileURLToPath(new URL("../fixtures/workers/worker.ts", import.meta.url));
|
||||
|
||||
const Session = hostKind("session", { id: str() });
|
||||
|
||||
/** The fixture worker forked and started with `behaviour`; its control messages and exit collected. */
|
||||
function started(behaviour: WorkerBehaviour) {
|
||||
const control: FromWorker[] = [];
|
||||
let exit: Exit | undefined;
|
||||
const child = new WorkerProcess(
|
||||
{
|
||||
entry: ENTRY,
|
||||
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
|
||||
onOutput: () => {},
|
||||
},
|
||||
{
|
||||
control: (message) => control.push(message),
|
||||
linkLost: () => {},
|
||||
linkClosed: () => {},
|
||||
exited: (value) => {
|
||||
exit = value;
|
||||
},
|
||||
},
|
||||
);
|
||||
child.send({
|
||||
kind: "start",
|
||||
address: Session({ id: "S1" }),
|
||||
start: behaviour,
|
||||
scripts: [],
|
||||
lock: `chord-test-main:${process.pid}:${Date.now()}:${Math.random()}`,
|
||||
});
|
||||
return { child, control, exited: () => exit };
|
||||
}
|
||||
|
||||
describe("the worker's side", () => {
|
||||
it("sends started when the app connects, then ready with its load report and idle state", async () => {
|
||||
const { child, control } = started({});
|
||||
await until(() => control.some(({ kind }) => kind === "ready"));
|
||||
expect(control.map(({ kind }) => kind)).toEqual(["started", "ready"]);
|
||||
expect(control[1]).toMatchObject({ idle: true, report: { plugins: { added: [] }, failed: [] } });
|
||||
child.kill();
|
||||
});
|
||||
|
||||
it("shuts down during its startup: aborts it, runs the closers registered so far in reverse order, exits 0", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-main-"));
|
||||
const { child, control, exited } = started({ hang: true, log: join(dir, "log") });
|
||||
await until(() => control.some(({ kind }) => kind === "started"));
|
||||
child.send({ kind: "shutdown" });
|
||||
await until(() => exited() !== undefined);
|
||||
expect(exited()).toEqual({ code: 0, signal: null });
|
||||
expect(await readFile(join(dir, "log"), "utf8")).toBe("store\nhost\n");
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("shuts down on SIGTERM and on the parent's disconnect", async () => {
|
||||
const term = started({});
|
||||
await until(() => term.control.some(({ kind }) => kind === "ready"));
|
||||
process.kill(term.child.pid, "SIGTERM");
|
||||
await until(() => term.exited() !== undefined);
|
||||
expect(term.exited()).toEqual({ code: 0, signal: null });
|
||||
const gone = started({});
|
||||
await until(() => gone.control.some(({ kind }) => kind === "ready"));
|
||||
gone.child.link.close();
|
||||
await until(() => gone.exited() !== undefined);
|
||||
expect(gone.exited()).toEqual({ code: 0, signal: null });
|
||||
});
|
||||
|
||||
it("sends fatal with the redacted error and exits 1", async () => {
|
||||
const { control, exited } = started({ fatal: "no secret-key-42 here" });
|
||||
await until(() => exited() !== undefined);
|
||||
expect(control).toEqual([{ kind: "fatal", error: { message: "no [redacted] here" } }]);
|
||||
expect(exited()?.code).toBe(1);
|
||||
});
|
||||
});
|
||||
46
packages/chord/test/next-node/workers/process.test.ts
Normal file
46
packages/chord/test/next-node/workers/process.test.ts
Normal file
|
|
@ -0,0 +1,46 @@
|
|||
// Tests of worker processes (slice 10 §3): forked with their output piped (the stderr tail kept), the link made at the
|
||||
// fork (an early frame queued), control messages told, and the whole tree killed.
|
||||
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { FromWorker } from "../../../src/next-node/workers/control.ts";
|
||||
import { WorkerProcess } from "../../../src/next-node/workers/process.ts";
|
||||
import { until } from "./support.ts";
|
||||
|
||||
const CHILD = fileURLToPath(new URL("../fixtures/workers/child.ts", import.meta.url));
|
||||
|
||||
/** Whether process `pid` lives. */
|
||||
function alive(pid: number): boolean {
|
||||
try {
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
describe("a worker process", () => {
|
||||
it("pipes its output, keeps its stderr tail, queues an early frame on its link, tells control messages", async () => {
|
||||
const output: string[] = [];
|
||||
const control: FromWorker[] = [];
|
||||
const child = new WorkerProcess(
|
||||
{
|
||||
entry: CHILD,
|
||||
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
|
||||
onOutput: (stream, chunk) => output.push(`${stream}: ${Buffer.from(chunk).toString().trim()}`),
|
||||
},
|
||||
{ control: (message) => control.push(message), linkLost: () => {}, linkClosed: () => {}, exited: () => {} },
|
||||
);
|
||||
await until(() => control.length > 0 && output.length >= 2);
|
||||
const frames: number[][] = [];
|
||||
child.link.onMessage((message) => frames.push([...message]));
|
||||
expect(frames).toEqual([[1, 2, 3]]);
|
||||
expect(control).toEqual([{ kind: "started" }]);
|
||||
expect(child.stderrTail).toBe("an error line\n");
|
||||
const grandchild = Number(/grandchild (\d+)/.exec(output.join("\n"))![1]);
|
||||
expect(alive(grandchild)).toBe(true);
|
||||
child.kill();
|
||||
expect(await child.exited).toEqual({ code: null, signal: "SIGKILL" });
|
||||
await until(() => !alive(grandchild), 2_000);
|
||||
});
|
||||
});
|
||||
151
packages/chord/test/next-node/workers/support.ts
Normal file
151
packages/chord/test/next-node/workers/support.ts
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
// Test support for the workers component (slice 10 §3, §4): a server routing `Session` to forked fixture workers, the
|
||||
// fixture plugins copied into a temporary root, each address's start data (its behaviour) chosen by the test.
|
||||
|
||||
import { cp, mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { afterEach } from "vitest";
|
||||
import { type Context, BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
|
||||
import type { JsonValue } from "../../../src/json.ts";
|
||||
import { hostKind, remote, str } from "../../../src/next/contracts/index.ts";
|
||||
import { createHost, type Host, type HostError } from "../../../src/next/host/index.ts";
|
||||
import * as chordNext from "../../../src/next/index.ts";
|
||||
import { createPlugins, type Plugins } from "../../../src/next-node/plugins/index.ts";
|
||||
import { createWorkers, type Workers, type WorkersError } from "../../../src/next-node/workers/index.ts";
|
||||
import { Session, type WorkerBehaviour } from "../fixtures/workers/control.ts";
|
||||
|
||||
/** The server's own kind. */
|
||||
export const Server = hostKind("server");
|
||||
|
||||
/** The fixture plugin's notes at a worker. */
|
||||
export const Notes = remote("test.notes", {
|
||||
version: 1,
|
||||
params: { id: str() },
|
||||
hostAddress: ({ id }) => Session({ id }),
|
||||
variants: { main: { state: { label: str() } } },
|
||||
});
|
||||
|
||||
/** The principals of these tests. */
|
||||
export type TestPrincipal =
|
||||
| { readonly kind: "server" }
|
||||
| { readonly kind: "worker"; readonly id: string }
|
||||
| { readonly kind: "guest"; readonly id: string };
|
||||
|
||||
const ENTRY = fileURLToPath(new URL("../fixtures/workers/worker.ts", import.meta.url));
|
||||
|
||||
const PLUGINS = fileURLToPath(new URL("../fixtures/workers/plugins/", import.meta.url));
|
||||
|
||||
const made: { readonly root: string; readonly close: () => Promise<void> }[] = [];
|
||||
|
||||
afterEach(async () => {
|
||||
for (const { root, close } of made.splice(0)) {
|
||||
await close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
/** A server with workers; closed after the test. */
|
||||
export type Fleet = {
|
||||
readonly server: Host<TestPrincipal>;
|
||||
readonly workers: Workers<typeof Session>;
|
||||
readonly plugins: Plugins;
|
||||
/** The copy of the fixture plugins. */
|
||||
readonly root: string;
|
||||
readonly errors: WorkersError[];
|
||||
readonly hostErrors: HostError[];
|
||||
/** The addresses `start` ran for, in order. */
|
||||
readonly starts: string[];
|
||||
/** The workers' stdout lines, each with its address key. */
|
||||
readonly output: string[];
|
||||
/** Each address's behaviour (its start data); none: a plain worker. */
|
||||
readonly behaviours: Map<string, WorkerBehaviour>;
|
||||
/** Writes `label` into the copy's notes plugin. */
|
||||
label(label: string): Promise<void>;
|
||||
};
|
||||
|
||||
/** Starts a server routing `Session` to fixture workers; `options` override the workers' defaults. */
|
||||
export async function fleet(
|
||||
options: {
|
||||
readonly idleMs?: number;
|
||||
readonly heartbeat?: { readonly ping?: number; readonly close?: number };
|
||||
readonly start?: (address: { readonly key: string }, context: Context) => Promise<JsonValue>;
|
||||
readonly principal?: () => TestPrincipal;
|
||||
readonly authorize?: () => boolean;
|
||||
} = {},
|
||||
): Promise<Fleet> {
|
||||
const root = await mkdtemp(join(tmpdir(), "chord-workers-"));
|
||||
await cp(PLUGINS, root, { recursive: true });
|
||||
const errors: WorkersError[] = [];
|
||||
const hostErrors: HostError[] = [];
|
||||
const starts: string[] = [];
|
||||
const behaviours = new Map<string, WorkerBehaviour>();
|
||||
const output: string[] = [];
|
||||
const modules = { "@earendil-works/chord/next": chordNext };
|
||||
const plugins = createPlugins(
|
||||
{ roots: [root], kinds: { worker: { platform: "node", hostPackages: Object.keys(modules) } }, onError: () => {} },
|
||||
context,
|
||||
);
|
||||
const server = createHost<TestPrincipal>({
|
||||
address: Server(),
|
||||
principal: { kind: "server" },
|
||||
onError: (error) => hostErrors.push(error),
|
||||
...(options.heartbeat === undefined ? {} : { heartbeat: options.heartbeat }),
|
||||
});
|
||||
const workers = createWorkers(
|
||||
server,
|
||||
Session,
|
||||
{
|
||||
kind: "worker",
|
||||
entry: ENTRY,
|
||||
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
|
||||
authorize: options.authorize ?? (() => true),
|
||||
principal: options.principal ?? ((address) => ({ kind: "worker", id: address.params.id }) as const),
|
||||
start: async (address, startContext) => {
|
||||
starts.push(address.key);
|
||||
if (options.start !== undefined) return options.start(address, startContext);
|
||||
return (behaviours.get(address.params.id) ?? {}) as JsonValue;
|
||||
},
|
||||
lock: (address) => `chord-test:${root}:${address.key}`,
|
||||
idleMs: () => options.idleMs ?? 60_000,
|
||||
plugins,
|
||||
onOutput: (address, stream, chunk) => {
|
||||
if (stream === "stdout") output.push(`${address.key} ${Buffer.from(chunk).toString().trim()}`);
|
||||
},
|
||||
onError: (error) => errors.push(error),
|
||||
},
|
||||
context,
|
||||
);
|
||||
await server.load([], context);
|
||||
// The worker kind's set made now (discovery, builds): the watcher marks edits from then on.
|
||||
await plugins.scripts("worker", context);
|
||||
const close = async () => {
|
||||
await workers.close(context);
|
||||
await server.close(context);
|
||||
plugins.close();
|
||||
};
|
||||
made.push({ root, close });
|
||||
return {
|
||||
server,
|
||||
workers,
|
||||
plugins,
|
||||
root,
|
||||
errors,
|
||||
hostErrors,
|
||||
starts,
|
||||
output,
|
||||
behaviours,
|
||||
label: (label) => writeFile(join(root, "notes", "label.ts"), `export const label = ${JSON.stringify(label)};\n`),
|
||||
};
|
||||
}
|
||||
|
||||
/** Waits until `ready()` holds, polling every 10 ms, for at most `ms`. */
|
||||
export async function until(ready: () => boolean, ms = 10_000): Promise<void> {
|
||||
const deadline = Date.now() + ms;
|
||||
while (!ready()) {
|
||||
if (Date.now() > deadline) throw new Error("until: timed out");
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
}
|
||||
|
||||
export { context, Session };
|
||||
605
packages/chord/test/next-node/workers/workers.test.ts
Normal file
605
packages/chord/test/next-node/workers/workers.test.ts
Normal file
|
|
@ -0,0 +1,605 @@
|
|||
// Tests of the workers component (slice 10 §3, §4): starts on demand (the link made at the fork, `started` on
|
||||
// `connect`, the start timer covering only the start), failures that stay (a crash, a `fatal`, a lost link, a failed
|
||||
// start: no fork for a new hold, holders `failed { waiting }`) until `retry` or a rollout, idle stops and `busy`, `stop`
|
||||
// and its suspension, the per-address queue, `close`, the rollout step, and nothing kept of 1,000 addresses.
|
||||
|
||||
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { withAbortSignal } from "../../../src/context/index.ts";
|
||||
import { createHost } from "../../../src/next/host/index.ts";
|
||||
import { memoryLinkPair } from "../../../src/next/link/index.ts";
|
||||
import type { RolloutHost } from "../../../src/next-node/plugins/index.ts";
|
||||
import { expectCollected } from "../../next/support.ts";
|
||||
import { Control } from "../fixtures/workers/control.ts";
|
||||
import { context, fleet, Notes, Session, type TestPrincipal, until } from "./support.ts";
|
||||
|
||||
/** A step's hosts as `[host, what]`. */
|
||||
function listed(hosts: readonly RolloutHost[]) {
|
||||
return hosts.map((entry) => [
|
||||
entry.host,
|
||||
"report" in entry ? entry.report.plugins : "stopped" in entry ? "stopped" : (entry.error as Error).message,
|
||||
]);
|
||||
}
|
||||
|
||||
/** The state entry of `id`. */
|
||||
const stateOf = (state: unknown, id: string) =>
|
||||
(state as Record<string, { status: string; error?: { message: string } }>)[`session|${id}`];
|
||||
|
||||
describe("starts on demand", () => {
|
||||
it("starts a worker for the first hold: starting, then running; its link binds the holder", async () => {
|
||||
const { server, workers, starts } = await fleet();
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1") !== undefined);
|
||||
expect(stateOf(workers.state.value, "S1")?.status).toBe("starting");
|
||||
await until(() => notes.state.value.kind === "live");
|
||||
expect(notes.label.value).toBe("S1 v1");
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "running");
|
||||
expect(starts).toEqual(["session|S1"]);
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("binds through a load longer than the heartbeat: the link is made at the fork, the worker connects first", async () => {
|
||||
const { server, behaviours } = await fleet({ heartbeat: { ping: 50, close: 200 } });
|
||||
behaviours.set("S1", { loadMs: 600 });
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
expect(notes.label.value).toBe("S1 v1");
|
||||
notes.release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("failures stay until retry or a rollout (R2)", () => {
|
||||
it("a start that throws fails the address: holders fail waiting with it, a new hold forks nothing; retry starts it", async () => {
|
||||
let fail = true;
|
||||
const { server, workers, errors, starts } = await fleet({
|
||||
start: async () => {
|
||||
if (fail) throw new Error("no data dir");
|
||||
return {};
|
||||
},
|
||||
});
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => notes.state.value.kind === "failed");
|
||||
expect(notes.state.value).toMatchObject({
|
||||
kind: "failed",
|
||||
cause: { kind: "waiting", error: { code: "unavailable", message: "worker failed to start" } },
|
||||
});
|
||||
expect(stateOf(workers.state.value, "S1")).toMatchObject({
|
||||
status: "failed",
|
||||
error: { message: "worker failed to start" },
|
||||
});
|
||||
const again = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => again.state.value.kind === "failed");
|
||||
expect(starts).toEqual(["session|S1"]);
|
||||
// The full error only for the app's logs (decision "Slice 10 fix round, worker errors in public").
|
||||
expect(errors).toMatchObject([
|
||||
{ kind: "worker_failed", address: "session|S1", error: { message: "no data dir" } },
|
||||
]);
|
||||
fail = false;
|
||||
await workers.retry(Session({ id: "S1" }), context);
|
||||
await until(() => notes.state.value.kind === "live" && again.state.value.kind === "live");
|
||||
expect(stateOf(workers.state.value, "S1")?.status).not.toBe("failed");
|
||||
notes.release();
|
||||
again.release();
|
||||
});
|
||||
|
||||
it("a crash after ready stays failed (worker_exited, worker_failed), nothing restarts", async () => {
|
||||
const { server, workers, errors, starts } = await fleet();
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
const control = await server.get(Control, { id: "S1" }, context);
|
||||
await control.crash(7, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed");
|
||||
expect(stateOf(workers.state.value, "S1")?.error?.message).toBe("worker exited (code 7)");
|
||||
expect(errors.map(({ kind }) => kind)).toEqual(["worker_exited", "worker_failed"]);
|
||||
await until(() => notes.state.value.kind === "failed");
|
||||
await new Promise((resolve) => setTimeout(resolve, 200));
|
||||
expect(starts).toEqual(["session|S1"]);
|
||||
notes.release();
|
||||
control.release();
|
||||
});
|
||||
|
||||
it("a fatal crosses redacted: the secret never reaches the public state", async () => {
|
||||
const { server, workers, errors, behaviours } = await fleet();
|
||||
behaviours.set("S1", { fatal: "boom secret-token-123" });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed");
|
||||
expect(stateOf(workers.state.value, "S1")?.error?.message).toBe("boom [redacted]");
|
||||
// Public: the state; the report's stderr tail is the app's to redact (a later test).
|
||||
expect(JSON.stringify(workers.state.value)).not.toContain("secret");
|
||||
expect(errors.find(({ kind }) => kind === "worker_failed")).toMatchObject({
|
||||
error: { message: "boom [redacted]" },
|
||||
});
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("a link lost while its child lives (a blocked event loop past the heartbeat): killed after the grace, failed", async () => {
|
||||
const { server, workers, starts, errors } = await fleet({ heartbeat: { ping: 100, close: 300 } });
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
const control = await server.get(Control, { id: "S1" }, context);
|
||||
await control.block(20_000, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "stopping", 5_000);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed", 10_000);
|
||||
expect(stateOf(workers.state.value, "S1")?.error?.message).toBe("worker exited (SIGKILL)");
|
||||
expect(errors.find(({ kind }) => kind === "worker_failed")).toMatchObject({
|
||||
error: { message: "worker: link lost" },
|
||||
});
|
||||
expect(starts).toEqual(["session|S1"]);
|
||||
notes.release();
|
||||
control.release();
|
||||
}, 20_000);
|
||||
|
||||
it("an exit before started fails the start; a worker that never connects is killed at the start timer", async () => {
|
||||
const { server, workers, behaviours, errors } = await fleet();
|
||||
behaviours.set("S1", { exit: 3 });
|
||||
const exited = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed");
|
||||
expect(stateOf(workers.state.value, "S1")?.error?.message).toBe("worker failed to start");
|
||||
expect(errors.find(({ kind }) => kind === "worker_failed")).toMatchObject({
|
||||
error: { message: expect.stringContaining("exited before it was ready (code 3)") },
|
||||
});
|
||||
exited.release();
|
||||
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
|
||||
try {
|
||||
behaviours.set("S2", { connect: false });
|
||||
const silent = server.hold(Notes, { id: "S2" }, context);
|
||||
await vi.waitFor(() => expect(stateOf(workers.state.value, "S2")?.status).toBe("starting"), {
|
||||
timeout: 5_000,
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(60_000);
|
||||
await vi.waitFor(() => expect(stateOf(workers.state.value, "S2")?.status).toBe("failed"), { timeout: 5_000 });
|
||||
expect(stateOf(workers.state.value, "S2")?.error?.message).toBe("worker failed to start");
|
||||
expect(errors.findLast(({ kind }) => kind === "worker_failed")).toMatchObject({
|
||||
error: { message: expect.stringContaining("did not start within 60 s") },
|
||||
});
|
||||
silent.release();
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it("the start timer covers only the start: a slow load after connecting is not killed (Q4)", async () => {
|
||||
const { server, workers, behaviours } = await fleet();
|
||||
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
|
||||
try {
|
||||
behaviours.set("S1", { hang: true });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await vi.waitFor(() => expect(stateOf(workers.state.value, "S1")?.status).toBe("starting"), {
|
||||
timeout: 5_000,
|
||||
});
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
await vi.advanceTimersByTimeAsync(120_000);
|
||||
expect(stateOf(workers.state.value, "S1")?.status).toBe("starting");
|
||||
expect(notes.state.value.kind).toBe("loading");
|
||||
notes.release();
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("keep-alive (D9, R3, Q5)", () => {
|
||||
it("stops a worker idle since idleMs; a hold keeps one running; busy keeps it too", async () => {
|
||||
const { server, workers } = await fleet({ idleMs: 200 });
|
||||
const held = await server.get(Notes, { id: "held" }, context);
|
||||
const released = await server.get(Notes, { id: "free" }, context);
|
||||
const control = await server.get(Control, { id: "busy" }, context);
|
||||
await control.busy(1_500, context);
|
||||
control.release();
|
||||
released.release();
|
||||
await until(() => stateOf(workers.state.value, "free") === undefined, 5_000);
|
||||
await new Promise((resolve) => setTimeout(resolve, 600));
|
||||
expect(stateOf(workers.state.value, "held")?.status).toBe("running");
|
||||
expect(stateOf(workers.state.value, "busy")?.status).toBe("running");
|
||||
await until(() => stateOf(workers.state.value, "busy") === undefined, 5_000);
|
||||
held.release();
|
||||
}, 20_000);
|
||||
|
||||
it("1,000 sequential local calls in a worker nobody holds send no message (Sol)", async () => {
|
||||
const { server, behaviours, output } = await fleet();
|
||||
behaviours.set("S1", { localCalls: 1_000 });
|
||||
// Released at once: the calls run 300 ms after the load, with no hold arriving (no reference keeps it busy).
|
||||
(await server.get(Notes, { id: "S1" }, context)).release();
|
||||
await until(() => output.some((line) => line.startsWith("session|S1 local calls sent")));
|
||||
expect(output).toContain("session|S1 local calls sent 0");
|
||||
});
|
||||
});
|
||||
|
||||
describe("stop", () => {
|
||||
it("suspends while held: holders fail waiting suspended, nothing starts; forgotten with the last holder (Q3)", async () => {
|
||||
const { server, workers, starts } = await fleet();
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
await workers.stop(Session({ id: "S1" }), context);
|
||||
expect(stateOf(workers.state.value, "S1")).toBeUndefined();
|
||||
await until(() => notes.state.value.kind === "failed");
|
||||
expect(notes.state.value).toMatchObject({
|
||||
cause: { kind: "waiting", error: { message: "session|S1: suspended" } },
|
||||
});
|
||||
expect(starts).toEqual(["session|S1"]);
|
||||
notes.release();
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
const again = await server.get(Notes, { id: "S1" }, context);
|
||||
expect(starts).toEqual(["session|S1", "session|S1"]);
|
||||
again.release();
|
||||
});
|
||||
|
||||
it("a stop during a start waits for it, then stops; a start during a stop waits for its exit", async () => {
|
||||
const { server, workers, behaviours, starts } = await fleet();
|
||||
behaviours.set("S1", { loadMs: 300 });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1") !== undefined);
|
||||
await workers.stop(Session({ id: "S1" }), context);
|
||||
expect(stateOf(workers.state.value, "S1")).toBeUndefined();
|
||||
notes.release();
|
||||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
behaviours.set("S1", { log: undefined });
|
||||
const again = await server.get(Notes, { id: "S1" }, context);
|
||||
const stopping = workers.stop(Session({ id: "S1" }), context);
|
||||
again.release();
|
||||
await stopping;
|
||||
const third = await server.get(Notes, { id: "S1" }, context);
|
||||
expect(starts).toHaveLength(3);
|
||||
third.release();
|
||||
});
|
||||
|
||||
it("runs the worker's closers in reverse order at its stop", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-workers-log-"));
|
||||
const { server, workers, behaviours } = await fleet();
|
||||
behaviours.set("S1", { log: join(dir, "log"), closeMs: 50 });
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
notes.release();
|
||||
await workers.stop(Session({ id: "S1" }), context);
|
||||
expect(await readFile(join(dir, "log"), "utf8")).toBe("store\nhost\n");
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe("close", () => {
|
||||
it("cancels a start waiting in start(address): nothing forks; new starts reject; no worker_failed at shutdown", async () => {
|
||||
let waiting = false;
|
||||
const { server, workers, errors } = await fleet({
|
||||
start: async (address, startContext) => {
|
||||
if (address.key === "session|wait") {
|
||||
waiting = true;
|
||||
await new Promise((_, reject) =>
|
||||
startContext.abortSignal?.addEventListener("abort", () => reject(startContext.abortSignal?.reason)),
|
||||
);
|
||||
}
|
||||
return {};
|
||||
},
|
||||
});
|
||||
const running = await server.get(Notes, { id: "S1" }, context);
|
||||
const waits = server.hold(Notes, { id: "wait" }, context);
|
||||
await until(() => waiting);
|
||||
await workers.close(context);
|
||||
expect(workers.state.value).toEqual({});
|
||||
expect(errors).toEqual([]);
|
||||
const late = server.hold(Notes, { id: "late" }, context);
|
||||
await until(() => late.state.value.kind === "failed");
|
||||
expect(late.state.value).toMatchObject({ cause: { error: { code: "released", reason: "closed" } } });
|
||||
for (const ref of [running, waits, late]) ref.release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("the rollout step (§4)", () => {
|
||||
it("ships the current set to running workers, retries held failed ones, lists each", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
const one = await server.get(Notes, { id: "S1" }, context);
|
||||
const two = await server.get(Notes, { id: "S2" }, context);
|
||||
const control = await server.get(Control, { id: "S3" }, context);
|
||||
const failed = await server.get(Notes, { id: "S3" }, context);
|
||||
await control.crash(1, context);
|
||||
await until(() => stateOf(workers.state.value, "S3")?.status === "failed");
|
||||
await label("v2");
|
||||
await until(() => plugins.state.value.dirty.includes("@test/notes"));
|
||||
const result = await plugins.rollout(
|
||||
[workers.step("workers")],
|
||||
await plugins.build(plugins.state.value.dirty, context),
|
||||
context,
|
||||
);
|
||||
const hosts = result.steps[0]!.hosts;
|
||||
expect(listed(hosts).sort()).toEqual([
|
||||
["session|S1", { added: [], changed: ["@test/notes"], removed: [] }],
|
||||
["session|S2", { added: [], changed: ["@test/notes"], removed: [] }],
|
||||
["session|S3", { added: ["@test/notes"], changed: [], removed: [] }],
|
||||
]);
|
||||
await until(() => one.label.value === "S1 v2" && two.label.value === "S2 v2" && failed.label.value === "S3 v2");
|
||||
expect(plugins.state.value.dirty).toEqual([]);
|
||||
for (const ref of [one, two, failed, control]) ref.release();
|
||||
});
|
||||
|
||||
it("lists a worker stopped cleanly during it as stopped and one that crashed with its error (it stays dirty)", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
const one = await server.get(Notes, { id: "S1" }, context);
|
||||
const control = await server.get(Control, { id: "S2" }, context);
|
||||
await label("v2");
|
||||
await until(() => plugins.state.value.dirty.includes("@test/notes"));
|
||||
const built = await plugins.build(plugins.state.value.dirty, context);
|
||||
// Both workers block their event loops: their reloads wait; S1 is stopped meanwhile, S2 crashes before reading it.
|
||||
const first = await server.get(Control, { id: "S1" }, context);
|
||||
await first.block(400, context);
|
||||
await control.blockThenCrash(400, context);
|
||||
// Their event loops are blocked once their timers ran.
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const rolling = plugins.rollout([workers.step("workers")], built, context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const stopped = workers.stop(Session({ id: "S1" }), context);
|
||||
const result = await rolling;
|
||||
await stopped;
|
||||
expect(listed(result.steps[0]!.hosts).sort()).toEqual([
|
||||
["session|S1", "stopped"],
|
||||
["session|S2", "worker: its link is gone"],
|
||||
]);
|
||||
expect(plugins.state.value.dirty).toEqual(["@test/notes"]);
|
||||
for (const ref of [one, first, control]) ref.release();
|
||||
});
|
||||
|
||||
it("a worker starting during it gets the new set: a start after the rollout loads it", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
await label("v2");
|
||||
await until(() => plugins.state.value.dirty.includes("@test/notes"));
|
||||
await plugins.rollout(
|
||||
[workers.step("workers")],
|
||||
await plugins.build(plugins.state.value.dirty, context),
|
||||
context,
|
||||
);
|
||||
const notes = await server.get(Notes, { id: "S9" }, context);
|
||||
expect(notes.label.value).toBe("S9 v2");
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("an aborted wait lists the worker aborted; its late reply keeps the child's slot until it arrives", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
const control = await server.get(Control, { id: "S1" }, context);
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
await label("v2");
|
||||
await until(() => plugins.state.value.dirty.includes("@test/notes"));
|
||||
const built = await plugins.build(plugins.state.value.dirty, context);
|
||||
await control.block(300, context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const abort = new AbortController();
|
||||
const step = workers.step("workers");
|
||||
const waited = plugins.rollout([step], built, withAbortSignal(abort.signal, context));
|
||||
setTimeout(() => abort.abort(new Error("stop waiting")), 50);
|
||||
const first = await waited;
|
||||
expect(listed(first.steps[0]!.hosts)).toEqual([["session|S1", "stop waiting"]]);
|
||||
expect(plugins.state.value.dirty).toEqual(["@test/notes"]);
|
||||
await label("v3");
|
||||
await until(() => plugins.state.value.dirty.includes("@test/notes"));
|
||||
const next = await plugins.rollout([step], await plugins.build(plugins.state.value.dirty, context), context);
|
||||
expect(listed(next.steps[0]!.hosts)).toEqual([
|
||||
["session|S1", { added: [], changed: ["@test/notes"], removed: [] }],
|
||||
]);
|
||||
await until(() => notes.label.value === "S1 v3");
|
||||
control.release();
|
||||
notes.release();
|
||||
});
|
||||
});
|
||||
|
||||
describe("GC", () => {
|
||||
// Real forks are costly (100 took the suite from 103 s to 700 s on one CPU at 1,000): 100 here; the 1,000 failed and
|
||||
// retried addresses below cover the per-address records and suspensions at scale.
|
||||
it("100 addresses started and stopped leave no state, no record, no suspension (Astra, Sol)", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const { server, workers } = await fleet({
|
||||
start: async (address) => {
|
||||
refs.push(new WeakRef(address));
|
||||
return {};
|
||||
},
|
||||
});
|
||||
// Twenty at a time: each a real fork, start, hold, stop and exit.
|
||||
for (let batch = 0; batch < 5; batch++) {
|
||||
await Promise.all(
|
||||
Array.from({ length: 20 }, async (_, index) => {
|
||||
const id = `R${batch * 20 + index}`;
|
||||
const notes = await server.get(Notes, { id }, context);
|
||||
notes.release();
|
||||
await workers.stop(Session({ id }), context);
|
||||
}),
|
||||
);
|
||||
}
|
||||
expect(workers.state.value).toEqual({});
|
||||
await expectCollected(...refs);
|
||||
}, 120_000);
|
||||
|
||||
it("1,000 failed and retried addresses leave no state, no record and no suspension", async () => {
|
||||
const refs: WeakRef<object>[] = [];
|
||||
const { server, workers } = await fleet({
|
||||
start: async (address) => {
|
||||
refs.push(new WeakRef(address));
|
||||
throw new Error("no worker");
|
||||
},
|
||||
});
|
||||
for (let i = 0; i < 1_000; i++) {
|
||||
const id = `A${i}`;
|
||||
const ref = server.hold(Notes, { id }, context);
|
||||
await until(() => ref.state.value.kind === "failed");
|
||||
ref.release();
|
||||
await workers.stop(Session({ id }), context);
|
||||
await workers.retry(Session({ id }), context);
|
||||
}
|
||||
expect(workers.state.value).toEqual({});
|
||||
await expectCollected(...refs.slice(0, 50));
|
||||
}, 120_000);
|
||||
});
|
||||
|
||||
describe("fix round 2", () => {
|
||||
// Astra's recheck R3: a throwing `principal(address)` reached the guest with its message, the child left starting.
|
||||
it("a throwing principal hook fails the start with the public message; the error stays private", async () => {
|
||||
const { server, workers, errors } = await fleet({
|
||||
principal: () => {
|
||||
throw new Error("secret token=sk-1 at /home/owner/private");
|
||||
},
|
||||
});
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => notes.state.value.kind === "failed");
|
||||
expect(JSON.stringify(notes.state.value)).not.toContain("secret");
|
||||
expect(notes.state.value).toMatchObject({
|
||||
kind: "failed",
|
||||
cause: { error: { message: "worker failed to start" } },
|
||||
});
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed");
|
||||
expect(stateOf(workers.state.value, "S1")).toMatchObject({ error: { message: "worker failed to start" } });
|
||||
// Opus's second recheck: the component's own kill of a failed start is no `worker_exited`.
|
||||
await until(() => errors.length > 0);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
expect(errors).toMatchObject([
|
||||
{ kind: "worker_failed", address: "session|S1", error: { message: expect.stringContaining("secret") } },
|
||||
]);
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("a throwing authorize refuses the hold (forbidden); the error stays private, reported authorize_failed", async () => {
|
||||
const { server, errors, starts } = await fleet({
|
||||
authorize: () => {
|
||||
throw new Error("secret lookup");
|
||||
},
|
||||
});
|
||||
const remote = createHost<TestPrincipal>({ onError: () => {} });
|
||||
const [up, down] = memoryLinkPair();
|
||||
server.accept(up, { principal: { kind: "guest", id: "g" } }, context);
|
||||
remote.connect(down, { principal: { kind: "server" } }, context);
|
||||
await remote.load([], context);
|
||||
const notes = remote.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => notes.state.value.kind !== "loading");
|
||||
expect(JSON.stringify(notes.state.value)).not.toContain("secret");
|
||||
expect(notes.state.value).toMatchObject({ kind: "failed", cause: { error: { code: "forbidden" } } });
|
||||
expect([errors, starts]).toMatchObject([[{ kind: "authorize_failed", address: "session|S1" }], []]);
|
||||
notes.release();
|
||||
await remote.close(context);
|
||||
});
|
||||
});
|
||||
|
||||
describe("fix round 1", () => {
|
||||
it("close during a forked start that never connects returns at once, no failure (Opus F1)", async () => {
|
||||
const { server, workers, behaviours, errors } = await fleet();
|
||||
behaviours.set("S1", { connect: false });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "starting");
|
||||
const started = Date.now();
|
||||
await workers.close(context);
|
||||
expect(Date.now() - started).toBeLessThan(5_000);
|
||||
expect(errors).toEqual([]);
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("close during an app's start that ignores its signal returns at once; nothing forks after it (Sol)", async () => {
|
||||
let forked = false;
|
||||
const { server, workers, errors } = await fleet({ start: () => new Promise<never>(() => {}) });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
await workers.close(context);
|
||||
forked = stateOf(workers.state.value, "S1") !== undefined;
|
||||
expect([forked, errors]).toEqual([false, []]);
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("a stop during the worker's startup is a stop: its closers run, nothing fails (Astra)", async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-workers-log-"));
|
||||
const { server, workers, behaviours, errors } = await fleet();
|
||||
behaviours.set("S1", { loadMs: 5_000, closeMs: 200, log: join(dir, "log") });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1") !== undefined);
|
||||
await new Promise((resolve) => setTimeout(resolve, 300));
|
||||
await workers.stop(Session({ id: "S1" }), context);
|
||||
expect(errors).toEqual([]);
|
||||
expect(await readFile(join(dir, "log"), "utf8")).toBe("store\nhost\n");
|
||||
notes.release();
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it("a reload queued behind one the child never answers settles when the child exits (Astra)", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
const control = await server.get(Control, { id: "S1" }, context);
|
||||
await label("v2");
|
||||
await until(() => plugins.state.value.dirty.length > 0);
|
||||
const built = await plugins.build(plugins.state.value.dirty, context);
|
||||
await control.blockThenCrash(600, context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const first = plugins.rollout([workers.step("one")], built, context);
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
const second = plugins.rollout([workers.step("two")], built, context);
|
||||
const outcomes = await Promise.all([first, second]);
|
||||
expect(outcomes.map((result) => listed(result.steps[0]!.hosts))).toEqual([
|
||||
[["session|S1", "worker: its link is gone"]],
|
||||
[["session|S1", "worker: its link is gone"]],
|
||||
]);
|
||||
control.release();
|
||||
});
|
||||
|
||||
it("a plugin whose evaluation failed is shipped again by the next rollout and stays dirty (Astra)", async () => {
|
||||
const { server, workers, plugins, root } = await fleet();
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
await writeFile(
|
||||
join(root, "notes", "label.ts"),
|
||||
'throw new Error("evaluation failed");\nexport const label = "v2";\n',
|
||||
);
|
||||
await until(() => plugins.state.value.dirty.length > 0);
|
||||
const built = await plugins.build(plugins.state.value.dirty, context);
|
||||
const failed = (result: { steps: readonly { hosts: readonly RolloutHost[] }[] }) => {
|
||||
const [entry] = result.steps[0]!.hosts;
|
||||
return entry !== undefined && "report" in entry ? entry.report.failed.map(({ plugin }) => plugin) : entry;
|
||||
};
|
||||
expect(failed(await plugins.rollout([workers.step("one")], built, context))).toEqual(["@test/notes"]);
|
||||
expect(failed(await plugins.rollout([workers.step("two")], built, context))).toEqual(["@test/notes"]);
|
||||
expect(plugins.state.value.dirty).toEqual(["@test/notes"]);
|
||||
expect(notes.label.value).toBe("S1 v1");
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("two rollouts started together each ship their own build; the later one stays (Astra)", async () => {
|
||||
const { server, workers, plugins, label } = await fleet();
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
await label("v2");
|
||||
const v2 = await plugins.build(["@test/notes"], context);
|
||||
await label("v3");
|
||||
const v3 = await plugins.build(["@test/notes"], context);
|
||||
const results = await Promise.all([
|
||||
plugins.rollout([workers.step("v2")], v2, context),
|
||||
plugins.rollout([workers.step("v3")], v3, context),
|
||||
]);
|
||||
for (const result of results) {
|
||||
expect(listed(result.steps[0]!.hosts)).toEqual([
|
||||
["session|S1", { added: [], changed: ["@test/notes"], removed: [] }],
|
||||
]);
|
||||
}
|
||||
await until(() => notes.label.value === "S1 v3");
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("reaps a worker that closed its own IPC channel and lives on: killed after the grace, failed (Sol)", async () => {
|
||||
const { server, workers, behaviours } = await fleet();
|
||||
behaviours.set("S1", { disconnectAndBlock: true });
|
||||
const notes = await server.get(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed", 10_000);
|
||||
expect(stateOf(workers.state.value, "S1")?.error?.message).toBe("worker exited (SIGKILL)");
|
||||
await until(() => notes.state.value.kind === "failed");
|
||||
notes.release();
|
||||
}, 15_000);
|
||||
|
||||
it("keeps no reload report once consumed: the reload queue settles to nothing (Sol)", async () => {
|
||||
const env = await fleet();
|
||||
const notes = await env.server.get(Notes, { id: "S1" }, context);
|
||||
const report = await (async () => {
|
||||
await env.label("v2");
|
||||
const built = await env.plugins.build(["@test/notes"], context);
|
||||
const result = await env.plugins.rollout([env.workers.step("workers")], built, context);
|
||||
const [entry] = result.steps[0]!.hosts;
|
||||
if (entry === undefined || !("report" in entry)) throw new Error("no report");
|
||||
return new WeakRef(entry.report);
|
||||
})();
|
||||
await expectCollected(report);
|
||||
notes.release();
|
||||
});
|
||||
|
||||
it("keeps a worker's stderr out of the public state: only worker_failed carries its tail (Astra)", async () => {
|
||||
const { server, workers, behaviours, errors } = await fleet();
|
||||
behaviours.set("S1", { fatal: "boom secret-token-123" });
|
||||
const notes = server.hold(Notes, { id: "S1" }, context);
|
||||
await until(() => stateOf(workers.state.value, "S1")?.status === "failed");
|
||||
expect(JSON.stringify(workers.state.value)).not.toContain("secret");
|
||||
expect(JSON.stringify(notes.state.value)).not.toContain("secret");
|
||||
const failure = errors.find(({ kind }) => kind === "worker_failed");
|
||||
expect(failure).toMatchObject({ stderrTail: expect.stringContaining("secret-token-123") });
|
||||
notes.release();
|
||||
});
|
||||
});
|
||||
120
packages/chord/test/next/bench/http.ts
Normal file
120
packages/chord/test/next/bench/http.ts
Normal file
|
|
@ -0,0 +1,120 @@
|
|||
// Benchmark of the HTTP component (slice 10 §7): a route's overhead per request against a plain Node handler (keep-alive
|
||||
// round trips, median µs per request over bursts), and `bytes` throughput for a 100 MiB file (`fileResponse`) against
|
||||
// `pipeline` of a read stream.
|
||||
// Run by hand from packages/chord (not in CI), pinned:
|
||||
// taskset -c 6 node --conditions=source --experimental-strip-types --expose-gc test/next/bench/http.ts
|
||||
import { createReadStream } from "node:fs";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { Agent, createServer, request } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { pipeline } from "node:stream/promises";
|
||||
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
|
||||
import { createHost, defineFacet } from "../../../src/next/host/index.ts";
|
||||
import { createHttp, fileResponse, Http } from "../../../src/next-node/http/index.ts";
|
||||
|
||||
const BURSTS = 7;
|
||||
const REQUESTS = 2_000;
|
||||
const FILE = 100 * 1024 * 1024;
|
||||
|
||||
/** GETs `path` on `port` over `agent`, reading the whole body; resolves with its length. */
|
||||
function get(port: number, path: string, agent: Agent): Promise<number> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = request({ host: "127.0.0.1", port, path, agent }, (res) => {
|
||||
let length = 0;
|
||||
res.on("data", (chunk: Buffer) => {
|
||||
length += chunk.length;
|
||||
});
|
||||
res.on("end", () => resolve(length));
|
||||
res.on("error", reject);
|
||||
});
|
||||
req.on("error", reject);
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
/** The median µs per request of `REQUESTS` sequential keep-alive requests, over bursts. */
|
||||
async function perRequest(port: number, path: string): Promise<number> {
|
||||
const agent = new Agent({ keepAlive: true, maxSockets: 1 });
|
||||
for (let i = 0; i < 200; i++) await get(port, path, agent);
|
||||
const times: number[] = [];
|
||||
for (let burst = 0; burst < BURSTS; burst++) {
|
||||
const start = performance.now();
|
||||
for (let i = 0; i < REQUESTS; i++) await get(port, path, agent);
|
||||
times.push(((performance.now() - start) * 1000) / REQUESTS);
|
||||
}
|
||||
agent.destroy();
|
||||
return median(times);
|
||||
}
|
||||
|
||||
/** MiB/s of downloading `path` once (after one warm-up). */
|
||||
async function throughput(port: number, path: string): Promise<number> {
|
||||
const agent = new Agent({ keepAlive: true, maxSockets: 1 });
|
||||
await get(port, path, agent);
|
||||
const times: number[] = [];
|
||||
for (let burst = 0; burst < 3; burst++) {
|
||||
const start = performance.now();
|
||||
await get(port, path, agent);
|
||||
times.push(performance.now() - start);
|
||||
}
|
||||
agent.destroy();
|
||||
return FILE / 1024 / 1024 / (median(times) / 1000);
|
||||
}
|
||||
|
||||
function median(values: readonly number[]): number {
|
||||
const sorted = [...values].sort((a, b) => a - b);
|
||||
return sorted[Math.floor(sorted.length / 2)]!;
|
||||
}
|
||||
|
||||
const dir = await mkdtemp(join(tmpdir(), "chord-bench-http-"));
|
||||
const file = join(dir, "big.bin");
|
||||
await writeFile(file, Buffer.alloc(FILE, 1));
|
||||
|
||||
// A plain Node server: the same JSON body, the same file through `pipeline`.
|
||||
const plain = createServer((req, res) => {
|
||||
if (req.url === "/file") {
|
||||
res.writeHead(200, { "Content-Type": "application/octet-stream", "Content-Length": String(FILE) });
|
||||
void pipeline(createReadStream(file), res).catch(() => {});
|
||||
return;
|
||||
}
|
||||
const body = JSON.stringify({ ok: true });
|
||||
res.writeHead(200, { "Content-Type": "application/json; charset=utf-8", "Content-Length": String(body.length) });
|
||||
res.end(body);
|
||||
});
|
||||
await new Promise<void>((resolve) => plain.listen(0, "127.0.0.1", resolve));
|
||||
const plainPort = (plain.address() as { port: number }).port;
|
||||
|
||||
// The component with Chord's own listener: one plugin with the same route and the file.
|
||||
const host = createHost({ principal: { kind: "server" }, onError: () => {} });
|
||||
const http = createHttp(host, { listen: { host: "127.0.0.1", port: 0 }, onError: () => {} }, context);
|
||||
const plugin = defineFacet((_host, self) => {
|
||||
self.contribute(Http, {
|
||||
routes: {
|
||||
"GET /json": async () => ({ kind: "json", value: { ok: true } }),
|
||||
"GET /file": async () => fileResponse(file),
|
||||
},
|
||||
});
|
||||
});
|
||||
await host.load(
|
||||
[
|
||||
{ name: "chord.http", facet: http.facet },
|
||||
{ name: "@bench/routes", facet: plugin },
|
||||
],
|
||||
context,
|
||||
);
|
||||
const { port } = await http.listening(context);
|
||||
|
||||
const plainRequest = await perRequest(plainPort, "/json");
|
||||
const routeRequest = await perRequest(port, "/json");
|
||||
console.log(
|
||||
`request round trip: plain ${plainRequest.toFixed(1)} µs, route ${routeRequest.toFixed(1)} µs (overhead ${(routeRequest - plainRequest).toFixed(1)} µs, ${(((routeRequest - plainRequest) / plainRequest) * 100).toFixed(1)}%)`,
|
||||
);
|
||||
const plainBytes = await throughput(plainPort, "/file");
|
||||
const routeBytes = await throughput(port, "/file");
|
||||
console.log(`100 MiB file: pipeline ${plainBytes.toFixed(0)} MiB/s, fileResponse ${routeBytes.toFixed(0)} MiB/s`);
|
||||
|
||||
await http.close(context);
|
||||
await host.close(context);
|
||||
plain.close();
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
|
|
@ -25,6 +25,7 @@ const RELOADS = 100;
|
|||
const gc = (globalThis as { gc?: () => void }).gc!;
|
||||
const HOST_PACKAGES = { "@earendil-works/chord/next": chordNext };
|
||||
const KINDS = ["server", "worker"];
|
||||
const NODE = { platform: "node", hostPackages: Object.keys(HOST_PACKAGES) } as const;
|
||||
|
||||
/** A plugin's worker facet: a store over a document, a derive, a method, a helper module and the shared contract. */
|
||||
function worker(index: number, label: string): string {
|
||||
|
|
@ -111,12 +112,9 @@ for (let run = 0; run < RUNS; run++) {
|
|||
console.log(`evaluation per facet: ${median(evaluations).toFixed(2)} ms (median of ${RUNS})`);
|
||||
|
||||
// The heap after 100 reloads of one plugin on a host, against after a warm-up (two builds alternating).
|
||||
const component = createPlugins(
|
||||
{ roots: [root], kinds: KINDS, hostPackages: HOST_PACKAGES, onError: () => {} },
|
||||
context,
|
||||
);
|
||||
const component = createPlugins({ roots: [root], kinds: { server: NODE, worker: NODE }, onError: () => {} }, context);
|
||||
const host = createHost({ onError: () => {} });
|
||||
await component.load(host, { kind: "worker", core: [] }, context);
|
||||
await component.load(host, { kind: "worker", core: [], modules: HOST_PACKAGES }, context);
|
||||
await writeFile(join(root, "p0", "worker.ts"), worker(0, "a"));
|
||||
const first = await component.build(["@bench/p0"], context);
|
||||
await writeFile(join(root, "p0", "worker.ts"), worker(0, "b"));
|
||||
|
|
|
|||
107
packages/chord/test/next/bench/workers.ts
Normal file
107
packages/chord/test/next/bench/workers.ts
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
// Benchmark of the workers component (slice 10 §7): a worker's cold start with shipped scripts (fork to `started`, fork
|
||||
// to `ready`, medians over runs) and the server's memory per idle worker (heap after a GC, 20 workers running idle).
|
||||
// Run by hand from packages/chord (not in CI), pinned:
|
||||
// taskset -c 6 node --conditions=source --experimental-strip-types --expose-gc test/next/bench/workers.ts
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
|
||||
import { hostKind, str } from "../../../src/next/contracts/index.ts";
|
||||
import { createHost } from "../../../src/next/host/index.ts";
|
||||
import * as chordNext from "../../../src/next/index.ts";
|
||||
import { createPlugins } from "../../../src/next-node/plugins/index.ts";
|
||||
import type { FromWorker } from "../../../src/next-node/workers/control.ts";
|
||||
import { createWorkers } from "../../../src/next-node/workers/index.ts";
|
||||
import { WorkerProcess } from "../../../src/next-node/workers/process.ts";
|
||||
import { Control } from "../../next-node/fixtures/workers/control.ts";
|
||||
|
||||
const RUNS = 7;
|
||||
const WORKERS = 20;
|
||||
const ENTRY = fileURLToPath(new URL("../../next-node/fixtures/workers/worker.ts", import.meta.url));
|
||||
const ROOT = fileURLToPath(new URL("../../next-node/fixtures/workers/plugins/", import.meta.url));
|
||||
const EXEC_ARGV = ["--conditions=source", "--experimental-strip-types", "--no-warnings"];
|
||||
|
||||
const gc = (globalThis as { gc?: () => void }).gc!;
|
||||
const Session = hostKind("session", { id: str() });
|
||||
const modules = { "@earendil-works/chord/next": chordNext };
|
||||
const plugins = createPlugins(
|
||||
{ roots: [ROOT], kinds: { worker: { platform: "node", hostPackages: Object.keys(modules) } }, onError: () => {} },
|
||||
context,
|
||||
);
|
||||
const scripts = await plugins.scripts("worker", context);
|
||||
|
||||
function median(values: readonly number[]): number {
|
||||
const sorted = [...values].sort((a, b) => a - b);
|
||||
return sorted[Math.floor(sorted.length / 2)]!;
|
||||
}
|
||||
|
||||
// Cold starts: a fork with the shipped scripts, timed to `started` and to `ready`.
|
||||
const started: number[] = [];
|
||||
const ready: number[] = [];
|
||||
for (let run = 0; run < RUNS; run++) {
|
||||
const forked = performance.now();
|
||||
const times: { started?: number; ready?: number } = {};
|
||||
const done = Promise.withResolvers<void>();
|
||||
const child = new WorkerProcess(
|
||||
{ entry: ENTRY, execArgv: EXEC_ARGV, onOutput: () => {} },
|
||||
{
|
||||
control: (message: FromWorker) => {
|
||||
if (message.kind === "started") times.started = performance.now() - forked;
|
||||
if (message.kind === "ready") {
|
||||
times.ready = performance.now() - forked;
|
||||
done.resolve();
|
||||
}
|
||||
},
|
||||
linkLost: () => {},
|
||||
linkClosed: () => {},
|
||||
exited: () => {},
|
||||
},
|
||||
);
|
||||
child.send({
|
||||
kind: "start",
|
||||
address: Session({ id: `R${run}` }),
|
||||
start: {},
|
||||
scripts,
|
||||
lock: `bench:${process.pid}:${run}`,
|
||||
});
|
||||
await done.promise;
|
||||
started.push(times.started!);
|
||||
ready.push(times.ready!);
|
||||
child.kill();
|
||||
await child.exited;
|
||||
}
|
||||
console.log(
|
||||
`cold start (${scripts.length} shipped script): fork to started ${median(started).toFixed(0)} ms, to ready ${median(ready).toFixed(0)} ms (median of ${RUNS})`,
|
||||
);
|
||||
|
||||
// Memory per idle worker on the server: its records, process handle, link and remote references.
|
||||
const server = createHost({ address: hostKind("server")(), principal: { kind: "server" }, onError: () => {} });
|
||||
const workers = createWorkers(
|
||||
server,
|
||||
Session,
|
||||
{
|
||||
kind: "worker",
|
||||
entry: ENTRY,
|
||||
execArgv: EXEC_ARGV,
|
||||
authorize: () => true,
|
||||
principal: () => ({ kind: "worker" }),
|
||||
start: () => ({}),
|
||||
lock: (address) => `bench:${process.pid}:${address.key}`,
|
||||
plugins,
|
||||
onOutput: () => {},
|
||||
onError: () => {},
|
||||
},
|
||||
context,
|
||||
);
|
||||
await server.load([], context);
|
||||
gc();
|
||||
const before = process.memoryUsage().heapUsed;
|
||||
const held = [];
|
||||
for (let i = 0; i < WORKERS; i++) held.push(await server.get(Control, { id: `W${i}` }, context));
|
||||
gc();
|
||||
const after = process.memoryUsage().heapUsed;
|
||||
console.log(
|
||||
`server heap per idle worker: ${((after - before) / WORKERS / 1024).toFixed(1)} KiB (${WORKERS} workers, one reference each)`,
|
||||
);
|
||||
for (const ref of held) ref.release();
|
||||
await workers.close(context);
|
||||
await server.close(context);
|
||||
plugins.close();
|
||||
|
|
@ -144,8 +144,8 @@ await guest.get(Notes, { session: "S2" }, context).catch((error) => {
|
|||
const lobby = await guest.get(Lobby, {}, context);
|
||||
console.log("guest via the lobby:", await lobby.peek({ session: "S1" }, context));
|
||||
|
||||
// 5. The worker drops: references go loading, the route resolves the address again after its backoff (1 s), and
|
||||
// the new worker (a fresh run, empty notes here) binds them again.
|
||||
// 5. The worker drops: references go loading, the route resolves the address again at once (its link had completed
|
||||
// its handshake), and the new worker (a fresh run, empty notes here) binds them again.
|
||||
const states: string[] = [];
|
||||
const watch = await notes.state.watch(context);
|
||||
watch.start(async (call) => {
|
||||
|
|
|
|||
|
|
@ -29,13 +29,15 @@ async function until(ready: () => boolean): Promise<void> {
|
|||
while (!ready()) await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
|
||||
// 1. The component: the roots, the facet kinds, and the host packages every script gets (this process's Chord: a
|
||||
// plugin's `import { defineFacet } from "@earendil-works/chord/next"` is the app's own instance).
|
||||
// 1. The component: the roots, and the facet kinds, each built for Node with Chord as a host package (kept external in
|
||||
// the builds). The hosts give the modules themselves when they load (this process's Chord: a plugin's
|
||||
// `import { defineFacet } from "@earendil-works/chord/next"` is the app's own instance).
|
||||
const modules = { "@earendil-works/chord/next": chordNext };
|
||||
const node = { platform: "node", hostPackages: Object.keys(modules) } as const;
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: ["server", "worker"],
|
||||
hostPackages: { "@earendil-works/chord/next": chordNext },
|
||||
kinds: { server: node, worker: node },
|
||||
onError: (error) => console.log("plugins:", error.kind),
|
||||
},
|
||||
context,
|
||||
|
|
@ -52,8 +54,8 @@ server.route(
|
|||
{ authorize: () => true, resolve: async () => ({ link: down, principal: { kind: "worker" } }) },
|
||||
context,
|
||||
);
|
||||
const workerLoad = await plugins.load(worker, { kind: "worker", core: [] }, context);
|
||||
const serverLoad = await plugins.load(server, { kind: "server", core: [] }, context);
|
||||
const workerLoad = await plugins.load(worker, { kind: "worker", core: [], modules }, context);
|
||||
const serverLoad = await plugins.load(server, { kind: "server", core: [], modules }, context);
|
||||
console.log(
|
||||
"1. discovered:",
|
||||
plugins.discovery.plugins.map(({ name, facets }) => `${name} [${Object.keys(facets)}]`).join(", "),
|
||||
|
|
@ -84,8 +86,17 @@ console.log(
|
|||
built.facets.map(({ plugin, facet, kind }) => `${plugin}/${facet}: ${kind}`),
|
||||
);
|
||||
const rolled = await plugins.rollout(steps, built, context);
|
||||
for (const { name, report } of rolled.steps) {
|
||||
console.log(` ${name}:`, JSON.stringify(report.plugins), "objects", JSON.stringify(report.objects));
|
||||
for (const { name, hosts } of rolled.steps) {
|
||||
for (const host of hosts) {
|
||||
if ("report" in host) {
|
||||
console.log(
|
||||
` ${name}:`,
|
||||
JSON.stringify(host.report.plugins),
|
||||
"objects",
|
||||
JSON.stringify(host.report.objects),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
await until(() => notes.label.value === "v2");
|
||||
console.log("4. notes:", notes.label.value, "count", notes.count.value, "| index:", index.label.value);
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
// this plugin, so a reload leaves it as it is: same script, same facet, same object.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { Clock } from "../contracts/contract.ts";
|
||||
import { Clock } from "@example/contracts/contract";
|
||||
|
||||
const evaluatedAt = new Date().toISOString();
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// The notes plugin's server facet: serves `Index` with the same label.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { Index } from "../contracts/contract.ts";
|
||||
import { Index } from "@example/contracts/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
export default defineFacet((host) => host.serve(Index, () => ({ label: constant(label) })));
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// The notes plugin's worker facet: serves `Notes`, its count kept in a document.
|
||||
|
||||
import { constant, defineFacet, defineState, int, obj } from "@earendil-works/chord/next";
|
||||
import { Notes } from "../contracts/contract.ts";
|
||||
import { Notes } from "@example/contracts/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
const Count = defineState("count", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 0 }) });
|
||||
|
|
|
|||
|
|
@ -63,11 +63,12 @@ const Counter = local("example.counter", { version: 1, params: {}, members: { st
|
|||
const Summary = local("example.summary", { version: 1, params: {}, members: { text: str() } });
|
||||
const Status = local("example.status", { version: 1, params: {}, members: { text: str() } });
|
||||
|
||||
// The host packages: this process's Chord.
|
||||
const modules = { "@earendil-works/chord/next": chordNext };
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: ["server"],
|
||||
hostPackages: { "@earendil-works/chord/next": chordNext },
|
||||
kinds: { server: { platform: "node", hostPackages: Object.keys(modules) } },
|
||||
onError: (error) => console.log("plugins:", error.kind),
|
||||
},
|
||||
context,
|
||||
|
|
@ -76,7 +77,7 @@ const host = createHost({ onError: () => {} });
|
|||
|
||||
// 1. Startup: the weather plugin's setup throws; it is left out and reported, the other plugins run. A reference to
|
||||
// what it would serve waits (`failed`, cause `waiting`: not served here), and so does its consumer's object.
|
||||
const loaded = await plugins.load(host, { kind: "server", core: [] }, context);
|
||||
const loaded = await plugins.load(host, { kind: "server", core: [], modules }, context);
|
||||
for (const { plugin, error } of loaded.failed) console.log(`1. left out ${plugin}: ${(error as Error).message}`);
|
||||
const weather = host.hold(Weather, {}, context);
|
||||
const forecast = host.hold(Forecast, {}, context);
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// The forecast plugin's server facet: a consumer of the weather plugin's `Weather`.
|
||||
|
||||
import { defineFacet, local, str } from "@earendil-works/chord/next";
|
||||
import { Weather } from "../weather/contract.ts";
|
||||
import { Weather } from "@example/weather/contract";
|
||||
|
||||
const Forecast = local("example.forecast", { version: 1, params: {}, members: { text: str() } });
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
// contract declares.
|
||||
|
||||
import { defineFacet, local, str } from "@earendil-works/chord/next";
|
||||
import { Price } from "../prices/contract.ts";
|
||||
import { Price } from "@example/prices/contract";
|
||||
|
||||
const Summary = local("example.summary", { version: 1, params: {}, members: { text: str() } });
|
||||
|
||||
|
|
|
|||
|
|
@ -29,17 +29,18 @@ for (const dir of ["plugins", "app", "deps"]) {
|
|||
}
|
||||
const root = join(copy, "plugins");
|
||||
|
||||
// 1. The component. Its host packages: Chord, the app's ES module and a CommonJS module of the app. Its npm command
|
||||
// (here a stand-in): what `installDependencies` runs in a plugin's directory.
|
||||
// 1. The component. Its host packages: Chord, the app's ES module and a CommonJS module of the app (their names kept
|
||||
// external in the server kind's builds, the modules given at load). Its npm command (here a stand-in): what
|
||||
// `installDependencies` runs in a plugin's directory.
|
||||
const modules = {
|
||||
"@earendil-works/chord/next": chordNext,
|
||||
"@example/app": app,
|
||||
"@example/legacy": createRequire(import.meta.url)("./app/legacy.cjs"),
|
||||
};
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: ["server"],
|
||||
hostPackages: {
|
||||
"@earendil-works/chord/next": chordNext,
|
||||
"@example/app": app,
|
||||
"@example/legacy": createRequire(import.meta.url)("./app/legacy.cjs"),
|
||||
},
|
||||
kinds: { server: { platform: "node", hostPackages: Object.keys(modules) } },
|
||||
install: { npmCommand: [process.execPath, fileURLToPath(new URL("./fake-npm.mjs", import.meta.url))] },
|
||||
onError: (error) => console.log("plugins:", error.kind),
|
||||
},
|
||||
|
|
@ -49,7 +50,7 @@ const plugins = createPlugins(
|
|||
// 2. Startup: every plugin whose build fails is left out with its errors (one per line: file, line, column, why). The
|
||||
// report plugin's dependency is not installed yet.
|
||||
const host = createHost({ onError: () => {} });
|
||||
const loaded = await plugins.load(host, { kind: "server", core: [] }, context);
|
||||
const loaded = await plugins.load(host, { kind: "server", core: [], modules }, context);
|
||||
console.log("1. left out at startup:");
|
||||
for (const { plugin, error } of loaded.failed) {
|
||||
const [first, ...errors] = (error as Error).message.split("\n");
|
||||
|
|
@ -67,7 +68,9 @@ for (const facet of built.facets) {
|
|||
console.log(` built ${facet.plugin}/${facet.facet}: read ${files.join(", ")}; copied ${facet.copied.length} file`);
|
||||
}
|
||||
const report = await plugins.reload(host, built, context);
|
||||
console.log(" reload:", JSON.stringify(report.plugins), "failed", report.failed.length);
|
||||
// The reload passes the complete set: the refused plugins fail again and stay out.
|
||||
const stillRefused = report.failed.map(({ plugin }) => plugin).join(", ");
|
||||
console.log(" reload:", JSON.stringify(report.plugins), "| still refused:", stillRefused);
|
||||
const text = (await host.get(Report, {}, context)).text.value;
|
||||
console.log("3. the report:", text);
|
||||
|
||||
|
|
|
|||
|
|
@ -6,12 +6,12 @@ import { pluginFile } from "@earendil-works/chord/next/plugins";
|
|||
// Host packages: an ES module (default and named imports) and a CommonJS one, the app's own instances.
|
||||
import app, { currency } from "@example/app";
|
||||
import legacy from "@example/legacy";
|
||||
// A contract of another plugin: a copy built into this script.
|
||||
import { Report } from "@example/shapes/contract";
|
||||
// An npm dependency: inlined into this script by the build (installed first).
|
||||
import { greet } from "example-greeting";
|
||||
// A `chord.external` package: left external, loaded from this plugin's node_modules when the script runs.
|
||||
import { where } from "example-native";
|
||||
// A contract of another plugin: a copy built into this script.
|
||||
import { Report } from "../shapes/contract.ts";
|
||||
|
||||
// A file of this plugin, read at run time by its absolute path.
|
||||
const rates = JSON.parse(readFileSync(pluginFile("data/rates.json"), "utf8")) as { readonly vat: number };
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
// copied file may import only host packages and files of its own plugin.
|
||||
|
||||
import { defineFacet } from "@earendil-works/chord/next";
|
||||
import { Millimetres } from "../shapes/units.ts";
|
||||
import { Millimetres } from "@example/shapes/units";
|
||||
|
||||
export default defineFacet(() => {
|
||||
console.log(Millimetres);
|
||||
|
|
|
|||
|
|
@ -48,17 +48,21 @@ const line = (report: ReloadReport) =>
|
|||
|
||||
/** A rollout, step by step. */
|
||||
function print(result: RolloutResult): void {
|
||||
for (const { name, report } of result.steps) console.log(` ${name}:`, line(report));
|
||||
for (const { name, hosts } of result.steps) {
|
||||
for (const host of hosts) if ("report" in host) console.log(` ${name}:`, line(host.report));
|
||||
}
|
||||
if (result.stopped !== undefined) {
|
||||
console.log(` stopped at ${result.stopped.step}: ${(result.stopped.error as Error).message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// The host packages: this process's Chord, kept external in Node builds.
|
||||
const modules = { "@earendil-works/chord/next": chordNext };
|
||||
const node = { platform: "node", hostPackages: Object.keys(modules) } as const;
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: ["server", "worker"],
|
||||
hostPackages: { "@earendil-works/chord/next": chordNext },
|
||||
kinds: { server: node, worker: node },
|
||||
// The app's own code: a change marks `base`.
|
||||
base: [join(copy, "app")],
|
||||
// A stand-in for npm (no network; these plugins have no dependencies).
|
||||
|
|
@ -69,8 +73,8 @@ const plugins = createPlugins(
|
|||
);
|
||||
const server = createHost({ onError: () => {} });
|
||||
const worker = createHost({ onError: () => {} });
|
||||
await plugins.load(server, { kind: "server", core: [] }, context);
|
||||
await plugins.load(worker, { kind: "worker", core: [] }, context);
|
||||
await plugins.load(server, { kind: "server", core: [], modules }, context);
|
||||
await plugins.load(worker, { kind: "worker", core: [], modules }, context);
|
||||
const steps = [plugins.step("server", server as Host), plugins.step("worker", worker as Host)];
|
||||
console.log("1. loaded:", plugins.discovery.plugins.map(({ name }) => name).join(", "), "| marks:", marks());
|
||||
|
||||
|
|
@ -79,7 +83,7 @@ await edit("plugins/notes/label.ts", "notes v1", "notes v2");
|
|||
await dirty("@example/notes");
|
||||
console.log("2. a file of notes edited:", marks());
|
||||
await edit("plugins/contracts/contract.ts", "one type per plugin", "one type per plugin (edited)");
|
||||
await dirty("@example/contracts", "@example/tags");
|
||||
await dirty("@example/shared", "@example/tags");
|
||||
console.log(" the shared contract edited:", marks());
|
||||
await edit("plugins/tags/package.json", '"private": true', '"private": true,\n\t"description": "tags"');
|
||||
await until(() => plugins.state.value.install.includes("@example/tags"));
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
{
|
||||
"name": "@example/contracts",
|
||||
"name": "@example/shared",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// The notes plugin's server facet: its label.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { labelOf } from "../contracts/contract.ts";
|
||||
import { labelOf } from "@example/shared/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
export default defineFacet((host) =>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
// The notes plugin's worker facet: its label.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { labelOf } from "../contracts/contract.ts";
|
||||
import { labelOf } from "@example/shared/contract";
|
||||
import { label } from "./label.ts";
|
||||
|
||||
export default defineFacet((host) =>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
// The old plugin's server facet; the example deletes the plugin's directory.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { labelOf } from "../contracts/contract.ts";
|
||||
import { labelOf } from "@example/shared/contract";
|
||||
|
||||
export default defineFacet((host) => host.serve(labelOf("old"), () => ({ text: constant("old on the server") })));
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
// The tags plugin's server facet.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { labelOf } from "../contracts/contract.ts";
|
||||
import { labelOf } from "@example/shared/contract";
|
||||
|
||||
export default defineFacet((host) => host.serve(labelOf("tags"), () => ({ text: constant("tags on the server") })));
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
// The tags plugin's worker facet.
|
||||
|
||||
import { constant, defineFacet } from "@earendil-works/chord/next";
|
||||
import { labelOf } from "../contracts/contract.ts";
|
||||
import { labelOf } from "@example/shared/contract";
|
||||
|
||||
export default defineFacet((host) => host.serve(labelOf("tags"), () => ({ text: constant("tags on the worker") })));
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
// A browser build (slice 7c §2): a facet of a browser kind (`browserKinds: ["web"]`) is built for the browser. Its
|
||||
// A browser build (slice 7c §2): a facet of a browser kind (`web: { platform: "browser", … }`) is built for the browser. Its
|
||||
// stylesheets become one CSS file (`css`), an asset imported by URL (`./logo.svg?url`) a content-hashed file whose
|
||||
// import evaluates to its URL under the app's `publicPath`. The build is plain data: script, stylesheet and assets,
|
||||
// what the app serves to its pages (slice 11 ships it); a page evaluates the script with its host packages, as Node
|
||||
|
|
@ -20,10 +20,8 @@ const hostPackages = { "@earendil-works/chord/next": chordNext };
|
|||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [root],
|
||||
kinds: ["web"],
|
||||
browserKinds: ["web"],
|
||||
kinds: { web: { platform: "browser", hostPackages: Object.keys(hostPackages) } },
|
||||
publicPath: "/plugin-assets/",
|
||||
hostPackages,
|
||||
onError: (error) => console.log("plugins:", error.kind),
|
||||
},
|
||||
context,
|
||||
|
|
|
|||
56
packages/chord/test/next/examples/40-http/main.ts
Normal file
56
packages/chord/test/next/examples/40-http/main.ts
Normal file
|
|
@ -0,0 +1,56 @@
|
|||
// HTTP (slice 10): a server answers HTTP requests with routes that plugins contribute. The notes plugin (`plugins/`)
|
||||
// contributes a page, a JSON route and a file download; the server's own code only builds the component, says where
|
||||
// Chord's own listener listens, and loads the plugins. Every answer carries the security baseline's headers.
|
||||
// Run from packages/chord:
|
||||
// node --conditions=source --experimental-strip-types test/next/examples/40-http/main.ts
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { BACKGROUND_CONTEXT as context } from "../../../../src/context/index.ts";
|
||||
import { createHost } from "../../../../src/next/host/index.ts";
|
||||
import * as chordNext from "../../../../src/next/index.ts";
|
||||
import * as chordHttp from "../../../../src/next-node/http/index.ts";
|
||||
import { createPlugins } from "../../../../src/next-node/plugins/index.ts";
|
||||
|
||||
// 1. A server host and its HTTP component; its settings are arguments: Chord's own listener listens on a free
|
||||
// loopback port. Nobody signs in here: no authenticator, every request is anonymous.
|
||||
const server = createHost({ principal: { kind: "server" }, onError: console.error });
|
||||
const http = chordHttp.createHttp(server, { listen: { host: "127.0.0.1", port: 0 }, onError: console.error }, context);
|
||||
|
||||
// 2. The plugins: the component's facet (its registries and the listener) goes first, then every plugin's server
|
||||
// facet. The listener opens once they are published.
|
||||
const modules = { "@earendil-works/chord/next": chordNext, "@earendil-works/chord/next/http": chordHttp };
|
||||
const plugins = createPlugins(
|
||||
{
|
||||
roots: [fileURLToPath(new URL("./plugins/", import.meta.url))],
|
||||
kinds: { server: { platform: "node", hostPackages: Object.keys(modules) } },
|
||||
onError: console.error,
|
||||
},
|
||||
context,
|
||||
);
|
||||
await plugins.load(server, { kind: "server", core: [{ name: "chord.http", facet: http.facet }], modules }, context);
|
||||
const { port } = await http.listening(context);
|
||||
|
||||
// 3. Some requests, printed like curl would.
|
||||
async function curl(path: string, headers: Record<string, string> = {}): Promise<void> {
|
||||
const response = await fetch(`http://127.0.0.1:${port}${path}`, { headers });
|
||||
const range = headers.Range === undefined ? "" : ` (Range: ${headers.Range})`;
|
||||
console.log(`GET ${path}${range} -> ${response.status} ${response.headers.get("content-type")}`);
|
||||
const download = response.headers.get("content-disposition");
|
||||
if (download !== null) console.log(` ${download}`);
|
||||
console.log(` ${JSON.stringify((await response.text()).replace(/nonce="[^"]+"/, 'nonce="…"'))}`);
|
||||
}
|
||||
|
||||
await curl("/");
|
||||
await curl("/notes/hello");
|
||||
await curl("/notes/hello/download", { Range: "bytes=0-6" });
|
||||
await curl("/notes/nope/download");
|
||||
await curl("/nowhere");
|
||||
const baseline = await fetch(`http://127.0.0.1:${port}/notes/hello`);
|
||||
for (const name of ["cache-control", "x-content-type-options", "x-frame-options"]) {
|
||||
console.log(` ${name}: ${baseline.headers.get(name)}`);
|
||||
}
|
||||
|
||||
// 4. Close the component first: it refuses new requests and waits for running ones (the host's close would seal them
|
||||
// at once). Then the host.
|
||||
await http.close(context);
|
||||
await server.close(context);
|
||||
plugins.close();
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
# Hello
|
||||
|
||||
A note served as a download.
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
{
|
||||
"name": "@example/notes",
|
||||
"private": true,
|
||||
"type": "module"
|
||||
}
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
// The notes plugin's server facet: a page, a JSON route and a file download, contributed to the HTTP component's
|
||||
// registry.
|
||||
|
||||
import { defineFacet } from "@earendil-works/chord/next";
|
||||
import { fileResponse, Http } from "@earendil-works/chord/next/http";
|
||||
import { pluginFile } from "@earendil-works/chord/next/plugins";
|
||||
|
||||
export default defineFacet((_host, self) =>
|
||||
self.contribute(Http, {
|
||||
routes: {
|
||||
// A page: a link from another site may open it. Its inline script carries the server's nonce (the CSP).
|
||||
"GET /": {
|
||||
page: true,
|
||||
handler: async (request) => ({
|
||||
kind: "html",
|
||||
body: `<h1>Notes</h1><script nonce="${request.nonce}">console.log("hi")</script>`,
|
||||
}),
|
||||
},
|
||||
// `:name` is one path segment, decoded.
|
||||
"GET /notes/:name": async (request) => ({ kind: "json", value: { note: request.params.name } }),
|
||||
// A file of this plugin, saved under its name, with ranges.
|
||||
"GET /notes/:name/download": async (request) =>
|
||||
fileResponse(pluginFile(`files/${request.params.name}.md`), {
|
||||
download: `${request.params.name}.md`,
|
||||
ranges: true,
|
||||
}),
|
||||
},
|
||||
}),
|
||||
);
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue