sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat(chord): Chord v2 slice 13, the app kit

Browse source 
chord/next/app: startApp and startWorker with the fixed app layout, worker kinds,
AppInfo and WorkerInfo; sessions (hashed tokens, rolling cookies, bearer tokens),
the one-time owner link, the owner token file, device pairing; Files per host
with ordered chunked uploads; export and import per address; plugin roots,
installs and applyChanges; optional users, invite and ephemeral login facets.
HTTP: deny by default (public routes and sockets), remoteAddress,
readSignedCookie. Examples 53-55.
This commit is contained in:
Mario Zechner 2026-11-02 15:13:36 +00:00
parent 0ec20e157b
commit 072e413bf3
74 changed files with 5437 additions and 187 deletions
Download patch file Download diff file Expand all files Collapse all files

4
packages/chord/CHANGELOG.md
View file

@ -5,9 +5,13 @@
### Added
- Chord v2's plugins component gains the plugin model (slice 12): `createPlugins({ api: { name, versions } })` checks every import of the app's versioned API (`@acme/app/v3`, per-kind modules such as `@acme/app/v3/web`): an unlisted version, another kind's module, two versions in one facet and a direct import of Chord are build errors naming the file and line. Roots may be `{ path, required }`: a required root's plugin failing at `load` rejects it with a `PluginError` after the load settled. Files named `<name>.<kind>.<ext>` (`contract.web.tsx`) are of that kind, refused in facets of another kind with the chain of imports named. Single-file plugins (`hello.server.ts` at a root's top, grouped by name, host packages only). `installPlugin({ git, ref } | { npm, version }, root, ctx)` installs plugins from git (a shallow clone of a branch, tag or commit; a repository of several plugins too) or npm; a package lists the plugin folders it provides in its `package.json` (`"chord": { "plugins": ["./notes", "plugins/*", "!plugins/draft"] }`), used by discovery and `installPlugin` (without it: one plugin when the package's top has facet files, else its subfolders with a `package.json`). `testPlugin(dir, { kind, modules, stubs, principal }, ctx)` builds a plugin as production does and loads it on an in-memory host beside stub facets, collecting its contributions to registries nobody serves and listing the contracts it waits for. Agents' guide: `docs/next/plugin-authoring.md`.
- Chord v2 gains the app kit (slice 13, `src/next-node/app`, to be exported as `@earendil-works/chord/next/app`): `startApp<P>({ name, dataDir?, root?, listen?, api, kinds, principal, workers?, files?, install?, onError }, ctx)` lays out the data directory (`server/store.sqlite`, `workers/<kind>/<address>/`, `plugins/`, `logs/`, `owner-token`; one server per data directory), creates the server's store and host, HTTP, one workers component per worker kind and the plugins component over the app's required root and the user's root, and loads the server's plugins with the kit's objects; `startWorker({ api, kinds, busy?, onError })` is a worker's entry (its directory and store, the link to the server first, its plugins). The start-data objects `AppInfo` and `WorkerInfo` serve process values to plugins. Sessions (`Sessions`: `issue`, `verify`, `revoke`, `list`; tokens kept as SHA-256 hashes, compared in constant time; persisted ones end after 400 days unused and their browser cookie is renewed daily while used, ephemeral ones live in memory, at most `limits.ephemeralSessions`) authenticate a signed `HttpOnly`, `SameSite=Strict` cookie or `Authorization: Bearer`; a one-time owner link per start (`app.ownerLink()`, the token in the URL's fragment, single use, 10 minutes), an owner token file (mode 0600, written atomically) and device pairing (`POST /_chord/pair`, owner approval, a poll returning the token once; bounded in all and per client address) sign the owner in. The data directory, its stores and secret files are made private (0700, 0600) at every start; the home directory, a file system's root and another user's directory are refused. `Files` per host: small and chunked writes at a stored offset, ranged reads, a watchable index, caps, at most `limits.uploads` uploads under way. `app.exportAddress(address, path, ctx)` and `app.importAddress(archive, address, ctx)` move a worker's store and files as a tar, holding the worker's process lock meanwhile (refusing an existing address, another app's archive, every path leaving the target, and archives past `limits.importBytes` or `limits.importEntries`). `app.applyChanges(ctx)` and `app.plugins.install(source, ctx)`. Optional plugins as exported facets: `defineUsers(profile)`, `inviteFacet()`, `ephemeralLoginFacet()`.
- Chord v2's `HttpRequest` gains `remoteAddress`, the peer's address as the listener knows it (for limits per client).
- Chord v2's HTTP component gains `readSignedCookie(request, name, secret)`: `readCookie`'s value with its signed expiry, for renewing a cookie before it expires.
### Changed
- Chord v2's HTTP component denies by default (slice 13): a request or plugin socket without a principal gets 401 before plugin code runs unless its route or socket is marked `public: true` (`{ handler, public: true }`; a `crossSite` route is public), and the `/chord` links endpoint refuses requests without a principal.
- Chord v2 plugins import each other by relative path within a root (copied into the importer's script) and across repositories by package name through the package's `exports`, pinned in `dependencies`; slice 10's imports by plugin name (`@app/notes/contract`) are removed, and a relative path leaving the root is a build error.
## [1.5.0] - 2026-11-02

101
packages/chord/docs/next/guide-plugins-hosts/book.md
View file

@ -1,7 +1,7 @@
---
title: Hosts and Plugins: a tutorial for Chord v2
cover: Hosts / and Plugins
kicker: A guide · Chord v2 · 2026-10-31
kicker: A guide · Chord v2 · 2026-11-02
subtitle: Build a small notes app step by step, from one host to plugins on a server and on workers, edited while it runs
stats:
- 3 | kinds of host in an app: server, worker, client
@ -10,10 +10,10 @@ stats:
- 1 | rule: plugins talk only through object types and registries
note: |
A tutorial for people building apps on Chord v2, written by the redesign session from `design.md`, `slice-7c.md`,
`slice-10.md`, the decision log and the code in `packages/chord/src/next` and `src/next-node`. API names are the
current ones; where slice 10's fix round or the app kit (`chord/next/app`, being designed) will change what app code
writes, a note says so.
colophon: Hosts and Plugins · second edition · 2026-11-01 · Chord v2 slices 1-10 and 12
`slice-10.md`, `slice-12.md`, `slice-13.md`, the decision log and the code in `packages/chord/src/next` and
`src/next-node`. API names are the current ones. The early chapters wire hosts by hand to show the parts; the last
chapter shows the app kit (`chord/next/app`), which does that wiring for an app.
colophon: Hosts and Plugins · third edition · 2026-11-02 · Chord v2 slices 1-10, 12 and 13
---
# Before we begin {.preface}
@ -265,10 +265,10 @@ tab.connect(webSocketLink(new WebSocket("/chord")), { principal: { kind: "server
`P` is your app's type of **principal**, who is calling: an owner, a guest, the server, a worker. Links stamp it;
a `remote` object is made per principal, so a guest can get a read-only variant of the same notes.
:::note | Coming with the app kit
Starting workers, routing a host kind to them, the WebSocket entry and authentication are slice 10's components today
(`createWorkers`, `chord.http`), wired by the app. The opinionated app kit (`chord/next/app`, being designed) will do
this wiring for you: `startApp(...)` and `startWorker()`.
:::note | The app kit does this wiring
Starting workers, routing a host kind to them, the WebSocket entry and authentication are slice 10's components
(`createWorkers`, `chord.http`). An app built with the app kit (`chord/next/app`, the last chapter) does not wire them:
`startApp(...)` on the server and `startWorker(...)` in a worker's entry do.
:::
# Client Hosts {#clients lede="Browser tabs and terminal UIs are hosts too: what they can do, and what they cannot."}
@ -633,6 +633,84 @@ same name, the first root wins, and the other is reported.
as production does, loads it on an in-memory host beside stub facets serving what it uses, collects what it
contributes to registries nobody serves, and lists the contracts it waits for (`docs/next/plugin-authoring.md`).
# The App Kit {#kit lede="startApp and startWorker: the hosts, stores, sign-in, files and plugin roots every app needs, so an app is its API, its plugins and a start script."}
The chapters so far wired an app by hand: a store, a host, `createHttp`, `createWorkers`, `createPlugins`, an
authenticator, a worker entry connecting to its parent. Every app needs the same wiring, and most of it has one right
answer. The **app kit** (`chord/next/app`) is that answer, opinionated on purpose: one data directory layout, one way
to sign in, one place for files.
## A whole app
The notes app of example 53 is four files of base code and two plugins:
```ts
// start.ts
const app = await startApp<NotesPrincipal>({
name: "notes", // the data directory: ~/.notes unless dataDir says otherwise
root: "./plugins", // the app's own plugins: a required root
api: { name: "@notes/app", versions: ["v1"], modules: { "@notes/app/v1": v1, "@notes/app/v1/server": v1server } },
kinds: { server: { platform: "node" }, worker: { platform: "node" } },
principal: { server: { kind: "server" }, worker: (address) => ({ kind: "worker", project: address.key }) },
workers: [{
kind: Project, // one worker per project
entry: "./worker.ts",
start: (address) => ({ project: address.key }),
authorize: (principal) => principal.kind === "owner" || principal.kind === "server",
}],
onError: (error) => log(error),
}, ctx);
console.log(`Open ${app.ownerLink()}`);
// worker.ts
startWorker({ api: API, kinds: KINDS, onError: (error) => log(error) });
```
`startApp` creates the data directory (`server/store.sqlite`, `workers/<kind>/<address>/`, `plugins/`, `logs/`,
`owner-token`), opens the server's store, creates the server host, HTTP, a workers component per worker kind and the
plugins component over two roots: the app's `plugins/` and the user's `<dataDir>/plugins/`. Then it loads the server's
plugins with the kit's own facets beside them and resolves. `startWorker` does the worker's half: the project's
directory and store, the host at the project's address, the link to the server first, the worker's plugins.
## What plugins get from the kit
Plugins still import only the app's API; the API re-exports the kit's objects it offers:
- `AppInfo` on the server (data directory, app name, API version) and `WorkerInfo` on workers (the address and the
start data `start(address)` made): the **start-data objects**. Process values reach plugins only through them.
- `Files`, one per host: `put` for small files, `begin`, `append` and `finish` for large ones, ranged `read`, `remove`
and a watchable `list`, stored in the host's directory with caps per host.
- `Sessions` on the server: `issue`, `revoke`, `list`.
This finishes what the previous chapter started. Base code is the API, the start script and the worker's entry. The
providers of the app's own registries live in its required root: example 53's API declares a `Sections` registry, and
its home plugin serves it and reads `AppInfo`, so it reloads like any other plugin.
## Who may come in
HTTP **denies by default**: a request without a principal gets 401 before any plugin code runs, unless the route says
`public: true`. Someone gets a principal through a **session**: a random token, kept in the store only as a hash, sent
as a signed cookie by a browser or as `Authorization: Bearer` by a command-line client.
The first session is the owner's. At each start the kit makes a one-time **owner link**,
`http://127.0.0.1:8788/_chord/owner#<token>`, valid ten minutes; the start script prints it. The token is in the
fragment, which browsers never send, so it never reaches a log: the page posts it, the kit consumes it and sets the
owner's cookie. A command-line client on the same machine reads `<dataDir>/owner-token` (mode 0600). One on another
machine **pairs**: it asks for a code, the owner approves the code in a signed-in browser, and the client's next poll
returns its token (example 54).
Users other than the owner come from optional plugins the app adds: `defineUsers(profile)` for accounts,
`inviteFacet()` for invite links, `ephemeralLoginFacet()` for "pick a name and you're in".
## Applying changes, moving projects
The kit never reloads by itself. An edit or an install marks plugins dirty; `app.plugins.state` drives the app's
"apply changes" button, and `app.applyChanges(ctx)` builds and rolls out to the server, then the workers.
A project's data is its directory. `app.exportAddress(Project({ id }), path, ctx)` stops its worker and writes one
archive (a manifest, the store's export, the files); `app.importAddress(path, Project({ id: "copy" }), ctx)` makes a new
project from it and refuses one that already has data (example 55).
# Reference {.appendix}
| Call | What it does |
@ -647,3 +725,8 @@ same name, the first root wins, and the other is reported.
| `plugins.load(host, { kind, core, modules }, ctx)`, `build(names)`, `rollout(steps, built)` | load, build, roll out |
| `plugins.installPlugin(source, root, ctx)`, `testPlugin(dir, options, ctx)` | install a plugin, test one alone |
| `hostKind(name, params?)`, `Kind(params)` | host kinds and addresses |
| `startApp<P>({ name, dataDir?, root?, listen?, api, kinds, principal, workers?, files?, onError }, ctx)` | an app's server, with the kit |
| `startWorker({ api, kinds, busy?, onError })` | a worker's entry, with the kit |
| `app.ownerLink()`, `app.applyChanges(ctx)`, `app.plugins.install(source, ctx)` | sign in, apply changes, install |
| `app.exportAddress(address, path, ctx)`, `app.importAddress(path, address, ctx)` | move a worker's data |
| `AppInfo`, `WorkerInfo`, `Files`, `Sessions` | the kit's objects plugins get |

100
packages/chord/docs/next/guide-plugins-hosts/index.html
View file

@ -109,14 +109,14 @@ table{max-width:100%}
</head>
<body><div class="book">
<section class="cover">
<div class="kicker">A guide · Chord v2 · 2026-10-31</div>
<div class="kicker">A guide · Chord v2 · 2026-11-02</div>
<h1>Hosts<br>and Plugins</h1>
<p class="subtitle">Build a small notes app step by step, from one host to plugins on a server and on workers, edited while it runs</p>
<div class="cover-stats"><div class="stat"><b>3</b><span>kinds of host in an app: server, worker, client</span></div><div class="stat"><b>1</b><span>file per host kind in a plugin</span></div><div class="stat"><b>0</b><span>restarts to apply a plugin edit</span></div><div class="stat"><b>1</b><span>rule: plugins talk only through object types and registries</span></div></div>
<div class="cover-note"><p>A tutorial for people building apps on Chord v2, written by the redesign session from <code>design.md</code>, <code>slice-7c.md</code>,
<code>slice-10.md</code>, the decision log and the code in <code>packages/chord/src/next</code> and <code>src/next-node</code>. API names are the
current ones; where slice 10&#39;s fix round or the app kit (<code>chord/next/app</code>, being designed) will change what app code
writes, a note says so.</p>
<code>slice-10.md</code>, <code>slice-12.md</code>, <code>slice-13.md</code>, the decision log and the code in <code>packages/chord/src/next</code> and
<code>src/next-node</code>. API names are the current ones. The early chapters wire hosts by hand to show the parts; the last
chapter shows the app kit (<code>chord/next/app</code>), which does that wiring for an app.</p>
</div>
</section><div class="inner">
<nav class="toc"><h2>Contents</h2><ol>
@ -130,6 +130,7 @@ writes, a note says so.</p>
<li><a href="#together">VII. Plugins Using Plugins</a></li>
<li><a href="#reload">VIII. Changing a Plugin</a></li>
<li><a href="#base">IX. Base Code and Plugins</a></li>
<li><a href="#kit">X. The App Kit</a></li>
<li><a href="#reference">Appendix A: Reference</a></li>
</ol></nav>
@ -306,9 +307,9 @@ worker.<span class="tk-f">connect</span>(parentLink, { principal: { kind: <span
tab.<span class="tk-f">connect</span>(<span class="tk-f">webSocketLink</span>(<span class="tk-k">new</span> <span class="tk-t">WebSocket</span>(<span class="tk-s">"/chord"</span>)), { principal: { kind: <span class="tk-s">"server"</span> } }, ctx);</code></pre>
<p><code>P</code> is your app&#39;s type of <strong>principal</strong>, who is calling: an owner, a guest, the server, a worker. Links stamp it;
a <code>remote</code> object is made per principal, so a guest can get a read-only variant of the same notes.</p>
<div class="note"><b class="tag">Coming with the app kit</b><p>Starting workers, routing a host kind to them, the WebSocket entry and authentication are slice 10&#39;s components today
(<code>createWorkers</code>, <code>chord.http</code>), wired by the app. The opinionated app kit (<code>chord/next/app</code>, being designed) will do
this wiring for you: <code>startApp(...)</code> and <code>startWorker()</code>.</p>
<div class="note"><b class="tag">The app kit does this wiring</b><p>Starting workers, routing a host kind to them, the WebSocket entry and authentication are slice 10&#39;s components
(<code>createWorkers</code>, <code>chord.http</code>). An app built with the app kit (<code>chord/next/app</code>, the last chapter) does not wire them:
<code>startApp(...)</code> on the server and <code>startWorker(...)</code> in a worker&#39;s entry do.</p>
</div>
</section>
@ -593,6 +594,69 @@ contributes to registries nobody serves, and lists the contracts it waits for (<
</ul>
</section>
<section class="chapter" id="kit"><header class="opener"><div class="numeral">X</div><h2>The App Kit</h2><p class="lede">startApp and startWorker: the hosts, stores, sign-in, files and plugin roots every app needs, so an app is its API, its plugins and a start script.</p></header>
<p class="dropcap">The chapters so far wired an app by hand: a store, a host, <code>createHttp</code>, <code>createWorkers</code>, <code>createPlugins</code>, an
authenticator, a worker entry connecting to its parent. Every app needs the same wiring, and most of it has one right
answer. The <strong>app kit</strong> (<code>chord/next/app</code>) is that answer, opinionated on purpose: one data directory layout, one way
to sign in, one place for files.</p>
<h3 id="a-whole-app">A whole app</h3>
<p>The notes app of example 53 is four files of base code and two plugins:</p>
<pre><code><span class="tk-c">// start.ts</span>
<span class="tk-k">const</span> app = <span class="tk-k">await</span> <span class="tk-f">startApp</span>&lt;<span class="tk-t">NotesPrincipal</span>&gt;({
name: <span class="tk-s">"notes"</span>, <span class="tk-c">// the data directory: ~/.notes unless dataDir says otherwise</span>
root: <span class="tk-s">"./plugins"</span>, <span class="tk-c">// the app's own plugins: a required root</span>
api: { name: <span class="tk-s">"@notes/app"</span>, versions: [<span class="tk-s">"v1"</span>], modules: { <span class="tk-s">"@notes/app/v1"</span>: v1, <span class="tk-s">"@notes/app/v1/server"</span>: v1server } },
kinds: { server: { platform: <span class="tk-s">"node"</span> }, worker: { platform: <span class="tk-s">"node"</span> } },
principal: { server: { kind: <span class="tk-s">"server"</span> }, worker: (address) =&gt; ({ kind: <span class="tk-s">"worker"</span>, project: address.key }) },
workers: [{
kind: <span class="tk-t">Project</span>, <span class="tk-c">// one worker per project</span>
entry: <span class="tk-s">"./worker.ts"</span>,
start: (address) =&gt; ({ project: address.key }),
authorize: (principal) =&gt; principal.kind === <span class="tk-s">"owner"</span> || principal.kind === <span class="tk-s">"server"</span>,
}],
onError: (error) =&gt; <span class="tk-f">log</span>(error),
}, ctx);
console.<span class="tk-f">log</span>(<span class="tk-s">`Open ${app.ownerLink()}`</span>);
<span class="tk-c">// worker.ts</span>
<span class="tk-f">startWorker</span>({ api: API, kinds: KINDS, onError: (error) =&gt; <span class="tk-f">log</span>(error) });</code></pre>
<p><code>startApp</code> creates the data directory (<code>server/store.sqlite</code>, <code>workers/&lt;kind&gt;/&lt;address&gt;/</code>, <code>plugins/</code>, <code>logs/</code>,
<code>owner-token</code>), opens the server&#39;s store, creates the server host, HTTP, a workers component per worker kind and the
plugins component over two roots: the app&#39;s <code>plugins/</code> and the user&#39;s <code>&lt;dataDir&gt;/plugins/</code>. Then it loads the server&#39;s
plugins with the kit&#39;s own facets beside them and resolves. <code>startWorker</code> does the worker&#39;s half: the project&#39;s
directory and store, the host at the project&#39;s address, the link to the server first, the worker&#39;s plugins.</p>
<h3 id="what-plugins-get-from-the-kit">What plugins get from the kit</h3>
<p>Plugins still import only the app&#39;s API; the API re-exports the kit&#39;s objects it offers:</p>
<ul>
<li><code>AppInfo</code> on the server (data directory, app name, API version) and <code>WorkerInfo</code> on workers (the address and the
start data <code>start(address)</code> made): the <strong>start-data objects</strong>. Process values reach plugins only through them.</li>
<li><code>Files</code>, one per host: <code>put</code> for small files, <code>begin</code>, <code>append</code> and <code>finish</code> for large ones, ranged <code>read</code>, <code>remove</code>
and a watchable <code>list</code>, stored in the host&#39;s directory with caps per host.</li>
<li><code>Sessions</code> on the server: <code>issue</code>, <code>revoke</code>, <code>list</code>.</li>
</ul>
<p>This finishes what the previous chapter started. Base code is the API, the start script and the worker&#39;s entry. The
providers of the app&#39;s own registries live in its required root: example 53&#39;s API declares a <code>Sections</code> registry, and
its home plugin serves it and reads <code>AppInfo</code>, so it reloads like any other plugin.</p>
<h3 id="who-may-come-in">Who may come in</h3>
<p>HTTP <strong>denies by default</strong>: a request without a principal gets 401 before any plugin code runs, unless the route says
<code>public: true</code>. Someone gets a principal through a <strong>session</strong>: a random token, kept in the store only as a hash, sent
as a signed cookie by a browser or as <code>Authorization: Bearer</code> by a command-line client.</p>
<p>The first session is the owner&#39;s. At each start the kit makes a one-time <strong>owner link</strong>,
<code>http://127.0.0.1:8788/_chord/owner#&lt;token&gt;</code>, valid ten minutes; the start script prints it. The token is in the
fragment, which browsers never send, so it never reaches a log: the page posts it, the kit consumes it and sets the
owner&#39;s cookie. A command-line client on the same machine reads <code>&lt;dataDir&gt;/owner-token</code> (mode 0600). One on another
machine <strong>pairs</strong>: it asks for a code, the owner approves the code in a signed-in browser, and the client&#39;s next poll
returns its token (example 54).</p>
<p>Users other than the owner come from optional plugins the app adds: <code>defineUsers(profile)</code> for accounts,
<code>inviteFacet()</code> for invite links, <code>ephemeralLoginFacet()</code> for &quot;pick a name and you&#39;re in&quot;.</p>
<h3 id="applying-changes-moving-projects">Applying changes, moving projects</h3>
<p>The kit never reloads by itself. An edit or an install marks plugins dirty; <code>app.plugins.state</code> drives the app&#39;s
&quot;apply changes&quot; button, and <code>app.applyChanges(ctx)</code> builds and rolls out to the server, then the workers.</p>
<p>A project&#39;s data is its directory. <code>app.exportAddress(Project({ id }), path, ctx)</code> stops its worker and writes one
archive (a manifest, the store&#39;s export, the files); <code>app.importAddress(path, Project({ id: &quot;copy&quot; }), ctx)</code> makes a new
project from it and refuses one that already has data (example 55).</p>
</section>
<section class="chapter appendix" id="reference"><header class="opener"><div class="numeral">A</div><h2>Reference</h2></header>
<table>
<thead>
@ -641,10 +705,30 @@ contributes to registries nobody serves, and lists the contracts it waits for (<
<td><code>hostKind(name, params?)</code>, <code>Kind(params)</code></td>
<td>host kinds and addresses</td>
</tr>
<tr>
<td><code>startApp&lt;P&gt;({ name, dataDir?, root?, listen?, api, kinds, principal, workers?, files?, onError }, ctx)</code></td>
<td>an app&#39;s server, with the kit</td>
</tr>
<tr>
<td><code>startWorker({ api, kinds, busy?, onError })</code></td>
<td>a worker&#39;s entry, with the kit</td>
</tr>
<tr>
<td><code>app.ownerLink()</code>, <code>app.applyChanges(ctx)</code>, <code>app.plugins.install(source, ctx)</code></td>
<td>sign in, apply changes, install</td>
</tr>
<tr>
<td><code>app.exportAddress(address, path, ctx)</code>, <code>app.importAddress(path, address, ctx)</code></td>
<td>move a worker&#39;s data</td>
</tr>
<tr>
<td><code>AppInfo</code>, <code>WorkerInfo</code>, <code>Files</code>, <code>Sessions</code></td>
<td>the kit&#39;s objects plugins get</td>
</tr>
</tbody></table>
</section>
<p class="colophon">Hosts and Plugins · second edition · 2026-11-01 · Chord v2 slices 1-10 and 12</p>
<p class="colophon">Hosts and Plugins · third edition · 2026-11-02 · Chord v2 slices 1-10, 12 and 13</p>
</div></div>
<script>(function activate(root) {
const replay = (f) => {

14
packages/chord/docs/next/slices.md
View file

@ -147,6 +147,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 22 | relays, size limits per part, compression at the origin, adapted bindings get diffs | 4a (relays), 4b (limits, compression), 7b |
| 22 | trace carriers (R5) | 9 |
| 23 | HTTP (routes, wraps, responses as descriptions, request bodies, plugin sockets), workers; channels and chunked transfers stay owner patterns until a second user | 10 |
| 23 | the app kit: `startApp`, `startWorker`, start-data objects, deny by default, sessions, owner link and token file, device pairing, files (with chunked transfers), export and import per address, plugin roots and "apply changes", optional users, invite and ephemeral-login plugins | 13 |
| 24 | Durable integration: `chord(fn, context)` (Q6), a `RegistryReader` over composed registries; the harness object and its model and env registries are p3's (base code, owner, 2026-10-27) | 8 (Chord parts), p3 |
| 24 | `StoreStorage` | 6b |
| 27 | versions table: contracts (L1 done), registries 8, documents 6, jobs 5, link protocol 3a | as listed |
@ -183,14 +184,13 @@ deleted (the slice's spec and code are the record). Moved here from `carry-over.
- **`rebuild` of tables** (key and column-type changes): `evolveTable` has no edit for them; build one only when a
real case needs it. (Slice 6a's second spec review; out of slice 7b's scope, `slice-7b.md` §12.)
### Slice 13: the app kit
### Slice 11: web and TUI
- **Deny by default in the HTTP component** (decision "The app kit, point 3: authentication"): routes and sockets
refuse requests without a principal unless marked public; the owner link with a one-time token; a session helper.
- **Base code reduced to the API, bootstrap and a start-data facet** (decision "The plugin model, points 3, 4, 5 and
8, imports and builds", point 8): the harness and the providers of core registries as plugins of the app's required
root, the kit's start-data objects serving process values. (Slice 12 built the required root, the API checks and
the agents' document `plugin-authoring.md`; the reduction itself is the kit's. Owner's main agent, 2026-11-01.)
- **Ephemeral sessions end with the connection** (slice 13 §3): the kit's ephemeral login drops its sessions when the
server starts and gives the browser a cookie without a lifetime; ending a session when its client's last link
closes needs the clients' connection loop, which slice 11 builds. (Slice 13 build.)
- **`startApp` gains the web surface** (slice 13 §6): serving the clients' bundles from the kit, behind its sessions.
(Slice 13 spec.)
### After slice 11

198
packages/chord/src/next-node/app/README.md Normal file
View file

@ -0,0 +1,198 @@
# app
The app kit (slice 13, design §23 "The app kit", decisions "The app kit, point 1" to "point 5"): an opinionated layer
over `http`, `workers` and `plugins` so an app is its API, its built-in plugins and a start script. `startApp` on the
server, `startWorker` in a worker's entry; the kit makes the data directory, the stores, the hosts, HTTP with deny by
default, sessions and the owner's sign-in, device pairing, each host's files, export and import per address, and the
plugin roots with "apply changes". Node-only. Exported as `@earendil-works/chord/next/app` once v2 has its entry
point. Examples 53 (`53-app/`, a whole small app), 54 (sessions and pairing) and 55 (export and import) run one app.
## For apps (`index.ts`)
```ts
// start.ts (the app's start script)
const app = await startApp<P>({
name: "acme", // the default data directory: ~/.acme
dataDir: process.argv[2], // optional; no environment variables
root: "./plugins", // the app's built-in plugins (required root); this is the default
listen: { host: "127.0.0.1", port: 8788 }, // 127.0.0.1 and a free port by default
api: { name: "@acme/app", versions: ["v3"], modules: { "@acme/app/v3": v3, "@acme/app/v3/server": v3server } },
kinds: { server: { platform: "node" }, worker: { platform: "node" } },
principal: { server: { kind: "server" }, worker: (address) => ({ kind: "worker", key: address.key }) },
workers: [{
kind: Project, // a host kind of the app, any params
entry: "./worker.ts", execArgv: [...], env: {...},
facet: "worker", // the facet kind its workers load (default)
start: (address, ctx) => ({ id: address.params.id }), // WorkerInfo.start
authorize: (principal, address) => mayReach(principal, address),
idleMs: () => 5 * 60_000,
}],
files: { maxFile: 256 * 2 ** 20, maxTotal: 4 * 2 ** 30 }, // every host's caps (these are the defaults)
install: { npmCommand: ["npm"] }, // how installs run npm
limits: { ephemeralSessions: 1000, pendingPairings: 100, pairingsPerAddress: 10, // the defaults
uploads: 64, importBytes: 4 * 2 ** 30, importEntries: 100_000 },
onError: (error) => log(error), // AppError: { kind: "host" | "http" | "workers" | "plugins", error }
}, ctx);
console.log(`Open ${app.ownerLink()}`);
app.host; app.http; app.workers.get("project"); app.plugins.state; app.dataDir; app.ownerTokenFile;
await app.plugins.install({ git: url, ref: "v1" }, ctx); // into <dataDir>/plugins/
await app.applyChanges(ctx); // build the dirty plugins, roll out: server, then workers
await app.exportAddress(Project({ id }), "p.tar", ctx);
await app.importAddress("p.tar", Project({ id: "copy" }), ctx);
await app.close(ctx); // http, workers, host, store, plugins, the lock
// worker.ts
startWorker<P>({ api, kinds, busy, onError });
```
- **The principal type** `P` is the app's; the kit needs `principal.server` and `principal.worker(address)` and adds
one variant, its owner: `{ kind: "owner" }` must be one of `P` (a type error otherwise, on `AppOptions`).
- **The API's modules** are named by convention: `<api>/<version>` for every kind and `<api>/<version>/<kind>` for a
kind's own; a kind's `packages` (React for a web kind) are host packages besides. The modules are this process's
instances; the worker's entry passes the same `api` and `kinds`.
- **`startApp`**, in order: the data directory, by its real path (refused when it is the home directory, a file
system's root, or another user's: the kit would make it private), its lock (one server per data directory, also
through a symbolic link: a second is refused after a second's wait), the layout made private (below), the cookie secret, the server's store, the host (address `Server()`,
`principal.server`), `createHttp` (the links endpoint at `/chord`), the plugins component with the two roots, one
workers component per worker kind (none without worker kinds), the server's plugins loaded with the kit's core
facets (`chord.http`, `chord.app.info`, `chord.app.sessions`, `chord.app.owner`, `chord.app.pairing`,
`chord.app.files`), the owner token file, the listening port, the owner link. A plugin of the required root failing
rejects with the plugins component's `PluginError`, everything closed. `close` is idempotent.
- **`startWorker`** is `workerMain` plus: the worker's directory `workers/<kind>/<key>/` and its store there, the host
at the address acting as `principal.worker(address)`, connected to the server first (trusted, as the server's
principal), its plugins loaded with the worker's core facets (`chord.app.worker-info`, `chord.app.files`), `busy`
from the app. A worker's errors reach the server as a fixed "worker failed" (the error stays in the worker's
`onError`).
## Start data
The kit's start-data objects serve process values to plugins: `AppInfo` (`chord.app.info`, on the server: `dataDir`,
`name`, `version`, the newest API version) and `WorkerInfo` (`chord.app.worker-info`, on workers: `address`, `start`,
the start data the app's `start(address)` made). Plugins `self.get(AppInfo, {})` through the app's API, which
re-exports them; nothing else of the process reaches a plugin.
## Base code
An app's base code is its API, the start script and the worker's entry; everything else is a plugin. The pattern
(decision "The plugin model", point 8): the providers of the app's core registries and its harness objects are plugins
of the app's required root, not facets the start script passes, so they reload like any plugin and the API only
declares the registries; process values (paths, the start data, the version) reach them through the start-data
objects, not through closures over the start script. Example 53: the API declares `Sections`, the home plugin serves it
and reads `AppInfo`.
## Sessions and sign-in
- **Deny by default** (`http`): a route or socket without a principal answers 401 before plugin code runs unless its
contribution says `public: true`; the `/chord` links endpoint needs a principal.
- **Sessions** (`Sessions`, `chord.app.sessions`, local, one per server): `issue({ principal, label, kind: "browser"
| "device" | "ephemeral", secure? })` returns `{ id, token, cookie }`, `verify(token)` the principal or `null`,
`revoke(id)`, `list` (browser and device sessions: `id`, `label`, `kind`, `principal`, `created`, `lastUsed`; never a
token or hash). A token is `<id>.<secret>` (32 random bytes, base64url); only the secret's SHA-256 is kept, compared
in constant time. The kit's authenticator reads `Authorization: Bearer <token>` or the session cookie
`chord-session` (signed with `server/cookie-secret`, `HttpOnly`, `SameSite=Strict`, `Secure` for a remote browser,
400 days, renewed by a response once older than a day, so a browser used within every 400 days stays signed in;
an ephemeral one only while the browser runs); a malformed token or an id that is no session (a prototype
key included) is a plain 401. Browser and device sessions persist in the server's store; `lastUsed` is written at
most once a minute, and a session unused for 400 days ends (checked when it is used, removed then). Ephemeral
sessions (a public login) live in memory only, never listed: at most `limits.ephemeralSessions`, none outliving the
server's run, so a login costs the same whatever came before. Past the cap the oldest ends: a flood of logins signs
out the earliest ephemeral users (the trade for bounded memory).
- **The owner link:** each start makes a one-time token (single use, 10 minutes, the previous start's no longer
valid), `app.ownerLink()` is `http://<the address listened on>:<port>/_chord/owner#<token>` (`127.0.0.1` for
`0.0.0.0`, `[::1]` for `::`, an IPv6 address in brackets). The page at `GET /_chord/owner` posts
the fragment's token to `POST /_chord/owner` (public), which consumes it and sets an owner browser session's cookie.
The token stays in the fragment: browsers never send it, so it never reaches a log. No owner by loopback alone.
- **The owner token file** `<dataDir>/owner-token`: a device session of the owner for command-line clients on the same
machine, mode 0600, written in one step (a temporary file made exclusively with mode 0600, then linked or renamed into
place). Kept while its token authenticates as the owner (its session exists, the secret matches, not expired; its
mode set to 0600 again); replaced at the next start otherwise.
- **Device pairing** for clients elsewhere: `POST /_chord/pair` (public, `{ label? }`) returns `{ code, poll }`; the
owner opens `/_chord/pair/<code>` signed in (owner only, a confirm page naming the device) and approves; the
client's `POST /_chord/pair/poll { poll }` answers `{ status: "waiting" }` until then, `{ status: "approved", token }`
once, then 404. Codes are `XXXX-XXXX` from 31 unambiguous characters (`crypto.randomInt`), single use, 10 minutes;
the poll secret is kept hashed. Asking is public, so pairings anonymous clients start are bounded:
`limits.pendingPairings` in all, `limits.pairingsPerAddress` per client address (`HttpRequest.remoteAddress`); past
them 429. The trade: whoever fills the caps locks anonymous pairing out for up to 10 minutes, and behind a tunnel or
relay every remote client shares one address (the tunnel's), so ten requests from one party block remote pairing
for all. A signed-in owner's own `POST /_chord/pair` is not counted: the owner can always start one. Pairings and the owner link live in memory: a restart drops them.
## Optional plugins
`defineUsers(profile)` (accounts by id with the app's profile schema: `Users` with `list`, `get`, `update`,
`remove`), `inviteFacet()` (the owner's `POST /_chord/invites { principal, label }` makes a 7-day single-use link,
whose page signs the invitee in as that principal) and `ephemeralLoginFacet<P>()` (a public `POST /_chord/login {
name }` signing in as `{ kind: "user", id: <fresh>, name }`, a type error unless `P` allows it). They are exported
facets, not plugin directories: each takes the app's settings as arguments (a profile schema, the principal type),
and a plugin directory could not import the kit (plugins import only the app's API). The app re-exports one through
its API and adds a one-line plugin to its root:
```ts
// app/v3/server.ts
export const users = defineUsers(obj({ name: str(), avatar: opt(str()) }));
// plugins/users/server.ts
import { users } from "@acme/app/v3/server";
export default users.facet;
```
## Storage and files
```
<dataDir>/ mode 0700, like every directory below
server/store.sqlite the server's store, mode 0600 (and its journal files)
server/cookie-secret mode 0600
server/files/ the server's files, mode 0600
workers/<kind>/<key>/ a worker's store.sqlite and files/
plugins/ the user's plugins root
logs/
owner-token mode 0600
```
Every start sets these modes again, on the server's part when the server starts and on a worker's when it starts:
a data directory made looser before (or restored with other modes) is tightened, so other local users can neither
read a store (it holds the sessions) nor replace it.
A worker's key is percent-encoded as one directory name (`/`, `\`, `.` and capital letters included, so keys differing
only in case stay apart on a case-insensitive file system), or `sha256-<hex>` past 160 characters.
`Files` (`chord.app.files`, local, one per host: the server's and each worker's) keeps a host's files in its `files/`
and their index (id, name, mime, size, created) in its store: `put({ name, mime, bytes })` (at most 4 MiB),
chunked transfers (design §23): `begin({ name, mime })`, `append({ upload, offset, bytes })` (at most 4 MiB a chunk,
about 1 MiB usual; `offset` must be the bytes stored so far, else `invalid` with the stored `{ offset }`),
`uploaded(upload)` (where a writer resumes after a drop), `finish(upload)`; `read({ id, offset, length })` (at most 4
MiB), `remove(id)`, `list`. Appends may be sent without waiting for the previous one: each is accepted at the offset
the accepted ones reached and written at its position (one file handle per upload); `uploaded`, `finish` and expiry
wait for the writes in flight, and a failed write ends its upload. Each upload under way holds a file handle, so at
most `limits.uploads` (64) are under way per `Files` object: past it `begin` is refused `unavailable` (`busy`) until
one finishes or an idle one expires. Caps per host (`files` option): a file over `maxFile` or a write past `maxTotal` (uploads
under way count) is refused `too_large`. An upload unwritten for an hour expires; uploads end with the object, and
those a previous run left are dropped when the object goes live. `Files` is `local`: a client uploads through the
plugin's own remote methods, which call it.
## Export and import
`exportAddress(address, path, ctx)` stops the address's worker (`workers.stop`) and takes its process lock (the one a
worker takes at start) for the whole pack, writes the archive to `path` (mode 0600, staged and renamed), then releases
the lock and lets the address start again (`workers.retry`): a hold arriving meanwhile starts a worker that waits for
the lock, then opens the unchanged state. `importAddress` holds the same lock while it unpacks. A failed address (a
crashed worker nobody retried) is not stopped or retried: it stays failed. The archive is a ustar of
`manifest.json` (`{ format: 1, app, api, address }`), `store.archive` (the store's export, JSONL) and `files/…`.
`importAddress(archive, address, ctx)` refuses an address that has a directory, unpacks beside it and renames into
place: the manifest must come first and name this app and format 1; only regular files and directories under
`manifest.json`, `store.archive` and `files/` are taken; absolute paths, `..`, `.`, empty segments, backslashes, NULs,
links, devices and bad checksums are refused (the kit's own tar reader, `tar.ts`); past `limits.importBytes` unpacked
bytes or `limits.importEntries` entries (counted from the headers before writing) it is refused. A refused or broken
archive leaves nothing. The next hold starts the worker on the imported store and files.
## Plugins
Roots: the app's `root` (required: a plugin of it failing stops `startApp`) and `<dataDir>/plugins/` (the user's:
installs go there). `app.plugins.state` is the plugins component's marks for the app's "apply changes" button;
`applyChanges` builds the dirty plugins (and installed ones) and rolls out to the server, then each worker kind;
nothing is applied by itself.
## Internals
`app.ts` (`startApp`, the `App`), `worker.ts` (`startWorker`), `layout.ts` (the layout, private directories and
stores), `secrets.ts` (tokens, hashes, files of mode 0600), `sessions.ts`, `owner.ts` (owner link and token file),
`pairing.ts`, `files.ts` and `uploads.ts` (chunked uploads), `info.ts` (start-data objects), `archive.ts` and `tar.ts`
(export and import), and the optional plugins `users.ts`, `invite.ts`, `ephemeral.ts`.

488
packages/chord/src/next-node/app/app.ts Normal file
View file

@ -0,0 +1,488 @@
// `startApp` (slice 13 §1, §5; decisions "The app kit, point 1" to "point 5"): the server of an app built with the kit.
// It lays out the data directory, opens the server's store, creates the server host, the HTTP component and one
// workers component per worker kind, the plugins component over the app's required root and the user's root, loads
// the server's plugins with the kit's own facets (the start-data object, sessions, the owner link, pairing, files),
// and gives the app the controls it needs: the owner link, `applyChanges`, installs, export and import per address.
import { join, resolve } from "node:path";
import { type Context, withAbortSignal } from "../../context/index.ts";
import type { JsonValue } from "../../json.ts";
import { type Address, type HostKind, hostKind, type KindShape } from "../../next/contracts/index.ts";
import type { Principal } from "../../next/data/index.ts";
import { createHost, type Host, type HostError, type PluginFacet } from "../../next/host/index.ts";
import { openStore, type Store, sqliteBackend } from "../../next/store/index.ts";
import { openNodeSqliteDatabase } from "../../sqlite/node.ts";
import type { Watchable } from "../../watch.ts";
import { createHttp, type HttpComponent, type HttpError, type ListenAddress } from "../http/index.ts";
import {
createPlugins,
type HostPackages,
type InstallOptions,
type PluginSource,
type Plugins,
type PluginsError,
type PluginsState,
type RolloutResult,
} from "../plugins/index.ts";
import { createWorkers, type ProcessLock, processLock, type Workers, type WorkersError } from "../workers/index.ts";
import { ARCHIVE_FORMAT, type ArchiveManifest, packAddress, unpackAddress } from "./archive.ts";
import { type FilesCaps, filesFacet } from "./files.ts";
import { appInfoFacet } from "./info.ts";
import { type AppLayout, createLayout, defaultDataDir, openDataDir, privateStore, workerDirectory } from "./layout.ts";
import { OWNER_PATH, OwnerLink, type OwnerPrincipal, ownerFacet, ownerTokenFile } from "./owner.ts";
import { Pairings, pairingFacet } from "./pairing.ts";
import { secretFile } from "./secrets.ts";
import { sessionsFacet } from "./sessions.ts";
/** The server's host kind: the kit's server host is `Server()`. */
export const Server = hostKind("server");
/**
* The app's API (slice 12): its package name, the versions it provides (newest first) and this process's modules of
* them by specifier (`@acme/app/v3`, `@acme/app/v3/server`).
*/
export type AppApi = {
readonly name: string;
readonly versions: readonly string[];
readonly modules: HostPackages;
};
/**
* A facet kind of the app: where its builds run, and the packages besides the API its plugins share (`react`). Its API
* modules are named by convention: `<api>/<version>` and `<api>/<version>/<kind>` for every version.
*/
export type AppKind = { readonly platform: "node" | "browser"; readonly packages?: HostPackages };
/** A worker kind: a host kind of the app and how its workers start (the workers component's options). */
export type AppWorkerKind<P extends Principal> = {
/** The host kind (`hostKind("project", { id: str() })`). */
readonly kind: KindShape;
/** The worker's entry module (it calls `startWorker`), its Node flags and environment. */
readonly entry: string;
readonly execArgv?: readonly string[];
readonly env?: Readonly<Record<string, string>>;
/** The facet kind its workers load: `worker` by default. */
readonly facet?: string;
/** The app's start data for the worker at `address` (JSON), before each start. */
readonly start: (address: Address, context: Context) => Promise<JsonValue> | JsonValue;
/** Whether `principal` may reach the worker at `address` (synchronous). */
readonly authorize: (principal: P, address: Address) => boolean;
/** How long a worker idles before it stops: 5 minutes by default. */
readonly idleMs?: () => number;
};
/** What the kit reports: from its host, its HTTP component, its workers or its plugins. */
export type AppError =
| { readonly kind: "host"; readonly error: HostError }
| { readonly kind: "http"; readonly error: HttpError }
| { readonly kind: "workers"; readonly error: WorkersError }
| { readonly kind: "plugins"; readonly error: PluginsError };
/** What `startApp` takes. `{ kind: "owner" }` must be one of the app's principals (the kit's owner). */
export type AppOptions<P extends Principal> = {
/** The app's name: its default data directory is `~/.<name>`, its archives carry it. */
readonly name: string;
/** The data directory (the start script's to decide; no environment variables): `~/.<name>` by default. */
readonly dataDir?: string;
/** The app's built-in plugins root (required): `plugins/` of the working directory by default. */
readonly root?: string;
/** Where the server listens: `127.0.0.1`, a free port by default. */
readonly listen?: ListenAddress;
readonly api: AppApi;
/** The app's facet kinds: `server` and each worker kind's facet kind among them. */
readonly kinds: Readonly<Record<string, AppKind>>;
/** The server's principal and each worker's (stamped on its link). */
readonly principal: { readonly server: P; readonly worker: (address: Address) => P };
/** The worker kinds; none: a server with clients only. */
readonly workers?: readonly AppWorkerKind<P>[];
/** The caps of each host's files (the server's and every worker's). */
readonly files?: FilesCaps;
/** The bounds of what public endpoints and imports may cost. */
readonly limits?: AppLimits;
/** How installs run npm (slice 12's `InstallOptions`): `npm` on the `PATH` by default. */
readonly install?: InstallOptions;
readonly onError: (error: AppError) => void;
} & (OwnerPrincipal extends P ? unknown : { readonly "the app's principal type must have { kind: 'owner' }": never });
/**
* The bounds of what public endpoints, clients and imports may cost: live ephemeral sessions (1,000; past it the
* oldest ends, signed out), pairings under way that anonymous clients started (100 in all, 10 per client address; past
* them 429; the owner's own are not counted), uploads under way per `Files` object (64; past it `unavailable`, `busy`),
* an archive's unpacked bytes (4 GiB) and entries (100,000).
*/
export type AppLimits = {
readonly uploads?: number;
readonly ephemeralSessions?: number;
readonly pendingPairings?: number;
readonly pairingsPerAddress?: number;
readonly importBytes?: number;
readonly importEntries?: number;
};
/** The plugins of a running app, for its "apply changes" control and its installs. */
export type AppPlugins = {
/** The marks (`dirty`, `install`, `set`, `base`): what the app's button shows. */
readonly state: Watchable<PluginsState>;
/** Installs plugins from git or npm into the user's root (slice 12's `installPlugin`); their names. */
install(source: PluginSource, context: Context): Promise<readonly string[]>;
/** The plugins component itself. */
readonly component: Plugins;
};
/** A running app. */
export interface App<P extends Principal> {
readonly host: Host<P>;
readonly http: HttpComponent<P>;
/** The workers components, by host kind name. */
readonly workers: ReadonlyMap<string, Workers<HostKind>>;
readonly plugins: AppPlugins;
/** The data directory. */
readonly dataDir: string;
/** The owner link of this start (single use, 10 minutes): `http://127.0.0.1:<port>/_chord/owner#<token>`. */
ownerLink(): string;
/** The owner token file's path (a device session of the owner, mode 0600). */
readonly ownerTokenFile: string;
/** Builds the dirty plugins and rolls them out: the server, then every worker kind; never by itself. */
applyChanges(context: Context): Promise<RolloutResult>;
/** Stops the worker at `address` and writes its archive (store and files) to `path`. */
exportAddress(address: Address, path: string, context: Context): Promise<void>;
/** Makes `address` (which must have no data yet) from the archive at `archive`; its next hold starts it. */
importAddress(archive: string, address: Address, context: Context): Promise<void>;
/** Closes the HTTP component, the workers, the host, the store, the watcher and the data directory's lock. */
close(context: Context): Promise<void>;
}
/** What `startWorker` reads from its start data besides the app's: the kit's part. */
export type KitStart = {
readonly dir: string;
readonly facet: string;
readonly principal: JsonValue;
readonly server: JsonValue;
readonly files: { readonly maxFile?: number; readonly maxTotal?: number };
/** The cap of uploads under way per `Files` object (`limits.uploads`). */
readonly uploads?: number;
};
/**
* Starts the app's server (§1): resolves once its plugins are loaded and it listens. A failing plugin of the app's
* root (or a problem discovery reports there) rejects with the plugins component's `PluginError`, everything closed.
*/
export async function startApp<P extends Principal>(options: AppOptions<P>, context: Context): Promise<App<P>> {
const { layout, cookieSecret, store, lock } = await openServer(
resolve(options.dataDir ?? defaultDataDir(options.name)),
context,
);
const { onError, api, limits = {} } = options;
const host = createHost<P>({
address: Server(),
principal: options.principal.server,
store,
onError: (error) => onError({ kind: "host", error }),
});
const http = createHttp(
host,
{
listen: options.listen ?? { host: "127.0.0.1", port: 0 },
links: { path: "/chord" },
onError: (error) => onError({ kind: "http", error }),
},
context,
);
const plugins = createPlugins(
{
roots: [{ path: resolve(options.root ?? "plugins"), required: true }, layout.plugins],
api: { name: api.name, versions: api.versions },
kinds: Object.fromEntries(
Object.entries(options.kinds).map(([name, kind]) => [
name,
{ platform: kind.platform, hostPackages: hostPackagesOf(api, name, kind) },
]),
),
install: options.install,
onError: (error) => onError({ kind: "plugins", error }),
},
context,
);
const workers = new Map<string, Workers<HostKind>>();
for (const kind of options.workers ?? []) {
workers.set(kind.kind.kind, workersOf(host, kind, options, layout.dataDir, plugins, context));
}
const link = new OwnerLink();
const core: PluginFacet<P>[] = [
{ name: "chord.http", facet: http.facet },
{
name: "chord.app.info",
facet: appInfoFacet({ dataDir: layout.dataDir, name: options.name, version: api.versions[0] ?? "" }),
},
{ name: "chord.app.sessions", facet: sessionsFacet(cookieSecret, { maxEphemeral: limits.ephemeralSessions }) },
{ name: "chord.app.owner", facet: ownerFacet(link) },
{
name: "chord.app.pairing",
facet: pairingFacet(
new Pairings({ maxPending: limits.pendingPairings, perAddress: limits.pairingsPerAddress }),
),
},
{ name: "chord.app.files", facet: filesFacet(join(layout.server, "files"), options.files, limits.uploads) },
];
const app = new RunningApp(host, http, workers, plugins, { store, lock }, layout, options);
try {
await plugins.load(host, { kind: "server", core, modules: modulesOf(api, options.kinds.server) }, context);
await ownerTokenFile(layout.ownerToken, host, context);
app.started(link.renew(), await http.listening(context));
} catch (error) {
await app.close(context);
throw error;
}
return app;
}
/** How long a second server waits for the data directory's lock before it is refused. */
const SERVER_LOCK_MS = 1000;
/**
* The server's own part of the data directory `path`: its lock (one server per data directory, by its real path: a
* second one, also through a symbolic link, would share its stores and secrets), the layout made private, the cookie
* secret and the server's store; the lock released when the rest fails.
*/
async function openServer(
path: string,
context: Context,
): Promise<{ layout: AppLayout; cookieSecret: string; store: Store; lock: ProcessLock }> {
const dataDir = await openDataDir(path);
const waited = AbortSignal.timeout(SERVER_LOCK_MS);
const lock = await processLock(`chord-app:${dataDir}`, withAbortSignal(waited, context)).catch((error: unknown) => {
throw waited.aborted ? new Error(`startApp: another server runs on ${dataDir}`) : error;
});
try {
const layout = await createLayout(dataDir);
const cookieSecret = await secretFile(layout.cookieSecret);
await privateStore(layout.serverStore);
const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(layout.serverStore)), context);
return { layout, cookieSecret, store, lock };
} catch (error) {
await lock.release();
throw error;
}
}
/** The host packages of kind `name`: the API's shared and per-kind module of every version, and its packages. */
function hostPackagesOf(api: AppApi, name: string, kind: AppKind): string[] {
const modules = api.versions.flatMap((version) => [`${api.name}/${version}`, `${api.name}/${version}/${name}`]);
return [...modules, ...Object.keys(kind.packages ?? {})];
}
/** This process's modules for a kind: the API's and the kind's packages. */
function modulesOf(api: AppApi, kind: AppKind | undefined): HostPackages {
return { ...api.modules, ...kind?.packages };
}
/** The workers component of one worker kind: started with the kit's part of the start data beside the app's. */
function workersOf<P extends Principal>(
host: Host<P>,
kind: AppWorkerKind<P>,
options: AppOptions<P>,
dataDir: string,
plugins: Plugins,
context: Context,
): Workers<HostKind> {
const facet = kind.facet ?? "worker";
return createWorkers(
host,
// A host kind of the app: any params; its addresses come back from its route as the kit hands them on.
kind.kind as HostKind,
{
kind: facet,
entry: kind.entry,
...(kind.execArgv === undefined ? {} : { execArgv: kind.execArgv }),
...(kind.env === undefined ? {} : { env: kind.env }),
authorize: kind.authorize,
principal: options.principal.worker,
start: async (address, ctx) => {
const kit: KitStart = {
dir: workerDirectory(dataDir, address),
facet,
// Principals are JSON (they cross links).
principal: options.principal.worker(address) as unknown as JsonValue,
server: options.principal.server as unknown as JsonValue,
files: { ...options.files },
...(options.limits?.uploads === undefined ? {} : { uploads: options.limits.uploads }),
};
return { kit, app: await kind.start(address, ctx) };
},
lock: (address) => workerLock(options.name, dataDir, address),
...(kind.idleMs === undefined ? {} : { idleMs: kind.idleMs }),
plugins,
onError: (error) => options.onError({ kind: "workers", error }),
},
context,
);
}
/** The process lock of the worker at `address`: its worker holds it while it runs, an export or import meanwhile. */
function workerLock(name: string, dataDir: string, address: Address): string {
return `${name}:${dataDir}:${address.key}`;
}
/** An address `#quiesce` made quiet: its worker's lock, and whether it was failed before. */
type Quiet = { readonly lock: ProcessLock; readonly failed: boolean };
/** How long an export or import waits for a stopped worker's lock before it stops the worker again. */
const ADDRESS_LOCK_MS = 2000;
/** The host of the owner link for the address listened on: loopback for a wildcard, an IPv6 address in brackets. */
function linkHost(host: string): string {
if (host === "0.0.0.0" || host === "") return "127.0.0.1";
if (host === "::") return "[::1]";
return host.includes(":") ? `[${host}]` : host;
}
class RunningApp<P extends Principal> implements App<P> {
readonly host: Host<P>;
readonly http: HttpComponent<P>;
readonly workers: ReadonlyMap<string, Workers<HostKind>>;
readonly plugins: AppPlugins;
readonly dataDir: string;
readonly ownerTokenFile: string;
readonly #component: Plugins;
readonly #store: Store;
readonly #lock: ProcessLock;
readonly #options: AppOptions<P>;
#link = "";
#closing: Promise<void> | undefined;
constructor(
host: Host<P>,
http: HttpComponent<P>,
workers: ReadonlyMap<string, Workers<HostKind>>,
plugins: Plugins,
server: { readonly store: Store; readonly lock: ProcessLock },
layout: { readonly dataDir: string; readonly plugins: string; readonly ownerToken: string },
options: AppOptions<P>,
) {
this.host = host;
this.http = http;
this.workers = workers;
this.#component = plugins;
this.#store = server.store;
this.#lock = server.lock;
this.#options = options;
this.dataDir = layout.dataDir;
this.ownerTokenFile = layout.ownerToken;
this.plugins = {
state: plugins.state,
install: (source, context) => plugins.installPlugin(source, layout.plugins, context),
component: plugins,
};
}
/** The start finished: the owner link's token and the address it names (the one listened on). */
started(token: string, listening: ListenAddress): void {
this.#link = `http://${linkHost(listening.host)}:${listening.port}${OWNER_PATH}#${token}`;
}
ownerLink(): string {
return this.#link;
}
async applyChanges(context: Context): Promise<RolloutResult> {
const plugins = this.#component;
const built = await plugins.build(plugins.state.value.dirty, context);
const steps = [plugins.step("server", this.host), ...[...this.workers].map(([name, each]) => each.step(name))];
return plugins.rollout(steps, built, context);
}
async exportAddress(address: Address, path: string, context: Context): Promise<void> {
const quiet = await this.#quiesce(address, context);
try {
const manifest = this.#manifest(address);
await packAddress(workerDirectory(this.dataDir, address), manifest, resolve(path), context);
} finally {
await this.#resume(address, quiet, context);
}
}
async importAddress(archive: string, address: Address, context: Context): Promise<void> {
const { name, limits = {} } = this.#options;
const accept = (manifest: ArchiveManifest): void => {
// A trust boundary: an archive from anywhere.
if (manifest?.format !== ARCHIVE_FORMAT)
throw new Error(`importAddress: not a format ${ARCHIVE_FORMAT} archive`);
if (manifest.app !== name) throw new Error(`importAddress: an archive of "${manifest.app}", not "${name}"`);
};
const quiet = await this.#quiesce(address, context);
try {
const bounds = { bytes: limits.importBytes, entries: limits.importEntries };
await unpackAddress(resolve(archive), workerDirectory(this.dataDir, address), accept, bounds, context);
} finally {
await this.#resume(address, quiet, context);
}
}
/**
* Takes the process lock of the worker at `address` (the one its worker takes at start) for an export or an import,
* stopping a worker that is not failed first: while it is held, a hold arriving starts a worker that waits for it,
* then opens the finished state. A worker started between the stop and the lock holds the lock first: it is
* stopped again (at most 5 times). A failed address stays failed: it starts nothing (its holds see its failure),
* and `#resume` leaves it so.
*/
async #quiesce(address: Address, context: Context): Promise<Quiet> {
const workers = this.#workersOf(address);
const failed = workers.state.value[address.key]?.status === "failed";
for (let attempt = 1; ; attempt++) {
if (!failed) await workers.stop(address, context);
const waited = AbortSignal.timeout(ADDRESS_LOCK_MS);
try {
const lock = await processLock(
workerLock(this.#options.name, this.dataDir, address),
withAbortSignal(waited, context),
);
return { lock, failed };
} catch (error) {
if (!waited.aborted) throw error;
if (attempt === 5) throw new Error(`app: the worker at ${address.key} keeps starting; try again`);
}
}
}
/** Releases the lock `#quiesce` took, and lets the address start again unless it was failed before. */
async #resume(address: Address, quiet: Quiet, context: Context): Promise<void> {
await quiet.lock.release();
if (!quiet.failed) await this.#workersOf(address).retry(address, context);
}
close(context: Context): Promise<void> {
this.#closing ??= this.#close(context);
return this.#closing;
}
/** The workers of `address`'s kind. */
#workersOf(address: Address): Workers<HostKind> {
const workers = this.workers.get(address.kind);
if (workers === undefined) throw new Error(`app: no worker kind "${address.kind}"`);
return workers;
}
/** The manifest of `address`'s archive. */
#manifest(address: Address): ArchiveManifest {
const { name, api } = this.#options;
return {
format: ARCHIVE_FORMAT,
app: name,
api: api.versions[0] ?? "",
address: { kind: address.kind, key: address.key },
};
}
/**
* The close, in order: HTTP first (it drains requests the host's close would seal), workers, host, store, the
* plugins' watcher, then the data directory's lock.
*/
async #close(context: Context): Promise<void> {
await this.http.close(context);
for (const workers of this.workers.values()) await workers.close(context);
await this.host.close(context);
await this.#store.close(context);
this.#component.close();
await this.#lock.release();
}
}

194
packages/chord/src/next-node/app/archive.ts Normal file
View file

@ -0,0 +1,194 @@
// Address archives (slice 13 §4, decision "The app kit, point 5" (export and import per address)): one tar of a
// worker's address, `manifest.json` (the app, its API version, the kit's format version), `store.archive` (the store's
// export archive, slice 6a, as JSONL) and `files/` (its files directory). Packing reads a stopped worker's directory;
// unpacking makes a new directory for an address that has none, refusing every path that would leave it.
import { createReadStream, createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readdir, rename, rm, stat } from "node:fs/promises";
import { dirname, join, relative, sep } from "node:path";
import { pipeline } from "node:stream/promises";
import type { Context } from "../../context/index.ts";
import { decodeJsonl, encodeJsonl, importStore, openStore, sqliteBackend } from "../../next/store/index.ts";
import { openNodeSqliteDatabase } from "../../sqlite/node.ts";
import { privateDirectory, privateStore } from "./layout.ts";
import { readTar, type TarEntry, writeTar } from "./tar.ts";
/** The kit's archive format version. */
export const ARCHIVE_FORMAT = 1;
/** What an archive says about itself. */
export type ArchiveManifest = {
readonly format: number;
readonly app: string;
readonly api: string;
readonly address: { readonly kind: string; readonly key: string };
};
/** The store file of a host directory. */
const STORE = "store.sqlite";
/**
* Packs the host directory `dir` (its worker stopped: nothing writes it) into the archive at `path`: `manifest`, its
* store's export (an empty store's when it has none yet) and its `files/` (uploads under way left out).
*/
export async function packAddress(
dir: string,
manifest: ArchiveManifest,
path: string,
context: Context,
): Promise<void> {
if (!(await exists(dir))) throw new Error(`exportAddress: ${manifest.address.key} has no data`);
const staging = await mkdtemp(join(dirname(path), ".chord-export-"));
try {
const exported = join(staging, "store.archive");
const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(join(dir, STORE))), context);
try {
await pipeline(encodeJsonl(store.export(context)), createWriteStream(exported));
} finally {
await store.close(context);
}
const text = new TextEncoder().encode(JSON.stringify(manifest));
const entries: TarEntry[] = [
{ path: "manifest.json", size: text.length, data: chunksOf(text) },
{ path: "store.archive", size: (await stat(exported)).size, data: createReadStream(exported) },
...(await filesOf(join(dir, "files"))),
];
const written = join(staging, "archive.tar");
await pipeline(writeTar(entries), createWriteStream(written, { mode: 0o600 }));
await rename(written, path);
} finally {
await rm(staging, { recursive: true, force: true });
}
}
/** One chunk. */
async function* chunksOf(bytes: Uint8Array): AsyncGenerator<Uint8Array> {
yield bytes;
}
/** The files under `root` as entries `files/<relative path>` (hidden ones, uploads under way, left out). */
async function filesOf(root: string): Promise<TarEntry[]> {
const entries: TarEntry[] = [];
const walk = async (dir: string): Promise<void> => {
let names: string[];
try {
names = (await readdir(dir)).sort();
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return;
throw error;
}
for (const name of names) {
if (name.startsWith(".")) continue;
const path = join(dir, name);
const info = await stat(path);
if (info.isDirectory()) await walk(path);
else if (info.isFile()) {
const inside = relative(root, path).split(sep).join("/");
entries.push({ path: `files/${inside}`, size: info.size, data: createReadStream(path) });
}
}
};
await walk(root);
return entries;
}
/** What an import takes at most: unpacked bytes (4 GiB by default) and entries (100,000 by default). */
export type ImportLimits = { readonly bytes?: number; readonly entries?: number };
/**
* Unpacks the archive at `archive` into the new host directory `dir` (refused when it exists): its manifest checked by
* `accept` (a throw refuses it), the store made from `store.archive`, `files/` restored, within `limits`. Staged beside
* `dir` (mode 0700, files 0600) and renamed into place at the end, so a refused or broken archive leaves nothing.
*/
export async function unpackAddress(
archive: string,
dir: string,
accept: (manifest: ArchiveManifest) => void,
limits: ImportLimits,
context: Context,
): Promise<void> {
if (await exists(dir)) throw new Error(`importAddress: ${dir} exists already (an address is imported once)`);
await privateDirectory(dirname(dir));
const staging = await mkdtemp(join(dirname(dir), ".chord-import-"));
try {
const storeArchive = await unpackEntries(archive, staging, accept, limits, context);
await privateStore(join(staging, STORE));
const backend = sqliteBackend(await openNodeSqliteDatabase(join(staging, STORE)));
const store = await importStore(backend, decodeJsonl(createReadStream(storeArchive)), context);
await store.close(context);
await rm(storeArchive);
await rename(staging, dir);
} catch (error) {
await rm(staging, { recursive: true, force: true });
throw error;
}
}
/**
* Writes the entries of `archive` into `staging`: the manifest first (checked by `accept` before any data is written),
* then `store.archive` and `files/…`, within `limits` (counted from the headers, before an entry is written). The path
* of the staged `store.archive`.
*/
async function unpackEntries(
archive: string,
staging: string,
accept: (manifest: ArchiveManifest) => void,
limits: ImportLimits,
context: Context,
): Promise<string> {
const maxBytes = limits.bytes ?? 4 * 1024 ** 3;
const maxEntries = limits.entries ?? 100_000;
let manifest: ArchiveManifest | undefined;
let storeArchive: string | undefined;
let bytes = 0;
let entries = 0;
for await (const entry of readTar(createReadStream(archive))) {
context.abortSignal?.throwIfAborted();
if (++entries > maxEntries) throw new Error(`importAddress: the archive has more than ${maxEntries} entries`);
if (entry.kind === "directory") continue;
bytes += entry.size;
if (bytes > maxBytes) throw new Error(`importAddress: the archive unpacks to more than ${maxBytes} bytes`);
if (manifest === undefined && entry.path !== "manifest.json") {
throw new Error("importAddress: the archive does not start with manifest.json");
}
if (entry.path === "manifest.json") {
// A trust boundary: the archive's own description, checked by `accept`.
manifest = JSON.parse(new TextDecoder().decode(await collect(entry.data, 64 * 1024))) as ArchiveManifest;
accept(manifest);
} else if (entry.path === "store.archive") {
storeArchive = join(staging, "store.archive");
await pipeline(entry.data, createWriteStream(storeArchive, { mode: 0o600 }));
} else if (entry.path.startsWith("files/")) {
// `readTar` refused absolute paths and `..`: `files/<path>` stays inside the staging directory.
const target = join(staging, ...entry.path.split("/"));
await mkdir(dirname(target), { recursive: true, mode: 0o700 });
await pipeline(entry.data, createWriteStream(target, { mode: 0o600 }));
} else {
throw new Error(`importAddress: the archive holds ${entry.path}, which an address archive does not`);
}
}
if (manifest === undefined || storeArchive === undefined) {
throw new Error("importAddress: the archive has no manifest.json or store.archive");
}
return storeArchive;
}
/** The bytes of `data`, refused past `limit`. */
async function collect(data: AsyncIterable<Uint8Array>, limit: number): Promise<Uint8Array> {
const chunks: Uint8Array[] = [];
let size = 0;
for await (const chunk of data) {
size += chunk.length;
if (size > limit) throw new Error("importAddress: manifest.json is too large");
chunks.push(chunk);
}
return Buffer.concat(chunks);
}
/** Whether `path` exists. */
async function exists(path: string): Promise<boolean> {
return stat(path).then(
() => true,
() => false,
);
}

43
packages/chord/src/next-node/app/ephemeral.ts Normal file
View file

@ -0,0 +1,43 @@
// The ephemeral login plugin (slice 13 §3, decision "The app kit, point 2"): an IRC-like "pick a name and you're in".
// `POST /_chord/login` (public: `{ name }`) issues an ephemeral session for `{ kind: "user", id: <fresh>, name }`
// (the app's principal type must allow it) and answers with its cookie. No account is stored. The session ends with
// the browser (its cookie has no lifetime) or the server (ephemeral sessions are dropped whenever it starts); ending it
// with the client's last link waits for slice 11's clients (carry-over). An exported facet, as `defineUsers`.
import type { Principal } from "../../next/data/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
import { Http } from "../http/index.ts";
import { newToken } from "./secrets.ts";
import { Sessions } from "./sessions.ts";
/** The principal an ephemeral login gives. */
export type EphemeralUser = { readonly kind: "user"; readonly id: string; readonly name: string };
/** The longest name taken. */
const MAX_NAME = 40;
/** The ephemeral login plugin's server facet; a type error unless the app's principal type holds `EphemeralUser`. */
export function ephemeralLoginFacet<P extends Principal>(
..._fits: EphemeralUser extends P ? [] : ["the app's principal type must hold { kind: 'user', id, name }"]
): Facet<P> {
return defineFacet<P>((_host, self) =>
self.contribute(Http, {
routes: {
"POST /_chord/login": {
public: true,
handler: async (request, ctx) => {
// A trust boundary: anyone may log in under a name.
const body = (await request.json(4096)) as { readonly name?: unknown } | null;
const name = typeof body?.name === "string" ? body.name.trim().slice(0, MAX_NAME) : "";
if (name === "") return { kind: "error", status: 400 };
const principal: EphemeralUser = { kind: "user", id: newToken(12), name };
const sessions = await ctx.get(Sessions, {});
const secure = request.provenance.kind !== "local";
const issued = await sessions.issue({ principal, label: name, kind: "ephemeral", secure }, ctx);
return { kind: "json", value: principal, headers: { "Set-Cookie": issued.cookie } };
},
},
},
}),
);
}

196
packages/chord/src/next-node/app/files.ts Normal file
View file

@ -0,0 +1,196 @@
// Files (slice 13 §4, decision "The app kit, point 5: plugins, files and export"): the kit's file storage, one `Files`
// object per host (`self.get(Files, {})`), its files in the host's directory (`files/`) and its index (id, name, mime,
// size, created) a watchable member kept in the store. Small files in one call (`put`), large ones in chunks (design
// §23 "Chunked transfers": `begin`, `append` at the stored offset, `uploaded` to resume after a drop, `finish`;
// unfinished uploads expire), reads by range, caps per host. Plugins never see a path.
import { mkdir, open, readdir, rm, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { setTimeout as sleep } from "node:timers/promises";
import { arr, bytes, int, local, map, method, num, obj, str } from "../../next/contracts/index.ts";
import { ChordError, type Principal } from "../../next/data/index.ts";
import { invalid } from "../../next/data/internal.ts";
import { defineState } from "../../next/documents/index.ts";
import { type Ctx, defineFacet, type Facet } from "../../next/host/index.ts";
import { newToken } from "./secrets.ts";
import { Uploads } from "./uploads.ts";
/** A file as the index lists it. */
export type FileInfo = {
readonly id: string;
readonly name: string;
readonly mime: string;
readonly size: number;
readonly created: number;
};
/** The caps of one host's files (bytes). */
export type FilesCaps = {
/** The largest file: 256 MiB by default. */
readonly maxFile?: number;
/** All files together: 4 GiB by default. */
readonly maxTotal?: number;
};
/** One file's fields in the index. */
const Entry = obj({ name: str(), mime: str(), size: int(), created: num() });
/** The host's files (one per host): the index, small and chunked writes, ranged reads, removal. */
export const Files = local("chord.app.files", {
version: 1,
params: {},
members: {
list: arr(obj({ id: str(), name: str(), mime: str(), size: int(), created: num() })),
/** Stores a small file (at most 4 MiB) in one call; its id. */
put: method(obj({ name: str(), mime: str(), bytes: bytes() }), str()),
/** Starts a chunked upload; its upload id. `unavailable` (`busy`) while the host's cap of uploads are under way. */
begin: method(obj({ name: str(), mime: str() }), str()),
/**
* Appends a chunk (at most 4 MiB; about 1 MiB is the usual size) to upload `upload` at `offset`, which must be
* the bytes stored so far (else `invalid`, its details `{ offset }` the stored one).
*/
append: method(obj({ upload: str(), offset: int({ min: 0 }), bytes: bytes() })),
/** The bytes stored so far of upload `upload`: where a writer resumes after a drop. */
uploaded: method(str(), int()),
/** Ends upload `upload`: the file it made, its id. */
finish: method(str(), str()),
/** `length` bytes of file `id` from `offset` (fewer at its end). */
read: method(obj({ id: str(), offset: int({ min: 0 }), length: int({ min: 0 }) }), bytes()),
/** Removes file `id` (an unknown one: nothing). */
remove: method(str()),
},
});
/** The index document. */
const FilesIndex = defineState("chord.app.files", {
schema: obj({ files: map(Entry) }),
version: 1,
initial: () => ({ files: {} }),
persist: true,
checkpointWhen: (_value, _ops, { deltasSinceBase }) => deltasSinceBase >= 50,
});
/** The largest piece a call carries (a `put`, an `append`, a `read`): 4 MiB, well under a link's part limit. */
const MAX_PIECE = 4 * 1024 * 1024;
/** How long an upload may go unwritten before it expires: an hour. */
const UPLOAD_IDLE_MS = 60 * 60_000;
/** How often expired uploads are dropped: every minute. */
const SWEEP_MS = 60_000;
/**
* The kit's facet serving `Files` with the host's files directory `dir` (`<host dir>/files`), `caps` and at most
* `uploads` uploads under way (64 by default). Uploads under way live in `dir/.uploads/<run>/` (`Uploads`) and end with
* the object (an unfinished one is dropped when the next one goes live).
*/
export function filesFacet<P extends Principal>(dir: string, caps: FilesCaps = {}, uploads = 64): Facet<P> {
const maxFile = caps.maxFile ?? 256 * 1024 * 1024;
const maxTotal = caps.maxTotal ?? 4 * 1024 * 1024 * 1024;
const uploadsDir = join(dir, ".uploads");
return defineFacet<P>((host) =>
host.serve(Files, (_params, self) => {
const index = self.state(FilesIndex);
// This run's uploads in their own directory: an upload left by an earlier run (its process ended) can never
// finish and is dropped, while this run's may already have begun.
const run = join(uploadsDir, newToken(12));
const underway = new Uploads(run, uploads);
// Uploads under way count against the total too: two of them cannot pass it together.
const room = (adding: number, fileSize: number): void => {
if (fileSize > maxFile)
throw tooLarge(`a file of ${fileSize} bytes is larger than this host's cap (${maxFile})`);
const stored = Object.values(index.value.files).reduce((sum, file) => sum + file.size, 0);
if (stored + underway.size + adding > maxTotal) {
throw tooLarge(`this host's files would exceed their cap (${maxTotal} bytes)`);
}
};
self.onLive((ctx) => sweep(uploadsDir, run, underway, ctx));
const fileOf = (id: string): FileInfo | undefined =>
// A trust boundary: `id` is the caller's (`Object.hasOwn`: no inherited member is a file).
Object.hasOwn(index.value.files, id) ? { id, ...index.value.files[id]! } : undefined;
return {
list: self.derive([index], ([value]) =>
Object.entries(value.files).map(([id, { name, mime, size, created }]) => ({
id,
name,
mime,
size,
created,
})),
),
put: async ({ name, mime, bytes }, ctx) => {
if (bytes.length > MAX_PIECE)
throw tooLarge(`put takes at most ${MAX_PIECE} bytes: use begin, append, finish`);
room(bytes.length, bytes.length);
const id = newToken(12);
await mkdir(dir, { recursive: true, mode: 0o700 });
await writeFile(join(dir, id), bytes, { mode: 0o600 });
await index.change((draft) => {
draft.files[id] = { name, mime, size: bytes.length, created: Date.now() };
}, ctx);
return id;
},
begin: ({ name, mime }) => underway.begin(name, mime),
append: async ({ upload, offset, bytes }) => {
if (bytes.length > MAX_PIECE) throw tooLarge(`append takes at most ${MAX_PIECE} bytes`);
await underway.append(upload, offset, bytes, room);
return null;
},
uploaded: (upload) => underway.uploaded(upload),
finish: async (upload, ctx) => {
const id = newToken(12);
await mkdir(dir, { recursive: true, mode: 0o700 });
const { name, mime, size } = await underway.finish(upload, join(dir, id));
await index.change((draft) => {
draft.files[id] = { name, mime, size, created: Date.now() };
}, ctx);
return id;
},
read: async ({ id, offset, length }) => {
const file = fileOf(id);
if (file === undefined) throw invalid("args", `files: no file ${id}`);
if (length > MAX_PIECE) throw tooLarge(`read takes at most ${MAX_PIECE} bytes`);
const handle = await open(join(dir, id), "r");
try {
const buffer = new Uint8Array(Math.max(0, Math.min(length, file.size - offset)));
const { bytesRead } = await handle.read(buffer, 0, buffer.length, offset);
return buffer.subarray(0, bytesRead);
} finally {
await handle.close();
}
},
remove: async (id, ctx) => {
if (fileOf(id) === undefined) return null;
await index.change((draft) => {
delete draft.files[id];
}, ctx);
await rm(join(dir, id), { force: true });
return null;
},
};
}),
);
}
/**
* The files object's routine once live: drops the uploads earlier runs left (every directory under `uploadsDir` but
* `run`), then every minute the uploads unwritten for an hour; when the object ends, every upload still under way.
*/
async function sweep(uploadsDir: string, run: string, uploads: Uploads, ctx: Ctx): Promise<void> {
for (const entry of await readdir(uploadsDir).catch(() => [])) {
if (join(uploadsDir, entry) !== run) await rm(join(uploadsDir, entry), { recursive: true, force: true });
}
try {
for (;;) {
await sleep(SWEEP_MS, undefined, { signal: ctx.abortSignal });
await uploads.expire(UPLOAD_IDLE_MS);
}
} finally {
if (ctx.abortSignal?.aborted === true) await uploads.close();
}
}
/** A cap refused a write. */
function tooLarge(message: string): ChordError {
return new ChordError({ code: "too_large", message: `files: ${message}` });
}

24
packages/chord/src/next-node/app/index.ts Normal file
View file

@ -0,0 +1,24 @@
// Chord v2's app kit (`@earendil-works/chord/next/app` once v2 has its entry point; slice 13): `startApp` on the server
// and `startWorker` in a worker's entry, the start-data objects, sessions and the owner's sign-in, device pairing, the
// files of each host, export and import per address, and the optional plugins (users, invites, ephemeral login).
export {
type App,
type AppApi,
type AppError,
type AppKind,
type AppOptions,
type AppPlugins,
type AppWorkerKind,
Server,
startApp,
} from "./app.ts";
export { ARCHIVE_FORMAT, type ArchiveManifest } from "./archive.ts";
export { type EphemeralUser, ephemeralLoginFacet } from "./ephemeral.ts";
export { type FileInfo, Files, type FilesCaps } from "./files.ts";
export { AppInfo, WorkerInfo } from "./info.ts";
export { inviteFacet } from "./invite.ts";
export { OWNER, type OwnerPrincipal } from "./owner.ts";
export { SESSION_COOKIE, type SessionInfo, type SessionKind, Sessions } from "./sessions.ts";
export { defineUsers } from "./users.ts";
export { startWorker, type WorkerOptions } from "./worker.ts";

52
packages/chord/src/next-node/app/info.ts Normal file
View file

@ -0,0 +1,52 @@
// The start-data objects (slice 13 §1; decision "The plugin model, points 3, 4, 5 and 8" (8)): base code shrinks to
// the API, the bootstrap and one facet serving process values as core objects plugins `get`. `AppInfo` on the
// server (the data directory, the app's name and its API version), `WorkerInfo` on a worker (its address and the
// app's start data for it).
import type { JsonValue } from "../../json.ts";
import { local, obj, str, unknown } from "../../next/contracts/index.ts";
import type { Principal } from "../../next/data/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
import { constant } from "../../next/state/index.ts";
/** What the server knows of the app: its data directory, its name, its newest API version. */
export const AppInfo = local("chord.app.info", {
version: 1,
params: {},
members: { dataDir: str(), name: str(), version: str() },
});
/** What a worker knows of itself: its address (kind, params, key) and the app's start data for it. */
export const WorkerInfo = local("chord.app.worker-info", {
version: 1,
params: {},
members: {
address: obj({ kind: str(), key: str(), params: unknown() }),
start: unknown(),
},
});
/** The facet serving `AppInfo` with `info`. */
export function appInfoFacet<P extends Principal>(info: {
readonly dataDir: string;
readonly name: string;
readonly version: string;
}): Facet<P> {
return defineFacet<P>((host) =>
host.serve(AppInfo, () => ({
dataDir: constant(info.dataDir),
name: constant(info.name),
version: constant(info.version),
})),
);
}
/** The facet serving `WorkerInfo` with the worker's address and start data. */
export function workerInfoFacet<P extends Principal>(info: {
readonly address: { readonly kind: string; readonly key: string; readonly params: JsonValue };
readonly start: JsonValue;
}): Facet<P> {
return defineFacet<P>((host) =>
host.serve(WorkerInfo, () => ({ address: constant(info.address), start: constant(info.start) })),
);
}

72
packages/chord/src/next-node/app/invite.ts Normal file
View file

@ -0,0 +1,72 @@
// The invite plugin (slice 13 §3, decision "The app kit, point 2"): the owner makes an invite link for a principal of
// the app it chooses (`POST /_chord/invites`, owner only: `{ principal, label }` → `{ path }`); the invitee opens it,
// its page posts the token (the URL's fragment, never logged) to `POST /_chord/invite`, which consumes it (single use,
// 7 days) and answers with a browser session for that principal (§2's sessions). Invites wait in memory: a restart
// drops the ones not used. An exported facet, as `defineUsers` (see there why).
import type { JsonValue } from "../../json.ts";
import type { Principal } from "../../next/data/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
import { Http } from "../http/index.ts";
import { fragmentPage } from "./owner.ts";
import { hashToken, newToken } from "./secrets.ts";
import { Sessions } from "./sessions.ts";
/** How long an invite is valid: 7 days. */
const INVITE_MS = 7 * 24 * 60 * 60_000;
/** The invite link's path: its page, and the post that consumes it. */
const INVITE_PATH = "/_chord/invite";
/** An invite waiting: the principal it signs in, its label, its expiry. */
type Invite = { readonly principal: JsonValue; readonly label: string; readonly expires: number };
/** The invite plugin's server facet. */
export function inviteFacet<P extends Principal>(): Facet<P> {
// By their token's hash: what the map holds is no credential.
const invites = new Map<string, Invite>();
return defineFacet<P>((_host, self) =>
self.contribute(Http, {
routes: {
"POST /_chord/invites": async (request, ctx) => {
if (ctx.principal?.kind !== "owner") return { kind: "error", status: 403 };
// A trust boundary: the owner's request, checked for its shape.
const body = (await request.json()) as {
readonly principal?: JsonValue;
readonly label?: unknown;
} | null;
const principal = body?.principal;
if (typeof principal !== "object" || principal === null || typeof body?.label !== "string") {
return { kind: "error", status: 400 };
}
const token = newToken();
invites.set(hashToken(token), { principal, label: body.label, expires: Date.now() + INVITE_MS });
return { kind: "json", value: { path: `${INVITE_PATH}#${token}` } };
},
[`GET ${INVITE_PATH}`]: {
public: true,
handler: async (request) => ({ kind: "html", body: fragmentPage(INVITE_PATH, request.nonce) }),
},
[`POST ${INVITE_PATH}`]: {
public: true,
handler: async (request, ctx) => {
// A trust boundary: anyone may post here.
const body = (await request.json()) as { readonly token?: unknown } | null;
const hash = typeof body?.token === "string" ? hashToken(body.token) : undefined;
const invite = hash === undefined ? undefined : invites.get(hash);
if (hash === undefined || invite === undefined) return { kind: "error", status: 403 };
invites.delete(hash);
if (invite.expires <= Date.now()) return { kind: "error", status: 403 };
const sessions = await ctx.get(Sessions, {});
const secure = request.provenance.kind !== "local";
const issued = await sessions.issue(
{ principal: invite.principal, label: invite.label, kind: "browser", secure },
ctx,
);
return { kind: "json", value: "signed in", headers: { "Set-Cookie": issued.cookie } };
},
},
},
}),
);
}

105
packages/chord/src/next-node/app/layout.ts Normal file
View file

@ -0,0 +1,105 @@
// The app kit's data directory (slice 13 §4, decision "The app kit, point 4: storage"): one fixed layout, made private
// at every start. `server/` holds the server's store and files, `workers/<kind>/<address>/` each worker's store and
// files, `plugins/` the user's plugins root, `logs/` the logs, `owner-token` the owner's token for command-line
// clients. Directories are 0700 and stores 0600, also when they existed looser before (other local users must not
// read the sessions in a store, nor replace it).
import { createHash } from "node:crypto";
import { chmod, mkdir, open, realpath, stat } from "node:fs/promises";
import { homedir } from "node:os";
import { join, parse } from "node:path";
import type { Address } from "../../next/contracts/index.ts";
/** The paths of a data directory. */
export type AppLayout = {
readonly dataDir: string;
readonly server: string;
readonly serverStore: string;
readonly workers: string;
readonly plugins: string;
readonly logs: string;
readonly ownerToken: string;
readonly cookieSecret: string;
};
/** The data directory of app `name` unless the start script names one: `~/.<name>`. */
export function defaultDataDir(name: string): string {
return join(homedir(), `.${name}`);
}
/**
* The real path of the data directory `path`, made when missing (mode 0700), refused when the kit must not make it
* private: the home directory, a file system's root, or a directory another user owns (a clear error, instead of
* tightening a directory others use or failing on `EPERM`). Its mode is set on the real path, never through a link.
*/
export async function openDataDir(path: string): Promise<string> {
await mkdir(path, { recursive: true, mode: 0o700 });
const dataDir = await realpath(path);
const home = await realpath(homedir());
if (dataDir === home || dataDir === parse(dataDir).root) {
const what = dataDir === home ? "the home directory" : "a file system's root";
throw new Error(`startApp: the data directory ${dataDir} is ${what}; name a directory of the app's own`);
}
const { uid } = await stat(dataDir);
if (process.getuid !== undefined && uid !== process.getuid()) {
throw new Error(`startApp: the data directory ${dataDir} belongs to another user (uid ${uid})`);
}
return dataDir;
}
/** The layout of `dataDir` (a real path, `openDataDir`'s), its directories made private (`privateDirectory`). */
export async function createLayout(dataDir: string): Promise<AppLayout> {
await privateDirectory(dataDir);
const layout: AppLayout = {
dataDir,
server: join(dataDir, "server"),
serverStore: join(dataDir, "server", "store.sqlite"),
workers: join(dataDir, "workers"),
plugins: join(dataDir, "plugins"),
logs: join(dataDir, "logs"),
ownerToken: join(dataDir, "owner-token"),
cookieSecret: join(dataDir, "server", "cookie-secret"),
};
for (const dir of [layout.server, layout.workers, layout.plugins, layout.logs]) await privateDirectory(dir);
return layout;
}
/** Makes `dir` (and missing parents) with mode 0700, and sets 0700 on it when it existed. */
export async function privateDirectory(dir: string): Promise<void> {
await mkdir(dir, { recursive: true, mode: 0o700 });
await chmod(dir, 0o700);
}
/**
* Makes the store file `path` private before SQLite opens it: created empty with mode 0600 when missing (SQLite gives
* its journal files the database's mode), else set to 0600 with the journal files it has.
*/
export async function privateStore(path: string): Promise<void> {
await (await open(path, "a", 0o600)).close();
for (const file of [path, `${path}-wal`, `${path}-shm`]) {
await chmod(file, 0o600).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error;
});
}
}
/** The directory of the worker at `address`: `workers/<kind>/<its key, encoded>`. */
export function workerDirectory(dataDir: string, address: Address): string {
return join(dataDir, "workers", address.kind, encodedKey(address.key));
}
/** The longest key kept readable as a directory name; a longer one is named by its hash. */
const READABLE_KEY = 160;
/**
* An address key as one directory name: percent-encoded (`/`, `\`, `.` and every other character a file system might
* treat specially included), or its SHA-256 when that is longer than `READABLE_KEY`. Capital letters are encoded too:
* on a case-insensitive file system (macOS) `p|A` and `p|a` would otherwise share a directory.
*/
function encodedKey(key: string): string {
const encoded = encodeURIComponent(key).replace(
/(%[0-9A-F]{2})|[.!'()*~A-Z]/g,
(match, escaped: string | undefined) => escaped ?? `%${match.charCodeAt(0).toString(16).toUpperCase()}`,
);
return encoded.length <= READABLE_KEY ? encoded : `sha256-${createHash("sha256").update(key).digest("hex")}`;
}

123
packages/chord/src/next-node/app/owner.ts Normal file
View file

@ -0,0 +1,123 @@
// The owner (slice 13 §2, decision "The app kit, point 3: authentication"): no owner by loopback alone. At each start
// the kit makes a one-time owner link (a random token, single use, valid 10 minutes, the previous one dropped); the
// start script prints it. Its page reads the token from the URL's fragment (never sent to a server, so never in a
// log) and posts it; the post consumes it and sets a browser session cookie for the owner. The owner token file
// (mode 0600) carries a device session of the owner for command-line clients on the same machine.
import { chmod } from "node:fs/promises";
import type { Context } from "../../context/index.ts";
import type { Principal } from "../../next/data/index.ts";
import { defineFacet, type Facet, type Host } from "../../next/host/index.ts";
import { Http } from "../http/index.ts";
import { hashToken, matchesHash, newToken, readSecret, writeSecret } from "./secrets.ts";
import { Sessions } from "./sessions.ts";
/** The kit's owner principal: a variant every app's principal type has. */
export type OwnerPrincipal = { readonly kind: "owner" };
/** The owner principal. */
export const OWNER: OwnerPrincipal = { kind: "owner" };
/** How long an owner link is valid: 10 minutes. */
const OWNER_LINK_MS = 10 * 60_000;
/** The owner link's one-time token: made at each start (the previous one dropped), consumed by its first use. */
export class OwnerLink {
#hash: string | undefined;
#expires = 0;
/** A fresh token (the previous one no longer valid), valid for 10 minutes. */
renew(): string {
const token = newToken();
this.#hash = hashToken(token);
this.#expires = Date.now() + OWNER_LINK_MS;
return token;
}
/** Whether `token` is the current one and has not expired; it is used up either way when it matches. */
consume(token: string): boolean {
const hash = this.#hash;
if (hash === undefined || !matchesHash(token, hash)) return false;
this.#hash = undefined;
return Date.now() < this.#expires;
}
}
/** The path of the owner link's page and post. */
export const OWNER_PATH = "/_chord/owner";
/**
* The kit's facet for the owner link: `GET /_chord/owner`, a page posting the fragment's token, and `POST
* /_chord/owner`, which consumes it and answers with the owner's session cookie.
*/
export function ownerFacet<P extends Principal>(link: OwnerLink): Facet<P> {
return defineFacet<P>((_host, self) =>
self.contribute(Http, {
routes: {
[`GET ${OWNER_PATH}`]: {
public: true,
handler: async (request) => ({ kind: "html", body: fragmentPage(OWNER_PATH, request.nonce) }),
},
[`POST ${OWNER_PATH}`]: {
public: true,
handler: async (request, ctx) => {
const body = (await request.json()) as { readonly token?: unknown } | null;
// A trust boundary: anyone may post here.
if (typeof body?.token !== "string" || !link.consume(body.token))
return { kind: "error", status: 403 };
const sessions = await ctx.get(Sessions, {});
const secure = request.provenance.kind !== "local";
const issued = await sessions.issue(
{ principal: OWNER, label: "owner browser", kind: "browser", secure },
ctx,
);
return { kind: "json", value: "signed in", headers: { "Set-Cookie": issued.cookie } };
},
},
},
}),
);
}
/**
* A sign-in page for a link with a token in its fragment (the owner link, an invite): its script (the response's nonce)
* posts the token to `path`, then opens the app.
*/
export function fragmentPage(path: string, nonce: string): string {
return [
"<!doctype html><title>Sign in</title><p id=status>Signing in…</p>",
`<script nonce="${nonce}">`,
"const token = location.hash.slice(1); history.replaceState(null, '', location.pathname);",
`fetch(${JSON.stringify(path)}, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ token }) })`,
" .then((r) => { if (r.ok) location.replace('/'); else document.getElementById('status').textContent = 'This link is used up or expired.'; });",
"</script>",
].join("\n");
}
/**
* The owner token file at `path`: kept when its token still authenticates as the owner (its session exists, its secret
* matches, it has not expired; its mode set to 0600 again), else a new device session of the owner written there (mode
* 0600, in one step). Its token, for the start script.
*/
export async function ownerTokenFile<P extends Principal>(
path: string,
host: Host<P>,
context: Context,
): Promise<string> {
const existing = await readSecret(path);
return host.root(async (ctx) => {
const sessions = await ctx.get(Sessions, {});
if (existing !== undefined && isOwner(await sessions.verify(existing, ctx))) {
await chmod(path, 0o600);
return existing;
}
const { token } = await sessions.issue({ principal: OWNER, label: "owner token file", kind: "device" }, ctx);
await writeSecret(path, token, existing === undefined ? "create" : "replace");
return token;
}, context);
}
/** Whether `principal` (as `Sessions.verify` gives it) is the owner. */
function isOwner(principal: unknown): boolean {
return typeof principal === "object" && principal !== null && (principal as { kind?: unknown }).kind === "owner";
}

205
packages/chord/src/next-node/app/pairing.ts Normal file
View file

@ -0,0 +1,205 @@
// Device pairing (slice 13 §2, decision "The app kit, point 3" (c)): a command-line or terminal client on another
// machine asks for a code (`POST /_chord/pair`, public) and polls with a secret it got with it; the owner opens
// `/_chord/pair/<code>` in a signed-in browser and approves; the next poll returns a device session's token, once.
// Codes are short, random, single use and expire after 10 minutes; the poll secret is kept only hashed. Asking is
// public, so pairings anonymous clients start are capped: `maxPending` in all (100 by default) and `perAddress` per
// client address (10). The trade: a party filling the caps locks pairing out for others for up to 10 minutes (behind a
// tunnel every remote client has the tunnel's address), instead of growing the server's memory without bound; a
// signed-in owner's pairings are not counted, so the owner can always pair.
import { randomInt } from "node:crypto";
import type { Principal } from "../../next/data/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
import { Http, type HttpRequest, type HttpResponse } from "../http/index.ts";
import { OWNER } from "./owner.ts";
import { hashToken, newToken } from "./secrets.ts";
import { Sessions } from "./sessions.ts";
/** How long a pairing code is valid: 10 minutes. */
const PAIRING_MS = 10 * 60_000;
/** The code's letters: no 0/O, 1/I/L. */
const ALPHABET = "ABCDEFGHJKMNPQRSTUVWXYZ23456789";
/** Who starts a pairing: the signed-in owner (not counted), or an anonymous client at an address (counted). */
export type PairingStarter = { readonly kind: "owner" } | { readonly kind: "anonymous"; readonly address: string };
/** One pairing under way: its code, the device's label, who started it, its expiry, its token once approved. */
type Pairing = {
readonly code: string;
readonly label: string;
readonly by: PairingStarter;
readonly expires: number;
token?: string;
};
/** The limits of pairings under way: in all (100 by default) and per client address (10 by default). */
export type PairingLimits = { readonly maxPending?: number; readonly perAddress?: number };
/** The pairings under way, by code and by their poll secret's hash, with the anonymous ones counted. */
export class Pairings {
readonly #byCode = new Map<string, Pairing>();
readonly #byPoll = new Map<string, Pairing>();
readonly #perAddress = new Map<string, number>();
#anonymous = 0;
readonly #maxPending: number;
readonly #maxPerAddress: number;
constructor(limits: PairingLimits = {}) {
this.#maxPending = limits.maxPending ?? 100;
this.#maxPerAddress = limits.perAddress ?? 10;
}
/**
* A new pairing for a device labelled `label`, started `by` the owner or an anonymous client: its code (for the
* owner) and its poll secret (for the device); `undefined` when an anonymous client reached a limit.
*/
begin(label: string, by: PairingStarter): { readonly code: string; readonly poll: string } | undefined {
this.#expire();
if (by.kind === "anonymous") {
const full = this.#anonymous >= this.#maxPending;
if (full || (this.#perAddress.get(by.address) ?? 0) >= this.#maxPerAddress) return undefined;
this.#anonymous++;
this.#perAddress.set(by.address, (this.#perAddress.get(by.address) ?? 0) + 1);
}
let code: string;
do code = `${letters(4)}-${letters(4)}`;
while (this.#byCode.has(code));
const poll = newToken();
const pairing: Pairing = { code, label, by, expires: Date.now() + PAIRING_MS };
this.#byCode.set(code, pairing);
this.#byPoll.set(hashToken(poll), pairing);
return { code, poll };
}
/** The pairing waiting for approval under `code` (any case), unexpired. */
waiting(code: string): Pairing | undefined {
this.#expire();
const pairing = this.#byCode.get(code.toUpperCase());
return pairing?.token === undefined ? pairing : undefined;
}
/** What a poll with `poll` learns: still waiting, the device's token (once: the pairing ends), or nothing. */
poll(
poll: string,
): { readonly kind: "waiting" } | { readonly kind: "approved"; readonly token: string } | undefined {
this.#expire();
const hash = hashToken(poll);
const pairing = this.#byPoll.get(hash);
if (pairing === undefined) return undefined;
if (pairing.token === undefined) return { kind: "waiting" };
this.#end(hash, pairing);
return { kind: "approved", token: pairing.token };
}
/** Drops the expired pairings. */
#expire(): void {
const now = Date.now();
for (const [hash, pairing] of this.#byPoll) {
if (pairing.expires <= now) this.#end(hash, pairing);
}
}
/** Ends `pairing` (its poll secret's hash `hash`). */
#end(hash: string, pairing: Pairing): void {
this.#byPoll.delete(hash);
this.#byCode.delete(pairing.code);
const { by } = pairing;
if (by.kind === "owner") return;
this.#anonymous--;
const left = (this.#perAddress.get(by.address) ?? 1) - 1;
if (left === 0) this.#perAddress.delete(by.address);
else this.#perAddress.set(by.address, left);
}
}
/** `count` random letters of `ALPHABET` (`crypto.randomInt`). */
function letters(count: number): string {
let text = "";
for (let index = 0; index < count; index++) text += ALPHABET[randomInt(ALPHABET.length)];
return text;
}
/** The pairing routes' prefix. */
export const PAIR_PATH = "/_chord/pair";
/**
* The kit's facet for device pairing: `POST /_chord/pair` (public: `{ label? }` → `{ code, poll }`, 429 past a limit
* unless the owner asks), `POST /_chord/pair/poll` (public: `{ poll }` → `{ status: "waiting" }`, `{ status:
* "approved", token }` once, else 404), `GET` and `POST /_chord/pair/:code` (the owner's confirm page and approval: a
* device session of the owner).
*/
export function pairingFacet<P extends Principal>(pairings: Pairings): Facet<P> {
return defineFacet<P>((_host, self) =>
self.contribute(Http, {
routes: {
[`POST ${PAIR_PATH}`]: {
public: true,
handler: async (request, ctx): Promise<HttpResponse> => {
const label = await labelOf(request);
const by: PairingStarter =
ctx.principal?.kind === "owner"
? { kind: "owner" }
: { kind: "anonymous", address: request.remoteAddress ?? "" };
const begun = pairings.begin(label, by);
return begun === undefined ? { kind: "error", status: 429 } : { kind: "json", value: begun };
},
},
[`POST ${PAIR_PATH}/poll`]: {
public: true,
handler: async (request): Promise<HttpResponse> => {
// A trust boundary: anyone may poll.
const body = (await request.json()) as { readonly poll?: unknown } | null;
const polled = typeof body?.poll === "string" ? pairings.poll(body.poll) : undefined;
if (polled === undefined) return { kind: "error", status: 404 };
if (polled.kind === "waiting") return { kind: "json", value: { status: "waiting" } };
return { kind: "json", value: { status: "approved", token: polled.token } };
},
},
[`GET ${PAIR_PATH}/:code`]: async (request, ctx) => {
if (ctx.principal?.kind !== "owner") return { kind: "error", status: 403 };
const pairing = pairings.waiting(request.params.code ?? "");
if (pairing === undefined)
return { kind: "html", status: 404, body: "<p>No such pairing (or it expired).</p>" };
return { kind: "html", body: confirmPage(pairing.code, pairing.label) };
},
[`POST ${PAIR_PATH}/:code`]: async (request, ctx) => {
if (ctx.principal?.kind !== "owner") return { kind: "error", status: 403 };
const pairing = pairings.waiting(request.params.code ?? "");
if (pairing === undefined)
return { kind: "html", status: 404, body: "<p>No such pairing (or it expired).</p>" };
const sessions = await ctx.get(Sessions, {});
const { token } = await sessions.issue({ principal: OWNER, label: pairing.label, kind: "device" }, ctx);
pairing.token = token;
return {
kind: "html",
body: `<p>Approved: ${escapeHtml(pairing.label)} is signed in as the owner.</p>`,
};
},
},
}),
);
}
/** The device's label from a pairing request's body (`{ label }`, at most 100 characters), else `device`. */
async function labelOf(request: HttpRequest): Promise<string> {
const declared = Number(request.headers.get("content-length") ?? "0");
if (declared === 0 && request.headers.get("transfer-encoding") === null) return "device";
// A trust boundary: anyone may ask for a code.
const body = (await request.json(4096)) as { readonly label?: unknown } | null;
return typeof body?.label === "string" && body.label.trim() !== "" ? body.label.trim().slice(0, 100) : "device";
}
/** The owner's confirm page for pairing `code` (a form posting to itself: same origin, the session cookie). */
function confirmPage(code: string, label: string): string {
return [
"<!doctype html><title>Pair a device</title>",
`<p>Sign in <b>${escapeHtml(label)}</b> as the owner? Its code: <b>${escapeHtml(code)}</b> (compare it with the device).</p>`,
'<form method="post"><button>Approve</button></form>',
].join("\n");
}
/** `text` safe in HTML. */
function escapeHtml(text: string): string {
return text.replace(/[&<>"']/g, (char) => `&#${char.charCodeAt(0)};`);
}

75
packages/chord/src/next-node/app/secrets.ts Normal file
View file

@ -0,0 +1,75 @@
// Secrets of the app kit (slice 13 §2): tokens from `crypto.randomBytes`, kept only as their SHA-256 hashes and
// compared in constant time, and secret files (the cookie key, the owner token file) created with mode 0600 in one
// step, so no other OS user and no half-written file ever reads them.
import { createHash, randomBytes, timingSafeEqual } from "node:crypto";
import { link, open, readFile, rename, unlink } from "node:fs/promises";
import { dirname, join } from "node:path";
/** A fresh random token of `bytes` bytes, base64url. */
export function newToken(bytes = 32): string {
return randomBytes(bytes).toString("base64url");
}
/** The hash a token is kept as (SHA-256, hex): what leaks from a store is no credential. */
export function hashToken(token: string): string {
return createHash("sha256").update(token).digest("hex");
}
/** Whether `token` hashes to `hash`, compared in constant time. */
export function matchesHash(token: string, hash: string): boolean {
const given = Buffer.from(hashToken(token), "hex");
const kept = Buffer.from(hash, "hex");
return given.length === kept.length && timingSafeEqual(given, kept);
}
/** The secret in the file at `path`, created (mode 0600, in one step) with a fresh token when it does not exist yet. */
export async function secretFile(path: string): Promise<string> {
const existing = await readSecret(path);
if (existing !== undefined) return existing;
const secret = newToken();
const written = await writeSecret(path, secret, "create");
return written === "exists" ? ((await readSecret(path)) ?? secret) : secret;
}
/** The text of the secret file at `path`, or `undefined` when it does not exist. */
export async function readSecret(path: string): Promise<string | undefined> {
try {
return (await readFile(path, "utf8")).trim();
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined;
throw error;
}
}
/**
* Writes `text` to `path` with mode 0600 in one step: a temporary file of the same directory (made exclusively, 0600)
* linked into place (`create`: refused when `path` exists, `"exists"` returned) or renamed over it (`replace`).
*/
export async function writeSecret(
path: string,
text: string,
mode: "create" | "replace",
): Promise<"written" | "exists"> {
const temporary = join(dirname(path), `.${newToken(9)}.tmp`);
const handle = await open(temporary, "wx", 0o600);
try {
await handle.writeFile(`${text}\n`);
await handle.sync();
} finally {
await handle.close();
}
try {
if (mode === "replace") {
await rename(temporary, path);
return "written";
}
await link(temporary, path);
return "written";
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "EEXIST") return "exists";
throw error;
} finally {
await unlink(temporary).catch(() => {});
}
}

267
packages/chord/src/next-node/app/sessions.ts Normal file
View file

@ -0,0 +1,267 @@
// Sessions (slice 13 §2, decision "The app kit, point 3: authentication"): the kit's object on the server that issues
// and revokes sessions, and its authenticator. A session is a principal behind a token `<id>.<secret>` (random); only
// the secret's hash is kept. A browser carries its token in the kit's signed session cookie (`HttpOnly`,
// `SameSite=Strict`), a command-line client in `Authorization: Bearer <token>`. Browser and device sessions persist in
// the server's store and expire after 400 days unused; ephemeral sessions (a pick-a-name login, a public route) live
// in memory only, at most `maxEphemeral` of them (past it the oldest ends), so a public login costs the same whatever
// came before, and none outlives the server's run.
import type { Context } from "../../context/index.ts";
import type { JsonValue } from "../../json.ts";
import { arr, bool, local, map, method, num, obj, oneOf, opt, str, unknown } from "../../next/contracts/index.ts";
import type { Principal } from "../../next/data/index.ts";
import { defineState } from "../../next/documents/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
import {
Authenticators,
cookie,
Http,
type HttpRequest,
type HttpWrap,
type Provenance,
readCookie,
readSignedCookie,
} from "../http/index.ts";
import { hashToken, matchesHash, newToken } from "./secrets.ts";
/** What a session is for: a browser (its cookie), a device (a command-line client's token), an ephemeral login. */
export type SessionKind = "browser" | "device" | "ephemeral";
/** A browser or device session as `Sessions.list` shows it: never its token or hash. */
export type SessionInfo = {
readonly id: string;
readonly label: string;
readonly kind: "browser" | "device";
readonly principal: unknown;
readonly created: number;
readonly lastUsed: number;
};
/** The kit's sessions (one per server): issue, verify, revoke, and the watchable list of persisted ones. */
export const Sessions = local("chord.app.sessions", {
version: 1,
params: {},
members: {
/** The browser and device sessions (ephemeral ones are not listed: there may be many, and they are short). */
list: arr(
obj({
id: str(),
label: str(),
kind: oneOf("browser", "device"),
principal: unknown(),
created: num(),
lastUsed: num(),
}),
),
/**
* A new session for `principal`: its id, its token (shown once, kept only hashed) and the session cookie carrying
* it for a browser (`Set-Cookie`; `secure` for a remote one).
*/
issue: method(
obj({
principal: unknown(),
label: str(),
kind: oneOf("browser", "device", "ephemeral"),
secure: opt(bool()),
}),
obj({ id: str(), token: str(), cookie: str() }),
),
/** The principal `token` authenticates as, or `null` (unknown, revoked, expired, malformed). */
verify: method(str(), unknown()),
/** Ends session `id` at once: its token authenticates nothing more. */
revoke: method(str()),
},
});
/** The name of the kit's session cookie. */
export const SESSION_COOKIE = "chord-session";
/** The stored sessions: by id, each with its secret's hash and its principal (as JSON text: any of the app's). */
const SessionsDocument = defineState("chord.app.sessions", {
schema: obj({
sessions: map(
obj({
hash: str(),
label: str(),
kind: oneOf("browser", "device"),
principal: str(),
created: num(),
lastUsed: num(),
}),
),
}),
version: 1,
initial: () => ({ sessions: {} }),
persist: true,
checkpointWhen: (_value, _ops, { deltasSinceBase }) => deltasSinceBase >= 50,
});
/** An ephemeral session, in memory: its secret's hash and its principal. */
type EphemeralSession = { readonly hash: string; readonly principal: Principal };
/** How often a session's `lastUsed` is written at most: once a minute (not one commit per request). */
const TOUCH_MS = 60_000;
/** How long a persisted session lives unused: 400 days, as its cookie. */
const SESSION_IDLE_MS = 400 * 24 * 60 * 60_000;
/** The sessions' limits: how many ephemeral sessions live at once (1,000 by default; past it the oldest ends). */
export type SessionLimits = { readonly maxEphemeral?: number };
/**
* The kit's facet serving `Sessions` (a host-lifetime object) and contributing its authenticator: the bearer token
* first, else the session cookie signed with `cookieSecret`.
*/
export function sessionsFacet<P extends Principal>(cookieSecret: string, limits: SessionLimits = {}): Facet<P> {
const maxEphemeral = limits.maxEphemeral ?? 1000;
return defineFacet<P>((host) =>
host.serve(Sessions, (_params, self) => {
const document = self.state(SessionsDocument);
// Insertion order is age order: the first entry is the oldest.
const ephemeral = new Map<string, EphemeralSession>();
const change = (fn: (sessions: Record<string, { lastUsed: number }>) => void, context: Context) =>
// Not awaited by authentication: the request goes on; a write that fails is made again by a later request.
void document.change((draft) => fn(draft.sessions), context).catch(() => {});
/** The principal of `token`, or `undefined`; touches or lazily removes the persisted session it names. */
const authenticate = (token: string, context: Context): Principal | undefined => {
const dot = token.indexOf(".");
if (dot <= 0) return undefined;
const id = token.slice(0, dot);
const secret = token.slice(dot + 1);
const short = ephemeral.get(id);
if (short !== undefined) return matchesHash(secret, short.hash) ? short.principal : undefined;
// A trust boundary: `id` is the client's (`Object.hasOwn`: no inherited member is a session).
const sessions = document.value.sessions;
if (!Object.hasOwn(sessions, id)) return undefined;
const session = sessions[id]!;
if (!matchesHash(secret, session.hash)) return undefined;
const now = Date.now();
if (now - session.lastUsed > SESSION_IDLE_MS) {
change((draft) => {
delete draft[id];
}, context);
return undefined;
}
if (now - session.lastUsed > TOUCH_MS) {
change((draft) => {
if (Object.hasOwn(draft, id)) draft[id]!.lastUsed = now;
}, context);
}
// Stored by `issue` from a principal of the app's type.
return JSON.parse(session.principal) as Principal;
};
self.contribute(Authenticators, (request, context) => {
const token = bearerToken(request) ?? readCookie(request, SESSION_COOKIE, cookieSecret);
return token === undefined ? undefined : authenticate(token, context);
});
self.contribute(Http, { wraps: [rollingCookie(cookieSecret, authenticate)] });
return {
list: self.derive([document], ([value]) =>
Object.entries(value.sessions).map(([id, { label, kind, principal, created, lastUsed }]) => ({
id,
label,
kind,
// Stored by `issue` as JSON text.
principal: JSON.parse(principal) as JsonValue,
created,
lastUsed,
})),
),
issue: async ({ principal, label, kind, secure }, ctx) => {
const id = newToken(12);
const secret = newToken(32);
if (kind === "ephemeral") {
// Principals are the app's: `issue`'s callers pass one of its type.
ephemeral.set(id, { hash: hashToken(secret), principal: principal as Principal });
for (const [oldest] of ephemeral) {
if (ephemeral.size <= maxEphemeral) break;
ephemeral.delete(oldest);
}
} else {
const now = Date.now();
const stored = JSON.stringify(principal);
await document.change((draft) => {
draft.sessions[id] = {
hash: hashToken(secret),
label,
kind,
principal: stored,
created: now,
lastUsed: now,
};
}, ctx);
}
const token = `${id}.${secret}`;
// Only whether the browser is remote matters to the cookie (`Secure`).
const provenance: Provenance = secure === true ? { kind: "remote", host: "" } : { kind: "local" };
const set = sessionCookie(token, kind, { provenance }, cookieSecret);
return { id, token, cookie: set };
},
verify: async (token, ctx) => (authenticate(token, ctx) as JsonValue | undefined) ?? null,
revoke: async (id, ctx) => {
if (ephemeral.delete(id)) return null;
await document.change((draft) => {
delete draft.sessions[id];
}, ctx);
return null;
},
};
}),
);
}
/**
* The session cookie carrying `token` (a session of `kind`) for a browser of `request`'s provenance: a persisted
* session's lives as long as the session does unused (400 days, renewed by `rollingCookie`), an ephemeral one's only
* while the browser runs.
*/
function sessionCookie(
token: string,
kind: SessionKind,
request: Pick<HttpRequest, "provenance">,
secret: string,
): string {
const maxAge = kind === "ephemeral" ? undefined : SESSION_IDLE_MS / 1000;
return cookie(SESSION_COOKIE, token, { secret, request, maxAge });
}
/** How old a session cookie gets before a response renews it: a day. */
const ROLL_MS = 24 * 60 * 60_000;
/**
* The wrap around every route renewing the session cookie a request signed in with once it is older than a day: the
* response sets it again with a fresh signed expiry, so a browser used within every 400 days stays signed in. Not when
* the route sets a cookie itself, nor for an ephemeral session's (it has no expiry) or a token that no longer
* authenticates.
*/
function rollingCookie(
secret: string,
authenticate: (token: string, context: Context) => Principal | undefined,
): HttpWrap {
return {
name: "chord.app.rolling-cookie",
phase: "headers",
scope: "all",
wrap: async (request, next, ctx) => {
const response = await next(request);
if (ctx.principal === undefined || Object.keys(response.headers ?? {}).some(isSetCookie)) return response;
const signed = readSignedCookie(request, SESSION_COOKIE, secret);
if (signed?.expires === undefined || signed.expires - Date.now() > SESSION_IDLE_MS - ROLL_MS) return response;
if (authenticate(signed.value, ctx) === undefined) return response;
const renewed = sessionCookie(signed.value, "browser", request, secret);
return { ...response, headers: { ...response.headers, "Set-Cookie": renewed } };
},
};
}
/** Whether a header name is `Set-Cookie` (any case). */
function isSetCookie(name: string): boolean {
return name.toLowerCase() === "set-cookie";
}
/** The token of an `Authorization: Bearer <token>` header. */
function bearerToken(request: HttpRequest): string | undefined {
const header = request.headers.get("authorization");
const match = header === null ? null : /^Bearer\s+(\S+)$/i.exec(header);
return match?.[1];
}

181
packages/chord/src/next-node/app/tar.ts Normal file
View file

@ -0,0 +1,181 @@
// A minimal ustar (POSIX tar) writer and reader for the kit's address archives (slice 13 §4): regular files and
// directories, paths of at most 255 bytes, nothing else. The reader is a trust boundary: an archive may come from
// anywhere, so it refuses every entry that is no regular file or directory, a path that is absolute, holds `..`, a
// backslash or NUL, or names an empty segment, and a header whose checksum does not hold.
/** One entry to write: its path (`/`-separated, relative) and its bytes, as a size and the chunks that make it up. */
export type TarEntry = { readonly path: string; readonly size: number; readonly data: AsyncIterable<Uint8Array> };
/** One entry read: a regular file (its data, to be consumed before the next entry) or a directory. */
export type TarRead =
| { readonly kind: "file"; readonly path: string; readonly size: number; readonly data: AsyncIterable<Uint8Array> }
| { readonly kind: "directory"; readonly path: string };
/** A tar block. */
const BLOCK = 512;
/** The archive of `entries`, as the chunks of its bytes (two zero blocks end it). */
export async function* writeTar(entries: AsyncIterable<TarEntry> | Iterable<TarEntry>): AsyncGenerator<Uint8Array> {
for await (const entry of entries) {
yield header(entry.path, entry.size);
let written = 0;
for await (const chunk of entry.data) {
written += chunk.length;
if (written > entry.size) throw new Error(`tar: ${entry.path} is larger than its size ${entry.size}`);
yield chunk;
}
if (written !== entry.size) throw new Error(`tar: ${entry.path} is ${written} bytes, not ${entry.size}`);
const pad = (BLOCK - (entry.size % BLOCK)) % BLOCK;
if (pad > 0) yield new Uint8Array(pad);
}
yield new Uint8Array(BLOCK * 2);
}
/** The ustar header of a regular file at `path` of `size` bytes. */
function header(path: string, size: number): Uint8Array {
checkPath(path);
const block = new Uint8Array(BLOCK);
const bytes = new TextEncoder().encode(path);
let name = bytes;
let prefix = new Uint8Array();
if (bytes.length > 100) {
// Split at a `/` so the name has at most 100 bytes and the prefix at most 155.
const at = path.lastIndexOf("/", path.length - 1);
const split = new TextEncoder().encode(path.slice(0, at)).length;
if (at < 0 || split > 155 || bytes.length - split - 1 > 100) throw new Error(`tar: the path ${path} is too long`);
prefix = bytes.subarray(0, split);
name = bytes.subarray(split + 1);
}
block.set(name, 0);
field(block, 100, 8, "0000644");
field(block, 108, 8, "0000000");
field(block, 116, 8, "0000000");
field(block, 124, 12, size.toString(8).padStart(11, "0"));
field(
block,
136,
12,
Math.floor(Date.now() / 1000)
.toString(8)
.padStart(11, "0"),
);
block.fill(0x20, 148, 156);
block[156] = 0x30;
block.set(new TextEncoder().encode("ustar\u000000"), 257);
block.set(prefix, 345);
const sum = block.reduce((total, byte) => total + byte, 0);
field(block, 148, 8, `${sum.toString(8).padStart(6, "0")}\u0000 `);
return block;
}
/** Writes `text` (ASCII) at `offset` of `block`, NUL-terminated within `length`. */
function field(block: Uint8Array, offset: number, length: number, text: string): void {
const bytes = new TextEncoder().encode(text);
if (bytes.length > length) throw new Error(`tar: ${text} does not fit ${length} bytes`);
block.set(bytes, offset);
}
/** Refuses a path an archive may not hold: absolute, with `..`, `.`, an empty segment, a backslash or a NUL. */
function checkPath(path: string): void {
const segments = path.split("/");
const bad = path === "" || path.includes("\\") || path.includes("\u0000") || path.startsWith("/");
if (bad || segments.some((segment) => segment === "" || segment === "." || segment === "..")) {
throw new Error(`tar: the path ${JSON.stringify(path)} is refused`);
}
}
/**
* Reads the archive `source` entry by entry (each file's data read fully before the next entry is asked for), refusing
* what it may not hold (see the file's header).
*/
export async function* readTar(source: AsyncIterable<Uint8Array>): AsyncGenerator<TarRead> {
const reader = new ByteReader(source);
for (;;) {
const block = await reader.read(BLOCK);
if (block.length < BLOCK) throw new Error("tar: the archive ends inside a header");
if (block.every((byte) => byte === 0)) return;
const { path, size, type } = parseHeader(block);
const padded = Math.ceil(size / BLOCK) * BLOCK;
if (type === "directory") {
yield { kind: "directory", path };
await reader.skip(padded);
continue;
}
let left = size;
const data = (async function* () {
while (left > 0) {
const chunk = await reader.read(Math.min(left, 64 * 1024));
if (chunk.length === 0) throw new Error(`tar: the archive ends inside ${path}`);
left -= chunk.length;
yield chunk;
}
})();
yield { kind: "file", path, size, data };
// What the consumer left unread (also when it ended the data's iteration early), then the padding.
await reader.skip(left + padded - size);
}
}
/** A header's path, size and type, its checksum checked; anything but a regular file or a directory refused. */
function parseHeader(block: Uint8Array): {
readonly path: string;
readonly size: number;
readonly type: "file" | "directory";
} {
const stored = Number.parseInt(text(block, 148, 8).trim(), 8);
let sum = 0;
for (let index = 0; index < BLOCK; index++) sum += index >= 148 && index < 156 ? 0x20 : block[index]!;
if (stored !== sum) throw new Error("tar: a header's checksum does not hold");
const flag = String.fromCharCode(block[156]!);
const type = flag === "0" || flag === "\u0000" ? "file" : flag === "5" ? "directory" : undefined;
if (type === undefined) throw new Error(`tar: an entry of type ${JSON.stringify(flag)} is refused (files only)`);
const prefix = text(block, 345, 155);
const name = text(block, 0, 100);
const path = (prefix === "" ? name : `${prefix}/${name}`).replace(/\/$/, "");
checkPath(path);
const size = Number.parseInt(text(block, 124, 12).trim() || "0", 8);
if (!Number.isSafeInteger(size) || size < 0) throw new Error(`tar: ${path} has no valid size`);
return { path, size: type === "directory" ? 0 : size, type };
}
/** The NUL-terminated text of a header field. */
function text(block: Uint8Array, offset: number, length: number): string {
const bytes = block.subarray(offset, offset + length);
const end = bytes.indexOf(0);
return new TextDecoder().decode(end < 0 ? bytes : bytes.subarray(0, end));
}
/** Reads exactly as many bytes as asked from a stream of chunks (fewer only at its end). */
class ByteReader {
readonly #source: AsyncIterator<Uint8Array>;
#buffer = new Uint8Array();
constructor(source: AsyncIterable<Uint8Array>) {
this.#source = source[Symbol.asyncIterator]();
}
/** Up to `length` bytes: `length` unless the stream ends first. */
async read(length: number): Promise<Uint8Array> {
while (this.#buffer.length < length) {
const next = await this.#source.next();
if (next.done === true) break;
const joined = new Uint8Array(this.#buffer.length + next.value.length);
joined.set(this.#buffer);
joined.set(next.value, this.#buffer.length);
this.#buffer = joined;
}
const taken = this.#buffer.subarray(0, Math.min(length, this.#buffer.length));
this.#buffer = this.#buffer.subarray(taken.length);
return taken;
}
/** Skips `length` bytes. */
async skip(length: number): Promise<void> {
let left = length;
while (left > 0) {
const skipped = await this.read(Math.min(left, 64 * 1024));
if (skipped.length === 0) throw new Error("tar: the archive ends early");
left -= skipped.length;
}
}
}

172
packages/chord/src/next-node/app/uploads.ts Normal file
View file

@ -0,0 +1,172 @@
// Chunked uploads of one `Files` object (slice 13 §4, design §23 "Chunked transfers"): each upload one file handle in
// the run's directory, written at positions. An append is accepted at the offset the accepted appends reached, so a
// writer may send several without waiting (the call window allows it): each write lands at its own position whatever
// order the thread pool completes them in. `uploaded`, `finish` and expiry wait for an upload's writes in flight; a
// failed write ends its upload.
import type { FileHandle } from "node:fs/promises";
import { mkdir, open, rename, rm } from "node:fs/promises";
import { join } from "node:path";
import { ChordError } from "../../next/data/index.ts";
import { invalid } from "../../next/data/internal.ts";
import { newToken } from "./secrets.ts";
/** An upload under way: its file, the bytes accepted, its writes in flight, its first failed write, its end. */
type Upload = {
readonly name: string;
readonly mime: string;
readonly handle: FileHandle;
size: number;
touched: number;
readonly writes: Set<Promise<unknown>>;
failed: boolean;
ended: Promise<void> | undefined;
};
/** A finished upload: its name, type and size, its bytes moved to the path `finish` was given. */
export type Finished = { readonly name: string; readonly mime: string; readonly size: number };
/**
* The uploads under way of one `Files` object, in the directory `run` (this object's own, made on first use), at most
* `max` at once: each holds a file handle until it ends, so a caller beginning uploads without end would exhaust the
* process's descriptors.
*/
export class Uploads {
readonly #run: string;
readonly #max: number;
readonly #uploads = new Map<string, Upload>();
constructor(run: string, max: number) {
this.#run = run;
this.#max = max;
}
/** The bytes accepted by every upload under way (they count against the host's cap). */
get size(): number {
let size = 0;
for (const upload of this.#uploads.values()) size += upload.size;
return size;
}
/** A new upload of `name` and `mime`: its id; `unavailable` (`busy`) while `max` are under way. */
async begin(name: string, mime: string): Promise<string> {
if (this.#uploads.size >= this.#max) {
throw new ChordError({
code: "unavailable",
reason: "busy",
message: `files: ${this.#max} uploads are under way; finish one, or wait for an idle one to expire`,
});
}
const id = newToken(12);
await mkdir(this.#run, { recursive: true, mode: 0o700 });
const handle = await open(join(this.#run, id), "wx", 0o600);
const upload: Upload = {
name,
mime,
handle,
size: 0,
touched: Date.now(),
writes: new Set(),
failed: false,
ended: undefined,
};
this.#uploads.set(id, upload);
return id;
}
/**
* Writes `bytes` at `offset` of upload `id`, which must be where the accepted appends reached (else `invalid`, its
* details the expected `{ offset }`); `room(adding, fileSize)` refuses what the caps do not allow, before anything
* is accepted. A failed write ends the upload.
*/
async append(
id: string,
offset: number,
bytes: Uint8Array,
room: (adding: number, fileSize: number) => void,
): Promise<void> {
const upload = this.#underway(id);
if (offset !== upload.size) {
throw invalid("args", `files: upload ${id} is at offset ${upload.size}, not ${offset}`, {
offset: upload.size,
});
}
room(bytes.length, upload.size + bytes.length);
// Accepted now, with no await before the write starts: the next append may follow at once, its write going to
// its own position.
upload.size += bytes.length;
upload.touched = Date.now();
const write = upload.handle.write(bytes, 0, bytes.length, offset);
upload.writes.add(write);
try {
await write;
} catch (error) {
upload.writes.delete(write);
if (!upload.failed) {
upload.failed = true;
this.#uploads.delete(id);
await this.#end(upload);
await rm(join(this.#run, id), { force: true });
}
throw error;
}
upload.writes.delete(write);
}
/** The bytes of upload `id` written so far, once its writes in flight landed: where a writer resumes. */
async uploaded(id: string): Promise<number> {
const upload = this.#underway(id);
await Promise.allSettled([...upload.writes]);
if (upload.failed) throw failedUpload(id);
return upload.size;
}
/** Ends upload `id` once its writes landed and moves its file to `path`; refused when a write of it failed. */
async finish(id: string, path: string): Promise<Finished> {
const upload = this.#underway(id);
// Taken out first: an append arriving now finds no upload.
this.#uploads.delete(id);
await this.#end(upload);
if (upload.failed) throw failedUpload(id);
await rename(join(this.#run, id), path);
return { name: upload.name, mime: upload.mime, size: upload.size };
}
/** Drops the uploads unwritten for `idleMs`, once their writes landed. */
async expire(idleMs: number): Promise<void> {
const now = Date.now();
for (const [id, upload] of this.#uploads) {
if (now - upload.touched >= idleMs) await this.#discard(id, upload);
}
}
/** Drops every upload under way (their object ended). */
async close(): Promise<void> {
for (const [id, upload] of this.#uploads) await this.#discard(id, upload);
}
/** The upload `id` under way; else `invalid`. */
#underway(id: string): Upload {
const upload = this.#uploads.get(id);
if (upload === undefined) throw invalid("args", `files: no upload ${id} (finished, failed, expired or unknown)`);
return upload;
}
/** Takes upload `id` out, closes its file once its writes landed and removes it. */
async #discard(id: string, upload: Upload): Promise<void> {
this.#uploads.delete(id);
await this.#end(upload);
await rm(join(this.#run, id), { force: true });
}
/** Closes `upload`'s file once its writes in flight landed; once, whoever ends it first. */
#end(upload: Upload): Promise<void> {
upload.ended ??= Promise.allSettled([...upload.writes]).then(() => upload.handle.close());
return upload.ended;
}
}
/** The refusal of an upload one of whose writes failed. */
function failedUpload(id: string): Error {
return invalid("args", `files: upload ${id} failed to write and was dropped`);
}

68
packages/chord/src/next-node/app/users.ts Normal file
View file

@ -0,0 +1,68 @@
// The users plugin (slice 13 §3, decision "The app kit, point 2: principals and users"): accounts keyed by the user's
// id with a profile the app describes (a schema it passes in: a Slack-like profile or just a name), kept in the
// server's store. User data never rides in the principal: a principal names the user, `Users` holds the rest.
// An exported facet, not a plugin directory: it takes the app's schema as an argument (settings are arguments), and a
// plugin directory could not import it (plugins import only the app's API). The app loads it in its root by
// re-exporting it through its API (`export const users = defineUsers(Profile)` in `/v3/server`, a one-line plugin).
import type { JsonValue } from "../../json.ts";
import { arr, local, map, method, nul, obj, type Schema, str, union } from "../../next/contracts/index.ts";
import type { Principal } from "../../next/data/index.ts";
import { invalid } from "../../next/data/internal.ts";
import { defineState } from "../../next/documents/index.ts";
import { defineFacet, type Facet } from "../../next/host/index.ts";
/** The users plugin for profiles of schema `profile`: its contract and the facet serving it on the server. */
export function defineUsers<T extends JsonValue, P extends Principal = Principal>(profile: Schema<T>) {
const Users = local("chord.app.users", {
version: 1,
params: {},
members: {
/** Every user: id and profile. */
list: arr(obj({ id: str(), profile })),
/** The profile of user `id`, or `null`. */
get: method(str(), union([profile, nul()])),
/** Sets the profile of user `id` (a new user: an account made). */
update: method(obj({ id: str(), profile })),
/** Removes user `id`'s account. */
remove: method(str()),
},
});
// Stored as JSON: the document's types cannot follow a generic profile, the contract's members do.
const stored = profile as Schema<JsonValue>;
const Accounts = defineState("chord.app.users", {
schema: obj({ users: map(stored) }),
version: 1,
initial: () => ({ users: {} }),
persist: true,
checkpointWhen: (_value, _ops, { deltasSinceBase }) => deltasSinceBase >= 50,
});
const facet: Facet<P> = defineFacet<P>((host) =>
host.serve(Users, (_params, self) => {
const accounts = self.state(Accounts);
return {
list: self.derive([accounts], ([value]) =>
Object.entries(value.users).map(([id, each]) => ({ id, profile: each as T })),
),
// A trust boundary: `id` is the caller's (`Object.hasOwn`: no inherited member is a user). Written by
// `update` from a `T`.
get: async (id) => (Object.hasOwn(accounts.value.users, id) ? (accounts.value.users[id] as T) : null),
update: async ({ id, profile: next }, ctx) => {
// As a key it would set the record's prototype, not add a user.
if (id === "__proto__") throw invalid("args", "users: __proto__ is no user id");
await accounts.change((draft) => {
draft.users[id] = next as JsonValue;
}, ctx);
return null;
},
remove: async (id, ctx) => {
await accounts.change((draft) => {
delete draft.users[id];
}, ctx);
return null;
},
};
}),
);
return { Users, facet };
}

72
packages/chord/src/next-node/app/worker.ts Normal file
View file

@ -0,0 +1,72 @@
// `startWorker` (slice 13 §1): a worker of an app built with the kit, in its entry module. `workerMain` plus the kit's
// part: the worker's directory under its kind and address with its store, the host at the worker's address acting as
// the principal the server gave it, connected to the server first, its plugins loaded with the kit's worker facets
// (`WorkerInfo`, `Files`).
import { join } from "node:path";
import type { JsonValue } from "../../json.ts";
import type { Principal } from "../../next/data/index.ts";
import { createHost } from "../../next/host/index.ts";
import { openStore, sqliteBackend } from "../../next/store/index.ts";
import { openNodeSqliteDatabase } from "../../sqlite/node.ts";
import type { Watchable } from "../../watch.ts";
import { workerMain } from "../workers/index.ts";
import type { AppApi, AppKind, KitStart } from "./app.ts";
import { filesFacet } from "./files.ts";
import { workerInfoFacet } from "./info.ts";
import { privateDirectory, privateStore } from "./layout.ts";
/** What `startWorker` takes: the app's API and kinds as the server's `startApp` has them, this process's modules. */
export type WorkerOptions = {
readonly api: AppApi;
readonly kinds: Readonly<Record<string, AppKind>>;
/** Extra keep-alive counts of the app (a record of numbers, all zero when idle). */
readonly busy?: Watchable<Readonly<Record<string, number>>>;
/** The worker host's errors (`HostError`) and a failed start's. */
readonly onError: (error: unknown) => void;
};
/** Runs this process as a worker of the app: once the start message arrived, its host is up and its plugins loaded. */
export function startWorker<P extends Principal>(options: WorkerOptions): void {
// The worker's facet kind comes with the start message: every kind's packages are offered, its own are imported.
const packages = Object.assign({}, ...Object.values(options.kinds).map((kind) => kind.packages ?? {}));
workerMain(
async ({ address, start, parent, plugins, onClose }, context) => {
// The server's `startApp` sends `{ kit, app }`.
const { kit, app } = start as { readonly kit: KitStart; readonly app: JsonValue };
await privateDirectory(kit.dir);
await privateStore(join(kit.dir, "store.sqlite"));
const store = await openStore(
sqliteBackend(await openNodeSqliteDatabase(join(kit.dir, "store.sqlite"))),
context,
);
onClose((ctx) => store.close(ctx));
// Principals are the app's: the server made these with its `principal` options.
const host = createHost<P>({
address,
principal: kit.principal as unknown as P,
store,
onError: options.onError,
});
onClose((ctx) => host.close(ctx));
// First: the server sees the worker started, then the load runs.
host.connect(parent, { principal: kit.server as unknown as P, trusted: true }, context);
const info = {
address: { kind: address.kind, key: address.key, params: address.params as JsonValue },
start: app,
};
const core = [
{ name: "chord.app.worker-info", facet: workerInfoFacet<P>(info) },
{ name: "chord.app.files", facet: filesFacet<P>(join(kit.dir, "files"), kit.files, kit.uploads) },
];
await plugins.load(host, { kind: kit.facet, core }, context);
return options.busy === undefined ? {} : { busy: options.busy };
},
{
hostPackages: { ...options.api.modules, ...packages },
// Worker errors are public (decision "worker errors in public"): a fixed message, never the error's own.
redact: () => ({ message: "worker failed" }),
onError: options.onError,
},
);
}

21
packages/chord/src/next-node/http/README.md
View file

@ -35,12 +35,16 @@ await server.close(ctx);
// a plugin's server facet
self.contribute(Http, {
routes: {
"GET /notes/:id": async (request, ctx) => ({ kind: "json", value: … }),
"GET /notes/:id": async (request, ctx) => ({ kind: "json", value: … }), // needs a principal (else 401)
"GET /status": { public: true, handler: async () => ({ kind: "json", value: "ok" }) },
"GET /": { page: true, handler: async (request) => ({ kind: "html", body: `<script nonce="${request.nonce}">…` }) },
"POST /hooks/ci": { crossSite: true, handler: async (request) => … }, // always anonymous
},
wraps: [{ name: "audit", phase: "observe", scope: "all", wrap: async (request, next, ctx) => next(request) }],
sockets: { "/terminal/:id": (socket, request, ctx) => attach(socket, ctx) },
sockets: {
"/terminal/:id": (socket, request, ctx) => attach(socket, ctx), // needs a principal (else 401)
"/lobby": { public: true, handler: (socket, request, ctx) => join(socket, ctx) },
},
maxPayload: 1024 * 1024, // its sockets' largest frame
hosts: ["notes.example.com"], // a writable (`self.writable`) to change it
framers: ["https://shell.example.com"], // who may frame the app's pages
@ -88,9 +92,13 @@ await http.close(ctx); // before host.close
- **Who:** the authenticators in contributor order (plugin name: an order that means nothing, so an app needing
precedence between credentials contributes one authenticator that decides); the first principal wins, a throwing one
is reported `hook_failed` and skipped; none: anonymous; once the request is gone or sealed no further one starts.
- **Deny by default** (slice 13 §2): an anonymous request to a route or socket not marked `public: true` answers 401
before any plugin code runs (wraps included); a `crossSite` route is public (it is always anonymous). The links
endpoint refuses anonymous upgrades with 401. A foreign request stays anonymous, so it reaches public routes only.
`cookie(name, value, { secret, request, sameSite?, maxAge? })` makes a signed `HttpOnly`, `Path=/`, `SameSite=Strict`
(or `Lax`) cookie, `Secure` on a remote origin, its expiry (`maxAge`) signed with it; `readCookie(request, name,
secret)` reads it back, its signature and expiry checked. A cookie without `maxAge` holds until the secret changes.
secret)` reads it back, its signature and expiry checked; `readSignedCookie` gives the value with its signed expiry
(to renew a cookie before it runs out). A cookie without `maxAge` holds until the secret changes.
Authenticators run raced against the request's signal: a client gone or `close`'s seal answers 503 (or refuses the
upgrade) without waiting for them.
- **A request's scope:** one `host.root` scope with its principal (an explicit `undefined`: anonymous) spans matching,
@ -100,7 +108,9 @@ await http.close(ctx); // before host.close
object it calls acts as its handler is written: a local helper calling remote types runs them as the host. A socket
handler's scope lasts until its socket closes.
- **`HttpRequest`:** `method`, `path` (dot segments resolved), `params`, `query`, `headers` (Web `Headers`), `origin`,
`provenance`, `nonce` (fresh for every response), `json(limit?)` (1 MiB; malformed 400, too large 413),
`provenance`, `nonce` (fresh for every response), `remoteAddress` (the peer's address as the listener knows it, a
relay's for a relayed request, none through `fetch` without a connection: for limits per client, never an
identity), `json(limit?)` (1 MiB; malformed 400, too large 413),
`body(limit?)` (chunks, 413 past the limit; a body cut short, by a disconnect or a broken stream, is a 400: a prefix
never parses as the whole). An `HttpFailure` escaping a route answers its status. Chord's listener drains a body
nobody read after the response up to 64 KiB, else closes the connection.
@ -129,7 +139,8 @@ await http.close(ctx); // before host.close
'none'`, `form-action 'self'`, `frame-ancestors` the framers or `'none'`, plus contributed sources). A response's own
headers replace same-named ones (a list sends one line per value: several `Set-Cookie`); a header with a CR, LF or
NUL, or a value that is no JSON, is a 500 reported `response_failed`.
- **Sockets:** after admission, the origin checks and the authenticators (no principal: 403 before plugin code), a
- **Sockets:** after admission, the origin checks and the authenticators (no principal and not public: 401 before
plugin code), a
`ws` WebSocket (its own server per upgrade, no compression) with an `error` listener attached (an oversized frame
closes it, 1009); a throwing handler is reported `socket_failed` and closes it (1011). Pinged after 15 s of silence,
terminated 15 s later without an answer. A client gone while its upgrade authenticates aborts that work.

23
packages/chord/src/next-node/http/cookies.ts
View file

@ -9,8 +9,8 @@ import type { HttpRequest } from "./request.ts";
export type CookieOptions = {
/** The key it is signed with (HMAC-SHA256). */
readonly secret: string;
/** The request it answers: a remote one gets `Secure`. */
readonly request: HttpRequest;
/** The request it answers (its provenance): a remote one gets `Secure`. */
readonly request: Pick<HttpRequest, "provenance">;
/** `Strict` by default; `Lax` lets a top-level navigation from another site carry it. */
readonly sameSite?: "Strict" | "Lax";
/** Seconds until it expires; none: a session cookie. */
@ -42,6 +42,18 @@ export function cookie(name: string, value: string, options: CookieOptions): str
* `undefined` otherwise.
*/
export function readCookie(request: HttpRequest, name: string, secret: string): string | undefined {
return readSignedCookie(request, name, secret)?.value;
}
/**
* Cookie `name` on `request` as `readCookie` checks it, with its signed expiry (milliseconds since the epoch; none for
* a cookie without `maxAge`): what renewing a cookie before it expires needs.
*/
export function readSignedCookie(
request: HttpRequest,
name: string,
secret: string,
): { readonly value: string; readonly expires: number | undefined } | undefined {
for (const pair of (request.headers.get("cookie") ?? "").split(";")) {
const equals = pair.indexOf("=");
if (equals < 0 || pair.slice(0, equals).trim() !== name) continue;
@ -54,10 +66,11 @@ export function readCookie(request: HttpRequest, name: string, secret: string):
const given = Buffer.from(signed.slice(dot + 1));
const expected = Buffer.from(signature(name, payload, secret));
if (given.length !== expected.length || !timingSafeEqual(given, expected)) return undefined;
const expires = payload.slice(expiresAt + 1);
if (expires !== "" && Number(expires) * 1000 <= Date.now()) return undefined;
const stamp = payload.slice(expiresAt + 1);
const expires = stamp === "" ? undefined : Number(stamp) * 1000;
if (expires !== undefined && expires <= Date.now()) return undefined;
try {
return decodeURIComponent(payload.slice(0, expiresAt));
return { value: decodeURIComponent(payload.slice(0, expiresAt)), expires };
} catch {
return undefined;
}

14
packages/chord/src/next-node/http/http.ts
View file

@ -233,7 +233,7 @@ class Component<P extends Principal> implements HttpComponent<P>, Handler {
return { answer: this.refusal(403, undefined, router, head) };
}
// A fresh script nonce per response (decision "A CSP nonce per response").
const webRequest = new WebRequest(request, admission, this.#nonces.next());
const webRequest = new WebRequest(request, admission, this.#nonces.next(), connection.remoteAddress);
const match = router.route(request.method, webRequest.path);
if (match === "bad_path" || match === undefined) {
return { answer: this.refusal(match === undefined ? 404 : 400, admission, router, head) };
@ -256,6 +256,8 @@ class Component<P extends Principal> implements HttpComponent<P>, Handler {
return { answer: this.refusal(503, admission, router, head) };
}
}
// Deny by default (decision "The app kit, point 3"): no plugin code runs for nobody unless the route is public.
if (principal === undefined && !match.public) return { answer: this.refusal(401, admission, router, head) };
const answered = Promise.withResolvers<Answer>();
const scope = this.#host
.root(
@ -402,17 +404,17 @@ class Component<P extends Principal> implements HttpComponent<P>, Handler {
}
if (this.#closing.signal.aborted || signal.aborted) return refuse(stream, 503);
const before = registries.router.value;
const admission = admit(request, connectionOf(message), before.hosts, before.relays);
const connection = connectionOf(message);
const admission = admit(request, connection, before.hosts, before.relays);
if (refused(admission)) {
if (admission.failed !== undefined) this.#hookFailed("relay", admission.failed.relay, admission.failed.error);
return refuse(stream, 403);
}
const sent = request.header("origin");
if (isForeign(request) || (sent !== null && sent !== admission.origin)) return refuse(stream, 403);
const webRequest = new WebRequest(request, admission, this.#nonces.next());
const webRequest = new WebRequest(request, admission, this.#nonces.next(), connection.remoteAddress);
const principal = await this.#principal(registries, webRequest, signal);
if (this.#closing.signal.aborted || signal.aborted) return refuse(stream, 503);
if (principal === undefined) return refuse(stream, 403);
// The router now: a host, a relay or a socket removed while authenticating refuses the upgrade.
const router = registries.router.value;
const { allowed, relay } = admission;
@ -424,6 +426,8 @@ class Component<P extends Principal> implements HttpComponent<P>, Handler {
}
const { links } = this.#options;
if (links !== undefined && webRequest.path === links.path) {
// The links endpoint always needs a principal: a link acts as one.
if (principal === undefined) return refuse(stream, 401);
let options: LinkOptions;
try {
options =
@ -438,6 +442,7 @@ class Component<P extends Principal> implements HttpComponent<P>, Handler {
const match = router.socket(webRequest.path);
if (match === "bad_path") return refuse(stream, 400);
if (match === undefined) return refuse(stream, 404);
if (principal === undefined && !match.public) return refuse(stream, 401);
webRequest.params = match.params;
const fail = ({ pattern, plugin, error }: { pattern: string; plugin: string; error: unknown }) =>
this.#onError({ kind: "socket_failed", pattern, plugin, error });
@ -517,6 +522,7 @@ function refuse(stream: Duplex, status: number): void {
/** The reason phrases of the statuses an upgrade is refused with. */
const STATUS: Readonly<Record<number, string>> = {
400: "Bad Request",
401: "Unauthorized",
403: "Forbidden",
404: "Not Found",
501: "Not Implemented",

2
packages/chord/src/next-node/http/index.ts
View file

@ -5,7 +5,7 @@
export type { Connection, Provenance } from "./admission.ts";
export { type Authenticator, Authenticators } from "./authenticate.ts";
export { type CookieOptions, cookie, readCookie } from "./cookies.ts";
export { type CookieOptions, cookie, readCookie, readSignedCookie } from "./cookies.ts";
export { type FileResponseOptions, fileResponse } from "./file.ts";
export {
createHttp,

8
packages/chord/src/next-node/http/request.ts
View file

@ -22,6 +22,11 @@ export interface HttpRequest {
readonly provenance: Provenance;
/** The server's script nonce: a page's inline scripts carry it (`<script nonce="…">`), the CSP allows them. */
readonly nonce: string;
/**
* The peer's address as the listener knows it (a relayed request: the relay's; `fetch` without a connection: none),
* for limits per client; never an identity.
*/
readonly remoteAddress: string | undefined;
/** The body as JSON, at most `limit` bytes (1 MiB by default): `HttpFailure` 400 when malformed, 413 when larger. */
json(limit?: number): Promise<JsonValue>;
/** The body's chunks, at most `limit` bytes (1 MiB by default): `HttpFailure` 413 past it, 400 when cut short. */
@ -53,6 +58,7 @@ export class WebRequest implements HttpRequest {
readonly origin: string;
readonly provenance: Provenance;
readonly nonce: string;
readonly remoteAddress: string | undefined;
readonly #incoming: Incoming;
/** The query string (without `?`), parsed when first read. */
readonly #search: string;
@ -64,6 +70,7 @@ export class WebRequest implements HttpRequest {
incoming: Incoming,
admitted: { readonly origin: string; readonly provenance: Provenance },
nonce: string,
remoteAddress: string | undefined,
) {
this.method = incoming.method;
// The URL is parsed and serialized already (dot segments resolved, `\` as `/`): only split it here.
@ -73,6 +80,7 @@ export class WebRequest implements HttpRequest {
this.origin = admitted.origin;
this.provenance = admitted.provenance;
this.nonce = nonce;
this.remoteAddress = remoteAddress;
this.#incoming = incoming;
}

24
packages/chord/src/next-node/http/router.ts
View file

@ -13,11 +13,13 @@ import type { HttpResponse } from "./response.ts";
export type HttpRoute = (request: HttpRequest, ctx: Ctx) => Promise<HttpResponse>;
/**
* A route with its options: `page` admits a top-level navigation from another site (a link to it) with its
* authentication; `crossSite` admits any foreign request (a webhook), always anonymous.
* A route with its options: `public` serves it without a principal (deny by default: a route without one answers 401
* to a request no authenticator recognized); `page` admits a top-level navigation from another site (a link to it)
* with its authentication; `crossSite` admits any foreign request (a webhook), always anonymous, so public too.
*/
export type HttpRouteOptions = {
readonly handler: HttpRoute;
readonly public?: true;
readonly page?: true;
readonly crossSite?: true;
};
@ -46,6 +48,9 @@ export type HttpWrap = {
*/
export type HttpSocket = (socket: WebSocketHandle, request: HttpRequest, ctx: Ctx) => void | Promise<void>;
/** A plugin socket with its options: `public` accepts it without a principal (else 401 before the handler runs). */
export type HttpSocketOptions = { readonly handler: HttpSocket; readonly public?: true };
/** A plugin socket: `ws`'s WebSocket, for the plugin's own protocol. */
export type WebSocketHandle = WebSocket;
@ -64,7 +69,7 @@ export type HttpContribution = {
readonly routes?: Readonly<Record<string, HttpRoute | HttpRouteOptions>>;
readonly wraps?: readonly HttpWrap[];
/** Plugin sockets by pattern (no method). */
readonly sockets?: Readonly<Record<string, HttpSocket>>;
readonly sockets?: Readonly<Record<string, HttpSocket | HttpSocketOptions>>;
/** The largest frame this contribution's sockets accept; 1 MiB by default. */
readonly maxPayload?: number;
/** Hosts allowed besides loopback, as the client names them (`notes.example.com`): requests through them are https. */
@ -101,6 +106,8 @@ export type RouteMatch = {
readonly plugin: string;
readonly params: Readonly<Record<string, string>>;
readonly handler: HttpRoute;
/** Served without a principal: marked `public`, or `crossSite` (always anonymous). */
readonly public: boolean;
readonly page: boolean;
readonly crossSite: boolean;
readonly wraps: readonly ResolvedWrap[];
@ -112,6 +119,7 @@ export type SocketMatch = {
readonly plugin: string;
readonly params: Readonly<Record<string, string>>;
readonly handler: HttpSocket;
readonly public: boolean;
readonly maxPayload: number;
};
@ -167,6 +175,7 @@ export function composeRouter(table: RegistryTable<HttpContribution>, { report }
pattern: key,
plugin,
handler: options.handler,
public: options.public === true || options.crossSite === true,
page: options.page === true,
crossSite: options.crossSite === true,
segments: parsed.segments,
@ -177,13 +186,16 @@ export function composeRouter(table: RegistryTable<HttpContribution>, { report }
else report({ message: `route "${key}" is contributed twice`, plugins: [taken.plugin, plugin] });
}
const maxPayload = value.maxPayload ?? DEFAULT_MAX_PAYLOAD;
for (const [pattern, handler] of Object.entries(value.sockets ?? {})) {
for (const [pattern, socket] of Object.entries(value.sockets ?? {})) {
const segments = parsePattern(pattern);
if (segments === undefined) {
report({ message: `socket "${pattern}" is no pattern`, plugins: [plugin] });
continue;
}
const taken = insert(sockets, segments, { pattern, plugin, handler, segments, maxPayload });
const { handler, public: open } =
typeof socket === "function" ? { handler: socket, public: undefined } : socket;
const entry = { pattern, plugin, handler, public: open === true, segments, maxPayload };
const taken = insert(sockets, segments, entry);
if (taken === undefined) socketHandlers.add(handler);
else report({ message: `socket "${pattern}" is contributed twice`, plugins: [taken.plugin, plugin] });
}
@ -218,6 +230,7 @@ type RouteEntry = {
readonly pattern: string;
readonly plugin: string;
readonly handler: HttpRoute;
readonly public: boolean;
readonly page: boolean;
readonly crossSite: boolean;
readonly segments: readonly Segment[];
@ -229,6 +242,7 @@ type SocketEntry = {
readonly pattern: string;
readonly plugin: string;
readonly handler: HttpSocket;
readonly public: boolean;
readonly segments: readonly Segment[];
readonly maxPayload: number;
};

7
packages/chord/src/next-node/http/upgrades.ts
View file

@ -81,7 +81,12 @@ export class Upgrades {
message: IncomingMessage,
stream: Duplex,
head: Buffer,
admission: Admission<P> & { readonly match: SocketMatch; readonly request: HttpRequest },
admission: Omit<Admission<P>, "principal"> & {
/** Anonymous only for a public socket. */
readonly principal: P | undefined;
readonly match: SocketMatch;
readonly request: HttpRequest;
},
host: Host<P>,
context: Context,
failed: (failure: SocketFailure) => void,

308
packages/chord/test/next-node/app/app.test.ts Normal file
View file

@ -0,0 +1,308 @@
// Tests of `startApp` (slice 13 §1, §4, §5): the layout and the start-data object, start and close, a server without
// workers, two worker kinds, `applyChanges`, a required root failing, and export and import per address (documents,
// tables and files round-trip; an existing address refused).
import { chmod, mkdir, readdir, stat, symlink, writeFile } from "node:fs/promises";
import { homedir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { hostKind, local, remote, str, unknown } from "../../../src/next/contracts/index.ts";
import { AppInfo } from "../../../src/next-node/app/index.ts";
import { rejection } from "../../next/support.ts";
import { send } from "../http/support.ts";
import { gitRepository, writePlugin } from "../plugins/support.ts";
import { ECHO_SERVER, NOTES_PLUGIN, Notes, Project, tempDir, testApp, WORKER_ENTRY } from "./support.ts";
const Echo = local("test.echo", { version: 1, params: {}, members: { text: str() } });
describe("starting and closing", () => {
it("lays out the data directory, serves AppInfo and the server's plugins, and closes everything", async () => {
const { app, dataDir, port } = await testApp({ plugins: { echo: { "server.ts": ECHO_SERVER } } });
expect((await readdir(dataDir)).sort()).toEqual(["logs", "owner-token", "plugins", "server", "workers"]);
expect((await readdir(join(dataDir, "server"))).filter((name) => !name.startsWith("."))).toEqual(
expect.arrayContaining(["cookie-secret", "store.sqlite"]),
);
const info = await app.host.get(AppInfo, {}, context);
expect([info.dataDir.value, info.name.value, info.version.value]).toEqual([dataDir, "chord-test", "v1"]);
expect((await app.host.get(Echo, {}, context)).text.value).toBe("echo");
expect(app.ownerLink()).toMatch(new RegExp(`^http://127\\.0\\.0\\.1:${port}/_chord/owner#[A-Za-z0-9_-]{43}$`));
expect(app.workers.size).toBe(0);
await app.close(context);
// HTTP closed first: the port answers nothing any more; a second close is the same close.
await expect(send(port, "GET", "/")).rejects.toThrow();
await app.close(context);
});
it("refuses a second server on the same data directory while the first runs", async () => {
const first = await testApp();
const second = await rejection(testApp({ dataDir: first.dataDir, root: first.root }));
expect(String(second)).toContain(`another server runs on ${first.dataDir}`);
await first.app.close(context);
const third = await testApp({ dataDir: first.dataDir, root: first.root });
await third.app.close(context);
});
it("refuses a second server on the same data directory through a symbolic link (review L3)", async () => {
const first = await testApp();
const link = join(await tempDir(), "link");
await symlink(first.dataDir, link);
expect(String(await rejection(testApp({ dataDir: link, root: first.root })))).toContain("another server runs on");
await first.app.close(context);
});
it("makes the data directory private at every start, also when it existed looser (review M3)", async () => {
const dataDir = join(await tempDir(), "data");
await mkdir(join(dataDir, "server"), { recursive: true, mode: 0o775 });
await chmod(dataDir, 0o755);
await chmod(join(dataDir, "server"), 0o775);
const { app } = await testApp({ dataDir });
const mode = async (path: string) => (await stat(join(dataDir, path))).mode & 0o777;
const paths = [
".",
"server",
"workers",
"plugins",
"logs",
"server/store.sqlite",
"server/cookie-secret",
"owner-token",
];
expect(await Promise.all(paths.map(mode))).toEqual([0o700, 0o700, 0o700, 0o700, 0o700, 0o600, 0o600, 0o600]);
await app.close(context);
await chmod(join(dataDir, "server", "store.sqlite"), 0o644);
await chmod(join(dataDir, "owner-token"), 0o644);
const again = await testApp({ dataDir });
expect([await mode("server/store.sqlite"), await mode("owner-token")]).toEqual([0o600, 0o600]);
await again.app.close(context);
});
it("refuses a data directory the kit must not tighten: home, root, another user's, also through a link (recheck N4)", async () => {
const homeMode = (await stat(homedir())).mode & 0o777;
expect(String(await rejection(testApp({ dataDir: homedir() })))).toContain("is the home directory");
expect(String(await rejection(testApp({ dataDir: "/" })))).toContain("is a file system's root");
const link = join(await tempDir(), "home-link");
await symlink(homedir(), link);
expect(String(await rejection(testApp({ dataDir: link })))).toContain("is the home directory");
expect((await stat(homedir())).mode & 0o777).toBe(homeMode);
if (process.getuid?.() !== 0) {
// `/tmp`'s owner is root.
expect(String(await rejection(testApp({ dataDir: "/tmp" })))).toContain("belongs to another user");
}
});
it("names the address listened on in the owner link (review L4)", async () => {
const { app } = await testApp({ listen: { host: "::1", port: 0 } });
expect(app.ownerLink()).toMatch(/^http:\/\/\[::1\]:\d+\/_chord\/owner#/);
await app.close(context);
});
it("rejects when a plugin of the app's root fails, with everything closed", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "broken", "@test/broken", { "server.ts": "export default {" });
const dataDir = join(base, "data");
const error = await rejection(testApp({ root, dataDir }));
expect(error).toMatchObject({ name: "PluginError" });
expect(String(error)).toContain('"@test/broken"');
// Closed: the data directory's lock is free again (a second start fails only on the same plugin).
expect(await rejection(testApp({ root, dataDir }))).toMatchObject({ name: "PluginError" });
});
});
describe("workers", () => {
it("starts a worker per address with the kit's start data and its own directory; two kinds side by side", async () => {
const Team = hostKind("team", { id: str() });
const Members = remote("test.members", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Team({ id }),
variants: { main: { state: { start: unknown() } } },
});
const team = [
'import { defineFacet, hostKind, remote, str, unknown, WorkerInfo } from "@test/app/v1";',
'const Team = hostKind("team", { id: str() });',
'const Members = remote("test.members", { version: 1, params: { id: str() }, hostAddress: ({ id }) => Team({ id }), variants: { main: { state: { start: unknown() } } } });',
"export default defineFacet((host) => host.serve(Members, (_params, _principal, self) => {",
"\tconst info = self.get(WorkerInfo, {});",
'\treturn { variant: "main", members: { start: self.derive([info.start], ([value]) => value) } };',
"}));",
].join("\n");
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "notes", "@test/notes", NOTES_PLUGIN);
await writePlugin(root, "team", "@test/team", { "teamworker.ts": team });
const { app, dataDir } = await testApp({
root,
withWorkers: true,
kinds: { server: { platform: "node" }, worker: { platform: "node" }, teamworker: { platform: "node" } },
workers: [
{
kind: Project,
entry: WORKER_ENTRY,
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
start: (address) => ({ project: address.key }),
authorize: () => true,
},
{
kind: Team,
entry: WORKER_ENTRY,
facet: "teamworker",
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
start: (address) => ({ team: address.key }),
authorize: () => true,
},
],
});
const notes = await app.host.get(Notes, { id: "p1" }, context);
const members = await app.host.get(Members, { id: "t1" }, context);
expect([notes.start.value, members.start.value]).toEqual([{ project: "project|p1" }, { team: "team|t1" }]);
expect(await readdir(join(dataDir, "workers"))).toEqual(["project", "team"]);
expect(await readdir(join(dataDir, "workers", "project"))).toEqual(["project%7Cp1"]);
notes.release();
members.release();
await app.close(context);
}, 60_000);
it("applies changes: builds the dirty plugins and rolls them out to the server, then the workers", async () => {
const { app, root } = await testApp({ plugins: { echo: { "server.ts": ECHO_SERVER } } });
const echo = await app.host.get(Echo, {}, context);
await writeFile(join(root, "echo", "server.ts"), ECHO_SERVER.replace('"echo"', '"changed"'));
while (!app.plugins.state.value.dirty.includes("@test/echo"))
await new Promise((resolve) => setTimeout(resolve, 10));
const result = await app.applyChanges(context);
expect(result.stopped).toBeUndefined();
expect(result.steps.map(({ name }) => name)).toEqual(["server"]);
expect(echo.text.value).toBe("changed");
expect(app.plugins.state.value.dirty).toEqual([]);
await app.close(context);
});
});
describe("installs", () => {
it("installs a plugin from git into the user's root; applying changes loads it", async () => {
const tools = await tempDir();
await writeFile(join(tools, "npm.mjs"), "process.exit(0);");
const repository = await gitRepository(
"echo",
{ "package.json": JSON.stringify({ name: "@bob/echo" }), "server.ts": ECHO_SERVER },
"v1",
);
const { app, dataDir } = await testApp({ install: { npmCommand: [process.execPath, join(tools, "npm.mjs")] } });
expect(await app.plugins.install({ git: repository.url, ref: "v1" }, context)).toEqual(["@bob/echo"]);
expect(await readdir(join(dataDir, "plugins"))).toEqual(["echo"]);
expect(app.plugins.state.value.set).toBe(true);
await app.applyChanges(context);
expect((await app.host.get(Echo, {}, context)).text.value).toBe("echo");
await app.close(context);
});
});
describe("export and import per address", () => {
it("holds the worker's lock while exporting: a hold arriving meanwhile starts the worker after it (review M1)", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "notes", "@test/notes", NOTES_PLUGIN);
const { app, dataDir } = await testApp({ root, withWorkers: true });
const notes = await app.host.get(Notes, { id: "p1" }, context);
await notes.write("one", context);
notes.release();
// A large file makes the pack take a while.
const files = join(dataDir, "workers", "project", "project%7Cp1", "files");
await mkdir(files, { recursive: true });
await writeFile(join(files, "big.bin"), new Uint8Array(200 * 1024 * 1024));
const workers = app.workers.get("project")!;
const status = () =>
(workers.state.value as Record<string, { status: string } | undefined>)["project|p1"]?.status;
let done = false;
const exporting = app.exportAddress(Project({ id: "p1" }), join(base, "p1.tar"), context).then(() => {
done = true;
});
while (status() !== undefined) await new Promise((resolve) => setTimeout(resolve, 5));
const holder = app.host.get(Notes, { id: "p1" }, context);
let heldWhileExporting = false;
let ranWhileExporting = false;
while (!done) {
heldWhileExporting ||= status() === "starting";
ranWhileExporting ||= status() === "running";
await new Promise((resolve) => setTimeout(resolve, 5));
}
await exporting;
const ref = await holder;
expect([heldWhileExporting, ranWhileExporting]).toEqual([true, false]);
expect(ref.text.value).toBe("one");
ref.release();
await app.close(context);
}, 60_000);
it("leaves a failed address failed after an export (recheck N5)", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "notes", "@test/notes", NOTES_PLUGIN);
let failing = false;
const { app } = await testApp({
root,
workers: [
{
kind: Project,
entry: WORKER_ENTRY,
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
start: () => {
if (failing) throw new Error("no start today");
return {};
},
authorize: () => true,
},
],
});
const notes = await app.host.get(Notes, { id: "p1" }, context);
await notes.write("kept", context);
notes.release();
const workers = app.workers.get("project")!;
await workers.stop(Project({ id: "p1" }), context);
await workers.retry(Project({ id: "p1" }), context);
failing = true;
const status = () =>
(workers.state.value as Record<string, { status: string } | undefined>)["project|p1"]?.status;
const hold = app.host.hold(Notes, { id: "p1" }, context);
while (status() !== "failed") await new Promise((resolve) => setTimeout(resolve, 5));
await app.exportAddress(Project({ id: "p1" }), join(base, "p1.tar"), context);
expect(status()).toBe("failed");
hold.release();
await app.close(context);
}, 60_000);
it("round-trips a worker's documents, tables and files; refuses an existing address", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "notes", "@test/notes", NOTES_PLUGIN);
const { app, dataDir } = await testApp({ root, withWorkers: true });
const notes = await app.host.get(Notes, { id: "p1" }, context);
await notes.write("buy milk", context);
expect(await notes.visit("/", context)).toBe(1);
const file = await notes.attach("a file of p1", context);
notes.release();
const archive = join(base, "p1.tar");
await app.exportAddress(Project({ id: "p1" }), archive, context);
expect((await stat(archive)).mode & 0o777).toBe(0o600);
await expect(app.importAddress(archive, Project({ id: "p1" }), context)).rejects.toThrow("exists already");
await app.importAddress(archive, Project({ id: "p2" }), context);
const copy = await app.host.get(Notes, { id: "p2" }, context);
expect(copy.text.value).toBe("buy milk");
expect(await copy.visit("/", context)).toBe(2);
expect(await copy.file(file, context)).toBe("a file of p1");
expect(await readdir(join(dataDir, "workers", "project"))).toEqual(["project%7Cp1", "project%7Cp2"]);
copy.release();
// The exported address goes on: its next hold starts its worker again.
const again = await app.host.get(Notes, { id: "p1" }, context);
expect(again.text.value).toBe("buy milk");
again.release();
await app.close(context);
// Another app refuses the archive.
const other = await testApp({ name: "other-app", withWorkers: true });
await expect(other.app.importAddress(archive, Project({ id: "p1" }), context)).rejects.toThrow(
'an archive of "chord-test", not "other-app"',
);
await other.app.close(context);
}, 60_000);
});

142
packages/chord/test/next-node/app/archive.test.ts Normal file
View file

@ -0,0 +1,142 @@
// Tests of address archives (slice 13 §4) on their own: pack and unpack round-trip a host directory (its store's
// export and its files, uploads under way left out); unpacking refuses an existing directory, a manifest `accept`
// refuses, an archive not starting with its manifest and an entry outside the format, each leaving nothing behind.
// The round trip of a live worker's documents, tables and files is in `app.test.ts`.
import { createWriteStream } from "node:fs";
import { mkdir, readdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pipeline } from "node:stream/promises";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { encodeJsonl, openStore, sqliteBackend } from "../../../src/next/store/index.ts";
import { type ArchiveManifest, packAddress, unpackAddress } from "../../../src/next-node/app/archive.ts";
import { writeTar } from "../../../src/next-node/app/tar.ts";
import { openNodeSqliteDatabase } from "../../../src/sqlite/node.ts";
import { tempDir } from "./support.ts";
const MANIFEST: ArchiveManifest = { format: 1, app: "acme", api: "v1", address: { kind: "doc", key: "doc|1" } };
/** A host directory with a fresh store and files `files/a` and `files/sub/b`, plus an upload under way. */
async function hostDirectory(base: string): Promise<string> {
const dir = join(base, "host");
await mkdir(join(dir, "files", "sub"), { recursive: true });
await mkdir(join(dir, "files", ".uploads", "run"), { recursive: true });
await (await openStore(sqliteBackend(await openNodeSqliteDatabase(join(dir, "store.sqlite"))), context)).close(
context,
);
await writeFile(join(dir, "files", "a"), "file a");
await writeFile(join(dir, "files", "sub", "b"), "file b");
await writeFile(join(dir, "files", ".uploads", "run", "u"), "half");
return dir;
}
/** The JSONL export of the store in `dir`. */
async function exported(dir: string): Promise<string> {
const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(join(dir, "store.sqlite"))), context);
const chunks: Uint8Array[] = [];
for await (const chunk of encodeJsonl(store.export(context))) chunks.push(chunk);
await store.close(context);
return Buffer.concat(chunks).toString();
}
/** An archive at `path` of `entries` (path and text), as an attacker could make one. */
async function forged(path: string, entries: ReadonlyArray<readonly [string, string]>): Promise<void> {
const tar = writeTar(
entries.map(([name, text]) => {
const bytes = new TextEncoder().encode(text);
return {
path: name,
size: bytes.length,
data: (async function* () {
yield bytes;
})(),
};
}),
);
await pipeline(tar, createWriteStream(path));
}
describe("pack and unpack", () => {
it("round-trips a host directory's store and files, without uploads under way", async () => {
const base = await tempDir();
const dir = await hostDirectory(base);
await packAddress(dir, MANIFEST, join(base, "out.tar"), context);
const seen: ArchiveManifest[] = [];
await unpackAddress(join(base, "out.tar"), join(base, "copy"), (manifest) => seen.push(manifest), {}, context);
expect(seen).toEqual([MANIFEST]);
expect((await readdir(join(base, "copy"))).sort()).toEqual(["files", "store.sqlite"]);
expect((await readdir(join(base, "copy", "files"))).sort()).toEqual(["a", "sub"]);
expect(await readFile(join(base, "copy", "files", "sub", "b"), "utf8")).toBe("file b");
expect(await exported(join(base, "copy"))).toBe(await exported(dir));
// No staging directory left beside either.
expect((await readdir(base)).sort()).toEqual(["copy", "host", "out.tar"]);
});
it("refuses to pack an address without data", async () => {
const base = await tempDir();
await expect(packAddress(join(base, "none"), MANIFEST, join(base, "out.tar"), context)).rejects.toThrow(
"has no data",
);
});
});
describe("refusals when unpacking", () => {
it("refuses an archive past its limits: unpacked bytes and entries, counted before writing", async () => {
const base = await tempDir();
const big = "x".repeat(2000);
const entries: Array<readonly [string, string]> = [
["manifest.json", JSON.stringify(MANIFEST)],
["store.archive", ""],
];
await forged(join(base, "big.tar"), [...entries, ["files/a", big], ["files/b", big]]);
await expect(
unpackAddress(join(base, "big.tar"), join(base, "a"), () => {}, { bytes: 3000 }, context),
).rejects.toThrow("unpacks to more than 3000 bytes");
await forged(join(base, "many.tar"), [
...entries,
...Array.from({ length: 5 }, (_, index) => [`files/${index}`, "x"] as const),
]);
await expect(
unpackAddress(join(base, "many.tar"), join(base, "b"), () => {}, { entries: 4 }, context),
).rejects.toThrow("more than 4 entries");
expect((await readdir(base)).sort()).toEqual(["big.tar", "many.tar"]);
});
it("refuses an existing directory", async () => {
const base = await tempDir();
const dir = await hostDirectory(base);
await packAddress(dir, MANIFEST, join(base, "out.tar"), context);
await expect(unpackAddress(join(base, "out.tar"), dir, () => {}, {}, context)).rejects.toThrow("exists already");
});
it("refuses what `accept` refuses, an archive not starting with its manifest, and a foreign entry; leaves nothing", async () => {
const base = await tempDir();
await packAddress(await hostDirectory(base), MANIFEST, join(base, "out.tar"), context);
const refuse = () => {
throw new Error("not this app");
};
await expect(unpackAddress(join(base, "out.tar"), join(base, "a"), refuse, {}, context)).rejects.toThrow(
"not this app",
);
await forged(join(base, "late.tar"), [
["files/x", "x"],
["manifest.json", JSON.stringify(MANIFEST)],
]);
await expect(unpackAddress(join(base, "late.tar"), join(base, "b"), () => {}, {}, context)).rejects.toThrow(
"start with manifest.json",
);
await forged(join(base, "extra.tar"), [
["manifest.json", JSON.stringify(MANIFEST)],
["plugins/evil.ts", "x"],
]);
await expect(unpackAddress(join(base, "extra.tar"), join(base, "c"), () => {}, {}, context)).rejects.toThrow(
"plugins/evil.ts",
);
await forged(join(base, "nostore.tar"), [["manifest.json", JSON.stringify(MANIFEST)]]);
await expect(unpackAddress(join(base, "nostore.tar"), join(base, "d"), () => {}, {}, context)).rejects.toThrow(
"no manifest.json or store.archive",
);
expect((await readdir(base)).sort()).toEqual(["extra.tar", "host", "late.tar", "nostore.tar", "out.tar"]);
});
});

55
packages/chord/test/next-node/app/ephemeral.test.ts Normal file
View file

@ -0,0 +1,55 @@
// Tests of the ephemeral login plugin (slice 13 §3): a public route taking a name signs the browser in as a fresh
// `{ kind: "user", id, name }` with a cookie that ends with the browser, and the session ends with the server.
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { ephemeralLoginFacet } from "../../../src/next-node/app/index.ts";
import { Http } from "../../../src/next-node/http/index.ts";
import { send } from "../http/support.ts";
import { API, type TestPrincipal, testApp, WHOAMI } from "./support.ts";
const login = ephemeralLoginFacet<TestPrincipal>();
/** The tests' API with the ephemeral login plugin in its server module. */
const WITH_LOGIN = { ...API, modules: { ...API.modules, "@test/app/v1/server": { Http, login } } };
const PLUGINS = {
login: { "server.ts": 'import { login } from "@test/app/v1/server";\nexport default login;' },
whoami: { "server.ts": WHOAMI },
};
/** Logs in as `name`. */
function logIn(port: number, name: unknown) {
return send(port, "POST", "/_chord/login", {
headers: { "content-type": "application/json" },
body: JSON.stringify({ name }),
});
}
describe("ephemeral login", () => {
it("signs in under a picked name as a fresh user, each login its own", async () => {
const { app, port } = await testApp({ api: WITH_LOGIN, plugins: PLUGINS });
const first = await logIn(port, " ada ");
const second = await logIn(port, "ada");
const principal = JSON.parse(first.body.toString()) as { kind: string; id: string; name: string };
expect(principal).toEqual({ kind: "user", id: expect.stringMatching(/^[A-Za-z0-9_-]{16}$/), name: "ada" });
expect(JSON.parse(second.body.toString()).id).not.toBe(principal.id);
const set = String(first.headers["set-cookie"]);
expect(set).not.toContain("Max-Age");
const whoami = await send(port, "GET", "/whoami", { headers: { cookie: set.split(";")[0]! } });
expect(JSON.parse(whoami.body.toString())).toEqual(principal);
expect((await logIn(port, " ")).status).toBe(400);
expect((await logIn(port, 7)).status).toBe(400);
expect(JSON.parse((await logIn(port, "n".repeat(100))).body.toString()).name).toHaveLength(40);
await app.close(context);
});
it("ends its sessions with the server", async () => {
const first = await testApp({ api: WITH_LOGIN, plugins: PLUGINS });
const set = String((await logIn(first.port, "ada")).headers["set-cookie"]);
await first.app.close(context);
const second = await testApp({ api: WITH_LOGIN, dataDir: first.dataDir, root: first.root });
expect((await send(second.port, "GET", "/whoami", { headers: { cookie: set.split(";")[0]! } })).status).toBe(401);
await second.app.close(context);
});
});

190
packages/chord/test/next-node/app/files.test.ts Normal file
View file

@ -0,0 +1,190 @@
// Tests of `Files` (slice 13 §4), on the server's host: small and chunked writes, ranged reads, the index, removal,
// the caps (per file, in total, per call), and uploads left by a previous run dropped.
import { readdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { Files } from "../../../src/next-node/app/index.ts";
import { rejection } from "../../next/support.ts";
import { testApp } from "./support.ts";
const encoder = new TextEncoder();
const decoder = new TextDecoder();
describe("writing, reading and removing", () => {
it("puts a small file, lists it, reads ranges of it, removes it", async () => {
const { app, dataDir } = await testApp();
const files = await app.host.get(Files, {}, context);
const id = await files.put({ name: "a.txt", mime: "text/plain", bytes: encoder.encode("hello files") }, context);
expect(files.list.value).toEqual([
{ id, name: "a.txt", mime: "text/plain", size: 11, created: expect.any(Number) },
]);
expect(decoder.decode(await files.read({ id, offset: 6, length: 100 }, context))).toBe("files");
expect(decoder.decode(await files.read({ id, offset: 0, length: 5 }, context))).toBe("hello");
expect((await files.read({ id, offset: 50, length: 5 }, context)).length).toBe(0);
expect(await readdir(join(dataDir, "server", "files"))).toContain(id);
await files.remove(id, context);
await files.remove(id, context);
expect(files.list.value).toEqual([]);
expect((await readdir(join(dataDir, "server", "files"))).filter((name) => name !== ".uploads")).toEqual([]);
expect(await rejection(files.read({ id, offset: 0, length: 1 }, context))).toMatchObject({ code: "invalid" });
files.release();
await app.close(context);
});
it("writes a large file in chunks: begin, append at the stored offset, finish", async () => {
const { app, errors } = await testApp();
const files = await app.host.get(Files, {}, context);
const upload = await files.begin({ name: "big.bin", mime: "application/octet-stream" }, context);
const chunk = new Uint8Array(1024 * 1024).fill(7);
for (let index = 0; index < 4; index++)
await files.append({ upload, offset: index * chunk.length, bytes: chunk }, context);
// A writer after a drop asks where to resume; an append elsewhere is refused with the stored offset.
const offset = await files.uploaded(upload, context);
expect(offset).toBe(4 * chunk.length);
expect(await rejection(files.append({ upload, offset: 3 * chunk.length, bytes: chunk }, context))).toMatchObject({
code: "invalid",
details: { offset },
});
await files.append({ upload, offset, bytes: chunk }, context);
expect(files.list.value).toEqual([]);
const id = await files.finish(upload, context);
expect(files.list.value.map(({ size }) => size)).toEqual([5 * 1024 * 1024]);
const tail = await files.read({ id, offset: 5 * 1024 * 1024 - 2, length: 10 }, context);
expect([...tail]).toEqual([7, 7]);
expect(await rejection(files.append({ upload, offset: 0, bytes: chunk }, context))).toMatchObject({
code: "invalid",
});
expect(await rejection(files.uploaded(upload, context))).toMatchObject({ code: "invalid" });
files.release();
await app.close(context);
// The routine sweeping expired uploads ends with the object, quietly.
expect(errors).toEqual([]);
});
it("writes appends sent without waiting at their own offsets, byte for byte (review H1)", async () => {
const { app } = await testApp();
const files = await app.host.get(Files, {}, context);
const upload = await files.begin({ name: "big.bin", mime: "application/octet-stream" }, context);
const size = 32 * 1024;
const chunks = Array.from({ length: 250 }, (_, index) => new Uint8Array(size).fill(index));
await Promise.all(chunks.map((bytes, index) => files.append({ upload, offset: index * size, bytes }, context)));
expect(await files.uploaded(upload, context)).toBe(250 * size);
const id = await files.finish(upload, context);
for (let index = 0; index < chunks.length; index++) {
const piece = await files.read({ id, offset: index * size, length: size }, context);
expect(piece.every((byte) => byte === index % 256)).toBe(true);
}
files.release();
await app.close(context);
});
it("finishes an upload only once the appends sent before it landed (review H1)", async () => {
const { app } = await testApp();
const files = await app.host.get(Files, {}, context);
const upload = await files.begin({ name: "x", mime: "x" }, context);
const size = 512 * 1024;
const appends = Array.from({ length: 8 }, (_, index) =>
files.append({ upload, offset: index * size, bytes: new Uint8Array(size).fill(index + 1) }, context),
);
const id = await files.finish(upload, context);
await Promise.all(appends);
expect(files.list.value.find((file) => file.id === id)?.size).toBe(8 * size);
expect([...(await files.read({ id, offset: 8 * size - 1, length: 10 }, context))]).toEqual([8]);
files.release();
await app.close(context);
});
it("answers ids that are prototype keys as unknown files (review L1)", async () => {
const { app, errors } = await testApp();
const files = await app.host.get(Files, {}, context);
for (const id of ["constructor", "__proto__", "toString"]) {
expect(await rejection(files.read({ id, offset: 0, length: 1 }, context))).toMatchObject({ code: "invalid" });
await files.remove(id, context);
}
expect(files.list.value).toEqual([]);
expect(errors).toEqual([]);
files.release();
await app.close(context);
});
it("caps the uploads under way per object: past it begin is refused busy; an ending one frees its slot (recheck N1)", async () => {
const { app } = await testApp({ limits: { uploads: 3 } });
const files = await app.host.get(Files, {}, context);
const fds = async () => (await readdir("/proc/self/fd")).length;
const before = await fds();
const begun = [];
for (let index = 0; index < 3; index++) begun.push(await files.begin({ name: `u${index}`, mime: "x" }, context));
expect(await rejection(files.begin({ name: "u3", mime: "x" }, context))).toMatchObject({
code: "unavailable",
reason: "busy",
});
expect((await fds()) - before).toBeLessThanOrEqual(3);
await files.finish(begun[0]!, context);
expect(await files.begin({ name: "u4", mime: "x" }, context)).toEqual(expect.any(String));
files.release();
await app.close(context);
});
it("drops the uploads a previous run left", async () => {
const first = await testApp();
const files = await first.app.host.get(Files, {}, context);
await files.begin({ name: "left", mime: "text/plain" }, context);
files.release();
await first.app.close(context);
const uploads = join(first.dataDir, "server", "files", ".uploads");
const [left] = await readdir(uploads);
await writeFile(join(uploads, "stray"), "x");
const second = await testApp({ dataDir: first.dataDir, root: first.root });
const again = await second.app.host.get(Files, {}, context);
const upload = await again.begin({ name: "new", mime: "text/plain" }, context);
// The routine dropping them runs once the object is live; this run's upload stays.
const dropped = async () => !(await readdir(uploads)).some((name) => name === left || name === "stray");
while (!(await dropped())) await new Promise((resolve) => setTimeout(resolve, 10));
expect((await readdir(uploads)).length).toBe(1);
await again.append({ upload, offset: 0, bytes: encoder.encode("kept") }, context);
expect(await again.finish(upload, context)).toEqual(expect.any(String));
again.release();
await second.app.close(context);
});
});
describe("caps", () => {
it("refuses a file over the per-file cap, writes over the total, and calls over 4 MiB", async () => {
const { app } = await testApp({ files: { maxFile: 1000, maxTotal: 1500 } });
const files = await app.host.get(Files, {}, context);
const put = (size: number) => files.put({ name: "f", mime: "x", bytes: new Uint8Array(size) }, context);
expect(await rejection(put(1001))).toMatchObject({ code: "too_large" });
await put(1000);
expect(await rejection(put(501))).toMatchObject({ code: "too_large" });
// An upload under way counts against the total.
const upload = await files.begin({ name: "u", mime: "x" }, context);
await files.append({ upload, offset: 0, bytes: new Uint8Array(400) }, context);
expect(await rejection(put(101))).toMatchObject({ code: "too_large" });
await put(100);
expect(await rejection(files.append({ upload, offset: 400, bytes: new Uint8Array(1) }, context))).toMatchObject({
code: "too_large",
});
expect(files.list.value.map(({ size }) => size).sort()).toEqual([100, 1000]);
files.release();
await app.close(context);
});
it("refuses a put and a read of more than 4 MiB", async () => {
const { app } = await testApp();
const files = await app.host.get(Files, {}, context);
const limit = 4 * 1024 * 1024;
const id = await files.put({ name: "f", mime: "x", bytes: new Uint8Array(10) }, context);
expect(await rejection(files.read({ id, offset: 0, length: limit + 1 }, context))).toMatchObject({
code: "too_large",
});
expect(
await rejection(files.put({ name: "f", mime: "x", bytes: new Uint8Array(limit + 1) }, context)),
).toMatchObject({
code: "too_large",
});
files.release();
await app.close(context);
});
});

56
packages/chord/test/next-node/app/info.test.ts Normal file
View file

@ -0,0 +1,56 @@
// Tests of the start-data objects (slice 13 §1): `AppInfo` on the server (data directory, app name, newest API
// version) and `WorkerInfo` on a worker (its address and the start data `start` made for it), as plugins get them.
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { remote, str, unknown } from "../../../src/next/contracts/index.ts";
import { AppInfo } from "../../../src/next-node/app/index.ts";
import { writePlugin } from "../plugins/support.ts";
import { API, Project, tempDir, testApp } from "./support.ts";
/** What the info plugin's worker facet shows of `WorkerInfo`. */
const Shown = remote("test.shown", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Project({ id }),
variants: { main: { state: { address: unknown(), start: unknown() } } },
});
const WORKER = [
'import { defineFacet, hostKind, remote, str, unknown, WorkerInfo } from "@test/app/v1";',
'const Project = hostKind("project", { id: str() });',
'const Shown = remote("test.shown", { version: 1, params: { id: str() }, hostAddress: ({ id }) => Project({ id }), variants: { main: { state: { address: unknown(), start: unknown() } } } });',
"export default defineFacet((host) => host.serve(Shown, (_params, _principal, self) => {",
"\tconst info = self.get(WorkerInfo, {});",
'\treturn { variant: "main", members: { address: self.derive([info.address], ([v]) => v), start: self.derive([info.start], ([v]) => v) } };',
"}));",
].join("\n");
describe("start data", () => {
it("serves AppInfo on the server: data directory, name, newest API version", async () => {
const { app, dataDir } = await testApp({
api: {
...API,
versions: ["v2", "v1"],
modules: { ...API.modules, "@test/app/v2": API.modules["@test/app/v1"] },
},
});
const info = await app.host.get(AppInfo, {}, context);
expect([info.dataDir.value, info.name.value, info.version.value]).toEqual([dataDir, "chord-test", "v2"]);
info.release();
await app.close(context);
});
it("serves WorkerInfo on a worker: its address and its start data", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "info", "@test/info", { "worker.ts": WORKER });
const { app } = await testApp({ root, withWorkers: true });
const shown = await app.host.get(Shown, { id: "p7" }, context);
expect(shown.address.value).toEqual({ kind: "project", key: "project|p7", params: { id: "p7" } });
expect(shown.start.value).toEqual({ project: "project|p7" });
shown.release();
await app.close(context);
}, 60_000);
});

55
packages/chord/test/next-node/app/invite.test.ts Normal file
View file

@ -0,0 +1,55 @@
// Tests of the invite plugin (slice 13 §3): the owner makes an invite for a principal, the invitee's page posts the
// token from the fragment and gets a browser session for it, once; only the owner makes invites.
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { inviteFacet, Sessions } from "../../../src/next-node/app/index.ts";
import { Http } from "../../../src/next-node/http/index.ts";
import { send } from "../http/support.ts";
import { API, testApp, WHOAMI } from "./support.ts";
const invite = inviteFacet();
/** The tests' API with the invite plugin and a whoami route in its server module. */
const WITH_INVITE = { ...API, modules: { ...API.modules, "@test/app/v1/server": { Http, invite } } };
const PLUGINS = {
invite: { "server.ts": 'import { invite } from "@test/app/v1/server";\nexport default invite;' },
whoami: { "server.ts": WHOAMI },
};
/** A JSON post to `path` with `headers`. */
function post(port: number, path: string, body: unknown, headers: Readonly<Record<string, string>> = {}) {
return send(port, "POST", path, {
headers: { "content-type": "application/json", ...headers },
body: JSON.stringify(body),
});
}
describe("invites", () => {
it("signs the invitee in as the chosen principal once; only the owner invites", async () => {
const { app, port } = await testApp({ api: WITH_INVITE, plugins: PLUGINS });
const token = async (principal: unknown) =>
app.host.root(
async (ctx) =>
(await (await ctx.get(Sessions, {})).issue({ principal, label: "t", kind: "device" }, ctx)).token,
context,
);
const owner = { authorization: `Bearer ${await token({ kind: "owner" })}` };
const user = { authorization: `Bearer ${await token({ kind: "user", id: "u", name: "u" })}` };
const guest = { kind: "user", id: "g1", name: "Guest" };
expect((await post(port, "/_chord/invites", { principal: guest, label: "guest" })).status).toBe(401);
expect((await post(port, "/_chord/invites", { principal: guest, label: "guest" }, user)).status).toBe(403);
expect((await post(port, "/_chord/invites", { principal: "x", label: "guest" }, owner)).status).toBe(400);
const made = await post(port, "/_chord/invites", { principal: guest, label: "guest" }, owner);
const { path } = JSON.parse(made.body.toString()) as { path: string };
const [page, secret] = path.split("#") as [string, string];
expect((await send(port, "GET", page)).body.toString()).toContain("location.hash");
const used = await post(port, page, { token: secret });
expect(used.status).toBe(200);
const cookie = String(used.headers["set-cookie"]).split(";")[0]!;
expect(JSON.parse((await send(port, "GET", "/whoami", { headers: { cookie } })).body.toString())).toEqual(guest);
expect((await post(port, page, { token: secret })).status).toBe(403);
await app.close(context);
});
});

47
packages/chord/test/next-node/app/layout.test.ts Normal file
View file

@ -0,0 +1,47 @@
// Tests of the data directory's layout (slice 13 §4): the directories made (the data directory 0700), the default
// data directory, and worker directories: one directory name per address key, distinct even on a case-insensitive
// file system, never leaving `workers/<kind>/`.
import { readdir, stat } from "node:fs/promises";
import { homedir } from "node:os";
import { basename, dirname, join } from "node:path";
import { describe, expect, it } from "vitest";
import { hostKind, str } from "../../../src/next/contracts/index.ts";
import { createLayout, defaultDataDir, workerDirectory } from "../../../src/next-node/app/layout.ts";
import { tempDir } from "./support.ts";
const Doc = hostKind("doc", { id: str() });
describe("the data directory", () => {
it("makes the layout's directories, the data directory with mode 0700", async () => {
const dataDir = join(await tempDir(), "data");
const layout = await createLayout(dataDir);
expect((await readdir(dataDir)).sort()).toEqual(["logs", "plugins", "server", "workers"]);
expect((await stat(dataDir)).mode & 0o777).toBe(0o700);
expect([layout.serverStore, layout.ownerToken, layout.cookieSecret]).toEqual([
join(dataDir, "server", "store.sqlite"),
join(dataDir, "owner-token"),
join(dataDir, "server", "cookie-secret"),
]);
expect(defaultDataDir("acme")).toBe(join(homedir(), ".acme"));
});
});
describe("worker directories", () => {
it("encodes an address key as one directory name under its kind", () => {
const dir = workerDirectory("/data", Doc({ id: "../../etc/x" }));
expect(dirname(dir)).toBe("/data/workers/doc");
expect(basename(dir)).toBe("doc%7C%2E%2E%2F%2E%2E%2Fetc%2Fx");
expect(basename(workerDirectory("/data", Doc({ id: "." })))).toBe("doc%7C%2E");
});
it("keeps keys differing only in case apart, lowercased as a case-insensitive file system sees them", () => {
const names = ["a", "A", "%41", "%2541"].map((id) => basename(workerDirectory("/data", Doc({ id }))));
expect(new Set(names.map((name) => name.toLowerCase())).size).toBe(4);
});
it("names a long key by its hash", () => {
const name = basename(workerDirectory("/data", Doc({ id: "x".repeat(200) })));
expect(name).toMatch(/^sha256-[0-9a-f]{64}$/);
});
});

112
packages/chord/test/next-node/app/owner.test.ts Normal file
View file

@ -0,0 +1,112 @@
// Tests of the owner's sign-in (slice 13 §2): the owner link (single use, expiry, replaced at each start, the token
// only in the fragment), and the owner token file (mode 0600, kept while its session lives, replaced once revoked).
import { readFile, stat, writeFile } from "node:fs/promises";
import { afterEach, describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { Sessions } from "../../../src/next-node/app/index.ts";
import { OwnerLink } from "../../../src/next-node/app/owner.ts";
import { send } from "../http/support.ts";
import { testApp, WHOAMI } from "./support.ts";
/** Posts `token` to the owner route as the link's page does. */
function postToken(port: number, token: string) {
return send(port, "POST", "/_chord/owner", {
headers: { "content-type": "application/json" },
body: JSON.stringify({ token }),
});
}
afterEach(() => {
vi.useRealTimers();
});
describe("the owner link", () => {
it("serves a page reading the fragment, signs the owner in once, and refuses the token a second time", async () => {
const { app, port } = await testApp({ plugins: { whoami: { "server.ts": WHOAMI } } });
expect((await send(port, "GET", "/whoami")).status).toBe(401);
const link = new URL(app.ownerLink());
expect(link.pathname).toBe("/_chord/owner");
const page = await send(port, "GET", link.pathname);
expect(page.status).toBe(200);
expect(page.body.toString()).toContain("location.hash");
const token = link.hash.slice(1);
const signed = await postToken(port, token);
expect(signed.status).toBe(200);
const cookie = String(signed.headers["set-cookie"]).split(";")[0]!;
const whoami = await send(port, "GET", "/whoami", { headers: { cookie } });
expect(JSON.parse(whoami.body.toString())).toEqual({ kind: "owner" });
expect((await postToken(port, token)).status).toBe(403);
const listed = await app.host.root(async (ctx) => (await ctx.get(Sessions, {})).list.value, context);
expect(listed.filter(({ kind }) => kind === "browser").map(({ label }) => label)).toEqual(["owner browser"]);
await app.close(context);
});
it("refuses an expired token, a wrong one, and the previous start's", () => {
vi.useFakeTimers({ toFake: ["Date"] });
const link = new OwnerLink();
const expiring = link.renew();
vi.advanceTimersByTime(10 * 60_000);
expect(link.consume(expiring)).toBe(false);
const previous = link.renew();
const current = link.renew();
expect(link.consume(previous)).toBe(false);
expect(link.consume("x".repeat(43))).toBe(false);
vi.advanceTimersByTime(10 * 60_000 - 1);
expect(link.consume(current)).toBe(true);
expect(link.consume(current)).toBe(false);
});
it("gives a new link at each start; the old one no longer signs in", async () => {
const first = await testApp();
const old = new URL(first.app.ownerLink()).hash.slice(1);
await first.app.close(context);
const second = await testApp({ dataDir: first.dataDir, root: first.root });
expect(new URL(second.app.ownerLink()).hash.slice(1)).not.toBe(old);
expect((await postToken(second.port, old)).status).toBe(403);
await second.app.close(context);
});
});
describe("the owner token file", () => {
it("is written at first start with mode 0600, signs in as the owner, and is kept across starts", async () => {
const first = await testApp({ plugins: { whoami: { "server.ts": WHOAMI } } });
const file = first.app.ownerTokenFile;
expect((await stat(file)).mode & 0o777).toBe(0o600);
const token = (await readFile(file, "utf8")).trim();
const whoami = await send(first.port, "GET", "/whoami", { headers: { authorization: `Bearer ${token}` } });
expect(JSON.parse(whoami.body.toString())).toEqual({ kind: "owner" });
await first.app.close(context);
const second = await testApp({ dataDir: first.dataDir, root: first.root });
expect((await readFile(file, "utf8")).trim()).toBe(token);
await second.app.close(context);
});
it("is replaced at the next start when its secret no longer matches its session (review L6)", async () => {
const first = await testApp();
const token = (await readFile(first.app.ownerTokenFile, "utf8")).trim();
await first.app.close(context);
const damaged = `${token.slice(0, token.indexOf("."))}.${"x".repeat(43)}`;
await writeFile(first.app.ownerTokenFile, `${damaged}\n`, { mode: 0o600 });
const second = await testApp({ dataDir: first.dataDir, root: first.root });
const replaced = (await readFile(second.app.ownerTokenFile, "utf8")).trim();
expect([replaced === damaged, replaced === token]).toEqual([false, false]);
await second.app.close(context);
});
it("is replaced at the next start once its session was revoked", async () => {
const first = await testApp();
const token = (await readFile(first.app.ownerTokenFile, "utf8")).trim();
await first.app.host.root(async (ctx) => {
const sessions = await ctx.get(Sessions, {});
const id = token.slice(0, token.indexOf("."));
await sessions.revoke(id, ctx);
}, context);
await first.app.close(context);
const second = await testApp({ dataDir: first.dataDir, root: first.root });
const replaced = (await readFile(second.app.ownerTokenFile, "utf8")).trim();
expect(replaced).not.toBe(token);
expect((await stat(second.app.ownerTokenFile)).mode & 0o777).toBe(0o600);
await second.app.close(context);
});
});

148
packages/chord/test/next-node/app/pairing.test.ts Normal file
View file

@ -0,0 +1,148 @@
// Tests of device pairing (slice 13 §2): a client asks for a code, the owner approves it in a signed-in browser, the
// client's poll gets a device token once; codes are unambiguous, owner-only to approve, single use, and expire.
import { afterEach, describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { Sessions } from "../../../src/next-node/app/index.ts";
import { Pairings } from "../../../src/next-node/app/pairing.ts";
import { send } from "../http/support.ts";
import { testApp } from "./support.ts";
/** A JSON post to `path`. */
function post(port: number, path: string, body: unknown, headers: Readonly<Record<string, string>> = {}) {
return send(port, "POST", path, {
headers: { "content-type": "application/json", ...headers },
body: JSON.stringify(body),
});
}
/** The JSON body of an answer. */
function json(answer: { readonly body: Buffer }): unknown {
return JSON.parse(answer.body.toString());
}
/** An anonymous client at `address`, as a pairing's starter. */
function anonymous(address: string) {
return { kind: "anonymous", address } as const;
}
afterEach(() => {
vi.useRealTimers();
});
describe("pairing over HTTP", () => {
it("pairs a client: code, owner approval, one poll with the token, then nothing", async () => {
const { app, port } = await testApp();
const owner = {
authorization: `Bearer ${await app.host.root(async (ctx) => {
const sessions = await ctx.get(Sessions, {});
return (await sessions.issue({ principal: { kind: "owner" }, label: "owner", kind: "device" }, ctx)).token;
}, context)}`,
};
const begun = (await post(port, "/_chord/pair", { label: "laptop" })) as { status: number; body: Buffer };
const { code, poll } = json(begun) as { code: string; poll: string };
expect(code).toMatch(/^[A-HJKMNP-Z2-9]{4}-[A-HJKMNP-Z2-9]{4}$/);
expect(json(await post(port, "/_chord/pair/poll", { poll }))).toEqual({ status: "waiting" });
// Anonymous and non-owner principals cannot see or approve it.
expect((await send(port, "GET", `/_chord/pair/${code}`)).status).toBe(401);
const user = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue(
{ principal: { kind: "user", id: "u", name: "u" }, label: "u", kind: "device" },
ctx,
),
context,
);
expect(
(await send(port, "POST", `/_chord/pair/${code}`, { headers: { authorization: `Bearer ${user.token}` } }))
.status,
).toBe(403);
const page = await send(port, "GET", `/_chord/pair/${code.toLowerCase()}`, { headers: owner });
expect(page.body.toString()).toContain("laptop");
expect((await send(port, "POST", `/_chord/pair/${code}`, { headers: owner })).status).toBe(200);
// Approved once: the code is no longer waiting.
expect((await send(port, "POST", `/_chord/pair/${code}`, { headers: owner })).status).toBe(404);
const approved = json(await post(port, "/_chord/pair/poll", { poll })) as { status: string; token: string };
expect(approved.status).toBe("approved");
expect((await post(port, "/_chord/pair/poll", { poll })).status).toBe(404);
const listed = await app.host.root(async (ctx) => (await ctx.get(Sessions, {})).list.value, context);
const id = approved.token.slice(0, approved.token.indexOf("."));
expect(listed.find((session) => session.id === id)).toMatchObject({
label: "laptop",
kind: "device",
principal: { kind: "owner" },
});
await app.close(context);
});
it("refuses an unknown poll secret", async () => {
const { app, port } = await testApp();
expect((await post(port, "/_chord/pair/poll", { poll: "nope" })).status).toBe(404);
expect((await post(port, "/_chord/pair/poll", {})).status).toBe(404);
await app.close(context);
});
});
describe("limits", () => {
it("refuses pairings past the limit per client address and in all with 429", async () => {
const { app, port } = await testApp({ limits: { pendingPairings: 5, pairingsPerAddress: 3 } });
const statuses = [];
for (let index = 0; index < 4; index++) statuses.push((await post(port, "/_chord/pair", {})).status);
expect(statuses).toEqual([200, 200, 200, 429]);
await app.close(context);
const pairings = new Pairings({ maxPending: 5, perAddress: 3 });
const begun = ["a", "a", "b", "b", "c", "c", "d"].map(
(address) => pairings.begin("x", anonymous(address)) !== undefined,
);
expect(begun).toEqual([true, true, true, true, true, false, false]);
});
it("frees a client address's place once its pairing ends", () => {
const pairings = new Pairings({ perAddress: 1 });
const first = pairings.begin("x", anonymous("a"))!;
expect(pairings.begin("x", anonymous("a"))).toBeUndefined();
const pairing = pairings.waiting(first.code)!;
pairing.token = "t";
expect(pairings.poll(first.poll)).toEqual({ kind: "approved", token: "t" });
expect(pairings.begin("x", anonymous("a"))).toBeDefined();
});
it("lets the signed-in owner pair past the caps, without counting it (recheck N2)", async () => {
const { app, port } = await testApp({ limits: { pendingPairings: 1, pairingsPerAddress: 1 } });
const { token } = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue({ principal: { kind: "owner" }, label: "o", kind: "device" }, ctx),
context,
);
const owner = { authorization: `Bearer ${token}` };
expect((await post(port, "/_chord/pair", {})).status).toBe(200);
expect((await post(port, "/_chord/pair", {})).status).toBe(429);
const statuses = [];
for (let index = 0; index < 3; index++) statuses.push((await post(port, "/_chord/pair", {}, owner)).status);
expect(statuses).toEqual([200, 200, 200]);
await app.close(context);
const pairings = new Pairings({ maxPending: 1 });
expect([pairings.begin("x", { kind: "owner" }), pairings.begin("x", anonymous("a"))].every(Boolean)).toBe(true);
});
});
describe("pairing codes", () => {
it("expire after 10 minutes, for the owner and the poll alike", () => {
vi.useFakeTimers({ toFake: ["Date"] });
const pairings = new Pairings();
const { code, poll } = pairings.begin("device", anonymous("1.2.3.4"))!;
vi.advanceTimersByTime(10 * 60_000 - 1);
expect(pairings.waiting(code)).toBeDefined();
vi.advanceTimersByTime(1);
expect(pairings.waiting(code)).toBeUndefined();
expect(pairings.poll(poll)).toBeUndefined();
});
it("are distinct", () => {
const pairings = new Pairings({ maxPending: 200 });
const codes = new Set(
Array.from({ length: 200 }, (_, index) => pairings.begin("d", anonymous(String(index)))!.code),
);
expect(codes.size).toBe(200);
});
});

65
packages/chord/test/next-node/app/secrets.test.ts Normal file
View file

@ -0,0 +1,65 @@
// Tests of the kit's secrets (slice 13 §2): random tokens, hashes compared in constant time, and secret files written
// with mode 0600 in one step (a create never overwrites, a replace swaps whole files, no temporary file left).
import { readdir, readFile, stat } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import {
hashToken,
matchesHash,
newToken,
readSecret,
secretFile,
writeSecret,
} from "../../../src/next-node/app/secrets.ts";
import { tempDir } from "./support.ts";
describe("tokens and hashes", () => {
it("makes distinct base64url tokens of the asked size and matches only their own hash", () => {
const token = newToken();
expect(token).toMatch(/^[A-Za-z0-9_-]{43}$/);
expect(newToken(12)).toMatch(/^[A-Za-z0-9_-]{16}$/);
expect(new Set(Array.from({ length: 100 }, () => newToken())).size).toBe(100);
const hash = hashToken(token);
expect(hash).toMatch(/^[0-9a-f]{64}$/);
expect(matchesHash(token, hash)).toBe(true);
expect(matchesHash(`${token}x`, hash)).toBe(false);
expect(matchesHash(token, "short")).toBe(false);
});
});
describe("secret files", () => {
it("creates a file with mode 0600, never overwrites it on create, replaces it whole, leaves no temporary file", async () => {
const dir = await tempDir();
const path = join(dir, "secret");
expect(await readSecret(path)).toBeUndefined();
expect(await writeSecret(path, "one", "create")).toBe("written");
expect((await stat(path)).mode & 0o777).toBe(0o600);
expect(await writeSecret(path, "two", "create")).toBe("exists");
expect(await readSecret(path)).toBe("one");
expect(await writeSecret(path, "three", "replace")).toBe("written");
expect(await readFile(path, "utf8")).toBe("three\n");
expect((await stat(path)).mode & 0o777).toBe(0o600);
expect(await readdir(dir)).toEqual(["secret"]);
});
it("keeps a secret file's first value: a fresh token at first, then that one", async () => {
const dir = await tempDir();
const path = join(dir, "cookie-secret");
const first = await secretFile(path);
expect(first).toMatch(/^[A-Za-z0-9_-]{43}$/);
expect(await secretFile(path)).toBe(first);
expect((await stat(path)).mode & 0o777).toBe(0o600);
});
it("keeps the mode 0600 under a permissive umask", async () => {
const dir = await tempDir();
const previous = process.umask(0);
try {
await writeSecret(join(dir, "s"), "x", "create");
} finally {
process.umask(previous);
}
expect((await stat(join(dir, "s"))).mode & 0o777).toBe(0o600);
});
});

227
packages/chord/test/next-node/app/sessions.test.ts Normal file
View file

@ -0,0 +1,227 @@
// Tests of sessions (slice 13 §2): issue, revoke and list through `Sessions`; tokens kept only as hashes; the
// authenticator taking a bearer token or the signed session cookie; ephemeral sessions dropped when the server starts.
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { Sessions } from "../../../src/next-node/app/index.ts";
import { hashToken } from "../../../src/next-node/app/secrets.ts";
import { send } from "../http/support.ts";
import { testApp, WHOAMI } from "./support.ts";
/** The app with the whoami plugin (its API's server module gives `Http`). */
async function withWhoami() {
return testApp({ plugins: { whoami: { "server.ts": WHOAMI } } });
}
describe("issuing, listing and revoking", () => {
it("issues a session whose token authenticates as a bearer and as the cookie, lists it without its hash, revokes it", async () => {
const { app, port } = await withWhoami();
const issued = await app.host.root(async (ctx) => {
const sessions = await ctx.get(Sessions, {});
return sessions.issue(
{ principal: { kind: "user", id: "u1", name: "ada" }, label: "ada's laptop", kind: "device" },
ctx,
);
}, context);
expect(issued.token).toMatch(/^[A-Za-z0-9_-]{16}\.[A-Za-z0-9_-]{43}$/);
const bearer = await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${issued.token}` } });
expect(JSON.parse(bearer.body.toString())).toEqual({ kind: "user", id: "u1", name: "ada" });
const cookie = issued.cookie.split(";")[0]!;
expect(issued.cookie).toMatch(/^chord-session=[^;]+; HttpOnly; Path=\/; SameSite=Strict; Max-Age=34560000$/);
const browser = await send(port, "GET", "/whoami", { headers: { cookie } });
expect(JSON.parse(browser.body.toString())).toMatchObject({ id: "u1" });
const listed = await app.host.root(async (ctx) => (await ctx.get(Sessions, {})).list.value, context);
expect(listed.find(({ id }) => id === issued.id)).toEqual({
id: issued.id,
label: "ada's laptop",
kind: "device",
principal: { kind: "user", id: "u1", name: "ada" },
created: expect.any(Number),
lastUsed: expect.any(Number),
});
await app.host.root(async (ctx) => (await ctx.get(Sessions, {})).revoke(issued.id, ctx), context);
expect(
(await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${issued.token}` } })).status,
).toBe(401);
await app.close(context);
});
it("refuses a token with a wrong secret, an unknown id and a forged cookie", async () => {
const { app, port } = await withWhoami();
const issued = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue({ principal: { kind: "owner" }, label: "x", kind: "device" }, ctx),
context,
);
const [id] = issued.token.split(".");
for (const token of [`${id}.wrong`, `nope.${issued.token.split(".")[1]}`, "garbage"]) {
expect((await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${token}` } })).status).toBe(
401,
);
}
expect(
(await send(port, "GET", "/whoami", { headers: { cookie: `chord-session=${issued.token}.x.y` } })).status,
).toBe(401);
await app.close(context);
});
it("keeps only the token's hash in the store", async () => {
const { app, dataDir } = await withWhoami();
const issued = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue({ principal: { kind: "owner" }, label: "x", kind: "device" }, ctx),
context,
);
await app.close(context);
const secret = issued.token.split(".")[1]!;
const stored = Buffer.concat(
await Promise.all(
["store.sqlite", "store.sqlite-wal"].map((name) =>
readFile(join(dataDir, "server", name)).catch(() => Buffer.alloc(0)),
),
),
).toString("latin1");
expect(stored.includes(secret)).toBe(false);
expect(stored.includes(hashToken(secret))).toBe(true);
});
});
describe("ephemeral sessions", () => {
it("authenticate in memory, are not listed, and end with the server; persisted ones stay", async () => {
const first = await withWhoami();
const issued = await first.app.host.root(async (ctx) => {
const sessions = await ctx.get(Sessions, {});
const principal = { kind: "user", id: "e", name: "e" };
const ephemeral = await sessions.issue({ principal, label: "e", kind: "ephemeral" }, ctx);
const device = await sessions.issue({ principal: { kind: "owner" }, label: "d", kind: "device" }, ctx);
return { ephemeral, device, listed: sessions.list.value.map(({ id }) => id) };
}, context);
expect(issued.listed).toEqual(expect.arrayContaining([issued.device.id]));
expect(issued.listed).not.toContain(issued.ephemeral.id);
const bearer = (token: string) => ({ headers: { authorization: `Bearer ${token}` } });
expect((await send(first.port, "GET", "/whoami", bearer(issued.ephemeral.token))).status).toBe(200);
await first.app.close(context);
const second = await testApp({ dataDir: first.dataDir, root: first.root });
expect((await send(second.port, "GET", "/whoami", bearer(issued.ephemeral.token))).status).toBe(401);
expect((await send(second.port, "GET", "/whoami", bearer(issued.device.token))).status).toBe(200);
await second.app.close(context);
});
it("live at most `limits.ephemeralSessions`: past it the oldest ends; an issue costs the same whatever came before", async () => {
const { app, port } = await testApp({
limits: { ephemeralSessions: 3 },
plugins: { whoami: { "server.ts": WHOAMI } },
});
const sessions = await app.host.get(Sessions, {}, context);
const issue = (index: number) =>
sessions.issue(
{ principal: { kind: "user", id: `u${index}`, name: "u" }, label: "u", kind: "ephemeral" },
context,
);
const tokens = [];
for (let index = 0; index < 4; index++) tokens.push((await issue(index)).token);
const status = async (token: string) =>
(await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${token}` } })).status;
expect(await Promise.all(tokens.map(status))).toEqual([401, 200, 200, 200]);
// Many more: the cap holds them, and nothing is persisted (no document grows).
for (let index = 4; index < 2000; index++) await issue(index);
expect(sessions.list.value.filter(({ kind }) => kind !== "browser" && kind !== "device")).toEqual([]);
sessions.release();
await app.close(context);
});
});
describe("rolling cookies", () => {
it("renews a browser's session cookie once it is older than a day, with a fresh signed expiry (recheck N3)", async () => {
const { app, port } = await withWhoami();
const issued = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue({ principal: { kind: "owner" }, label: "b", kind: "browser" }, ctx),
context,
);
const cookie = issued.cookie.split(";")[0]!;
const whoami = async (header: string) => send(port, "GET", "/whoami", { headers: { cookie: header } });
// Within the day: nothing set.
expect((await whoami(cookie)).headers["set-cookie"]).toBeUndefined();
const start = Date.now();
vi.useFakeTimers({ toFake: ["Date"], now: start + 2 * 24 * 60 * 60_000 });
try {
const renewed = await whoami(cookie);
expect(renewed.status).toBe(200);
const set = String(renewed.headers["set-cookie"]);
expect(set).toMatch(/^chord-session=[^;]+; HttpOnly; Path=\/; SameSite=Strict; Max-Age=34560000$/);
const expires = Number(set.split(";")[0]!.split(".").at(-2)) * 1000;
expect(expires).toBeGreaterThan(start + 401 * 24 * 60 * 60_000);
// 399 days later the old cookie has expired, the renewed one still signs in.
vi.setSystemTime(start + 401 * 24 * 60 * 60_000);
expect((await whoami(cookie)).status).toBe(401);
expect((await whoami(set.split(";")[0]!)).status).toBe(200);
} finally {
vi.useRealTimers();
}
await app.close(context);
});
it("does not renew an ephemeral session's cookie, nor a revoked session's", async () => {
const { app, port } = await withWhoami();
const sessions = await app.host.get(Sessions, {}, context);
const ephemeral = await sessions.issue(
{ principal: { kind: "user", id: "e", name: "e" }, label: "e", kind: "ephemeral" },
context,
);
const browser = await sessions.issue({ principal: { kind: "owner" }, label: "b", kind: "browser" }, context);
await sessions.revoke(browser.id, context);
vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 2 * 24 * 60 * 60_000 });
try {
const short = await send(port, "GET", "/whoami", { headers: { cookie: ephemeral.cookie.split(";")[0]! } });
expect([short.status, short.headers["set-cookie"]]).toEqual([200, undefined]);
const revoked = await send(port, "GET", "/whoami", { headers: { cookie: browser.cookie.split(";")[0]! } });
expect([revoked.status, revoked.headers["set-cookie"]]).toEqual([401, undefined]);
} finally {
vi.useRealTimers();
}
sessions.release();
await app.close(context);
});
});
describe("expiry and malformed tokens", () => {
it("ends a persisted session unused for 400 days, at its next use", async () => {
const { app, port } = await withWhoami();
const { token } = await app.host.root(
async (ctx) =>
(await ctx.get(Sessions, {})).issue({ principal: { kind: "owner" }, label: "d", kind: "device" }, ctx),
context,
);
const now = Date.now();
vi.useFakeTimers({ toFake: ["Date"], now: now + 400 * 24 * 60 * 60_000 + 1 });
try {
expect((await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${token}` } })).status).toBe(
401,
);
} finally {
vi.useRealTimers();
}
const sessions = await app.host.get(Sessions, {}, context);
while (sessions.list.value.some(({ label }) => label === "d"))
await new Promise((resolve) => setTimeout(resolve, 10));
sessions.release();
await app.close(context);
});
it("answers a malformed token or a prototype key as an id with a plain 401, reporting nothing", async () => {
const { app, port, errors } = await withWhoami();
for (const token of ["constructor.x", "__proto__.x", "toString.x", "hasOwnProperty.x", ".x", "x", "a.b.c"]) {
expect((await send(port, "GET", "/whoami", { headers: { authorization: `Bearer ${token}` } })).status).toBe(
401,
);
}
expect((await send(port, "GET", "/whoami", { headers: { cookie: "chord-session=__proto__.x" } })).status).toBe(
401,
);
expect(errors).toEqual([]);
await app.close(context);
});
});

187
packages/chord/test/next-node/app/support.ts Normal file
View file

@ -0,0 +1,187 @@
// Test support for the app kit (slice 13): an app started in a temporary data directory with a temporary plugins root,
// the tests' API (Chord's surface as `@test/app/v1`), a worker kind whose entry is the fixture worker, and requests to
// it as a browser or a command-line client would make them.
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { afterEach } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { hostKind, int, method, remote, str, unknown } from "../../../src/next/contracts/index.ts";
import * as chordNext from "../../../src/next/index.ts";
import {
type App,
AppInfo,
type AppOptions,
Files,
Sessions,
startApp,
WorkerInfo,
} from "../../../src/next-node/app/index.ts";
import { Http } from "../../../src/next-node/http/index.ts";
import { writeFiles, writePlugin } from "../plugins/support.ts";
/** The principals of these tests. */
export type TestPrincipal =
| { readonly kind: "server" }
| { readonly kind: "owner" }
| { readonly kind: "worker"; readonly key: string }
| { readonly kind: "user"; readonly id: string; readonly name: string };
/** The tests' worker kind: a project by id. */
export const Project = hostKind("project", { id: str() });
/** The tests' API v1: Chord's surface and the kit's objects, as an app's API re-exports them. */
export const API_V1 = { ...chordNext, AppInfo, WorkerInfo, Files, Sessions };
/** The tests' API, as the server and the fixture worker have it. */
export const API = {
name: "@test/app",
versions: ["v1"],
modules: { "@test/app/v1": API_V1, "@test/app/v1/server": { Http } },
} as const;
/** The fixture worker's entry. */
export const WORKER_ENTRY = fileURLToPath(new URL("../fixtures/app/worker.ts", import.meta.url));
const made: string[] = [];
afterEach(async () => {
for (const dir of made.splice(0)) await rm(dir, { recursive: true, force: true });
});
/** A temporary directory removed after the test. */
export async function tempDir(): Promise<string> {
const dir = await mkdtemp(join(tmpdir(), "chord-app-"));
made.push(dir);
return dir;
}
/** A started test app and what it reported; `plugins` (directory name to files) written into its root first. */
export async function testApp(
options: Partial<AppOptions<TestPrincipal>> & {
readonly plugins?: Readonly<Record<string, Readonly<Record<string, string>>>>;
readonly withWorkers?: boolean;
} = {},
): Promise<{ app: App<TestPrincipal>; errors: unknown[]; dataDir: string; root: string; port: number }> {
const base = await tempDir();
const dataDir = options.dataDir ?? join(base, "data");
const root = options.root ?? join(base, "plugins");
await writeFiles(root, { ".keep": "" });
for (const [dir, files] of Object.entries(options.plugins ?? {}))
await writePlugin(root, dir, `@test/${dir}`, files);
const errors: unknown[] = [];
const app = await startApp<TestPrincipal>(
{
name: "chord-test",
dataDir,
root,
listen: { host: "127.0.0.1", port: 0 },
api: API,
kinds: { server: { platform: "node" }, worker: { platform: "node" } },
principal: { server: { kind: "server" }, worker: (address) => ({ kind: "worker", key: address.key }) },
workers:
options.withWorkers === true
? [
{
kind: Project,
entry: WORKER_ENTRY,
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
start: (address) => ({ project: address.key }),
authorize: () => true,
idleMs: () => 60_000,
},
]
: [],
onError: (error) => errors.push(error),
...options,
},
context,
);
const port = Number(new URL(app.ownerLink()).port);
return { app, errors, dataDir, root, port };
}
/** A plugin's server facet serving `Echo` (a `local` type) through the tests' API. */
export const ECHO_SERVER = [
'import { constant, defineFacet, local, str } from "@test/app/v1";',
'const Echo = local("test.echo", { version: 1, params: {}, members: { text: str() } });',
'export default defineFacet((host) => host.serve(Echo, () => ({ text: constant("echo") })));',
].join("\n");
/** The notes plugin's remote contract (`NOTES_PLUGIN`), declared again for the tests (contracts are data). */
export const Notes = remote("test.notes", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Project({ id }),
variants: {
main: {
state: { text: str(), start: unknown() },
methods: {
write: method(str()),
visit: method(str(), int()),
attach: method(str(), str()),
file: method(str(), str()),
},
},
},
});
/**
* The notes plugin of the export tests (`withWorkers`): on each project's worker a store object with a persisted
* document, a table and the worker's files, reached from the server through the remote `Notes`.
*/
export const NOTES_PLUGIN: Readonly<Record<string, string>> = {
"contract.ts": [
'import { hostKind, int, method, remote, str, unknown } from "@test/app/v1";',
'export const Project = hostKind("project", { id: str() });',
'export const Notes = remote("test.notes", {',
"\tversion: 1,",
"\tparams: { id: str() },",
"\thostAddress: ({ id }) => Project({ id }),",
"\tvariants: { main: { state: { text: str(), start: unknown() }, methods: {",
"\t\twrite: method(str()), visit: method(str(), int()), attach: method(str(), str()), file: method(str(), str()),",
"\t} } },",
"});",
].join("\n"),
"worker.ts": [
'import { defineFacet, defineState, defineTable, Files, int, local, method, obj, str, WorkerInfo } from "@test/app/v1";',
'import { Notes } from "./contract.ts";',
'const Text = defineState("test.text", { schema: obj({ text: str() }), version: 1, initial: () => ({ text: "" }), persist: true, checkpointWhen: () => false });',
'const Visits = defineTable("test.visits", { version: 1, columns: { page: str(), count: int() }, key: ["page"] });',
'const Store = local("test.store", { version: 1, params: {}, members: { text: str(), write: method(str()), visit: method(str(), int()) } });',
"export default defineFacet((host) => {",
"\thost.serve(Store, (_params, self) => {",
"\t\tconst text = self.state(Text);",
"\t\tconst visits = self.table(Visits);",
"\t\treturn {",
"\t\t\ttext: self.derive([text], ([value]) => value.text),",
"\t\t\twrite: async (next, ctx) => { await text.change((draft) => { draft.text = next; }, ctx); return null; },",
"\t\t\tvisit: async (page, ctx) => self.tx(async (tx) => { const count = ((await tx.get(visits, { page }))?.count ?? 0) + 1; tx.put(visits, { page, count }); return count; }, ctx),",
"\t\t};",
"\t});",
"\thost.serve(Notes, (_params, _principal, self) => {",
"\t\tconst store = self.get(Store, {});",
"\t\tconst info = self.get(WorkerInfo, {});",
"\t\tconst files = self.get(Files, {});",
"\t\tconst decoder = new TextDecoder();",
'\t\treturn { variant: "main", members: {',
"\t\t\ttext: self.derive([store.text], ([value]) => value),",
"\t\t\tstart: self.derive([info.start], ([value]) => value),",
"\t\t\twrite: (next, ctx) => store.write(next, ctx),",
"\t\t\tvisit: (page, ctx) => store.visit(page, ctx),",
'\t\t\tattach: (body, ctx) => files.put({ name: "note.txt", mime: "text/plain", bytes: new TextEncoder().encode(body) }, ctx),',
"\t\t\tfile: async (id, ctx) => decoder.decode(await files.read({ id, offset: 0, length: 1024 }, ctx)),",
"\t\t} };",
"\t});",
"});",
].join("\n"),
};
/** A plugin's server facet with a route (not public) answering who it runs as: `GET /whoami`. */
export const WHOAMI = [
'import { defineFacet } from "@test/app/v1";',
'import { Http } from "@test/app/v1/server";',
'export default defineFacet((_host, self) => self.contribute(Http, { routes: { "GET /whoami": async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }) } }));',
].join("\n");

107
packages/chord/test/next-node/app/tar.test.ts Normal file
View file

@ -0,0 +1,107 @@
// Tests of the kit's tar writer and reader (slice 13 §4): a round trip (long paths, empty and multi-block files, data
// left unread), and the reader refusing what an address archive may not hold: paths leaving the target (absolute,
// `..`), symbolic and hard links, devices, a bad checksum, a truncated archive.
import { describe, expect, it } from "vitest";
import { readTar, type TarEntry, writeTar } from "../../../src/next-node/app/tar.ts";
/** The bytes `writeTar` makes of `entries`. */
async function archiveOf(entries: readonly TarEntry[]): Promise<Uint8Array> {
const chunks: Uint8Array[] = [];
for await (const chunk of writeTar(entries)) chunks.push(chunk);
return Buffer.concat(chunks);
}
/** An entry of `text`. */
function entry(path: string, text: string): TarEntry {
const bytes = new TextEncoder().encode(text);
return {
path,
size: bytes.length,
data: (async function* () {
yield bytes;
})(),
};
}
/** Every entry of `archive`: path and text (files) or `dir`; `skip` names files whose data is left unread. */
async function entriesOf(archive: Uint8Array, skip: readonly string[] = []): Promise<Array<[string, string]>> {
const read: Array<[string, string]> = [];
for await (const each of readTar(
(async function* () {
yield archive;
})(),
)) {
if (each.kind === "directory") read.push([each.path, "dir"]);
else if (skip.includes(each.path)) read.push([each.path, "skipped"]);
else {
const chunks: Uint8Array[] = [];
for await (const chunk of each.data) chunks.push(chunk);
read.push([each.path, Buffer.concat(chunks).toString()]);
}
}
return read;
}
/** A single-entry archive with header field `offset` set to `value`, its checksum recomputed. */
async function patched(offset: number, value: string): Promise<Uint8Array> {
const archive = await archiveOf([entry("files/a", "x")]);
const block = archive.subarray(0, 512);
block.fill(0, offset, offset + (offset === 0 ? 100 : 1));
block.set(new TextEncoder().encode(value), offset);
block.fill(0x20, 148, 156);
const sum = block.reduce((total, byte) => total + byte, 0);
block.set(new TextEncoder().encode(`${sum.toString(8).padStart(6, "0")}\u0000 `), 148);
return archive;
}
describe("round trip", () => {
it("reads back what it wrote: long paths, empty and multi-block files, data left unread", async () => {
const long = `files/${"d".repeat(90)}/${"f".repeat(90)}`;
const big = "y".repeat(1500);
const archive = await archiveOf([
entry("manifest.json", "{}"),
entry(long, "long"),
entry("files/empty", ""),
entry("files/big", big),
entry("files/last", "z"),
]);
expect(archive.length % 512).toBe(0);
expect(await entriesOf(archive, ["files/big"])).toEqual([
["manifest.json", "{}"],
[long, "long"],
["files/empty", ""],
["files/big", "skipped"],
["files/last", "z"],
]);
});
it("refuses to write a path it would refuse to read", async () => {
for (const path of ["/abs", "a/../b", "a//b", "./a", "a\\b"]) {
await expect(archiveOf([entry(path, "x")])).rejects.toThrow("is refused");
}
});
});
describe("refusals when reading", () => {
it("refuses absolute paths and `..`", async () => {
await expect(entriesOf(await patched(0, "/etc/passwd"))).rejects.toThrow("is refused");
await expect(entriesOf(await patched(0, "files/../../evil"))).rejects.toThrow("is refused");
await expect(entriesOf(await patched(0, "../evil"))).rejects.toThrow("is refused");
});
it("refuses symbolic links, hard links, devices and FIFOs", async () => {
for (const flag of ["1", "2", "3", "4", "6"]) {
await expect(entriesOf(await patched(156, flag))).rejects.toThrow("is refused (files only)");
}
});
it("refuses a header whose checksum does not hold, and a truncated archive", async () => {
const tampered = await archiveOf([entry("files/a", "x")]);
tampered[6] = "b".charCodeAt(0);
await expect(entriesOf(tampered)).rejects.toThrow("checksum");
const archive = await archiveOf([entry("files/a", "x".repeat(600))]);
await expect(entriesOf(archive.subarray(0, 700))).rejects.toThrow("ends");
await expect(entriesOf(archive.subarray(0, 100))).rejects.toThrow("ends inside a header");
});
});

28
packages/chord/test/next-node/app/uploads.test.ts Normal file
View file

@ -0,0 +1,28 @@
// Tests of chunked uploads on their own (slice 13 §4): the cap of uploads under way, a slot freed by expiry or by the
// object's end, the files of ended uploads removed. Their use through `Files` (pipelined appends, finish waiting for
// writes, caps) is in `files.test.ts`.
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { Uploads } from "../../../src/next-node/app/uploads.ts";
import { tempDir } from "./support.ts";
describe("uploads under way", () => {
it("refuses a begin past the cap busy; an expired upload frees its slot and its file (recheck N1)", async () => {
const run = join(await tempDir(), "run");
const uploads = new Uploads(run, 1);
const first = await uploads.begin("a", "x");
await expect(uploads.begin("b", "x")).rejects.toMatchObject({ code: "unavailable", reason: "busy" });
await uploads.expire(60_000);
expect(await uploads.uploaded(first)).toBe(0);
await uploads.expire(0);
await expect(uploads.uploaded(first)).rejects.toMatchObject({ code: "invalid" });
const second = await uploads.begin("b", "x");
expect(await readdir(run)).toEqual([second]);
await uploads.close();
expect(await readdir(run)).toEqual([]);
expect(await uploads.begin("c", "x")).toEqual(expect.any(String));
await uploads.close();
});
});

42
packages/chord/test/next-node/app/users.test.ts Normal file
View file

@ -0,0 +1,42 @@
// Tests of the users plugin (slice 13 §3): the app makes it with its profile schema, re-exports it through its API and
// adds it to its root as a one-line plugin; other plugins and the server reach `Users`; profiles persist.
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { obj, opt, str } from "../../../src/next/contracts/index.ts";
import { defineUsers } from "../../../src/next-node/app/index.ts";
import { Http } from "../../../src/next-node/http/index.ts";
import { rejection } from "../../next/support.ts";
import { API, testApp } from "./support.ts";
const users = defineUsers(obj({ name: str(), title: opt(str()) }));
/** The tests' API with the users plugin in its server module. */
const WITH_USERS = { ...API, modules: { ...API.modules, "@test/app/v1/server": { Http, users } } };
/** The app's one-line plugin adding it. */
const PLUGIN = { "server.ts": 'import { users } from "@test/app/v1/server";\nexport default users.facet;' };
describe("users", () => {
it("keeps accounts by id with the app's profile, listed, updated, removed, and kept across starts", async () => {
const first = await testApp({ api: WITH_USERS, plugins: { users: PLUGIN } });
const ref = await first.app.host.get(users.Users, {}, context);
await ref.update({ id: "u1", profile: { name: "Ada" } }, context);
await ref.update({ id: "u2", profile: { name: "Grace", title: "Admiral" } }, context);
await ref.update({ id: "u1", profile: { name: "Ada L." } }, context);
expect(await ref.get("u1", context)).toEqual({ name: "Ada L." });
expect(await ref.get("nobody", context)).toBeNull();
await ref.remove("u2", context);
expect(ref.list.value).toEqual([{ id: "u1", profile: { name: "Ada L." } }]);
expect(await rejection(ref.update({ id: "u3", profile: { title: "no name" } as never }, context))).toMatchObject({
code: "invalid",
});
ref.release();
await first.app.close(context);
const second = await testApp({ api: WITH_USERS, dataDir: first.dataDir, root: first.root });
const again = await second.app.host.get(users.Users, {}, context);
expect(again.list.value).toEqual([{ id: "u1", profile: { name: "Ada L." } }]);
again.release();
await second.app.close(context);
});
});

40
packages/chord/test/next-node/app/worker.test.ts Normal file
View file

@ -0,0 +1,40 @@
// Tests of `startWorker` (slice 13 §1, §4) through a running app: a worker's store and files live in its own directory
// and outlast its process (stopped, then started again by the next hold), and its `Files` are its own.
import { readdir, stat } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { writePlugin } from "../plugins/support.ts";
import { NOTES_PLUGIN, Notes, Project, tempDir, testApp } from "./support.ts";
describe("a worker's directory", () => {
it("keeps its store and files in workers/<kind>/<key>/ across its process's restarts", async () => {
const base = await tempDir();
const root = join(base, "plugins");
await writePlugin(root, "notes", "@test/notes", NOTES_PLUGIN);
const { app, dataDir } = await testApp({ root, withWorkers: true });
const notes = await app.host.get(Notes, { id: "p1" }, context);
await notes.write("kept", context);
const file = await notes.attach("in p1's files", context);
notes.release();
const dir = join(dataDir, "workers", "project", "project%7Cp1");
expect((await readdir(dir)).sort()).toEqual(expect.arrayContaining(["files", "store.sqlite"]));
expect(await readdir(join(dir, "files"))).toContain(file);
const mode = async (path: string) => (await stat(path)).mode & 0o777;
const paths = [dir, join(dir, ".."), join(dir, "store.sqlite"), join(dir, "files"), join(dir, "files", file)];
expect(await Promise.all(paths.map(mode))).toEqual([0o700, 0o700, 0o600, 0o700, 0o600]);
const workers = app.workers.get("project")!;
await workers.stop(Project({ id: "p1" }), context);
await workers.retry(Project({ id: "p1" }), context);
const again = await app.host.get(Notes, { id: "p1" }, context);
expect(again.text.value).toBe("kept");
expect(await again.file(file, context)).toBe("in p1's files");
// Another address has its own files: p1's id is unknown there.
const other = await app.host.get(Notes, { id: "p2" }, context);
await expect(other.file(file, context)).rejects.toMatchObject({ code: "invalid" });
again.release();
other.release();
await app.close(context);
}, 60_000);
});

13
packages/chord/test/next-node/fixtures/app/worker.ts Normal file
View file

@ -0,0 +1,13 @@
// The worker entry of the app kit's tests: `startWorker` with the tests' API (Chord's surface and the kit's objects as
// `@test/app/v1`, as the server's tests have it).
import * as chordNext from "../../../../src/next/index.ts";
import { AppInfo, Files, Sessions, startWorker, WorkerInfo } from "../../../../src/next-node/app/index.ts";
const v1 = { ...chordNext, AppInfo, WorkerInfo, Files, Sessions };
startWorker({
api: { name: "@test/app", versions: ["v1"], modules: { "@test/app/v1": v1 } },
kinds: { server: { platform: "node" }, worker: { platform: "node" }, teamworker: { platform: "node" } },
onError: (error) => console.error(error),
});

7
packages/chord/test/next-node/http/file.test.ts
View file

@ -97,8 +97,11 @@ describe("fileResponse", () => {
const server = await served({
"@app/a": contributing({
routes: {
"GET /f": async () => fileResponse(path, { ranges: true, download: "big.bin" }),
"GET /wrapped": async () => fileResponse(path),
"GET /f": {
public: true,
handler: async () => fileResponse(path, { ranges: true, download: "big.bin" }),
},
"GET /wrapped": { public: true, handler: async () => fileResponse(path) },
},
wraps: [
{

283
packages/chord/test/next-node/http/http.test.ts
View file

@ -16,7 +16,12 @@ import { context, contributing, send, served, type TestPrincipal } from "./suppo
/** A route answering who it runs as. */
const whoami = contributing({
routes: { "GET /whoami": async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }) },
routes: {
"GET /whoami": {
public: true,
handler: async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }),
},
},
});
/** A loopback request to `fetch` directly, as a mounted server would make it. */
@ -138,11 +143,17 @@ describe("Host and principals", () => {
const server = await served({
"@app/notes": contributing({
routes: {
"POST /delete": async (_request, ctx) => {
if (ctx.principal?.kind === "owner") deleted++;
return { kind: "json", value: deleted };
"POST /delete": {
public: true,
handler: async (_request, ctx) => {
if (ctx.principal?.kind === "owner") deleted++;
return { kind: "json", value: deleted };
},
},
"GET /whoami": {
public: true,
handler: async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }),
},
"GET /whoami": async (_request, ctx) => ({ kind: "json", value: ctx.principal ?? null }),
},
}),
});
@ -173,6 +184,7 @@ describe("Host and principals", () => {
"@app/pages": contributing({
routes: {
"GET /app": {
public: true,
page: true,
handler: async (_request, ctx) => {
seen.push(ctx.principal ?? null);
@ -180,6 +192,7 @@ describe("Host and principals", () => {
},
},
"POST /hook": {
public: true,
crossSite: true,
handler: async (_request, ctx) => {
seen.push(ctx.principal ?? null);
@ -205,15 +218,48 @@ describe("Host and principals", () => {
});
});
describe("deny by default (slice 13 §2)", () => {
it("answers 401 to a request nobody authenticates before a route not marked public runs; serves a public one", async () => {
let ran = 0;
const server = await served({
"@app/notes": contributing({
routes: {
"GET /private": async () => {
ran++;
return { kind: "json", value: "private" };
},
"GET /public": { public: true, handler: async () => ({ kind: "json", value: "public" }) },
"POST /hook": { public: true, crossSite: true, handler: async () => ({ kind: "json", value: "hook" }) },
},
}),
});
const nobody = await send(server.port, "GET", "/private");
expect(nobody.status).toBe(401);
expect(nobody.headers["cache-control"]).toBe("no-store");
expect(ran).toBe(0);
expect((await send(server.port, "GET", "/private", { headers: { "x-principal": "owner" } })).status).toBe(200);
expect((await send(server.port, "GET", "/public")).body.toString()).toBe('"public"');
// A cross-site route always runs anonymous: public by being one.
const hook = await send(server.port, "POST", "/hook", {
headers: { "sec-fetch-site": "cross-site", origin: "https://ci.example.com" },
});
expect(hook.body.toString()).toBe('"hook"');
await server.close();
});
});
describe("default headers", () => {
it("sends the baseline on every answer, a CSP with the response's own nonce and the contributed framers and sources", async () => {
let nonce = "";
const server = await served({
"@app/page": contributing({
routes: {
"GET /page": async (request) => {
nonce = request.nonce;
return { kind: "html", body: `<script nonce="${request.nonce}"></script>` };
"GET /page": {
public: true,
handler: async (request) => {
nonce = request.nonce;
return { kind: "html", body: `<script nonce="${request.nonce}"></script>` };
},
},
},
framers: ["https://shell.example.com"],
@ -273,16 +319,22 @@ describe("routes and wraps failing", () => {
const server = await served({
"@app/a": contributing({
routes: {
"GET /throws": async () => {
throw new Error("secret");
"GET /throws": {
public: true,
handler: async () => {
throw new Error("secret");
},
},
"GET /failure": async () => {
throw new HttpFailure(422, "no");
"GET /failure": {
public: true,
handler: async () => {
throw new HttpFailure(422, "no");
},
},
},
}),
"@app/b": contributing({
routes: { "GET /wrapped": async () => ({ kind: "json", value: 1 }) },
routes: { "GET /wrapped": { public: true, handler: async () => ({ kind: "json", value: 1 }) } },
wraps: [
{
name: "broken",
@ -310,9 +362,12 @@ describe("routes and wraps failing", () => {
const server = await served({
"@app/a": contributing({
routes: {
"GET /r": async (_request, ctx) => {
order.push(`route as ${ctx.principal?.kind ?? "anonymous"}`);
return { kind: "json", value: 1 };
"GET /r": {
public: true,
handler: async (_request, ctx) => {
order.push(`route as ${ctx.principal?.kind ?? "anonymous"}`);
return { kind: "json", value: 1 };
},
},
},
wraps: [
@ -347,16 +402,21 @@ describe("reloads and close", () => {
const gate = Promise.withResolvers<void>();
const old = contributing({
routes: {
"GET /v": async () => {
await gate.promise;
return { kind: "html", body: "old" };
"GET /v": {
public: true,
handler: async () => {
await gate.promise;
return { kind: "html", body: "old" };
},
},
},
});
const server = await served({ "@app/v": old });
const running = send(server.port, "GET", "/v");
await new Promise((resolve) => setTimeout(resolve, 50));
const next = contributing({ routes: { "GET /v": async () => ({ kind: "html", body: "new" }) } });
const next = contributing({
routes: { "GET /v": { public: true, handler: async () => ({ kind: "html", body: "new" }) } },
});
await server.host.reload(server.facets({ "@app/v": next }), context);
expect((await send(server.port, "GET", "/v")).body.toString()).toBe("new");
gate.resolve();
@ -370,13 +430,19 @@ describe("reloads and close", () => {
{
"@app/slow": contributing({
routes: {
"GET /quick": async () => {
await new Promise((resolve) => setTimeout(resolve, 30));
return { kind: "html", body: "done" };
"GET /quick": {
public: true,
handler: async () => {
await new Promise((resolve) => setTimeout(resolve, 30));
return { kind: "html", body: "done" };
},
},
"GET /stuck": async (_request, ctx: Ctx) => {
sealed = ctx.abortSignal;
return new Promise(() => {});
"GET /stuck": {
public: true,
handler: async (_request, ctx: Ctx) => {
sealed = ctx.abortSignal;
return new Promise(() => {});
},
},
},
}),
@ -405,9 +471,12 @@ describe("reloads and close", () => {
{
"@app/r": contributing({
routes: {
"GET /": async () => {
calls++;
return { kind: "json", value: 1 };
"GET /": {
public: true,
handler: async () => {
calls++;
return { kind: "json", value: 1 };
},
},
},
}),
@ -446,7 +515,9 @@ describe("cancellation", () => {
let later = 0;
const server = await served(
{
"@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }),
"@app/r": contributing({
routes: { "GET /": { public: true, handler: async () => ({ kind: "html", body: "ok" }) } },
}),
"@app/z-later": defineFacet<TestPrincipal>((_host, self) =>
self.contribute(Authenticators, () => {
later++;
@ -476,7 +547,11 @@ describe("cancellation", () => {
it("asks no authenticator for a request aborted before it arrived", async () => {
let calls = 0;
const server = await served(
{ "@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }) },
{
"@app/r": contributing({
routes: { "GET /": { public: true, handler: async () => ({ kind: "html", body: "ok" }) } },
}),
},
{
config: { listen: undefined },
authenticate: () => {
@ -503,18 +578,21 @@ describe("cancellation", () => {
{
"@app/b": contributing({
routes: {
"GET /": async (_request, ctx) => {
entered.resolve();
if (when === "inside the route") abort.abort(new Error("client gone"));
bornSealed = ctx.abortSignal?.aborted;
return {
kind: "bytes",
mime: "application/octet-stream",
read: async () => {
reads++;
return new Uint8Array([1]);
},
};
"GET /": {
public: true,
handler: async (_request, ctx) => {
entered.resolve();
if (when === "inside the route") abort.abort(new Error("client gone"));
bornSealed = ctx.abortSignal?.aborted;
return {
kind: "bytes",
mime: "application/octet-stream",
read: async () => {
reads++;
return new Uint8Array([1]);
},
};
},
},
},
}),
@ -539,11 +617,14 @@ describe("cancellation", () => {
{
"@app/b": contributing({
routes: {
"GET /": async () => ({
kind: "bytes",
mime: "application/octet-stream",
read: async () => new Uint8Array([1]),
}),
"GET /": {
public: true,
handler: async () => ({
kind: "bytes",
mime: "application/octet-stream",
read: async () => new Uint8Array([1]),
}),
},
},
}),
},
@ -587,10 +668,13 @@ describe("cancellation", () => {
{
"@app/wait": contributing({
routes: {
"GET /wait": async () => {
entered.resolve();
await gate.promise;
return { kind: "html", body: "done" };
"GET /wait": {
public: true,
handler: async () => {
entered.resolve();
await gate.promise;
return { kind: "html", body: "done" };
},
},
},
}),
@ -612,7 +696,11 @@ describe("cancellation", () => {
const entered = Promise.withResolvers<void>();
const gate = Promise.withResolvers<void>();
const server = await served(
{ "@app/r": contributing({ routes: { "GET /": async () => ({ kind: "html", body: "ok" }) } }) },
{
"@app/r": contributing({
routes: { "GET /": { public: true, handler: async () => ({ kind: "html", body: "ok" }) } },
}),
},
{
config: { listen: undefined, closeTimeout: 10 },
authenticate: async () => {
@ -640,15 +728,18 @@ describe("cancellation", () => {
{
"@app/bytes": contributing({
routes: {
"GET /stream": async () => ({
kind: "bytes",
mime: "application/octet-stream",
read: async (_offset, _length, ctx) => {
reads++;
readAfterSeal ||= ctx.abortSignal?.aborted ?? false;
return new Uint8Array([reads]);
},
}),
"GET /stream": {
public: true,
handler: async () => ({
kind: "bytes",
mime: "application/octet-stream",
read: async (_offset, _length, ctx) => {
reads++;
readAfterSeal ||= ctx.abortSignal?.aborted ?? false;
return new Uint8Array([reads]);
},
}),
},
},
}),
},
@ -672,10 +763,13 @@ describe("cancellation", () => {
{
"@app/body": contributing({
routes: {
"POST /body": async (request) => {
const value = await request.json();
writes++;
return { kind: "json", value };
"POST /body": {
public: true,
handler: async (request) => {
const value = await request.json();
writes++;
return { kind: "json", value };
},
},
},
}),
@ -700,7 +794,11 @@ describe("answers", () => {
// Sol's recheck N8: `Response` threw on a body for these statuses.
it.each([204, 205, 304])("answers %s without a body", async (status) => {
const server = await served(
{ "@app/s": contributing({ routes: { "GET /": async () => ({ kind: "json", value: null, status }) } }) },
{
"@app/s": contributing({
routes: { "GET /": { public: true, handler: async () => ({ kind: "json", value: null, status }) } },
}),
},
{ config: { listen: undefined } },
);
const response = await server.http.fetch(loopback("/"), LOOPBACK);
@ -714,8 +812,11 @@ describe("answers", () => {
{
"@app/r": contributing({
routes: {
"GET /": async () => {
throw new ChordError({ code: "forbidden", message: "no" });
"GET /": {
public: true,
handler: async () => {
throw new ChordError({ code: "forbidden", message: "no" });
},
},
},
}),
@ -731,7 +832,12 @@ describe("answers", () => {
it("sends several Set-Cookie headers", async () => {
const server = await served({
"@app/r": contributing({
routes: { "GET /": async () => ({ kind: "json", value: 1, headers: { "Set-Cookie": ["a=1", "b=2"] } }) },
routes: {
"GET /": {
public: true,
handler: async () => ({ kind: "json", value: 1, headers: { "Set-Cookie": ["a=1", "b=2"] } }),
},
},
}),
});
expect((await send(server.port, "GET", "/")).headers["set-cookie"]).toEqual(["a=1", "b=2"]);
@ -752,9 +858,16 @@ describe("GC", () => {
{
"@app/b": contributing({
routes: {
"GET /": async (_request, ctx) => {
refs.push(new WeakRef(ctx));
return { kind: "bytes", mime: "application/octet-stream", read: async () => new Uint8Array([1]) };
"GET /": {
public: true,
handler: async (_request, ctx) => {
refs.push(new WeakRef(ctx));
return {
kind: "bytes",
mime: "application/octet-stream",
read: async () => new Uint8Array([1]),
};
},
},
},
}),
@ -777,10 +890,17 @@ describe("GC", () => {
{
"@app/b": contributing({
routes: {
"GET /": async (_request, ctx) => {
refs.push(new WeakRef(ctx));
aborts.at(-1)?.abort(new Error("client gone"));
return { kind: "bytes", mime: "application/octet-stream", read: async () => new Uint8Array([1]) };
"GET /": {
public: true,
handler: async (_request, ctx) => {
refs.push(new WeakRef(ctx));
aborts.at(-1)?.abort(new Error("client gone"));
return {
kind: "bytes",
mime: "application/octet-stream",
read: async () => new Uint8Array([1]),
};
},
},
},
}),
@ -804,9 +924,12 @@ describe("GC", () => {
const server = await served({
"@app/a": contributing({
routes: {
"GET /r": async (request, ctx) => {
refs.push(new WeakRef(ctx), new WeakRef(request));
return { kind: "json", value: 1 };
"GET /r": {
public: true,
handler: async (request, ctx) => {
refs.push(new WeakRef(ctx), new WeakRef(request));
return { kind: "json", value: 1 };
},
},
},
}),

34
packages/chord/test/next-node/http/node.test.ts
View file

@ -27,8 +27,8 @@ async function raw(port: number, text: string): Promise<{ readonly status: strin
/** A route answering its path and params, under `GET /who` and `GET /n/:name`. */
const echo = contributing({
routes: {
"GET /who": async (request) => ({ kind: "json", value: request.path }),
"GET /n/:name": async (request) => ({ kind: "json", value: request.params }),
"GET /who": { public: true, handler: async (request) => ({ kind: "json", value: request.path }) },
"GET /n/:name": { public: true, handler: async (request) => ({ kind: "json", value: request.params }) },
},
});
@ -118,9 +118,15 @@ describe("Web requests", () => {
{
"@app/r": contributing({
routes: {
"GET /plain": async (request) => ({ kind: "json", value: request.path }),
"GET /headers": async (request) => ({ kind: "json", value: request.headers.get("x-thing") }),
"POST /body": async (request) => ({ kind: "json", value: await request.json() }),
"GET /plain": { public: true, handler: async (request) => ({ kind: "json", value: request.path }) },
"GET /headers": {
public: true,
handler: async (request) => ({ kind: "json", value: request.headers.get("x-thing") }),
},
"POST /body": {
public: true,
handler: async (request) => ({ kind: "json", value: await request.json() }),
},
},
}),
},
@ -157,9 +163,12 @@ describe("the request's scope", () => {
{
"@app/large": contributing({
routes: {
"GET /large": async (_request, ctx) => {
signal = ctx.abortSignal;
return { kind: "html", body: "x".repeat(size) };
"GET /large": {
public: true,
handler: async (_request, ctx) => {
signal = ctx.abortSignal;
return { kind: "html", body: "x".repeat(size) };
},
},
},
}),
@ -195,9 +204,12 @@ describe("the request's scope", () => {
const server = await served({
"@app/slow": contributing({
routes: {
"GET /slow": async (_request, ctx) => {
entered.resolve(ctx.abortSignal!);
return new Promise(() => {});
"GET /slow": {
public: true,
handler: async (_request, ctx) => {
entered.resolve(ctx.abortSignal!);
return new Promise(() => {});
},
},
},
}),

29
packages/chord/test/next-node/http/request.test.ts
View file

@ -12,9 +12,12 @@ describe("the request a route gets", () => {
const server = await served({
"@app/a": contributing({
routes: {
"GET /notes/:id": async (request) => {
seen = request;
return { kind: "json", value: null };
"GET /notes/:id": {
public: true,
handler: async (request) => {
seen = request;
return { kind: "json", value: null };
},
},
},
}),
@ -39,11 +42,17 @@ describe("bodies", () => {
served({
"@app/a": contributing({
routes: {
"POST /json": async (request) => ({ kind: "json", value: await request.json(16) }),
"POST /body": async (request) => {
let bytes = 0;
for await (const chunk of request.body(16)) bytes += chunk.byteLength;
return { kind: "json", value: bytes };
"POST /json": {
public: true,
handler: async (request) => ({ kind: "json", value: await request.json(16) }),
},
"POST /body": {
public: true,
handler: async (request) => {
let bytes = 0;
for await (const chunk of request.body(16)) bytes += chunk.byteLength;
return { kind: "json", value: bytes };
},
},
},
}),
@ -91,7 +100,9 @@ describe("bodies", () => {
it("drains a small body nobody read and keeps the connection; a large one closes it", async () => {
const server = await served({
"@app/a": contributing({ routes: { "POST /ignore": async () => ({ kind: "json", value: 1 }) } }),
"@app/a": contributing({
routes: { "POST /ignore": { public: true, handler: async () => ({ kind: "json", value: 1 }) } },
}),
});
const agent = new Agent({ keepAlive: true, maxSockets: 1 });
const post = (body: string) =>

43
packages/chord/test/next-node/http/response.test.ts
View file

@ -11,7 +11,7 @@ import { contributing, send, served } from "./support.ts";
/** A server answering `GET /r` with `response`. */
async function answering(response: HttpResponse) {
return served({ "@app/r": contributing({ routes: { "GET /r": async () => response } }) });
return served({ "@app/r": contributing({ routes: { "GET /r": { public: true, handler: async () => response } } }) });
}
/** A `bytes` response over `data` that records its reads. */
@ -64,8 +64,11 @@ describe("descriptions", () => {
const server = await served({
"@app/r": contributing({
routes: {
"GET /header": async () => ({ kind: "json", value: 1, headers: { "X-Split": "a\r\nSet-Cookie: x=1" } }),
"GET /bigint": async () => ({ kind: "json", value: { n: 1n } as never }),
"GET /header": {
public: true,
handler: async () => ({ kind: "json", value: 1, headers: { "X-Split": "a\r\nSet-Cookie: x=1" } }),
},
"GET /bigint": { public: true, handler: async () => ({ kind: "json", value: { n: 1n } as never }) },
},
}),
});
@ -185,21 +188,27 @@ describe("bytes", () => {
const server = await served({
"@app/r": contributing({
routes: {
"GET /short": async () => ({
kind: "bytes",
mime: "text/plain",
size: 10,
read: async () => Buffer.from("abc"),
}),
"GET /short": {
public: true,
handler: async () => ({
kind: "bytes",
mime: "text/plain",
size: 10,
read: async () => Buffer.from("abc"),
}),
},
// Astra's build review: a throwing read hung the response, unreported.
"GET /throws": async () => ({
kind: "bytes",
mime: "text/plain",
size: 10,
read: async () => {
throw new Error("read failed");
},
}),
"GET /throws": {
public: true,
handler: async () => ({
kind: "bytes",
mime: "text/plain",
size: 10,
read: async () => {
throw new Error("read failed");
},
}),
},
},
}),
});

13
packages/chord/test/next-node/http/router.test.ts
View file

@ -104,18 +104,25 @@ describe("patterns and precedence (D8)", () => {
expect(matched(router, "GET", "/v/b")).toEqual({ pattern: "GET /:name/b", params: { name: "v" } });
});
it("takes a route with options: page and crossSite", () => {
it("takes a route with options: page, crossSite and public (a cross-site route is public: always anonymous)", () => {
const { router } = compose([
"@app/a",
{
routes: {
"GET /p": { handler: route("p"), page: true },
"POST /hook": { handler: route("h"), crossSite: true },
"GET /open": { handler: route("o"), public: true },
"GET /closed": route("c"),
},
sockets: { "/s": { handler: () => {}, public: true }, "/t": () => {} },
},
]);
expect(router.route("GET", "/p")).toMatchObject({ page: true, crossSite: false });
expect(router.route("POST", "/hook")).toMatchObject({ page: false, crossSite: true });
expect(router.route("GET", "/p")).toMatchObject({ page: true, crossSite: false, public: false });
expect(router.route("POST", "/hook")).toMatchObject({ page: false, crossSite: true, public: true });
expect(router.route("GET", "/open")).toMatchObject({ public: true });
expect(router.route("GET", "/closed")).toMatchObject({ public: false });
expect(router.socket("/s")).toMatchObject({ public: true });
expect(router.socket("/t")).toMatchObject({ public: false });
expect(router.route("GET", "/notes")).toBeUndefined();
});
});

23
packages/chord/test/next-node/http/upgrades.test.ts
View file

@ -75,7 +75,7 @@ describe("the links endpoint", () => {
await server.close();
});
it("refuses a socket without a principal, from another Origin or Host, foreign, or whose links.options throws (403)", async () => {
it("refuses a socket without a principal (401), from another Origin or Host, foreign, or whose links.options throws (403)", async () => {
const server = await served(
{ "@app/hosts": contributing({ hosts: ["a.example.com"] }) },
{
@ -92,8 +92,9 @@ describe("the links endpoint", () => {
);
const { port } = server;
const owner = { "x-principal": "owner" };
// Nobody: the links endpoint needs a principal (deny by default, slice 13).
await expect(socketTo(port, "/chord")).rejects.toThrow("refused 401");
const refusals = [
socketTo(port, "/chord"),
socketTo(port, "/chord", { ...owner, origin: "http://evil.com" }),
socketTo(port, "/chord", { ...owner, origin: "null" }),
socketTo(port, "/chord", { ...owner, origin: `HTTP://127.0.0.1:${port}` }),
@ -139,7 +140,7 @@ describe("the links endpoint", () => {
describe("plugin sockets", () => {
it("hands the socket and its request to the handler in a scope with its principal; no principal is refused", async () => {
const server = await served({ "@app/echo": echo });
await expect(socketTo(server.port, "/echo/lobby")).rejects.toThrow("refused 403");
await expect(socketTo(server.port, "/echo/lobby")).rejects.toThrow("refused 401");
await expect(socketTo(server.port, "/nope", { "x-principal": "owner" })).rejects.toThrow("refused 404");
const socket = await socketTo(server.port, "/echo/lobby", { "x-principal": "owner" });
expect(await nextMessage(socket)).toBe("lobby as owner");
@ -149,6 +150,22 @@ describe("plugin sockets", () => {
await server.close();
});
it("accepts a public socket without a principal, its scope anonymous (deny by default, slice 13 §2)", async () => {
const lobby = contributing({
sockets: {
"/lobby": {
public: true,
handler: (socket, _request, ctx) => socket.send(`as ${ctx.principal?.kind ?? "nobody"}`),
},
},
});
const server = await served({ "@app/lobby": lobby });
const socket = await socketTo(server.port, "/lobby");
expect(await nextMessage(socket)).toBe("as nobody");
socket.close();
await server.close();
});
it("an oversized frame closes only that socket (1009), never the server", async () => {
const server = await served({ "@app/echo": echo });
const [big, other] = await Promise.all([

19
packages/chord/test/next/examples/40-http/plugins/notes/server.ts
View file

@ -10,6 +10,7 @@ export default defineFacet((_host, self) =>
routes: {
// A page: a link from another site may open it. Its inline script carries the server's nonce (the CSP).
"GET /": {
public: true,
page: true,
handler: async (request) => ({
kind: "html",
@ -17,13 +18,19 @@ export default defineFacet((_host, self) =>
}),
},
// `:name` is one path segment, decoded.
"GET /notes/:name": async (request) => ({ kind: "json", value: { note: request.params.name } }),
"GET /notes/:name": {
public: true,
handler: async (request) => ({ kind: "json", value: { note: request.params.name } }),
},
// A file of this plugin, saved under its name, with ranges.
"GET /notes/:name/download": async (request) =>
fileResponse(pluginFile(`files/${request.params.name}.md`), {
download: `${request.params.name}.md`,
ranges: true,
}),
"GET /notes/:name/download": {
public: true,
handler: async (request) =>
fileResponse(pluginFile(`files/${request.params.name}.md`), {
download: `${request.params.name}.md`,
ranges: true,
}),
},
},
}),
);

4
packages/chord/test/next/examples/44-http-wraps/plugins/notes/server.ts
View file

@ -5,7 +5,9 @@ import { Http } from "@earendil-works/chord/next/http";
export default defineFacet((_host, self) =>
self.contribute(Http, {
routes: { "GET /notes": async () => ({ kind: "json", value: ["buy milk", "call mom"] }) },
routes: {
"GET /notes": { public: true, handler: async () => ({ kind: "json", value: ["buy milk", "call mom"] }) },
},
wraps: [
{
name: "cache",

7
packages/chord/test/next/examples/46-allowed-hosts/plugins/tunnel/server.ts
View file

@ -12,7 +12,12 @@ export const Tunnel = local("example.tunnel", {
export default defineFacet((host, self) => {
self.contribute(Http, {
routes: { "GET /hello": async (request) => ({ kind: "json", value: { origin: request.origin } }) },
routes: {
"GET /hello": {
public: true,
handler: async (request) => ({ kind: "json", value: { origin: request.origin } }),
},
},
});
host.serve(Tunnel, (_params, tunnel) => {
const allowed = tunnel.writable<HttpContribution>({ hosts: [] });

5
packages/chord/test/next/examples/47-http-own-server/plugins/notes/server.ts
View file

@ -6,7 +6,10 @@ import { Http } from "@earendil-works/chord/next/http";
export default defineFacet((_host, self) =>
self.contribute(Http, {
routes: {
"GET /notes": async (request) => ({ kind: "json", value: { notes: ["buy milk"], via: request.origin } }),
"GET /notes": {
public: true,
handler: async (request) => ({ kind: "json", value: { notes: ["buy milk"], via: request.origin } }),
},
},
}),
);

14
packages/chord/test/next/examples/48-http-baseline/main.ts
View file

@ -1,7 +1,8 @@
// The security baseline (slice 10): on by default, the app decides only who someone is. Here the app signs people in
// with a signed cookie (`cookie`, `readCookie`) and the notes plugin contributes a page, routes and a webhook. Requests
// another site's page makes in a browser (Fetch Metadata says so) are never authenticated, a foreign `Origin` is
// refused, but a link to a page still opens it signed in, and a cross-site webhook runs anonymous.
// another site's page makes in a browser (Fetch Metadata says so) are never authenticated (so a route not marked
// public answers them 401: deny by default, slice 13), a foreign `Origin` is refused, but a link to a page still opens
// it signed in, and a cross-site webhook runs anonymous.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/48-http-baseline/main.ts
import { request } from "node:http";
@ -21,9 +22,12 @@ const http = chordHttp.createHttp(server, { listen: { host: "127.0.0.1", port: 0
const app = defineFacet<Principal>((_host, self) => {
self.contribute(chordHttp.Http, {
routes: {
"POST /sign-in": async (request) => {
const set = chordHttp.cookie("user", String(await request.json()), { secret, request, sameSite: "Lax" });
return { kind: "json", value: "signed in", headers: { "Set-Cookie": set } };
"POST /sign-in": {
public: true,
handler: async (request) => {
const set = chordHttp.cookie("user", String(await request.json()), { secret, request, sameSite: "Lax" });
return { kind: "json", value: "signed in", headers: { "Set-Cookie": set } };
},
},
},
});

6
packages/chord/test/next/examples/48-http-baseline/plugins/notes/server.ts
View file

@ -1,5 +1,6 @@
// The notes plugin's server facet: the notes page (a link from another site may open it), the notes as JSON, adding
// a note, and a webhook another site calls (cross-site: it always runs anonymous).
// a note, and a webhook another site calls (cross-site: it always runs anonymous, so it is public). Nothing else is
// public: a request nobody signed in gets 401 before this code runs (deny by default).
import { defineFacet } from "@earendil-works/chord/next";
import { Http } from "@earendil-works/chord/next/http";
@ -14,8 +15,7 @@ export default defineFacet((_host, self) =>
handler: async (_request, ctx) => ({ kind: "html", body: `hi ${who(ctx.principal)}` }),
},
"GET /notes": async (_request, ctx) => ({ kind: "json", value: { notes, as: who(ctx.principal) } }),
"POST /notes": async (request, ctx) => {
if (ctx.principal === undefined) return { kind: "error", status: 401 };
"POST /notes": async (request) => {
notes.push(String(await request.json()));
return { kind: "json", value: notes };
},

8
packages/chord/test/next/examples/53-app/app/principals.ts Normal file
View file

@ -0,0 +1,8 @@
// The notes app's principals: its server and workers, the kit's owner, and the users its plugins may sign in.
/** Who someone is in the notes app. */
export type NotesPrincipal =
| { readonly kind: "server" }
| { readonly kind: "owner" }
| { readonly kind: "worker"; readonly project: string }
| { readonly kind: "user"; readonly id: string; readonly name: string };

31
packages/chord/test/next/examples/53-app/app/v1.ts Normal file
View file

@ -0,0 +1,31 @@
// The notes app's API, version 1, shared by every host kind (`@notes/app/v1`): what its plugins import from their
// host. The part of Chord the app supports, the kit's start-data objects and files, the app's own host kind (a
// project, one worker per project) and its registry of home-page sections. The API only declares the registry: its
// provider is the home plugin of the app's required root, not base code. Examples 53 to 55 run this app; a real app
// keeps it in `src/api/v1.ts`.
import { defineRegistry, hostKind, str } from "../../../../../src/next/index.ts";
export {
arr,
constant,
defineFacet,
defineRegistry,
defineState,
defineTable,
int,
local,
method,
obj,
remote,
str,
} from "../../../../../src/next/index.ts";
export { AppInfo, Files, WorkerInfo } from "../../../../../src/next-node/app/index.ts";
/** A project of the app: its own worker process, store and files. */
export const Project = hostKind("project", { id: str() });
/** The home page's sections, one per contributing plugin (title and the path it links to). */
export const Sections = defineRegistry<{ readonly title: string; readonly path: string }, string[]>("notes.sections", {
version: 1,
});

5
packages/chord/test/next/examples/53-app/app/v1/server.ts Normal file
View file

@ -0,0 +1,5 @@
// The notes app's API v1 for server facets only (`@notes/app/v1/server`): the HTTP registry plugins contribute routes
// to, and the kit's sessions.
export { Sessions } from "../../../../../../src/next-node/app/index.ts";
export { Http } from "../../../../../../src/next-node/http/index.ts";

44
packages/chord/test/next/examples/53-app/client.ts Normal file
View file

@ -0,0 +1,44 @@
// A small HTTP client for examples 53 to 55, standing in for a browser and a command-line client: a request to the
// app on loopback, its status, `Set-Cookie` and JSON body.
import { request } from "node:http";
/** An answer: status, the cookie it set (name=value), JSON body (or text). */
export type Answer = { readonly status: number; readonly cookie: string | undefined; readonly body: unknown };
/** `method path` on the app at `port`, with `headers` and a JSON `body`. */
export function call(
port: number,
method: string,
path: string,
headers: Readonly<Record<string, string>> = {},
body?: unknown,
): Promise<Answer> {
return new Promise((resolve, reject) => {
const sent = request(
{
host: "127.0.0.1",
port,
method,
path,
headers: { ...headers, ...(body === undefined ? {} : { "content-type": "application/json" }) },
},
(response) => {
const chunks: Buffer[] = [];
response.on("data", (chunk: Buffer) => chunks.push(chunk));
response.on("end", () => {
const text = Buffer.concat(chunks).toString();
const type = response.headers["content-type"] ?? "";
const cookie = response.headers["set-cookie"]?.[0]?.split(";")[0];
resolve({
status: response.statusCode ?? 0,
cookie,
body: type.includes("json") ? JSON.parse(text) : text,
});
});
},
);
sent.on("error", reject);
sent.end(body === undefined ? undefined : JSON.stringify(body));
});
}

56
packages/chord/test/next/examples/53-app/main.ts Normal file
View file

@ -0,0 +1,56 @@
// The app kit (slice 13): an app is its API (`app/`), its built-in plugins (`plugins/`: home and notes) and a start
// script (`start.ts`) calling `startApp`; its workers' entry (`worker.ts`) calls `startWorker`. The kit makes the data
// directory, the server's store and host, HTTP (deny by default: no session, 401), one worker per project with its own
// store and files, and the owner's sign-in: a one-time owner link for the browser, an owner token file for command-line
// clients on the same machine.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/53-app/main.ts
import { mkdtemp, readdir, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { BACKGROUND_CONTEXT as context } from "../../../../src/context/index.ts";
import { call } from "./client.ts";
import { startNotes } from "./start.ts";
const dataDir = join(await mkdtemp(join(tmpdir(), "notes-")), "data");
// 1. Start the app. The start script prints the owner link (single use, 10 minutes, a new one at each start).
const app = await startNotes(dataDir, context);
const link = new URL(app.ownerLink());
const port = Number(link.port);
console.log("1. owner link:", `${link.origin}${link.pathname}#…`);
console.log(" data directory:", (await readdir(dataDir)).sort().join(", "));
// 2. Nobody is signed in: every plugin route answers 401 before the plugin runs.
console.log("2. anonymous GET /:", (await call(port, "GET", "/")).status);
// 3. The owner opens the link: the page posts the token from the URL's fragment (never sent in a URL, so never in a
// log) and gets the session cookie. The link is used up.
const page = await call(port, "GET", link.pathname);
console.log(
"3. the link's page:",
page.status,
String(page.body).includes("location.hash") ? "(reads the fragment)" : "",
);
const signedIn = await call(port, "POST", link.pathname, {}, { token: link.hash.slice(1) });
const browser = { cookie: signedIn.cookie ?? "" };
console.log(" signed in:", signedIn.status, signedIn.body);
console.log(" the link again:", (await call(port, "POST", link.pathname, {}, { token: link.hash.slice(1) })).status);
// 4. Signed in, the home plugin reads the kit's `AppInfo` and lists the sections of its registry (the notes plugin's);
// the notes plugin's routes reach the project's worker (it starts on the first request) with its store and the
// start data the app gave it.
console.log("4. GET /:", JSON.stringify((await call(port, "GET", "/", browser)).body));
await call(port, "POST", "/notes/p1", browser, "milk, eggs");
console.log(" GET /notes/p1:", JSON.stringify((await call(port, "GET", "/notes/p1", browser)).body));
console.log(" worker directories:", (await readdir(join(dataDir, "workers", "project"))).join(", "));
// 5. A command-line client on the same machine reads the owner token file (mode 0600) and sends it as a bearer token.
const token = (await readFile(app.ownerTokenFile, "utf8")).trim();
const cli = { authorization: `Bearer ${token}` };
console.log("5. CLI GET /:", JSON.stringify((await call(port, "GET", "/", cli)).body));
// 6. Close: HTTP, workers, host, store, in that order.
await app.close(context);
await rm(join(dataDir, ".."), { recursive: true, force: true });
console.log("6. closed");

5
packages/chord/test/next/examples/53-app/plugins/home/package.json Normal file
View file

@ -0,0 +1,5 @@
{
"name": "@notes/home",
"private": true,
"type": "module"
}

27
packages/chord/test/next/examples/53-app/plugins/home/server.ts Normal file
View file

@ -0,0 +1,27 @@
// The home plugin's server facet: it serves the app's `Sections` registry (the provider of a core registry is a plugin
// of the app's required root, so it reloads like any plugin) and an object reading it, and `GET /` says who is signed
// in to which app (the kit's `AppInfo`, the start data every server plugin can read) with the sections plugins
// contributed.
import { AppInfo, arr, defineFacet, local, Sections, str } from "@notes/app/v1";
import { Http } from "@notes/app/v1/server";
/** The home page's data: the composed sections. */
const Home = local("notes.home", { version: 1, params: {}, members: { sections: arr(str()) } });
export default defineFacet((host, self) => {
host.serve(Sections, { compose: (table) => table.map(({ value }) => `${value.title}: ${value.path}`) });
host.serve(Home, (_params, home) => ({
sections: home.derive([home.registry(Sections)], ([sections]) => sections),
}));
self.contribute(Http, {
routes: {
"GET /": async (_request, ctx) => {
const info = await ctx.get(AppInfo, {});
const page = await ctx.get(Home, {});
const app = `${info.name.value} ${info.version.value}`;
return { kind: "json", value: { app, you: ctx.principal ?? null, sections: page.sections.value } };
},
},
});
});

22
packages/chord/test/next/examples/53-app/plugins/notes/contract.ts Normal file
View file

@ -0,0 +1,22 @@
// A project's notes: served by the project's worker (`Project({ id })`), reached from the server by id.
import { method, obj, Project, remote, str } from "@notes/app/v1";
export const Notes = remote("notes.notes", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Project({ id }),
variants: {
main: {
state: { text: str(), start: str() },
methods: {
/** Replaces the project's text. */
write: method(str()),
/** Stores an attachment in the project's files; its id. */
attach: method(obj({ name: str(), body: str() }), str()),
/** An attachment's text. */
attachment: method(str(), str()),
},
},
},
});

5
packages/chord/test/next/examples/53-app/plugins/notes/package.json Normal file
View file

@ -0,0 +1,5 @@
{
"name": "@notes/notes",
"private": true,
"type": "module"
}

23
packages/chord/test/next/examples/53-app/plugins/notes/server.ts Normal file
View file

@ -0,0 +1,23 @@
// The notes plugin's server facet: routes reading and writing a project's notes, and its section of the home page.
// No route is public: the kit refuses anyone without a session (401) before they run.
import { defineFacet, Sections } from "@notes/app/v1";
import { Http } from "@notes/app/v1/server";
import { Notes } from "./contract.ts";
export default defineFacet((_host, self) => {
self.contribute(Sections, { title: "Notes", path: "/notes/:id" });
self.contribute(Http, {
routes: {
"GET /notes/:id": async (request, ctx) => {
const notes = await ctx.get(Notes, { id: request.params.id ?? "" });
return { kind: "json", value: { text: notes.text.value, start: notes.start.value } };
},
"POST /notes/:id": async (request, ctx) => {
const notes = await ctx.get(Notes, { id: request.params.id ?? "" });
await notes.write(String(await request.json()), ctx);
return { kind: "json", value: "saved" };
},
},
});
});

49
packages/chord/test/next/examples/53-app/plugins/notes/worker.ts Normal file
View file

@ -0,0 +1,49 @@
// The notes plugin's worker facet: the project's text in a persisted document (kept by a local object with empty
// params, the worker's one copy: a persisted state needs a stable identity), its attachments in the project's files
// (the kit's `Files`), and the start data the app gave the worker (`WorkerInfo`).
import { defineFacet, defineState, Files, local, method, obj, str, WorkerInfo } from "@notes/app/v1";
import { Notes } from "./contract.ts";
const Text = defineState("notes.text", {
schema: obj({ text: str() }),
version: 1,
initial: () => ({ text: "" }),
persist: true,
checkpointWhen: (_value, _ops, { deltasSinceBase }) => deltasSinceBase >= 20,
});
/** The project's text, one per worker. */
const Store = local("notes.store", { version: 1, params: {}, members: { text: str(), write: method(str()) } });
export default defineFacet((host) => {
host.serve(Store, (_params, self) => {
const text = self.state(Text);
return {
text: self.derive([text], ([value]) => value.text),
write: async (next, ctx) => {
await text.change((draft) => {
draft.text = next;
}, ctx);
return null;
},
};
});
host.serve(Notes, (_params, _principal, self) => {
const store = self.get(Store, {});
const info = self.get(WorkerInfo, {});
const files = self.get(Files, {});
return {
variant: "main",
members: {
text: self.derive([store.text], ([text]) => text),
start: self.derive([info.start], ([start]) => JSON.stringify(start)),
write: (next, ctx) => store.write(next, ctx),
attach: ({ name, body }, ctx) =>
files.put({ name, mime: "text/plain", bytes: new TextEncoder().encode(body) }, ctx),
attachment: async (id, ctx) =>
new TextDecoder().decode(await files.read({ id, offset: 0, length: 65536 }, ctx)),
},
};
});
});

50
packages/chord/test/next/examples/53-app/start.ts Normal file
View file

@ -0,0 +1,50 @@
// The notes app's start: everything the app itself decides, given to `startApp`. The kit does the rest: the data
// directory, the store, the host, HTTP with sessions and the owner link, a workers component for projects, and the
// plugins (`plugins/` here, the app's required root, and `<dataDir>/plugins/`, the user's).
// Examples 53 to 55 call `startNotes`; a real app's `start.ts` calls it with `process.argv[2]` and prints the link.
import { fileURLToPath } from "node:url";
import type { Context } from "../../../../src/context/index.ts";
import { type App, startApp } from "../../../../src/next-node/app/index.ts";
import type { NotesPrincipal } from "./app/principals.ts";
import * as v1server from "./app/v1/server.ts";
import * as v1 from "./app/v1.ts";
/** The app's API: its name, versions (newest first) and this process's modules (the worker's entry has the same). */
export const API = {
name: "@notes/app",
versions: ["v1"],
modules: { "@notes/app/v1": v1, "@notes/app/v1/server": v1server },
};
/** The app's facet kinds. */
export const KINDS = { server: { platform: "node" }, worker: { platform: "node" } } as const;
/** Starts the notes app on a free loopback port with its data in `dataDir`. */
export function startNotes(dataDir: string, context: Context): Promise<App<NotesPrincipal>> {
return startApp<NotesPrincipal>(
{
name: "notes",
dataDir,
root: fileURLToPath(new URL("./plugins/", import.meta.url)),
listen: { host: "127.0.0.1", port: 0 },
api: API,
kinds: KINDS,
principal: { server: { kind: "server" }, worker: (address) => ({ kind: "worker", project: address.key }) },
workers: [
{
kind: v1.Project,
entry: fileURLToPath(new URL("./worker.ts", import.meta.url)),
execArgv: ["--conditions=source", "--experimental-strip-types", "--no-warnings"],
// What each project's worker gets as `WorkerInfo.start`.
start: (address) => ({ project: address.key }),
// Who may reach a project: the owner (through the server's routes) and the server itself.
authorize: (principal) => principal.kind === "owner" || principal.kind === "server",
idleMs: () => 60_000,
},
],
onError: (error) => console.error("app:", error.kind, error.error),
},
context,
);
}

7
packages/chord/test/next/examples/53-app/worker.ts Normal file
View file

@ -0,0 +1,7 @@
// The notes app's worker entry: one process per project. `startWorker` opens the project's store in its directory,
// connects to the server and loads the plugins' worker facets.
import { startWorker } from "../../../../src/next-node/app/index.ts";
import { API, KINDS } from "./start.ts";
startWorker({ api: API, kinds: KINDS, onError: (error) => console.error("worker:", error) });

62
packages/chord/test/next/examples/54-app-sessions/main.ts Normal file
View file

@ -0,0 +1,62 @@
// Sessions and device pairing (slice 13 §2), with example 53's notes app. A command-line client on another machine
// has no owner token file: it asks for a pairing code (`POST /_chord/pair`, public), the owner approves the code in a
// signed-in browser (`/_chord/pair/<code>`, owner only), and the client's next poll returns its device token, once.
// The owner lists the sessions (`Sessions`, the kit's object; never a token or its hash) and revokes the device's.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/54-app-sessions/main.ts
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { BACKGROUND_CONTEXT as context } from "../../../../src/context/index.ts";
import { Sessions } from "../../../../src/next-node/app/index.ts";
import { call } from "../53-app/client.ts";
import { startNotes } from "../53-app/start.ts";
const base = await mkdtemp(join(tmpdir(), "notes-"));
const app = await startNotes(join(base, "data"), context);
const link = new URL(app.ownerLink());
const port = Number(link.port);
// 1. The owner signs in in the browser with the owner link (example 53).
const browser = { cookie: (await call(port, "POST", link.pathname, {}, { token: link.hash.slice(1) })).cookie ?? "" };
// 2. The client asks for a code and shows it; it keeps the poll secret to itself.
const { code, poll } = (await call(port, "POST", "/_chord/pair", {}, { label: "laptop CLI" })).body as {
code: string;
poll: string;
};
console.log("1. pairing code:", code.replace(/[A-Z0-9]/g, "X"));
console.log(" poll:", JSON.stringify((await call(port, "POST", "/_chord/pair/poll", {}, { poll })).body));
// 3. Only the owner may see and approve it: an anonymous request is refused before the kit's route runs.
console.log("2. anonymous approval:", (await call(port, "POST", `/_chord/pair/${code}`)).status);
const confirm = await call(port, "GET", `/_chord/pair/${code}`, browser);
console.log(
" the owner's confirm page:",
confirm.status,
String(confirm.body).includes("laptop CLI") ? "(names the device)" : "",
);
console.log(" approved:", (await call(port, "POST", `/_chord/pair/${code}`, browser)).status);
// 4. The client's next poll returns its token, once; it signs in with it as a bearer token.
const approved = (await call(port, "POST", "/_chord/pair/poll", {}, { poll })).body as {
status: string;
token: string;
};
console.log("3. poll:", approved.status);
console.log(" poll again:", (await call(port, "POST", "/_chord/pair/poll", {}, { poll })).status);
const device = { authorization: `Bearer ${approved.token}` };
console.log(" CLI GET /:", JSON.stringify((await call(port, "GET", "/", device)).body));
// 5. The owner lists the sessions (a server plugin would do this with `self.get(Sessions, {})`) and revokes the
// device's: its next request is refused.
const sessions = await app.host.get(Sessions, {}, context);
console.log("4. sessions:");
for (const { kind, label } of sessions.list.value) console.log(` ${kind}: ${label}`);
const laptop = sessions.list.value.find(({ label }) => label === "laptop CLI");
await sessions.revoke(laptop?.id ?? "", context);
console.log(" revoked; CLI GET /:", (await call(port, "GET", "/", device)).status);
sessions.release();
await app.close(context);
await rm(base, { recursive: true, force: true });

66
packages/chord/test/next/examples/55-app-export/main.ts Normal file
View file

@ -0,0 +1,66 @@
// Export and import per address (slice 13 §4), with example 53's notes app. `app.exportAddress` stops a project's
// worker and writes one archive: a manifest (app, API version, format), the project's store export and its files.
// `app.importAddress` makes another address from it (refusing one that has data) and the next request starts its
// worker on the imported store and files. Archives of another app are refused.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/55-app-export/main.ts
import { mkdtemp, rm, stat } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { BACKGROUND_CONTEXT as context } from "../../../../src/context/index.ts";
import { method, obj, Project, remote, str } from "../53-app/app/v1.ts";
import { startNotes } from "../53-app/start.ts";
// The notes plugin's contract, declared again with the API's builders (contracts are data: the same name, version and
// schema bind). The plugin's own files import `@notes/app/v1`, a host package this process has no module path for.
const Notes = remote("notes.notes", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Project({ id }),
variants: {
main: {
state: { text: str(), start: str() },
methods: {
write: method(str()),
attach: method(obj({ name: str(), body: str() }), str()),
attachment: method(str(), str()),
},
},
},
});
const base = await mkdtemp(join(tmpdir(), "notes-"));
const app = await startNotes(join(base, "data"), context);
// 1. Project p1 gets a text and an attachment (its worker's store and files). The server reaches it as `Notes`.
const p1 = await app.host.get(Notes, { id: "p1" }, context);
await p1.write("the plan", context);
const attachment = await p1.attach({ name: "plan.txt", body: "step 1, step 2" }, context);
p1.release();
// 2. Export p1: its worker stops, the archive is written (mode 0600), and p1 goes on (the next hold starts it).
const archive = join(base, "p1.tar");
await app.exportAddress(Project({ id: "p1" }), archive, context);
console.log(
"1. exported:",
`${(await stat(archive)).size} bytes`,
`mode ${((await stat(archive)).mode & 0o777).toString(8)}`,
);
// 3. Import it as p2: a copy with p1's text and attachment, its own worker and directory.
await app.importAddress(archive, Project({ id: "p2" }), context);
const p2 = await app.host.get(Notes, { id: "p2" }, context);
console.log("2. p2:", JSON.stringify(p2.text.value), JSON.stringify(await p2.attachment(attachment, context)));
await p2.write("p2's own plan", context);
p2.release();
const again = await app.host.get(Notes, { id: "p1" }, context);
console.log(" p1 unchanged:", JSON.stringify(again.text.value));
again.release();
// 4. An address that has data is never overwritten.
await app
.importAddress(archive, Project({ id: "p1" }), context)
.catch((error: Error) => console.log("3. refused:", error.message.replace(base, "…")));
await app.close(context);
await rm(base, { recursive: true, force: true });

2
tsconfig.json
View file

@ -12,8 +12,10 @@
"@earendil-works/chord/next/plugins": ["./packages/chord/src/next-node/plugins/index.ts"],
"@earendil-works/chord/next/plugins/runtime": ["./packages/chord/src/next-node/plugins/runtime.ts"],
"@acme/app/*": ["./packages/chord/test/next/examples/49-plugin-api/app/*.ts"],
"@notes/app/*": ["./packages/chord/test/next/examples/53-app/app/*.ts"],
"@earendil-works/chord/next/http": ["./packages/chord/src/next-node/http/index.ts"],
"@earendil-works/chord/next/workers": ["./packages/chord/src/next-node/workers/index.ts"],
"@earendil-works/chord/next/app": ["./packages/chord/src/next-node/app/index.ts"],
"@earendil-works/chord/node": ["./packages/chord/src/node.ts"],
"@earendil-works/chord/output": ["./packages/chord/src/output.ts"],
"@earendil-works/chord/telemetry-sqlite": ["./packages/chord/src/telemetry-sqlite/index.ts"],