From ebce9debd4206e5a0c176ca6bed28fcb40d0edc1 Mon Sep 17 00:00:00 2001 From: Mario Zechner Date: Mon, 2 Nov 2026 11:13:08 +0000 Subject: [PATCH] docs(chord): fold the plugin model, app kit and slice 11 decisions into the design; slice 12 spec --- packages/chord/docs/next/design.md | 72 +++++++++++++++ packages/chord/docs/next/slice-12.md | 131 +++++++++++++++++++++++++++ packages/chord/docs/next/slices.md | 8 +- 3 files changed, 208 insertions(+), 3 deletions(-) create mode 100644 packages/chord/docs/next/slice-12.md diff --git a/packages/chord/docs/next/design.md b/packages/chord/docs/next/design.md index c11cb89e4..413674592 100644 --- a/packages/chord/docs/next/design.md +++ b/packages/chord/docs/next/design.md @@ -1980,6 +1980,39 @@ type ReloadReport = { - Project plugins, trust, snapshots, scopes and npm/git package installs: `apps/p2/docs/plugin-loading.md` (still the reference for those; not in 7c). +### The plugin model (2026-11-01; slice 12) + +Decisions "Contracts come from where the plugin's code came from", "Plugins import only the app's versioned API", +"The plugin model, points 3, 4, 5 and 8, imports and builds", "Single-file plugins", "Testing a plugin alone". The +rule underneath: a plugin carries what it was written against, never the running app's newest file. + +- **What a plugin may import:** its own files; the app's versioned API (`@acme/app/v3`, never Chord directly: Chord is + not a host package plugins may name; the API re-exports the Chord surface the app supports); its npm `dependencies` + (bundled at the installed version); other plugins' contract files: by relative path within one repository + (`../notes/contract.ts`, the same commit holds both sides), by package name across repositories (`@bob/notes`, + `@bob/notes/web`, pinned by an npm version or a git ref `github:bob/notes#`; a subfolder of a multi-plugin + repository is cloned and linked by the plugins component). One facet imports one API version only. +- **Versions:** the app lists the API versions it provides; an unlisted one is a build error naming the available ones. + The process runs one Chord: an older API version is a layer over today's, re-exporting Chord's compatibility module + for its previous API while Chord keeps one; contracts bind through `accepts` whatever Chord runs. +- **Split by host kind:** the API has a shared module and per-kind modules (`/v3/web`, `/v3/tui`, `/v3/server`, + `/v3/worker`); a plugin's contract files mirror it (`contract.ts`, `contract.web.tsx`, `contract.tui.ts`, ...). A + contract file mentions native things only through `import type`. The build checks every import by the file it + resolves to: a per-kind file or module only for its kind, a package the kind does not provide refused. A token or + `local` contract used on one kind lives in that kind's file. Shared UI between plugins: an imported file (inlined, + frozen at the importer's build) or a registry entry (one live instance). +- **Base code** is the API modules, bootstrap (hosts, store, components such as `createHttp` and `createWorkers`, which + take settings and process values and return facets) and one start-data facet serving process values as core + objects. Every implementation, the harness and core registry providers included, is a plugin; the app's own root is + required (a failure at startup stops the app). +- **Single-file plugins:** `..ts(x)` at a root's top, grouped by name, no `package.json`, importing only the + API; a directory of the same name replaces it as the same plugin. +- **Installs** from git (a branch, tag or commit cloned into a root) or npm. **Builds** are one esbuild call per facet + from source (TS, TSX, JS, JSON, CSS, assets and WASM by URL); plugins bring no build hooks. +- **`testPlugin(dir, { kind, modules, stubs, start?, principal? }, ctx)`:** the plugin built as in production on an + in-memory host and store, its dependencies served by stub facets, contributions to registries collected for the + test, the contracts nobody serves listed. + - **UI reload vs. leaving:** "leaving" is never derived from an unmount, only from explicit navigation or presence (a dropped reference); a component unmounted by a reload has not left. @@ -2236,6 +2269,45 @@ fails closed when stale; a start waits for a draining old holder without polling complete sets (`createPluginRuntime`, no esbuild); `workers.step(name)` ships a rollout's set to every worker (D12: `{ host, report } | { host, stopped: true } | { host, error }` per worker). +### The app kit (`chord/next/app`; slice 13) + +Decisions "The app kit, point 1" to "point 5". `startApp(options)` on the server, `startWorker(options)` in a worker +entry. Opinionated and small: an app is its API, its built-in plugins and a start script. + +- **Shape:** the app's layout is fixed: `src/api/v/` (shared and per-kind modules) and the built-in plugins root + `plugins/`. The data directory is a `startApp` option (no environment variables), default a dot directory named after + the app in the home directory. One server; worker kinds are a list the app defines, each a host kind with its own + params (sharding is the app's) and how it starts (entry, start data, principal, idle time); zero kinds is a server + with clients only. +- **Principals and users:** the principal is tiny (a kind and an id) and the app's type; the kit needs only its server + and worker variants. User data lives in objects keyed by the user's id. Optional kit plugins: users (accounts with an + app-supplied profile schema), authenticators (device pairing, invite links, an ephemeral pick-a-name login). +- **Authentication:** no owner by loopback alone: the start script prints a link with a one-time token that sets a + signed owner cookie; deny by default (the HTTP component refuses requests without a principal unless the route or + socket is public); a session helper issues and revokes sessions; command-line clients send a token in a header (an + owner token file readable only by the app's OS user; a device pairing flow from other machines). +- **Storage:** one store for the server, a directory per worker under its kind and address with its store; plugins + never use data-directory paths. A files plugin per address (`put`, chunked transfers, a watchable index, caps) is + the default file storage; raw directories only where a real path is needed, owned by one object, under the worker's + directory. Export and import per address: the store's archive plus the worker's directory. +- **Plugins:** the app's root (required) and the user's root in the data directory; installs from git or npm; one + "apply changes" call builds and rolls out to the server, then every running worker, exposed by the app as it likes. + +### Web and TUI (slice 11) + +Decision "Slice 11's points" and "Slice 11, P3" to "P9". + +- `chord/next/web`: serving the surface, a page's client host with its connection loop (p2's `keepConnected`), loading + and applying the plugins' web facets. A base bundle per client kind built on the server from the app's entry + (content-hashed); at page load the kind's current plugin set as one combined file named by its hash, later changed + plugins one script each, each a script that registers itself (no eval); new scripts apply in place (`host.reload` in + the page); pages learn of them through an object the web component serves (the current set); a base code change + reloads the page. +- `chord/next/react`: `HostProvider`, `useHold`, `useWatch`, `useRegistry`, `useRefState`. +- `chord/next/tui`: a Node client connecting with a token, always running the downloaded code in a sandbox (a + permission-dropped worker thread, a bridge terminal, the trusted launcher outside); a component scope (`hold`, + `watch`, `registry`, released on dispose, one `changed` callback for the app's redraw), no pi-tui dependency. + ### Channels WebSocket byte channels with acknowledged windows, separate from object links, wait for a second user after the diff --git a/packages/chord/docs/next/slice-12.md b/packages/chord/docs/next/slice-12.md new file mode 100644 index 000000000..4bfd790e7 --- /dev/null +++ b/packages/chord/docs/next/slice-12.md @@ -0,0 +1,131 @@ +# Slice 12: the plugin model (spec) + +Status: **approved** (owner, 2026-11-01: the plugin model's decisions, then "go ahead, just do all the things"; this +spec written by the redesign session from them, no separate spec review). Builds on 7c's plugins component +(`src/next-node/plugins/`) and slice 10's kinds and `createPluginRuntime`. + +Read in full before writing code: this spec; `decisions.md` entries of 2026-11-01 "Contracts come from where the +plugin's code came from", "Plugins import only the app's versioned API", "The plugin model, points 3, 4, 5 and 8, +imports and builds", "Single-file plugins", "Testing a plugin alone"; `design.md` §21 (all of it, "The plugin model" +included) and §16-§17; `slices.md` (carry-over "Slice 12"); `slice-7c.md`, `slice-10.md` §3 "Plugins in workers"; +`src/next-node/plugins/README.md` and its sources; the primer `docs/next/plugin-primer/book.md`. + +## 1. The app's API (`createPlugins` option `api`) + +```ts +const plugins = createPlugins({ + roots: [{ path: "./plugins", required: true }, { path: join(dataDir, "plugins") }], + api: { name: "@acme/app", versions: ["v3", "v2"] }, // newest first + kinds: { + server: { platform: "node", hostPackages: ["@acme/app/v3", "@acme/app/v3/server", "@acme/app/v2"] }, + worker: { platform: "node", hostPackages: ["@acme/app/v3", "@acme/app/v3/worker", "@acme/app/v2"] }, + web: { platform: "browser", hostPackages: ["@acme/app/v3", "@acme/app/v3/web", "@acme/app/v2", "react", "react/jsx-runtime"] }, + }, + onError, +}, ctx); +``` + +- `api` names the app's API package and the versions it provides. A plugin import of `/[/]`: + a version not listed is a build error naming the listed ones ("@acme/app/v1 is not available: this app provides v3, + v2"); a listed one that the kind's `hostPackages` lacks (`@acme/app/v3/web` from a worker facet) is a build error + naming the kind. One facet's imports of the API all use one version (a mix is a build error naming both). +- Chord itself (`@earendil-works/chord/next` and its subpaths other than `/plugins`, which stays as in 7c for + `pluginFile`) is not importable by plugins: a build error pointing to the app's API. Apps that do not set `api` keep + 7c's behaviour (host packages as listed), so Chord's own tests and examples keep importing Chord directly where they + model base code. +- `roots` entries may be strings (as today) or `{ path, required? }`. A plugin of a required root whose build, + evaluation or setup fails at `plugins.load` makes `load` reject (`PluginError` naming it), after the host's own load + settled; failures in other roots are reported as today. At a reload, a required plugin failing keeps its old code as + any plugin (reported). + +## 2. Imports between plugins (replaces slice 10's by-name lookup) + +- **Relative paths within a root:** an import from one plugin into another plugin's directory under the same root + (`../notes/contract.ts`) is inlined (listed in `copied`), the copied file under 7c's rule (only host packages and + files of its own plugin). A relative path leaving the root (base code, another root) is a build error naming the file + and pointing to the API or a dependency. A git checkout of several plugins cloned into a root is "within a root". +- **Package names across repositories:** an import of a package in the plugin's `dependencies` resolves as npm does + through that package's `exports` (`@bob/notes` → its `contract.ts`, `@bob/notes/web` → `contract.web.tsx`), inlined + at the installed version. A dependency may be an npm version or a git URL with a ref (`github:bob/notes#`, + `git+https://...#`); `installDependencies` installs it with npm as today. A package from a subfolder of a git + repository (which npm cannot install) is declared as `"chord": { "plugins": { "@bob/notes": "github:bob/plugins#:notes" } }` + and installed by `installDependencies` itself: a shallow clone at the ref into the plugin's `node_modules/.chord-git/` + and a link `node_modules/@bob/notes` to the subfolder. +- The by-name lookup (`build.ts` `#byName`), its error texts and the six `tsconfig.json` path aliases go; fixtures and + examples 35-38 import relatively again. + +## 3. Per-kind files and the import check + +- A file whose name has a kind segment before its extension (`contract.web.tsx`, `ui.tui.ts`, any `*..` with + `` one of `kinds`) is a file of that kind: importing it (directly or through other files) from a facet of + another kind is a build error naming the chain. Facet files keep their names (`web.tsx`); contract files are + `contract.ts` (shared) and `contract..ts(x)`. +- The check runs in the existing esbuild `onResolve` path, on the resolved file, for relative imports, dependencies and + copied files alike. +- Documented, not checked: a contract file mentions native things only through `import type` (TypeScript erases it; + a runtime import of a package the kind lacks already fails as above). + +## 4. Single-file plugins + +- Discovery also lists files at a root's top named `..(ts|tsx|js|mjs)`; files of one `` form one + plugin named `` (an unscoped valid npm name, else reported), one facet per file, `dir` the root, no + `package.json` (no dependencies, no `chord.external`). A directory plugin of the same name in the same root is a + reported duplicate (the directory wins). +- Its build allows imports of host packages only (the app's API); anything else is a build error saying to make it a + directory. `pluginFile` refuses (it has no directory of its own). +- The watcher: such a file appearing or disappearing at a root's top sets `set`; a change marks the plugin dirty. + +## 5. Installing plugins + +`installPlugin(source, root, ctx)`: `source` is `{ git: url, ref }` (a shallow clone of the ref into +`/`; a repository whose top has a `package.json` is one plugin, otherwise its subdirectories +are plugins) or `{ npm: name, version }` (`npm pack` and extract into `/`), then `installDependencies` for +each new plugin; marks `set`. Returns the installed plugin names. Removal is deleting the directory (`set`). No +updating in place: install another ref (a new directory) or delete and install. + +## 6. `testPlugin` + +```ts +const t = await testPlugin("./plugins/stats", { + kind: "worker", + api: { name: "@acme/app", versions: ["v3"] }, + modules: { "@acme/app/v3": apiV3, "@acme/app/v3/worker": apiV3Worker }, // the real API modules + stubs: [{ name: "@acme/notes", facet: notesStub }], // facets serving what it uses + principal: { kind: "user", id: "u1" }, // the host's principal (optional) +}, ctx); +const stats = await t.host.get(Stats, { project: "p1" }, ctx); +t.contributions(Panels); // what the plugin contributed to a registry nobody serves (the helper serves it) +t.unserved(); // contract names requests wait for unbound `not_served` +await t.close(ctx); +``` + +- Built with the same build as production (`buildFacet` with the kind's options), evaluated with `modules`, loaded on a + host with `address`/`principal` as given, an in-memory store (`:memory:` SQLite); stubs loaded beside it. +- The helper serves, as a collecting registry, every registry the plugin contributes to that no stub or the plugin + itself serves (`t.contributions(R)`: the current entries). +- `t.unserved()` lists the names of contracts whose requests wait unbound `not_served` on the host (the unbound + index), so a test knows what to stub. `t.root(fn, ctx, { principal })` runs as another principal. +- Exported from `chord/next/plugins` (`testing.ts`); no test-only modes elsewhere. + +## 7. Docs for authors and agents + +- `docs/next/plugin-authoring.md`: a plugin's layout (directory and single-file), its facet and contract files per + kind, what it may import and how each import resolves, versions, how the server builds it (esbuild, what the build + handles, no build hooks), installing from git or npm, testing with `testPlugin`; written for agents (short, exact, + with one complete example plugin). +- Update `src/next-node/plugins/README.md`, the guide `docs/next/guide-plugins-hosts/book.md` (chapters "Plugins Using + Plugins" and "Base Code and Plugins"; rebuild `index.html`), design §21's slice-10 note. + +## 8. Tests, examples + +- One test file per source file; build tests for every refusal and allowance of §1-§4 (an unlisted version, a kind + module from another kind, mixed versions, a Chord import, a relative path leaving the root, a per-kind file through + a chain, a package from `dependencies` through `exports`, a git dependency with a ref (a local bare repository in a + temp directory, no network), a subfolder dependency); discovery of single-file plugins; required roots at load; + `installPlugin` from a local git repository and a local npm tarball; `testPlugin` (stubs, collected contributions, + `unserved`, a principal). +- Examples: `49-plugin-api` (an app API with two versions and per-kind modules, a plugin per version, the refusals), + `50-plugin-imports` (relative within a root, a git dependency pinned to a commit, per-kind contract files), + `51-single-file-plugin`, `52-test-plugin` (a plugin tested with a stub). Examples 35-38 updated. +- No hot path changes; no benchmark beyond a cold build of the 20-plugin set (as 7c's) to show the checks cost nothing + measurable. diff --git a/packages/chord/docs/next/slices.md b/packages/chord/docs/next/slices.md index 766d32211..edd891029 100644 --- a/packages/chord/docs/next/slices.md +++ b/packages/chord/docs/next/slices.md @@ -45,7 +45,7 @@ slice spec's section citations are for orientation, not a reading limit (owner, Order (owner, 2026-10-22: routines and jobs before the store and reload; 2026-10-27: slice 8 before slice 7, whose reload has to reload real registries; 2026-10-29: slice 7 split into 7a, 7b, 7c; the app kit as slices 10 and 11; -telemetry last, since it instruments the app kit too). +telemetry last, since it instruments the app kit too; 2026-11-01: the plugin model (12) and the app kit (13) before web and TUI (11), which builds into the kit). | # | Slice | In short | | --- | --- | --- | @@ -63,6 +63,8 @@ telemetry last, since it instruments the app kit too). | 7b | **Reload in the kernel** | `host.reload`: staging, commit, rollback; swaps, restarts, removal and re-serving; holds and `accepts` across a reload; jobs and registries across a reload; per-name wakes (`slice-7b/` brief) | | 7c | **Plugins** | `chord/next/plugins` (discovery, `buildFacet`, `evaluateFacet`, `pluginFile`, `installDependencies`, the watcher, `rollout`, `mapStack`; no dev/prod split; the kernel's `load`/`reload` per plugin; `slice-7c.md`) | | 10 | **HTTP and workers** | `chord/next/http` (server, `Http` registry, responses as descriptions, WebSocket entry) and `chord/next/workers` (a worker per address, IPC, idle stop) | +| 12 | **The plugin model** | imports (relative within a repository, package names pinned across), versioned app API with Chord re-exported, per-kind API modules and contract files with the build's import check, single-file plugins, git installs, `testPlugin`, base code as API, bootstrap and a start-data facet (`slice-12.md`) | +| 13 | **The app kit** | `chord/next/app`: `startApp`, `startWorker`, layout, worker kinds, owner link, deny by default, sessions, CLI tokens, storage, the files plugin, export per address, plugin roots and "apply changes" (`slice-13.md`) | | 11 | **Web and TUI** | `chord/next/web` (page host, React hooks) and `chord/next/tui` (launcher, sandbox), from p2's code | | 9 | **Telemetry** | spans, events, metrics across hosts | @@ -181,12 +183,12 @@ deleted (the slice's spec and code are the record). Moved here from `carry-over. - **`rebuild` of tables** (key and column-type changes): `evolveTable` has no edit for them; build one only when a real case needs it. (Slice 6a's second spec review; out of slice 7b's scope, `slice-7b.md` §12.) -### The app kit +### Slice 13: the app kit - **Deny by default in the HTTP component** (decision "The app kit, point 3: authentication"): routes and sockets refuse requests without a principal unless marked public; the owner link with a one-time token; a session helper. -### The plugin slice (plugin model) +### Slice 12: the plugin model - **Cross-plugin imports by relative path and npm** (decision "Contracts come from where the plugin's code came from"): remove `build.ts`'s by-name lookup, allow relative imports into another plugin of the same root, refuse