From 18a155e2f52acb2b513a2dede59b41079ea2408d Mon Sep 17 00:00:00 2001 From: Mario Zechner Date: Fri, 30 Oct 2026 14:56:23 +0000 Subject: [PATCH] docs(chord): slice 7c spec (plugins), per plugin at startup and reload --- packages/chord/docs/next/decisions.md | 31 ++++- packages/chord/docs/next/design.md | 62 ++++++---- packages/chord/docs/next/slice-7b.md | 2 +- packages/chord/docs/next/slice-7c.md | 157 ++++++++++++++++++++++++++ packages/chord/docs/next/slices.md | 2 +- 5 files changed, 230 insertions(+), 24 deletions(-) create mode 100644 packages/chord/docs/next/slice-7c.md diff --git a/packages/chord/docs/next/decisions.md b/packages/chord/docs/next/decisions.md index 5bbb62b16..5377f93a2 100644 --- a/packages/chord/docs/next/decisions.md +++ b/packages/chord/docs/next/decisions.md @@ -1838,6 +1838,35 @@ validation and copies the v2 kernel added around Delta. key hold two pins (counted on the key's one source; the document is loaded and kept once). **Binary append commit-to-frame median +10% accepted** (owner, 2026-10-29; CPU 10, 0.031 → 0.034-0.035 ms): equal with the young generation pinned, fewer scavenges over the whole benchmark (V8's young-generation sizing, slice 7b fix round 3). +- **Per plugin at startup and reload (owner, 2026-10-30: "yes", "1. lgtm 2. yes"; analyses + `/tmp/per-plugin-{opus,fable}.md`):** supersedes "Rollback is all-or-nothing per host" (2026-10-2x, 7b §2, design §21) + and 7c's Q2 ("a facet throwing while evaluated fails its host's whole step"). Why all or nothing was chosen: per plugin + was decided first and p2's plugin reviewers found counter-traces (a dropped argument field written into a transcript; a + copied Durable document written by two versions; a provider's failed factory leaving the harness hanging; mixed copied + code and registries); v2 has since closed them (closed argument schemas for every caller, Q12, Q15; version binding + with waiting, 7a; ownership of documents; registry versions with `accepts`, Q8; copied cross-plugin files limited to + contracts without dependencies, 7c). Rules: (1) **at startup** a plugin whose build, evaluation or setup fails is left + out; `load` resolves with a report `{ failed: [{ plugin, error }] }` and never closes the host for it (the core facet + included: base code decides whether that is fatal); its consumers wait `failed { waiting }` as for any missing plugin. + (2) **At reload** a plugin whose new code fails (build, evaluation, setup, a factory in staging, a declaration error, + `checkAccepts`) keeps its old code (its loaded facet, unchanged), and every other plugin of the reload applies; the + report lists it (`kept: [{ plugin, error }]`); staging retries without it. (3) **The closure rule** (Opus): after + staging, a changed plugin whose recorded requests would not bind because the plugin they need was kept old or left + out is kept old too, so a consumer edited together with a broken provider keeps working on its old code. (4) What no + rule detects: a consumer relying on new behaviour of a kept provider whose contract (version, fingerprint) did not + change: the same risk as a rollout across hosts, documented. (5) `rollout` keeps a plugin dirty while it was kept old + on any host. Built with slice 7c (the kernel's `load` and `reload` change with it). +- **Slice 7c's points (owner, 2026-10-30; `slice-7c.md` §5):** (1) esbuild is an optional peer dependency of Chord, + imported only by the plugins component. (2) **No dev/prod split** (owner: "chord apps are supposed to not have a + prod/dev split. they are supposed to be self modifying applications"): watching, rebuilding and rolling out are core + machinery; one component `chord/next/plugins` (discovery, build, evaluation, `pluginFile`, the watcher, `rollout`) in + `src/next-node/plugins/`, exported with v2's entry point; it turns V8's compilation cache off when created; reloads + stay explicit (the watcher marks, never applies). Supersedes the separate `loader` and `dev` subpaths of "Slice 7b's + open points" (5), (6). (3) `installDependencies(plugin, ctx)` as coding-agent installs package dependencies (`npm + install --omit=dev --legacy-peer-deps --ignore-scripts --no-audit --no-fund` in the plugin's real directory, the + command overridable). (4) Cross-plugin imports: any file of a discovered plugin, copied, documented as contracts and + side-effect-free modules only, listed by the build, no static check yet. (5) `pluginFile` answered per facet by the + `require` evaluation passes. - **A plugin's npm dependencies are bundled; external only by opt-in (owner, 2026-10-29: "explicit opt in + (a)"):** a plugin's `dependencies` are installed into its own `node_modules` (`npm install --ignore-scripts`) and bundled into each facet that imports them (host packages, its `peerDependencies`, stay external and are passed in). On Node a @@ -2395,7 +2424,7 @@ From `research/chord-on-durable-objects.md`, adopted for the Chord v2 kernel bec - **Per host:** load (bundle; a load error rejects) → staging (run the facet setup and every live object's new factory with a recording `self`, declarations only, nothing activated; a throw rejects) → commit (atomic) → after. -- **Rollback is all-or-nothing per host:** any load or staging error leaves the host entirely on its previous code. +- ~~**Rollback is all-or-nothing per host:**~~ (**superseded** by "Per plugin at startup and reload", 2026-10-30) any load or staging error leaves the host entirely on its previous code. A host is completely at revision R or R−1, so status per host suffices. - **Pure objects** (**superseded in part** by R15 (old code touching a document migrated past its version throws)) (no `self.open`, old or new) are swapped in place: new methods and derives; documents, references and jobs stay; routines restart serialized; new references become live; then the object's gate opens. Running calls diff --git a/packages/chord/docs/next/design.md b/packages/chord/docs/next/design.md index f222c5b87..756fad951 100644 --- a/packages/chord/docs/next/design.md +++ b/packages/chord/docs/next/design.md @@ -231,8 +231,8 @@ parent link closes. `resolve` returns the link before the worker has loaded (the | Call | Meaning | | --- | --- | | `createHost

({ address?, principal?, ...options })` | with an address: addressable, serves only objects placed there, acts as `principal` on its own host; without: a client. `P` is the app's principal type | -| `load([{ name, facet }], context)` | facets with their plugin names (the plugin's `package.json` name, npm style; duplicates and invalid names are refused); runs every facet's setup, publishes all of them in one batch (types and registries served, registries composed once, contributions; requests waiting for the publication are placed after it), then waits for the start barrier (section 21: the first outcomes of its host-lifetime objects, slice 7a); a throwing setup rejects and closes the host; runs once. Loaders are adapters; base code passes its core facet the same way | -| `reload([{ name, facet }], context)` | the complete new plugin set after `load` (a facet passed as the same object is an unchanged plugin): staging, then one commit, then every object of a changed or removed plugin takes its fate; resolves with a report, rejects with a staging error and nothing changed; one at a time (section 21, slice 7b) | +| `load([{ name, facet }], context)` | facets with their plugin names (the plugin's `package.json` name, npm style; duplicates and invalid names are refused); runs every facet's setup, publishes all of them in one batch (types and registries served, registries composed once, contributions; requests waiting for the publication are placed after it), then waits for the start barrier (section 21: the first outcomes of its host-lifetime objects, slice 7a); a plugin whose setup throws is left out and reported (`load` resolves with `{ failed: [{ plugin, error }] }`, decision "Per plugin at startup and reload"); runs once. Loaders are adapters; base code passes its core facet the same way | +| `reload([{ name, facet }], context)` | the complete new plugin set after `load` (a facet passed as the same object is an unchanged plugin): staging, then one commit, then every object of a changed or removed plugin takes its fate; resolves with a report (a plugin whose new code fails keeps its old code, listed as `kept`); one at a time (section 21, slice 7b) | | `connect(link, { principal, trusted?, codec? }, context)` | the link to the parent, at most one open; once it closed (a drop), `connect` takes the next, and every remote reference over the parent is held again on it. `principal` = who the parent is | | `accept(link, { principal, trusted?, codec? }, context)` | a link from a child; holds and calls on it act as `principal` | | `route(Kind, { authorize, resolve }, context)` | how to reach hosts of a kind (below); called before `load`, once per kind; work arriving on its links runs under `context` | @@ -1810,7 +1810,8 @@ agent asks everyone present. Its store must live in an object (it predates "nati changed (Chord compares no code); one passed as the very same facet object is unchanged and untouched (owner, 2026-10-29: base code passes its core facet value every time); one loaded and not in it is removed. It resolves with a report once the commit happened and every object took its fate (swaps and retries had their first dependency check; -it does not wait for objects to go live), and rejects with the code error when staging fails, nothing changed. +it does not wait for objects to go live); a plugin whose new code fails keeps its old code and is listed in the +report's `kept` (per plugin, below). Reloads run one after another per host; before `load` resolved it throws, after `close` it rejects `released/closed`; `context` bounds it (decision 4: no install watchdog): its abort before the commit rejects `aborted` and changes nothing, after the commit it only ends the wait. @@ -1818,6 +1819,7 @@ nothing, after the commit it only ends the wait. ```ts type ReloadReport = { readonly plugins: { readonly added: readonly string[]; readonly changed: readonly string[]; readonly removed: readonly string[] }; + readonly kept: readonly { readonly plugin: string; readonly error: unknown }[]; // new code failed: old code kept (per plugin) readonly objects: readonly ({ readonly object: string } & ( | { readonly kind: "swapped" } | { readonly kind: "restarted" } | { readonly kind: "ended" } | { readonly kind: "parked"; readonly cause: FailureCause } | { readonly kind: "retried" }))[]; @@ -1832,11 +1834,18 @@ type ReloadReport = { 4. after each object takes its fate (below) ``` -- **Rollback is all-or-nothing per host:** any load or staging error (a setup's or factory's throw, a declaration - error, `checkAccepts`, a type or registry served twice, a remote type on a host without principal or address) - rejects the reload naming the plugin (and object), and the host stays entirely on its old code; a host is at - revision R or R−1. Runtime states (registry conflicts, failed opens, migrations) are not rollback reasons: they are - per object after the commit (`onError` as always; no new `HostError` kind, the report is the summary). +- **Per plugin, never all or nothing** (owner, 2026-10-30, decision "Per plugin at startup and reload"; supersedes 7b's + rollback per host): a plugin whose new code fails at a reload (its build or evaluation, a setup's or factory's throw + in staging, a declaration error, `checkAccepts`, a type or registry served twice, a remote type on a host without + principal or address) keeps its old code (its loaded facet, unchanged) and is reported in the reload's report + (`kept: [{ plugin, error }]`); staging runs again without it, and every other plugin of the reload applies. **The + closure rule:** a changed plugin whose recorded requests would not bind because the plugin they need was kept old (or + is missing) is kept old too, so a consumer edited together with a broken provider keeps working on its old code. At + startup the same holds: `load` leaves a failing plugin out and resolves with `{ failed: [{ plugin, error }] }`; its + consumers wait `failed { waiting }`; `load` never closes the host for a plugin (base code decides whether its core + failing is fatal). What no rule detects: a consumer relying on new behaviour of a kept provider whose contract did not + change (as in a rollout across hosts). Runtime states (registry conflicts, failed opens, migrations) stay per object + after the commit (`onError` as always). - **Staging** runs every facet's setup into a staged table beside the live one (types with their factory, `linger` and provider, `bindProvider(contract, options.accepts)`; registry providers; facet contributions by plugin), then, for every live, activating, parked or failed object of a changed or removed plugin whose type is still served, its new @@ -1922,18 +1931,30 @@ type ReloadReport = { unchanged server restart reinstalls nothing), serialized installs, a host that starts or reconnects installing the current set before anything is routed to it, a rejected own install failing its own requests with the reason. -### Loading and the dev loop +### Loading plugins and the reload loop (slice 7c) -- Plugins come from built-in, global (`/plugins`), package (npm/git, installed with `--ignore-scripts`) and - project roots, found by directory listing (no source parsing); the server selects; dependencies resolve by contract - name at runtime; names are prefixed (`.`; tools `_`). -- **Loader D:** one base bundle per surface (host packages and the app runtime) plus one script per facet (host - packages passed in, no module map); old generations are reclaimed by the GC; stable filename per (facet, digest) and - V8's compilation cache off on hosts that reload in place. The loader is an adapter: static imports (tests, Durable - Objects) work the same. -- Project plugins, trust, snapshots and scopes: `apps/p2/docs/plugin-loading.md` (still the reference for those). -- **Dev loop:** a source watcher marks plugins dirty (never applies changes); a reload control (playground web shell, - CLI, agent tool) reloads the changed plugins or restarts all hosts for base code. +- **No dev/prod split** (owner, 2026-10-30): a Chord app is self-modifying (the owner or an agent edits a plugin while + it runs), so finding, building, loading, watching and rolling out plugins is core machinery: one component, + `chord/next/plugins` (`src/next-node/plugins/`, Node-side; the kernel stays portable and takes facets however they + were made: static imports in tests and Durable Objects work the same). Reloads stay explicit: the watcher marks, + never applies. +- **A plugin** is a directory with a `package.json` (its `name` is the plugin's name, npm style) and one file per facet + kind the app names; plugins come from roots found by directory listing (no source parsing; the first root wins a + duplicate name). Dependencies between plugins resolve by contract name at runtime; names are prefixed + (`.`; tools `_`). +- **Loader D:** one esbuild script per facet (its own files, copied files of other plugins, its npm `dependencies` + inlined; host packages, `chord.external` packages and Node builtins external, passed in at evaluation); evaluated with + `vm.Script` under a stable filename per (plugin, kind, digest); no module map: old generations are reclaimed by the + GC; V8's compilation cache turned off when the component is created. `pluginFile(path)` resolves a file of the + calling facet's own plugin. A plugin's dependencies are installed into its directory (`installDependencies`, as + coding-agent does: `--omit=dev --legacy-peer-deps --ignore-scripts`). esbuild is an optional peer dependency. +- **The reload loop:** a watcher marks plugins dirty from the files each facet used (a copied contract dirties every + user) and flags changes of base code (the app restarts its hosts); a build is all or nothing; `rollout` runs the app's + steps in order (server, workers, clients; decision "Slice 7b's open points" (3)). A reload control (a button, a CLI, + an agent tool) is the app's. One base bundle per surface (host packages and the app runtime) is an app's startup + optimization, the web's in slice 11. +- Project plugins, trust, snapshots, scopes and npm/git package installs: `apps/p2/docs/plugin-loading.md` (still the + reference for those; not in 7c). - **UI reload vs. leaving:** "leaving" is never derived from an unmount, only from explicit navigation or presence (a dropped reference); a component unmounted by a reload has not left. @@ -2409,8 +2430,7 @@ with their contributor, readable contributions and readers looking up the curren registry keys with a `conflict` flag; `restart_required`; detached references counting toward presence; contracts taken from the running app; ceremony (`native/` directories, `defineNative`, lints); weakening type checking; "magic" input-field matching for delete; a -factory trial; a migrate dry run; principal field patterns and re-stamping for revocation; `storeKey`; a consumer contract check during staging; reload votes; per-plugin rollback within a -host; `defaultProjectTrust: "always"`; loader variants A and B; a topology layer above the host API; a stream type in +factory trial; a migrate dry run; principal field patterns and re-stamping for revocation; `storeKey`; a consumer contract check during staging; reload votes; `defaultProjectTrust: "always"`; loader variants A and B; a topology layer above the host API; a stream type in Chord; callbacks as method arguments; `opaque` members; numeric middleware priorities; host addresses as data naming an input field; host-name strings in app code; host values (start data and platform bindings are core objects); consumer-side linger; registry composition as a staging check; a JSONL document storage; `tx.get` for documents; validating state and documents on commits or reads (migration results are validated once, R8); copying values on same-host calls; op-rewriting projections and a diff --git a/packages/chord/docs/next/slice-7b.md b/packages/chord/docs/next/slice-7b.md index 4dd6d556c..7b7dff063 100644 --- a/packages/chord/docs/next/slice-7b.md +++ b/packages/chord/docs/next/slice-7b.md @@ -45,7 +45,7 @@ type ReloadedObject = { readonly object: string } & ( (decision 4: no install watchdog); its abort before the commit rejects `aborted` and changes nothing; after the commit it only ends the wait. -## 2. Staging (all or nothing) +## 2. Staging (all or nothing; **superseded** 2026-10-30 by "Per plugin at startup and reload": a failing plugin keeps its old code, slice 7c §4) 1. **Setups.** Every facet's setup runs into a **staged** table, beside the live one: served types (with their factory, options and `bindProvider(contract, options.accepts)`, which runs `checkAccepts`), served registries (provider diff --git a/packages/chord/docs/next/slice-7c.md b/packages/chord/docs/next/slice-7c.md new file mode 100644 index 000000000..110378d8e --- /dev/null +++ b/packages/chord/docs/next/slice-7c.md @@ -0,0 +1,157 @@ +# Slice 7c: plugins (spec) + +Status: **ready to build** (owner, 2026-10-30: decisions "Slice 7c's points" and "Per plugin at startup and reload"; +spec reviews by Fable and Opus, `/tmp/s7c-spec-review-{fable,opus}.md`, folded in). 7c builds `chord/next/plugins`: the +code that turns plugin directories into facets, loads and reloads them per plugin, and watches them. A Chord app is +self-modifying (the owner or an agent edits a plugin while it runs), so this is core machinery, not a dev tool (no +dev/prod split). It also changes the kernel's `load` and `reload` to per-plugin failure (§4). HTTP and workers are slice +10, the web page host and the TUI slice 11: 7c builds scripts for every surface and evaluates them on Node; its build +results are plain data those slices ship. + +Read in full before writing code: this spec; `decisions.md` (2026-10-29/30: "Slice 7b's open points, as recommended", +"A plugin's npm dependencies are bundled", "Per plugin at startup and reload", "Slice 7c's points"); `design.md` (§3 +`load`/`reload`, §16, §18, §21 in full); the 7b brief `slice-7b/book.md` (chapters "A Plugin, File by File", "Building +and Loading", "Assets and Files", "Performance", "A Mini p2 from Chord"); the two spec reviews; p2's +`apps/p2/src/hosts/{bundles,plugins}.ts` and `apps/p2/docs/plugin-loading.md` §4.1, §7.1, §8, §9, §14; coding-agent's +`packages/coding-agent/src/core/package-manager.ts` (dependency installs). Code lives in +`packages/chord/src/next-node/plugins/` (Node-only: `fs`, esbuild, `vm`; the kernel in `src/next` stays portable), +exported as `./next/plugins` when v2 gets its entry point; until then tests import the source. + +## 1. A plugin on disk + +```text +plugins/notes/ + package.json "name": the plugin's name (required, npm style); "dependencies": bundled; "peerDependencies": + host packages; "chord": { "external": ["better-sqlite3"] } (optional, Node facets only) + server.ts worker.ts web.tsx tui.ts one file per facet kind the app names; default export: a facet + contract.ts store.ts NoteList.tsx notes.css assets/… plain files + node_modules/ its own npm dependencies (`installDependencies`) +``` + +- The plugin's name is its `package.json` `name`, never the directory name. A directory without a `package.json`, or + with an invalid name, is reported, not a plugin. +- Facet kinds are the app's (`["server", "worker", "web", "tui"]` in p3), each `.ts`, `.tsx`, `.js` or `.mjs`; two + files of one kind are reported (not chosen by extension). A plugin with none of the kinds (contracts only) is a + plugin with no facets: imported by others, never loaded. +- Discovery lists each root one level deep, reads only `package.json`, follows symlinks to their real paths, and keeps + the first root's plugin of a duplicate name (the other reported). + +## 2. Building + +```ts +const plugins = await discoverPlugins(["./plugins", join(dataDir, "plugins")], { kinds: KINDS }, ctx); +const built = await buildFacet(plugins[0], "worker", { hostPackages: HOST_NAMES, platform: "node" }, ctx); +// { kind: "script", plugin, facet: "worker", source, digest, inputs: [{ file, hash }], copied: [file], +// css?: { name, text }, assets: [{ name, bytes }] } | { kind: "failed", plugin, facet, errors } +``` + +One esbuild build per (plugin, kind), from cold (no kept contexts: about 10 ms per facet; p2 measured 237 ms for 25 +worker facets). A failed build is a result, never a throw. The result is plain data (strings, bytes, arrays): the same +value a worker (slice 10) or a page (slice 11) evaluates where it runs. + +| An import of | Becomes | +| --- | --- | +| the plugin's own files | inlined | +| a file of another discovered plugin (`../sessions/contract.ts`) | inlined (a copy), listed in `copied`; it may import only host packages and files of its own plugin, never packages (a build error otherwise): contracts and side-effect-free modules only (decision 4) | +| a package in the plugin's `dependencies` | inlined from its `node_modules` | +| a package in `chord.external` (Node builds) | external, `require`d at evaluation from the plugin's `node_modules`; a browser build fails | +| a host package (`options.hostPackages`, e.g. `@earendil-works/chord/next`, `react`, `react/jsx-runtime`, `@app/core`) | external, from the map at evaluation | +| `@earendil-works/chord/next/plugins` | external always (its `pluginFile` is per facet, §3) | +| a Node builtin (`node:fs`, `fs`) | external (Node builds); a browser build fails | +| CSS (browser builds) | one stylesheet per facet (`css`) | +| an asset (`import url from "./icon.svg?url"`, `.wasm`, browser builds) | a content-hashed file in `assets`; the import is its URL under the app's `options.publicPath` | +| `import.meta.url`, `__dirname`, `__filename`, dynamic `import()`, top-level `await` | a build error pointing to `pluginFile` (§3) | +| anything else (a relative path into base code, a package not in `dependencies`) | a build error naming the file and the rule | + +- Output: `format: "cjs"`, wrapped `(function (require, module, exports) { … })`; TypeScript stripped, JSX automatic + (`react/jsx-runtime`); no type check; minified; an inline source map whose `sources` are relative to the plugin's + directory (so the script and its `digest`, a hash of the source, are the same on every machine and checkout). +- `inputs` are captured where esbuild loads each file (an `onLoad` that hashes the exact bytes it hands esbuild, as p2's + `captureBundleInputs`; the metafile has no hashes), absolute paths, for the watcher. + +## 3. Evaluating (Node) + +```ts +const facet = evaluateFacet(built, { hostPackages: HOST, plugin }); // returns the facet, or throws its error +``` + +- `vm.Script` under a stable filename per (plugin, kind, digest), path-like (`/chord-plugins/@pi/notes/worker.ts` + plus the digest), run in this context; the default export, checked to be a facet (a declaration error otherwise). +- **The `require` it passes answers, in order:** the host-package map (each value is what Node's `require` would give: + a CommonJS package's exports, an ESM package's namespace wrapped once with `__esModule: true`, so default and named + imports both work); `@earendil-works/chord/next/plugins` → `{ pluginFile }` bound to this plugin's directory; + `chord.external` packages and Node builtins through `createRequire(join(plugin.dir, "package.json"))`; anything else + throws naming the facet. +- **`pluginFile(path)`** gives the absolute path of a file inside the plugin's directory; `..` out of it is refused. +- **No module map, no cache:** an old facet's closures are collected once nothing holds them. The component turns V8's + compilation cache off when it is created (`v8.setFlagsFromString("--no-compilation-cache")`, once per process; an + app may pass the flag itself). +- **Stack traces** (decision Q1 (b)): `mapStack(error)` returns the error's stack with plugin scripts' positions mapped + through their source maps (a `module.SourceMap` per evaluated script, kept while the script is) to `notes/worker.ts:12`; + the app's reporters call it. Nothing global (no `Error.prepareStackTrace`, no process-wide source-map switch). + +## 4. Loading and reloading, per plugin (kernel change) + +Decision "Per plugin at startup and reload" changes `host.load` and `host.reload` (slice 7b) and the component's +`load`/`reload` helpers: + +- **`host.load(facets, ctx)`** resolves with `{ failed: [{ plugin, error }] }`: a plugin whose setup throws (or that is + refused at declaration: invalid name, a type or registry served twice, a remote type on a host without principal or + address, `checkAccepts`) is left out and reported; the others publish; its consumers wait `failed { waiting }`. `load` + never closes the host for a plugin (base code decides whether its core failing is fatal). +- **`host.reload(facets, ctx)`**: staging per plugin. A plugin whose new code fails in staging (setup, a factory against + the recording `self`, a declaration error, `checkAccepts`) keeps its loaded facet (as if passed unchanged), and + staging runs again without its new code. **The closure rule:** a changed plugin whose recorded requests (staging + records them with their contract versions) would not bind because the plugin they need was kept old, or is missing, + is kept old too (repeat until stable). The report gains `kept: [{ plugin, error }]`; the commit applies the rest. +- **The component** builds and evaluates per plugin: a plugin whose build fails or whose evaluation throws is passed to + the host as its loaded facet (kept) at a reload, left out at startup, and reported with the host's report. + `plugins.load(host, ctx)` and `plugins.reload(host, names, ctx)` keep, per host, the facet loaded for every plugin, so + `reload` always passes the complete set (unchanged plugins as the same facet object, removed ones left out) and never + restarts an unchanged plugin. + +## 5. Watching and rollout + +```ts +const plugins = createPlugins({ roots: ["./plugins"], kinds: KINDS, hostPackages: HOST, base: ["./src"] }, ctx); +plugins.state // a JSON watchable: { dirty: ["@pi/notes"], install: ["@pi/notes"], set: false, base: false } +const next = await plugins.build(["@pi/notes"], ctx); // built results per (plugin, kind), plain data +await rollout([serverStep, ...workerSteps, ...clientSteps], next, ctx); // the app's order (decision 3) +``` + +- **The watcher** (`fs.watch` recursive over roots and `base`) ignores `node_modules` and `.git`. A changed file marks + dirty every plugin whose facets used it (`inputs`; a copied contract dirties every user) or whose directory contains + it; a changed `package.json` marks `install`; a plugin directory added or removed sets `set` (discovery runs again + at the next build); a file under `base` sets `base` (the app restarts its hosts; Chord restarts nothing). A change + during a build keeps its mark. If the watcher cannot watch (the system's limit), it reports once and the app builds + on demand. It marks, never applies. +- **`installDependencies(plugin, ctx)`**: `npm install --omit=dev --legacy-peer-deps --ignore-scripts --no-audit + --no-fund` in the plugin's real directory, the command overridable (decision 3); it clears `install`. +- **`rollout(steps, built, ctx)`**: each step is `{ name, kinds, reload(built, ctx) }` and returns the host's report; + steps run in order; a step's `reload` evaluates the facets of its kinds where its host runs (an in-process step through + the component's per-host state, §4; a remote one, slice 10/11, gets the built data). It stops at a step that rejects + (a host closed, an abort); kept plugins do not stop it. Marks clear for plugins applied on every step; a plugin kept + on any host stays dirty. + +## 6. Not in 7c + +Installing plugins from npm or git (p2 §8), project plugins with trust and snapshots (design §26 #14), a base bundle per +surface, evaluating in a browser (slice 11), shipping built scripts to workers and pages (slices 10, 11), static checks +of cross-plugin imports beyond §2's rule. + +## 7. Tests, examples, cost + +- One test file per source file; fixture plugins under `test/next-node/fixtures/plugins/`. Discovery (names, duplicates, + missing or invalid `package.json`, two files of a kind, contracts-only plugins, symlinks); build (every row of §2's + table, failures as results, relative source maps and stable digests across two checkout paths, inputs hashed at + load); evaluation (host packages as the host's instances: `instanceof ChordError` across a plugin; default and named + imports of ESM and CommonJS host packages; `pluginFile` and its `..` refusal; an import-time throw; a non-facet + export); `mapStack` naming plugin files and lines; per plugin at startup and reload (§4: a failing setup, factory, + declaration, build, evaluation; the closure rule with a provider and consumer changed together; the report's `failed` + and `kept`; `load` never closing the host); the watcher (dirty marks from inputs and directories, `install`, `set`, + `base`, a change during a build, `node_modules` ignored); `rollout` (order, stopping at a rejecting step, marks). +- **GC:** 200 reloads of an evaluated plugin facet: old closures collected, heap flat after warm-up. +- **End to end:** a server and a worker (memory links) loading plugin directories, a plugin edited on disk, built and + rolled out (server, then worker), a holder seeing one frame; a broken edit kept old on both hosts and reported. +- Example `35-plugins.ts`. Benchmark (`benchmarks.md`): cold build of a 20-plugin set, a rebuild of one facet, + evaluation per facet, heap after 100 reloads. diff --git a/packages/chord/docs/next/slices.md b/packages/chord/docs/next/slices.md index 6fbb544d5..8960bc67d 100644 --- a/packages/chord/docs/next/slices.md +++ b/packages/chord/docs/next/slices.md @@ -56,7 +56,7 @@ telemetry last, since it instruments the app kit too). | 8 | **Registries and `host.root`** | registries, `self.writable`, `retryOn`, Durable's `registryReader`, `host.root` (`chord(fn, context)` and the harness object are p3's base code) | | 7a | **The dependency model** | requests that outlive providers, parked attempts without timers, wakes, the publication gate, getters, registries as dependencies, `parked_long` (`slice-7a.md`, `dependencies/model.md`) | | 7b | **Reload in the kernel** | `host.reload`: staging, commit, rollback; swaps, restarts, removal and re-serving; holds and `accepts` across a reload; jobs and registries across a reload; per-name wakes (`slice-7b/` brief) | -| 7c | **Loader and dev** | `chord/next/loader` (discovery, `buildFacet`, `evaluateFacet`, `pluginFile`, dependencies) and `chord/next/dev` (watcher, reload control across hosts) | +| 7c | **Plugins** | `chord/next/plugins` (discovery, `buildFacet`, `evaluateFacet`, `pluginFile`, `installDependencies`, the watcher, `rollout`, `mapStack`; no dev/prod split; the kernel's `load`/`reload` per plugin; `slice-7c.md`) | | 10 | **HTTP and workers** | `chord/next/http` (server, `Http` registry, responses as descriptions, WebSocket entry) and `chord/next/workers` (a worker per address, IPC, idle stop) | | 11 | **Web and TUI** | `chord/next/web` (page host, React hooks) and `chord/next/tui` (launcher, sandbox), from p2's code | | 9 | **Telemetry** | spans, events, metrics across hosts |