diff --git a/packages/chord/CHANGELOG.md b/packages/chord/CHANGELOG.md index f475c4c8a..d0c9d0eff 100644 --- a/packages/chord/CHANGELOG.md +++ b/packages/chord/CHANGELOG.md @@ -14,6 +14,7 @@ ### Added - Chord v2 has a package export again, `@earendil-works/chord/next` (experimental): every layer's API for apps (contracts, data, documents, hosts, jobs, links, state, the store), and, marked internal, the store's surface for adapters (`StoreOwnerTables`, `OwnerTable`, the documents engine's reads and writes, the commit step, `applyBatch`). v2's store gains one id allocator for every id in it, claims per table and per document namespace, Durable's Storage semantics for a refused commit (it rejects that commit only; an unknown outcome stops the store), document histories `none`, `latest` and `rewindable` with reads at a point, archives carrying document revisions, and batch apply; v2 objects gain `self.follow` (another runtime's synchronous observers as sources) and persisted jobs (`persist: { resume }`, a stored `abort(id, ctx)`, deferred state changes). +- Chord v2 (`@earendil-works/chord/next`) gains registries, writables, wake signals and root scopes (slice 8): `defineRegistry(name, { version })` tokens shared by plugins, served by the defining plugin's facet with `host.serve(R, { compose, accepts? })`; contributions from facets (`defineFacet((host, self) => self.contribute(R, value))`) and host-lifetime objects (`StoreSelf.contribute`), plain values or Chord watchables, in a table sorted by plugin name and composed lazily (once at the end of `load`'s setups, then when read or followed, in the batch that changed the table); readers through `self.registry(R)` (a declared dependency: an object activates once the registry has a value, and fails `transient` with the new reason `unavailable/undeclared` when nobody serves it) and `host.registry(R, ctx)`, a `RegistryValue` with `observe(observer, ctx)`; older contributions and readers adapted by `accepts`; the helpers `keyedEntries` and `orderedEntries`; the `onError` kinds `registry_conflict`, `registry_failed`, `contribution_failed` and `contribution_refused`. `self.writable(initial)` is an in-process value an attempt sets (identity compared); native sources (writables, registry values, `ref.state`) are derive sources. `self.retryOn(...sources)` retries a failed object when a registry value changes. `host.root(fn, context, { principal? })` runs code outside objects with a handler's `ctx`, sealed when `fn` settles, `context` aborts or the host closes. - Chord v2, experimental and not exported yet: the next object runtime in `src/next` (contracts, hosts and objects, documents, derives, links and relays, routines and jobs, the store), with its tests and runnable examples in `test/next` and its design in `docs/next` (`design.md`, the decision log `decisions.md`, the slice plan `slices.md`). Its API changes with every slice until it gets an entry point. - `NativeWatchable`: an in-process value that changes and can be watched (`value`, `watch(context)` returning a `NativeWatchHandle`), the in-process counterpart of `Watchable`. - `WatchHandle` and `WatchEnd`, the one watch API, moved from Durable: a serialized observation with `value`, `start(listener)`, `stop()`, and `closed`; the listener gets a `WatchChange` (`{ value, ops, retired }`) and the context. `WatchEnd` reasons are `stopped`, `cancelled`, `ended`, `listener_error`, and `failed`. diff --git a/packages/chord/docs/next/benchmarks.md b/packages/chord/docs/next/benchmarks.md index 8e63d8d36..916c70838 100644 --- a/packages/chord/docs/next/benchmarks.md +++ b/packages/chord/docs/next/benchmarks.md @@ -583,3 +583,53 @@ each; the head lookup and scans under it are at parity or faster in the micro-be memory, 55.5 µs persisted (two commits: the record and state created, then the reservation); a state change inside `run` 2.7 µs in memory, 21.3 µs persisted (one delta commit, as the gate's (c)), 3.2 µs deferred (stored with the outcome). + +## Slice 8: registries and `host.root` (2026-10-28) + +Node 24.4.1, one CPU (`taskset -c 6`), runs alternating. HEAD is `391e83fb1`, extracted at a path of the same length +(`/home/badlogic/pi-h8`); the build was measured in the worktree and, where the worktree itself skewed a result (below), +in an extract of the same sources at `/home/badlogic/pi-x8`. + +**Paths under every object** (`test/next/bench/calls.ts`, new: runs on HEAD as it is), median per operation over nine +bursts, heap allocated per operation, three alternating runs each, HEAD → build: + +| Case | HEAD | build | +| --- | --- | --- | +| local method call (its ctx) | 2.80-2.81 µs, 6,894 B | 2.81-2.83 µs, 6,894 B | +| commit through a watched derive (`batch.compute`) | 10.34-10.43 µs, 15,312 B | 10.46-10.51 µs, 15,310 B | +| activation churn: get, live, release (an object with params) | 57.5-59.3 µs, 43,696-43,717 B | 58.1-58.9 µs, 43,782-43,787 B | +| retained per live object (10,000 kept) | 6,586 B | 6,602 B | + +The attempt keeps its registry reads, contributions, wake signals and writables in one record made by the first such +declaration, and iterates nothing when there is none: the first build allocated 1,020 B more per activation (six +arrays per attempt, an array iterator per `for … of` over an empty list, a copy from `splice`) and retained 72 B more +per live object; now +70 B allocated and +16 B retained (one field on the attempt, one on the object). + +**Existing benchmarks**, HEAD → build: the derive diamond (`derives.ts`, three runs) 9.4-9.8 → 9.5-9.9 µs median; its +pooled p95 16.5-17.1 → 18.4-20.8 µs comes from the run in which V8 tiers the commit path up (per run, after it: HEAD +10.7-11.3 µs, build 10.1-11.3 µs; the first measured runs 15.0-25.6 µs on either side, in another run each), the same +with HEAD's `batch.ts` and `derive.ts` in the build. The store gate (`store.ts`, three runs): every median within +0.4 µs (Chord store 13.0-13.9 → 13.0-14.2 µs, Chord host 17.3-18.2 → 17.3-17.8 µs, in-memory document 2.1 → 2.2 µs), +heap 3,498 → 3,144 KiB. Jobs (`jobs.ts`): start 8.4 → 8.5 µs in memory, 55.4 → 56.2 µs persisted; state change 2.7 → +2.7, 21.3 → 21.8, 3.4 → 3.5 µs. Links (`links.ts`, three runs): wire sizes, encodes, decodes and heap per message +unchanged; its round trips are one cold burst each and vary as much between HEAD runs (commit to frame, text codec, +0.055-0.076 ms). A warmed remote call (2,000 calls per burst, nine bursts, a memory link): binary 7.16-7.24 → 7.17-7.29 +µs, text 7.66-7.72 → 7.70-7.83 µs in the extracts; the worktree itself runs the same sources at 7.8-7.9 µs (binary), +an effect of the tree, not the code. + +**Registries and writables** (`test/next/bench/registries.ts`, new), median over nine bursts: `writable.set` delivered +to one watch 0.81-0.91 µs (2.9 KB allocated: the batch and the watch's call); a set of the same value 0.008 µs, 0 B +(one comparison: its batch work is a method of its own, so `set` allocates no closure context); one of 10 contributions +changed while the registry is watched 0.74-0.77 µs, of 100 1.52-1.59 µs (the table built again over every entry, one +compose per set: the counted `chord.registry.composes` is 1.00 per set). + +**Fix round 1** (2026-10-28), HEAD extract → build extract (`/home/badlogic/pi-x8`), CPU 6. `calls.ts`, two runs: local +method call 2.80 → 2.80-2.81 µs, 6,894 B both; commit through a watched derive 10.51-10.65 → 10.72 µs; activation churn +57.9-59.0 → 59.3-60.0 µs, 43,714 → 43,769-43,787 B. The derive diamond (five runs): median 9.4-9.8 → 9.5 µs, pooled p95 +16.4-16.7 → 19.0-19.2 µs. Per run, both have one slow run while V8 tiers the commit path up (HEAD its second, median/p95 +15.5/22.6-22.9 µs; the build its first, 16.4/26.7-27.2 µs) and equal runs after it (HEAD 8.8-9.6/9.6-11.2 µs, build +8.9-9.8/9.7-10.6 µs). `batch.compute` is not the cause: HEAD's `batch.ts` and `derive.ts` in the build give the same +p95, and swapping host files moves it without a pattern (code layout, not work on the path). `registries.ts`: +`writable.set` 0.87 µs, the same value 0.011-0.016 µs and 0 B (the seal check now comes first: a set of an ended +attempt's writable throws whatever its value); one of 10 contributions set 0.71-0.78 µs, of 100 1.43-1.61 µs (one run +in six 3.4 µs on either variant tried: JIT). diff --git a/packages/chord/docs/next/design.md b/packages/chord/docs/next/design.md index ba7b8378b..b3ec994d1 100644 --- a/packages/chord/docs/next/design.md +++ b/packages/chord/docs/next/design.md @@ -207,7 +207,8 @@ parent link closes. `resolve` returns the link before the worker has loaded (the | `hold(Type, params, context, { readyTimeout? })` | a reference at once that follows its object until released (section 7) | | `close(context)` | closes the links (children, routes, then the parent), then ends every object at once and tears them down, the last to go live first (reason `close`); rejects with an `AggregateError` of the native closes that threw | | `revoke(r, context)` | revocation (section 4; slice 4b) | -| `root(fn, …)` | a call scope for code outside objects (a Durable hook, a CLI); built in slice 8 (**decision needed** on its signature, section 26) | +| `root(fn, context, { principal? })` | a call scope for code outside objects (a Durable hook, a CLI): `fn` gets a handler's `ctx`, sealed when `fn` settles, `context` aborts or the host closes (slice 8 D3, section 8) | +| `registry(R, context)` | registry `R`'s composed value for code outside objects; waits for `load`, rejects when it has none (slice 8, section 17) | **Options of `createHost`:** `address`, `principal`, `principalKey` (section 4; the same function on every host), `telemetry` (section 29), `onError`, `closeTimeout`, `readyTimeout`, `heartbeat: { ping?, close? }`; later `revokedBy` @@ -216,7 +217,7 @@ and closed by the app with `store.close(ctx)` after `host.close`; the host never background, as a tagged `HostError`: `listener_error` (a watch listener threw), `delivery_failed` (a frame delivery threw), `activation_failed { object, failure }`, `close_failed { object, native, error }`, `close_timeout { object, native }`, `drain_timeout { object }`, `protocol_error { error }`, `resolve_failed { address, error }`; later routine -errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6) and `storage_failed { error }` +errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), the registries' `registry_conflict { registry, message, plugins }`, `registry_failed { registry, error }`, `contribution_failed { registry, plugin, type?, error }` and `contribution_refused { registry, plugin, type?, version }` (each once per episode, slice 8), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6) and `storage_failed { error }` (slice 6a), which every app must handle: Chord never ends the process, the app exits with its own code for its supervisor or reopens. @@ -546,7 +547,7 @@ host.serve(Notes, ({ session }, principal, self) => link protocol (adding one bumps the protocol version; a receiver rejects unknown reasons; a test pins the list): `invalid` (`args`, `params`, `member`, `placement`, `reentrant`, `read_after_bulk`, `cycle`), `contract_mismatch` (`updating`), `forbidden` (`revoked`), `gone`, `not_ready`, `released` (`ended`, `closed`, `settled`, `version`, `discarded`; none for a holder's own release), `too_large` (`part`), `unavailable` (`locked`, - `park_timeout`, `busy`, `storage_failed`), `aborted`. A failed check names TypeBox's instance path ("unknown field", "missing", else TypeBox's + `park_timeout`, `busy`, `storage_failed`, `undeclared`: a registry no plugin serves here, slice 8), `aborted`. A failed check names TypeBox's instance path ("unknown field", "missing", else TypeBox's message; keys containing `/` or `~` make the path ambiguous, documented). Plugins throw `PluginError(code, message, details?)` for codes of their own; anything else crosses a link as the plugin code `internal` with its message, never a stack or a cause; plugin errors never use Chord's codes. An error rebuilt from the wire is not copied and only its @@ -1505,7 +1506,7 @@ host.serve(Http, { // the defining p accepts: { 1: { contribution: (old: HttpContributionV1) => upgrade(old), value: (router) => routerV1(router) } }, }); self.contribute(Http, { routes: { … }, wraps: [ … ] }); -const router = self.registry(Http).value; // live (host.registry(Http) outside objects) +const router = self.registry(Http); // a declared dependency; read router.value at use (await host.registry(Http, ctx) outside objects) ``` - Each contributor (a facet or an object) contributes **at most one value per registry**; the table is an immutable, @@ -1521,7 +1522,9 @@ const router = self.registry(Http).value; // live (host.registry(Http) outsi a plain value and never handle conflicts. If `compose` throws, the failure is reported and its watches get a `failed` call, while `.value` keeps the last good composition (slice 8); only before any good composition do readers get the error. Work that started earlier keeps the value it read (a pinned Durable phase keeps - its old table). Helpers: `keyedEntries(table, entriesOf, report)`, `orderedEntries(table, itemsOf, orderOf?)`. + its old table). Helpers: `keyedEntries(table, entriesOf, report)` (a map, the first by plugin name wins, a duplicate + reported), `orderedEntries(table, itemsOf, orderOf?)`; both keep each item's plugin (named so beside the contracts' + `keyed`, slice 8 build). - **Who contributes (R21):** only facets and host-lifetime objects (`local`, empty params); their factory's `self` has `contribute`, a referenced object's `self` does not (a type error). Reading a registry's composed value stays open to every object. - **A contribution is in the table exactly while its contributor is live:** a facet's while its plugin is loaded; an @@ -1962,14 +1965,20 @@ loop. Chord has no stream type (it would need its own rules for drops, cancellat loading object (refines R17). A missing tool gives its call an error result the model sees; a missing model makes the generation report "model unavailable" and wait (not fail) until a model contribution appears; a task whose definition is missing stays blocked (`missing_task`) until the plugin is installed or the task is aborted, as Durable does. +- **Durable's views from other objects** (slice 8 §5): the harness object contributes its `Native` to a + registry of p3's; another object declares that registry (`self.registry`), wakes on it (`self.retryOn`) and follows + a view through its value (`self.follow`). A Session close ends Durable's synchronous observers `failed` ("session + closed"; decision "Slice 8 S1"; app watches still end `ended`), so the follower fails `transient` and follows the + next Harness once its contribution arrives. `registryReader`'s listener runs inside Chord's batch: it only wakes + Durable's scheduler (a Chord call there throws `invalid/reentrant`). - `HarnessModels` and `HarnessEnvironments` start with one contribution each, from the models and envs plugins' objects that own the natives; several providers or environment kinds are their `compose`. After a models reload the next request uses the new runtime; a running stream finishes on the old one. - **Chord from Durable code (Q6):** tools, hooks and sections get Durable's `ToolExecutionApi` and a `Context`, never a Chord `ctx`. p3's core opens the harness with a base context holding the host under a typed key (`withContextValue(ChordHostKey, host, …)`); every phase context inherits it. A core helper opens a call scope from - any Durable context: `chord(async (ctx) => …, context)` = `host.root(fn, { signal: context.abortSignal, telemetry: - context.telemetry })` (host principal; attribution as data; references released when `fn` settles). No wrapping of + any Durable context: `chord(async (ctx) => …, context)` = `host.root(fn, context)` (host principal; attribution as + data; references released when `fn` settles; slice 8 D3). No wrapping of tools and no p3 `defineTool`. Clients are never reachable this way (they are not addressable): a tool that needs the user goes through an owner's state, e.g. the Questions service (`apps/p2/docs/questions.md`): the model-facing `ask_user` tool and any other tool call `Questions.ask`/`wait`, tabs render the open questions and `answer` them. diff --git a/packages/chord/docs/next/slice-8.md b/packages/chord/docs/next/slice-8.md index 64c682534..65f1c494f 100644 --- a/packages/chord/docs/next/slice-8.md +++ b/packages/chord/docs/next/slice-8.md @@ -1,6 +1,7 @@ # Slice 8: registries and `host.root` (spec) -Status: **ready to build** (2026-10-27; D1-D3, the registry failures and S1-S3 decided by the owner; two spec reviews, +Status: **built** (2026-10-28; build review and two fix rounds by the same four reviewers, +`/tmp/s8-build-review-*.md`, `/tmp/s8-build-rereview-*.md`). Before: ready to build (2026-10-27; D1-D3, the registry failures and S1-S3 decided by the owner; two spec reviews, `/tmp/s8-review{,2}-{astra,fable,opus,sol}.md`, found no remaining blocker; S3 refined after the second: with no value after `load`, a reader fails at once, as `self.get` of a type not served here) diff --git a/packages/chord/docs/next/slices.md b/packages/chord/docs/next/slices.md index 87d395336..cf346eb9e 100644 --- a/packages/chord/docs/next/slices.md +++ b/packages/chord/docs/next/slices.md @@ -27,15 +27,16 @@ slice spec's section citations are for orientation, not a reading limit (owner, - **L1** (`src/next/contracts`, `src/next/data`): contracts, params and identity, argument checks, principal keys, errors. `index.ts` for apps, `internal.ts` for Chord's own code. -- **Slices 1, 2, 3a, 3b, 4a, 4b, 5, 6a, 6b** (committed; 5 `4a5b45c45`, 6a and 6b 2026-10-27/28): `src/next/documents`, `state`, `host`, `jobs`, `link`, - `remote`, `store`; examples 01-25; benchmarks in `benchmarks.md`. +- **Slices 1, 2, 3a, 3b, 4a, 4b, 5, 6a, 6b, 8** (committed; 5 `4a5b45c45`, 6a and 6b 2026-10-27/28, 8 2026-10-28): `src/next/documents`, `state`, `host`, `jobs`, `link`, + `remote`, `store`, `registries`; examples 01-30 (Durable's 35-37); benchmarks in `benchmarks.md`. - **Chord's shared code** (upstream pi): `src/watch.ts` (`Watch`, `NativeWatch`, `WatchHandle`, `WatchEnd`, `NativeWatchable`), Delta (`track`, `diffRevisions`, `applyImmutable`, encoders, `PendingOps`, `appendText`), context, JSON, output, SQLite drivers. ## The slices -Order (owner, 2026-10-22: routines and jobs before the store and reload). +Order (owner, 2026-10-22: routines and jobs before the store and reload; 2026-10-27: slice 8 before slice 7, whose +reload has to reload real registries). | # | Slice | In short | | --- | --- | --- | @@ -48,8 +49,8 @@ Order (owner, 2026-10-22: routines and jobs before the store and reload). | 5 | **Routines and jobs** | `self.routines`, `self.job` | | 6a | **The store** | the store (backends, tables, queries, writes, commit queue, outbox, documents engine, migrations, export), persisted documents, `self.table`, keyed watches and retirement | | 6b | **Durable and jobs on the store** | persisted jobs, Durable's `StoreStorage`, the Chord source over Durable's views (`self.follow`) | +| 8 | **Registries and `host.root`** | registries, `self.writable`, `retryOn`, Durable's `registryReader`, `host.root` (`chord(fn, context)` and the harness object are p3's base code) | | 7 | **Reload and rollout** | plugin loading, staging, swaps and restarts, revisions, `accepts` in use | -| 8 | **Registries and `host.root`** | registries, `self.writable`, `retryOn`, a `RegistryReader` for Durable, `host.root` (`chord(fn, context)` and the harness object are p3's base code) | | 9 | **Telemetry** | spans, events, metrics across hosts | The playground (`apps/chord-playground`) moves to the new code once slice 4 exists. Round-3 cuts (C3-C16) and @@ -211,14 +212,13 @@ deleted (the slice's spec and code are the record). Moved here from `carry-over. internal ones are rebuilt; add matching only if a real need appears. (W1.) - **Retrying broken failures:** reload retries a broken object as `ref.retry()` does (`ChordObject.retry`: the backoff skipped, an attempt now; holders are told `loading`). (Slices 2, 3b.) - -### Slice 8: registries and `host.root` - -- **Durable's views from other objects** (from slice 6b): the harness carried in a registry entry, so an object other - than the one that opened Durable can `self.follow` its views. - -- Specified in `slice-8.md` (decided 2026-10-27): `host.root` (D3), `retryOn` (D2), registries recomposing in the - owner's flush (W4); `host.activity` on the facet's host dropped (nothing needs it). +- **Registries on reload** (from slice 8 §6): a reload replaces a plugin's contributions and a provider atomically, in + one batch (staging composes nothing, decision D1 of "Review round 1"); the record keeps the provider, facet + contributions by plugin and object contributions by object apart for it (`RegistryRecord.serve` refuses a second + serve today). Older registry readers across a reload of the provider (their mapped values follow the new + `accepts[old].value`). A `writable` across a pure swap: the new factory makes a new `self.writable(initial)`, so a + contribution goes back to its initial value until the restarted routine sets it (MCP's tools would vanish briefly). + `retryOn` watches across a reload (a reload retries failed objects anyway). (Slice 8 spec.) ### Slice 9: telemetry diff --git a/packages/chord/src/next/data/errors.ts b/packages/chord/src/next/data/errors.ts index 697f3b578..623c05fe2 100644 --- a/packages/chord/src/next/data/errors.ts +++ b/packages/chord/src/next/data/errors.ts @@ -18,7 +18,8 @@ export const CHORD_ERROR_REASONS = { // `discarded`: a transaction cut at the seal. released: ["ended", "closed", "settled", "version", "discarded"], too_large: ["part"], - unavailable: ["locked", "park_timeout", "busy", "storage_failed"], + // `undeclared`: a registry no plugin serves on this host. + unavailable: ["locked", "park_timeout", "busy", "storage_failed", "undeclared"], aborted: [], } as const; diff --git a/packages/chord/src/next/host/README.md b/packages/chord/src/next/host/README.md index 2474ca16a..da13258bc 100644 --- a/packages/chord/src/next/host/README.md +++ b/packages/chord/src/next/host/README.md @@ -4,8 +4,10 @@ Hosts, facets, objects, natives and references: hosts serving `local` and `remot principals and variants, links to children, to a parent and over routes to addressable hosts (with relays between them), the heartbeat, the `ctx` of method handlers, (slice 4b) revocation, cycles across hosts, presence, keep-alive counts, size limits per part and compression at the origin, (slice 5) routines and the objects' jobs -(declared here, built in `jobs/`), and (slice 6a) the host's store: tables, persisted documents and eager table -migrations at activation (built in `store/` and `documents/`), and live objects that fail. +(declared here, built in `jobs/`), (slice 6a) the host's store: tables, persisted documents and eager table +migrations at activation (built in `store/` and `documents/`), and live objects that fail, and (slice 8) registries +(served, contributed and read here, built in `registries/`), writables, wake signals (`retryOn`) and root scopes +(`host.root`). ## For apps (`index.ts`) @@ -189,8 +191,41 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded: route's `resolve` rejected; the address waits its backoff), `routine_failed` (`object`, `routine`: its position, `error`: a routine call threw, or its watch could not be taken; a source's failure calls never reach a routine), `job_failed` (`object`, `job`, `id`, `error`: a run threw, not after an abort), `job_finish_failed` (the same: a `finish` threw; it is retried) and - `job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end). A - throwing `onError` is ignored. + `job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end), and + the registries' (slice 8, each once per episode) `registry_conflict` (`registry`, `message`, `plugins`: a conflict + `compose` resolved), `registry_failed` (`registry`, `error`: `compose` or an older version's `value` adapter threw), + `contribution_failed` (`registry`, `plugin`, `type?`, `error`: a watchable contribution failed or ended, or its + adapter threw) and `contribution_refused` (`registry`, `plugin`, `type?`, `version`). A throwing `onError` is + ignored. +- **Registries** (slice 8, `registries/`): a facet serves a registry it defines with `host.serve(R, { compose, + accepts? })` (one provider per host; a second serve throws at `load`); a facet's `self.contribute(R, value)` (its + setup's second argument, `FacetSelf`) is in the table while the plugin is loaded, a host-lifetime object's (`local`, + empty params: `StoreSelf.contribute`; any other factory throws, R21) from `live` until it fails, ends or restarts, + entering before anyone sees it live and leaving at its seal. One value per registry each; a Chord watchable (a + source, a constant, a document handle, a reference member, a writable, a registry value) is followed through the + batch, anything else is a plain value whatever methods it has (registries carry natives); a registry value closing + a cycle of registries (through the tables and the factory's own earlier contributions) is refused `invalid/cycle` + at `contribute` (an edge lives as long as its contribution is in a table). Every served registry composes once at the end of `load`'s setups, before any object activates. + `self.registry(R)` is a declared dependency: with no value at activation (no provider, a first `compose` that threw, + a version not served here) the attempt fails `dependency_failed` (`transient`, the registry's error as the cause, + the references kept) at once, and the registry's first value retries it (R18); a version the provider does not + serve fails it `broken` (`contract_mismatch`: only a reload helps). `host.registry(R, ctx)` waits for `load` and + rejects without a value. +- **Wake signals** (`self.retryOn(...sources)`, D2): registry values (constants are accepted and never wake; anything + else throws: a reference's member, the attempt's writable, `ref.state`). The object follows them from the factory's + return (before hydration, references and natives, so a failed open still has them) until the next attempt's + declaration replaces them or it ends. A new value retries a failed object (`transient` or `broken`) as `ref.retry()` + does, after the batch; one while it activates is remembered and retries it at once if that attempt fails; one while + live is ignored; failure calls wake nothing; a composition made by the object's own contributions leaving (it + failed live) does not wake it. There is no backoff between wakes: wake an object on sources that change rarely + (models, a harness), not on every event (a writable set per message retries a failing object per message). +- `host.root(fn, ctx, { principal? })` (slice 8 D3): `fn(ctx)` with a handler's `Ctx

`: the host's principal or the + given one, this host's incarnation, `ctx`'s telemetry, a signal linked to `ctx`'s, `ctx.get`. When `fn` settles, + `ctx` aborts or the host closes, the scope is sealed: its references released (their calls reject `released`), a + waiting `ctx.get` gives up, a later one is refused `released/settled`, its signal aborted. An abort rejects + `host.root` at once (an uncooperative `fn` runs on and fails at its next use of Chord); an aborted `ctx` runs no + `fn`; `host.close` rejects it at once `released/closed`, as a method's caller when its object ends; after `close` + it rejects `released/closed`. Not counted in `host.activity`. - `host.load([{ name, facet }], ctx)`: once per host; runs every setup, then activates every type with empty params and resolves when each has had its first outcome, live or failed (the start barrier; later retries never hold it). A setup that throws rejects it with that error and closes the host as `close` does, its links included. Gets wait @@ -281,7 +316,8 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded: `broken`; a live object whose read fails so fails while live (`object_failed`), keeping its documents for the next attempt. - A factory `(params, self) => implementation` declares with `self.state`, `self.derive`, `self.get`, `self.open`, - `self.on`, `self.onLive` and `self.job` (`jobs/`), and writes with `self.tx`. It runs once per attempt. The implementation has exactly the contract's members: state + `self.on`, `self.onLive`, `self.job` (`jobs/`), `self.registry`, `self.writable` and `self.retryOn` (below), and + writes with `self.tx`. It runs once per attempt. The implementation has exactly the contract's members: state members are JSON watchables (document handles, derives, reference members, constants), methods `(args, ctx) => Promise`, or `(ctx) => Promise` for a method without arguments (the reference omits them too: `notes.clear(ctx)`). A remote factory `(params, principal, self) => ({ variant, members })` gets the principal its @@ -456,8 +492,9 @@ caller), `Placement` (where a reference or an arriving hold goes: here, a route' served here is placed here), `Route` (one per routed kind: per address its `Path`, at most one running `resolve`, the backoff per address; an address is kept while it has holders, a link or a running `resolve`), `ChordObject` (one per identity: its state across attempts, holders, retries, cycle checks with its instance id and -the chain it was created for, lingering, ending and revocation, and the count of its running and finishing job runs, -which keeps it like a reference: `countRuns`), `Activity` (`host.activity`: counters, one +the chain it was created for, lingering, ending and revocation, the count of its running and finishing job runs, +which keeps it like a reference: `countRuns`, its contributions while live and its wake signals), `RetryWakes` (an +object's `retryOn` subscriptions), `RootScopes` (the open `host.root` scopes, sealed once each), `Activity` (`host.activity`: counters, one publication per microtask turn), `References` (one type's presence list), `RevocationKeys` (each principal's keys, in a `WeakMap`), `Attempt` (one activation: it is the @@ -471,9 +508,9 @@ native: opened once, closed once, a late value closed at once; `within` bounds a windows per identity and when each ends, holding no object), `Binding` (a reference's link to its object or remote object: its state, the object that declared it, its per-reference watches, calls, parked calls and the waits for a state; released, it keeps only its name), `ParkedCalls` (a reference's queue of calls made while loading, a timer -each), `ReferenceState` (`ref.state`: its native watches, told of each change), `CallContext` (a handler's -`ctx`), `withLinkedAbortSignal` (a call's or a get's context aborted by the caller and by a long-lived signal, its +each), `ReferenceState` (`ref.state`, a native source: its native watches, told of each change; derives over it +marked), `CallContext` (a handler's or a root scope's `ctx`), `withLinkedAbortSignal` (a call's or a get's context aborted by the caller and by a long-lived signal, its reference's or its attempt's, linked only until it settles: `AbortSignal.any` keeps cleanup entries per call while its -sources live before Node 24.16.0), `checkImplementation` (the variant and member check), `setUp` (runs one facet's setup) and -`resolveWatchable` (the member check, Q14), and `nextDelay` (the 1 s to 60 s doubling every backoff shares). Cycle checks follow the waits of activating objects with a visited set, +sources live before Node 24.16.0), `checkImplementation` (the variant and member check), `setUp` (runs one facet's setup), +`resolveWatchable` (the member check, Q14), `contributionOf` (a contribution as Chord follows it, the cycle check), and `nextDelay` (the 1 s to 60 s doubling every backoff shares). Cycle checks follow the waits of activating objects with a visited set, linear in the waits. diff --git a/packages/chord/src/next/host/attempt.ts b/packages/chord/src/next/host/attempt.ts index b6d04a898..259b6a0cf 100644 --- a/packages/chord/src/next/host/attempt.ts +++ b/packages/chord/src/next/host/attempt.ts @@ -8,7 +8,7 @@ import type { JsonValue } from "../../json.ts"; import type { NativeWatchable, Watchable, WatchEnd } from "../../watch.ts"; import type { Contract, Implementation, ParamsOf } from "../contracts/index.ts"; import { contractData, type ShapeOf } from "../contracts/internal.ts"; -import { ChordError, type FailureCause, type Principal } from "../data/index.ts"; +import { ChordError, type FailureCause, isChordError, type Principal } from "../data/index.ts"; import { asChordError, checkArgs, checkParams, invalid, openFailure } from "../data/internal.ts"; import type { HandleOf, JsonObject, StateDefinition, Tx } from "../documents/index.ts"; import { @@ -21,7 +21,10 @@ import { } from "../documents/internal.ts"; import type { JobDefinition, JobHandle, JobImplementation } from "../jobs/index.ts"; import { type AnyJobImplementation, Job } from "../jobs/internal.ts"; -import { Derive, type DeriveSource } from "../state/internal.ts"; +import type { Contribution, Registry, RegistryValue } from "../registries/index.ts"; +import { type RegistryContribution, type RegistryEdge, RegistryValueSource } from "../registries/internal.ts"; +import type { Writable } from "../state/index.ts"; +import { Derive, type DeriveSource, WritableSource } from "../state/internal.ts"; import type { Columns } from "../store/columns.ts"; import { tableSqlName } from "../store/sql.ts"; import type { ChordStore } from "../store/store.ts"; @@ -32,7 +35,7 @@ import { CallContext, type Caller, type Ctx, withLinkedAbortSignal } from "./ctx import { Deadline } from "./deadline.ts"; import { type FollowAttach, FollowSource } from "./follow.ts"; import type { HostError } from "./host.ts"; -import { attachDerives, checkImplementation, type KeyedBinding, type Members } from "./members.ts"; +import { attachDerives, checkImplementation, contributionOf, type KeyedBinding, type Members } from "./members.ts"; import { type CloseFailed, drained, @@ -44,6 +47,7 @@ import { } from "./native.ts"; import type { ChordObject } from "./object.ts"; import type { Binding, CheckedParams, Reference, Target } from "./reference.ts"; +import { type RetrySource, retrySource } from "./retry-on.ts"; import { type OnHandler, type OnLiveHandler, @@ -53,19 +57,48 @@ import { } from "./routines.ts"; /** - * What a factory declares with. Declarations (`state`, `derive`, `get`, `open`, `on`, `onLive`, `job`) are valid only - * while it runs. `P` is the app's principal type, which the ctx of routine calls and job runs carry. + * What a factory declares with. Declarations (`state`, `derive`, `get`, `registry`, `writable`, `retryOn`, `open`, + * `on`, `onLive`, `job`) are valid only while it runs. `P` is the app's principal type, which the ctx of routine calls + * and job runs carry. */ export interface Self

extends RoutineDeclarations

{ /** This object's documents of `definition`, in memory for its life (across its attempts). */ state>(definition: D): HandleOf; - /** A value computed from `sources` by the pure, synchronous `compute`. */ - derive[], T extends JsonValue>( + /** + * A value computed from `sources` (JSON watchables and native ones: a `writable`, a registry value, `ref.state`) by + * the pure, synchronous `compute`; always JSON. + */ + derive | NativeWatchable)[], T extends JsonValue>( sources: S, - compute: (values: { readonly [I in keyof S]: S[I] extends Watchable ? V : never }) => T, + compute: ( + values: { + readonly [I in keyof S]: S[I] extends Watchable + ? V + : S[I] extends NativeWatchable + ? V + : never; + }, + ) => T, ): Watchable; /** A reference to another object, live once this object is, released when it ends. */ get(Type: C, params: ParamsOf): Reference; + /** + * Registry `R`'s composed value as this plugin's version of `R` sees it: a declared dependency, as `get` is + * (activation waits until it has a value; with none it fails `transient` and is retried when one arrives, R18). + * Read `.value` at use; once live, a later failure of the registry keeps its last good value. + */ + registry(R: Registry): RegistryValue; + /** + * An in-process value this attempt sets (`set`, identity compared): a contribution, a derive source and an `on` + * source, never a state member; its watches end with the attempt, and `set` after its end throws `released`. + */ + writable(initial: T): Writable; + /** + * Wake signals: a new value of any of `sources` (registry values; constants never change) retries this object + * while it is failed, as `ref.retry()` does, and once more at once if its current activation fails. Followed from + * the factory's return until the next attempt's declaration or the object's end; failures wake nothing. + */ + retryOn(...sources: readonly (Watchable | NativeWatchable)[]): void; /** * A native, opened during activation after the references are live and the earlier natives are open, and closed * in reverse when the attempt ends. `open` gets a context aborted when the attempt is abandoned; `close` one aborted @@ -101,6 +134,12 @@ export interface StoreSelf

extends Self

{ state>(definition: D): HandleOf; /** This object's table `definition` in the host's store, once per table name. */ table(definition: TableDefinition): Table; + /** + * Contributes `value` to registry `R` from the moment this object is live until it ends, fails or restarts (R21: a + * host-lifetime object): a plain value, or a Chord watchable of one (followed: its change recomposes). At most one + * per registry. + */ + contribute(R: Registry, value: Contribution): void; } /** @@ -111,6 +150,27 @@ export type Factory = Shape ? (params: ParamsOf, self: keyof ParamsOf extends never ? StoreSelf

: Self

) => Implementation> : (params: ParamsOf, principal: P, self: Self

) => Implementation>; +/** How an activation fails on a declared registry without a value (`#readRegistries`). */ +type UnreadRegistry = Extract; + +/** No declarations of a kind: the shared empty list. */ +const NONE: readonly never[] = []; + +/** An attempt's registry reads, contributions and wake signals, and its writables (slice 8). */ +type RegistryDeclarations = { + /** The registry values it declared (`self.registry`), registered as read during activation (`reads`). */ + readonly registries: RegistryValueSource[]; + /** Releases its registry reads (with its references). */ + readonly reads: (() => void)[]; + /** Its wake signals (`self.retryOn`), handed to its object right after the factory. */ + readonly retrySources: RetrySource[]; + /** Its writables, whose watches end with it. */ + readonly writables: { detach(error: Error): void }[]; + /** Its contributions as declared, resolved during activation into `contributions`. */ + readonly declared: { readonly registry: Registry; readonly value: unknown }[]; + contributions: readonly RegistryContribution[]; +}; + /** How an attempt's activation ended. `dependency_failed` (R18) keeps the attempt's references. */ type Activation = | { readonly kind: "live" } @@ -174,6 +234,8 @@ export class Attempt implements StoreSelf, KeyDeclarer { #abandoned: Exclude | undefined; /** A followed source's end while this attempt activated: it fails as it goes live (if its activation returned). */ #endedEarly: ChordError | undefined; + /** Its registry reads, contributions, wake signals and writables, made by the first: most attempts have none. */ + #registryDeclarations: RegistryDeclarations | undefined; constructor(object: ChordObject, previous: Attempt | undefined, context: Context) { this.#object = object; @@ -221,13 +283,21 @@ export class Attempt implements StoreSelf, KeyDeclarer { return handle as unknown as Table; } - derive[], T extends JsonValue>( + derive | NativeWatchable)[], T extends JsonValue>( sources: S, - compute: (values: { readonly [I in keyof S]: S[I] extends Watchable ? V : never }) => T, + compute: ( + values: { + readonly [I in keyof S]: S[I] extends Watchable + ? V + : S[I] extends NativeWatchable + ? V + : never; + }, + ) => T, ): Watchable { this.#assertDeclaring("derive"); // The values are the sources' values, in order; the result is `T`. - const derive = new Derive(sources, compute as (values: readonly JsonValue[]) => T, this.#object.host.batch); + const derive = new Derive(sources, compute as (values: readonly unknown[]) => T, this.#object.host.batch); this.#derives.push(derive); return derive as unknown as Watchable; } @@ -242,6 +312,50 @@ export class Attempt implements StoreSelf, KeyDeclarer { return binding.reference as Reference; } + registry(R: Registry): RegistryValue { + this.#assertDeclaring("registry"); + const value = this.#object.host.registryRecord(R.name).value(R.version); + this.#declarations().registries.push(value); + // Its values are the composition as `R`'s version has it, `V`. + return value as RegistryValue; + } + + writable(initial: T): Writable { + this.#assertDeclaring("writable"); + const owner = { + context: this.#context, + assertUsable: () => { + if (this.#token.sealed) throw this.#object.ended(); + }, + }; + const writable = new WritableSource(initial, owner, this.#object.host.batch); + this.#declarations().writables.push(writable); + return writable; + } + + retryOn(...sources: readonly (Watchable | NativeWatchable)[]): void { + this.#assertDeclaring("retryOn"); + const { retrySources } = this.#declarations(); + for (const source of sources) retrySources.push(retrySource(source, this.#object.label)); + } + + contribute(R: Registry, value: Contribution): void { + this.#assertDeclaring("contribute"); + const { served, label, host } = this.#object; + // R21: the types allow it only to host-lifetime objects; plugin code is bundled without type checks. + if (served.kind !== "local" || !served.hostLifetime) { + throw new Error(`${label}: self.contribute is valid only in a local type with empty params`); + } + const { declared } = this.#declarations(); + if (declared.some(({ registry }) => registry.name === R.name)) { + throw new Error(`${label}: registry ${R.name} is contributed to twice`); + } + // Refused at `contribute` (declaration policy), also a cycle within this factory's own declarations; the edges + // live with the contributions in the tables, from live on. + if (value instanceof RegistryValueSource) host.registryRecord(R.name).assertNoCycle(value, this.#pendingEdges()); + declared.push({ registry: R as Registry, value }); + } + open( open: (context: Context) => Promise, close: (value: T, reason: NativeCloseReason, context: Context) => void | Promise, @@ -384,6 +498,11 @@ export class Attempt implements StoreSelf, KeyDeclarer { }); } + /** Its contributions, resolved by its activation: in the registries' tables while its object is live. */ + get contributions(): readonly RegistryContribution[] { + return this.#registryDeclarations?.contributions ?? NONE; + } + /** The variant a remote object's factory chose; `undefined` for a local object. */ get variant(): string | undefined { return this.#variant; @@ -494,27 +613,34 @@ export class Attempt implements StoreSelf, KeyDeclarer { } /** - * Detaches its derives (once their object no longer needs them: reads throw `released`, their watches end) and its - * followed sources (their observers detached, their watches ended). + * Detaches its derives (once their object no longer needs them: reads throw `released`, their watches end), its + * followed sources (their observers detached, their watches ended) and its writables (their watches ended). */ detach(): void { const ended = this.#object.ended(); for (const derive of this.#derives) derive.detach(ended); for (const follow of this.#follows) follow.detach(); + const declarations = this.#registryDeclarations; + if (declarations !== undefined) for (const writable of declarations.writables) writable.detach(ended); } - /** Releases its references, and the previous attempt's if this one never bound its own. */ + /** Releases its references and registry reads, and the previous attempt's if this one never bound its own. */ releaseReferences(): void { this.#releasePrevious(); for (const { binding } of this.#references) binding.release(); + const declarations = this.#registryDeclarations; + if (declarations !== undefined) for (const release of declarations.reads.splice(0)) release(); } /** The activation steps; each catches its own failures with their kind (the table in slice 2's spec). */ async #activate(deadline: Deadline): Promise { let implementation: unknown; let dependencies: Target[]; + let unread: UnreadRegistry | undefined; try { implementation = this.#declare(); + // Right after the factory, before anything can fail: a failed attempt keeps its wake signals. + this.#object.retryOn(this.#registryDeclarations?.retrySources ?? NONE); const { documents } = this.#object; // The object holds the pins from now on. documents.hydrate(this.#declared, this.#pins.splice(0), this.#opening); @@ -528,12 +654,15 @@ export class Attempt implements StoreSelf, KeyDeclarer { host.bind(binding, contract, params, this.#opening), ); for (const dependency of dependencies) this.#object.assertNoCycle(dependency, this.#opening); + if (this.#registryDeclarations !== undefined) unread = this.#readRegistries(this.#registryDeclarations); } catch (error) { return this.#broken(error); } finally { // Bound first: a dependency held only by the previous attempt never ends in between. this.#releasePrevious(); } + // No value after `load`'s setups: none comes before a first composition (which retries it, R18) or a reload. + if (unread !== undefined) return this.#abandoned ?? unread; try { await Promise.all(dependencies.map((dependency) => this.#object.waitFor(dependency, this.#opening))); } catch (error) { @@ -586,6 +715,8 @@ export class Attempt implements StoreSelf, KeyDeclarer { const checked = checkImplementation(this.#object.served.contract, implementation); this.#members = checked.members; this.#variant = checked.variant; + if (this.#registryDeclarations !== undefined) + this.#registryDeclarations.contributions = this.#resolveContributions(this.#registryDeclarations); const { host, label } = this.#object; attachDerives(this.#derives, this.#members.state, (member) => (error) => { // During activation a throwing derive fails the attempt, which `activation_failed` reports alone. @@ -607,6 +738,41 @@ export class Attempt implements StoreSelf, KeyDeclarer { return this.#abandoned ?? { kind: "live" }; } + /** + * Registers this attempt's object as a reader of every registry it declared (woken by a first value, R18, until its + * references are released); returns how the activation fails on the first with no value, `undefined` when every + * one has a value. A version the provider does not serve (S2) is final until a reload: broken, the + * `contract_mismatch` itself the cause. Otherwise (no provider, a first `compose` that threw) a value may still come: + * `transient` (R18, the references kept), naming the registry, its error as the cause. + */ + #readRegistries({ registries, reads }: RegistryDeclarations): UnreadRegistry | undefined { + let unread: UnreadRegistry | undefined; + for (const value of registries) { + reads.push(value.read(this.#object)); + const start = value.start(); + if (unread !== undefined || start.kind === "value" || start.last !== undefined) continue; + const { error } = start; + if (isChordError(error, "contract_mismatch")) { + unread = { kind: "failed", cause: { kind: "broken", error } }; + continue; + } + const message = `${this.#object.label}: registry ${value.name} has no value`; + const cause = new ChordError({ code: "unavailable", message }, { cause: error }); + unread = { kind: "dependency_failed", cause: { kind: "transient", error: cause } }; + } + return unread; + } + + /** Its declared contributions, as Chord follows them (its references are live now), by its plugin and type. */ + #resolveContributions({ declared }: RegistryDeclarations): readonly RegistryContribution[] { + const { served, host } = this.#object; + const contributor = { plugin: served.plugin, type: served.contract.name }; + const pending = this.#pendingEdges(); + return declared.map(({ registry, value }) => + contributionOf(host.registryRecord(registry.name), contributor, registry.version, value, pending), + ); + } + /** * Brings the declared tables to their definitions before the references are bound (slice 6a §9): created fresh or * migrated eagerly, with the `readyTimeout` deadline paused meanwhile (resumed with the budget it had left). @@ -686,6 +852,29 @@ export class Attempt implements StoreSelf, KeyDeclarer { this.#previous = undefined; } + /** The registry edges its declared contributions would add (another registry's value contributed). */ + #pendingEdges(): RegistryEdge[] { + const edges: RegistryEdge[] = []; + const { host } = this.#object; + for (const { registry, value } of this.#registryDeclarations?.declared ?? NONE) { + const to = value instanceof RegistryValueSource ? value.record : undefined; + if (to !== undefined) edges.push({ from: host.registryRecord(registry.name), to }); + } + return edges; + } + + #declarations(): RegistryDeclarations { + this.#registryDeclarations ??= { + registries: [], + reads: [], + retrySources: [], + writables: [], + declared: [], + contributions: NONE, + }; + return this.#registryDeclarations; + } + #assertDeclaring(declaration: string): void { if (!this.#declaring) { throw new Error(`${this.#object.label}: self.${declaration} is valid only in the factory`); diff --git a/packages/chord/src/next/host/ctx.ts b/packages/chord/src/next/host/ctx.ts index 4eb44020b..d14d3470c 100644 --- a/packages/chord/src/next/host/ctx.ts +++ b/packages/chord/src/next/host/ctx.ts @@ -23,7 +23,7 @@ export interface Ctx

extends Context { /** Who makes a call: the principal it acts as and its host's incarnation. */ export type Caller = { readonly principal: Principal | undefined; readonly incarnation: string }; -/** The ctx of one handler run: its references are released by `settle` when the handler settled. */ +/** The ctx of one handler run (or root scope): its references are released by `settle` when it settled. */ export class CallContext implements Ctx { readonly principal: Principal | undefined; readonly incarnation: string; @@ -58,9 +58,9 @@ export class CallContext implements Ctx { } async get(Type: C, params: ParamsOf, options: GetOptions = {}): Promise> { - // An ended handler creates nothing: its call's abort signal is checked before anything else. - this.#context.abortSignal?.throwIfAborted(); + // A settled scope refuses `released` whatever aborted it; an aborted call creates nothing. if (this.#references === undefined) throw settled(Type); + this.#context.abortSignal?.throwIfAborted(); // Made for the caller: its principal and its origin's incarnation travel with the reference's holds. const caller = { principal: this.principal, incarnation: this.incarnation }; this.#settling ??= new AbortController(); @@ -96,7 +96,7 @@ function settled(Type: Contract): ChordError { return new ChordError({ code: "released", reason: "settled", - message: `${Type.name}: ctx.get after its handler settled`, + message: `${Type.name}: ctx.get after its call settled`, }); } diff --git a/packages/chord/src/next/host/facet.ts b/packages/chord/src/next/host/facet.ts index 0055285ff..4e9615852 100644 --- a/packages/chord/src/next/host/facet.ts +++ b/packages/chord/src/next/host/facet.ts @@ -1,16 +1,23 @@ -// Facets (§16): a plugin's setup for one host, typed by the app's principal type. A setup only declares: the types it -// serves and the host's presence lists; it has no documents, routines or natives, so a facet cannot fail after `load`. +// Facets (§16): a plugin's setup for one host, typed by the app's principal type. A setup only declares: the types and +// registries it serves, its contributions to registries (slice 8) and the host's presence lists; it has no documents, +// routines or natives, so a facet cannot fail after `load`. import type { Watchable } from "../../watch.ts"; import type { Contract } from "../contracts/index.ts"; import type { Principal } from "../data/index.ts"; +import type { Contribution, Registry, RegistryProvider } from "../registries/index.ts"; import type { Factory } from "./attempt.ts"; import type { Presence } from "./presence.ts"; -/** What a setup serves types with (valid only while the setup runs), and the host's presence lists (always). */ +/** What a setup serves types and registries with (valid only while the setup runs), and the host's presence lists. */ export interface FacetHost

{ /** Serves a type on this host with `factory`; a remote type's factory gets the principal its object is for. */ serve(Type: C, factory: Factory, options?: ServeOptions): void; + /** + * Serves registry `R` on this host (its defining plugin's facet, the one provider, D1): `compose` builds the value + * from the table, `accepts` adapts older versions. A second serve of a registry is refused at `load`. + */ + serve(R: Registry, provider: RegistryProvider): void; /** `host.references(Type)`: the references to objects of `Type` on this host (§19), for factories to derive from. */ references(Type: C): Watchable[]>; } @@ -24,48 +31,92 @@ export type ServeOptions = { readonly linger?: number; }; -/** A type a setup served: its contract, its factory and its options. */ -export type ServedEntry = { - readonly contract: Contract; - readonly factory: ServedFactory; - readonly options: ServeOptions; -}; +/** What a facet's setup declares with besides the host: its contributions to registries (R21). */ +export interface FacetSelf { + /** + * Contributes `value` to registry `R` while this plugin is loaded: a plain value, or a Chord watchable of one + * (followed: its change recomposes). At most one per registry. + */ + contribute(R: Registry, value: Contribution): void; +} + +/** What a setup served, by kind: a type (its contract, factory and options), or a registry (its provider). */ +export type ServedEntry = + | { + readonly kind: "type"; + readonly contract: Contract; + readonly factory: ServedFactory; + readonly options: ServeOptions; + } + | { + readonly kind: "registry"; + readonly registry: Registry; + readonly provider: RegistryProvider; + }; + +/** A contribution a setup declared. */ +export type FacetContribution = { readonly registry: Registry; readonly value: unknown }; /** A factory as a setup served it, of either kind (the host dispatches on its contract's kind). */ type ServedFactory = (params: never, principalOrSelf: never, self: never) => unknown; /** A plugin's facet for hosts of the app's principal type `P` (type only: nothing is checked at runtime). */ export type Facet

= { - readonly setup: (host: FacetHost

) => void; + readonly setup: (host: FacetHost

, self: FacetSelf) => void; /** Type only: the principal type, invariant, so a host takes only facets typed for exactly its `P`. */ readonly "~principal"?: (principal: P) => P; }; /** Defines a facet; base code binds `P` once, next to its principal type. */ -export function defineFacet

(setup: (host: FacetHost

) => void): Facet

{ +export function defineFacet

(setup: (host: FacetHost

, self: FacetSelf) => void): Facet

{ return { setup }; } -/** Runs a facet's setup and returns the types it served, in order; `references` is the host's. */ +/** + * Runs a facet's setup and returns the types and registries it served and its contributions, in order; `references` + * is the host's. + */ export function setUp

( name: string, facet: Facet

, references: (Type: C) => Watchable[]>, -): readonly ServedEntry[] { +): { readonly served: readonly ServedEntry[]; readonly contributions: readonly FacetContribution[] } { const served: ServedEntry[] = []; + const contributions: FacetContribution[] = []; let open = true; + const assertOpen = (what: string): void => { + if (!open) throw new Error(`${name}: ${what} is valid only in the facet's setup`); + }; const host: FacetHost

= { - serve: (Type, factory, options = {}) => { - if (!open) throw new Error(`${name}: host.serve is valid only in the facet's setup`); - // Served types are dispatched by name and kind; the factory matches its own contract. - served.push({ contract: Type, factory: factory as ServedFactory, options }); + serve: (Type: Contract | Registry, implementation: unknown, options: ServeOptions = {}) => { + assertOpen("host.serve"); + // Served types and registries are dispatched by their kind; a factory matches its own contract, a provider + // its own registry. + if (Type.kind === "registry") { + served.push({ + kind: "registry", + registry: Type, + provider: implementation as RegistryProvider, + }); + } else { + served.push({ kind: "type", contract: Type, factory: implementation as ServedFactory, options }); + } }, references, }; + const self: FacetSelf = { + contribute: (registry, value) => { + assertOpen("self.contribute"); + if (contributions.some((each) => each.registry.name === registry.name)) { + throw new Error(`${name}: registry ${registry.name} is contributed to twice`); + } + contributions.push({ registry: registry as Registry, value }); + }, + }; try { - facet.setup(host); + facet.setup(host, self); } finally { open = false; } - return served; + return { served, contributions }; } diff --git a/packages/chord/src/next/host/host.ts b/packages/chord/src/next/host/host.ts index cbafd799b..5b360e4ba 100644 --- a/packages/chord/src/next/host/host.ts +++ b/packages/chord/src/next/host/host.ts @@ -1,10 +1,16 @@ // Hosts (§3): `createHost`, loading facets and the start barrier, links to children, the parent (connected again after // a drop) and routes, the origins it relays holds for, getting and holding references as a principal, the heartbeat -// of every link, revocation (§4), presence (§19) and keep-alive counts, closing; one batch per host, the backoff map -// and the endings new instances wait for. +// of every link, revocation (§4), presence (§19) and keep-alive counts, registries and root scopes (slice 8), closing; +// one batch per host, the backoff map and the endings new instances wait for. import type { TelemetryContext } from "@earendil-works/pi-telemetry"; -import { BACKGROUND_CONTEXT, type Context, withAbortSignal, withoutAbortSignal } from "../../context/index.ts"; +import { + awaitWithContext, + BACKGROUND_CONTEXT, + type Context, + withAbortSignal, + withoutAbortSignal, +} from "../../context/index.ts"; import type { JsonValue } from "../../json.ts"; import type { Watchable } from "../../watch.ts"; import type { Address, Contract, HostKind, ParamsOf } from "../contracts/index.ts"; @@ -21,6 +27,13 @@ import { } from "../data/internal.ts"; import { binaryCodec, type Codec, type Compression, type Link, type PartLimits } from "../link/index.ts"; import { linkLimits } from "../link/internal.ts"; +import type { Registry, RegistryValue } from "../registries/index.ts"; +import { + type RegistryContribution, + type RegistryEdge, + type RegistryError, + RegistryRecord, +} from "../registries/internal.ts"; import { type HoldRequest, HostLink, @@ -34,14 +47,16 @@ import type { ChordStore, Store } from "../store/store.ts"; import { Activity, type ActivityCounts } from "./activity.ts"; import { notLive } from "./attempt.ts"; import { Backoff } from "./backoff.ts"; -import { type Caller, withLinkedAbortSignal } from "./ctx.ts"; -import { type Facet, type ServedEntry, setUp } from "./facet.ts"; +import { type Caller, type Ctx, withLinkedAbortSignal } from "./ctx.ts"; +import { type Facet, type FacetContribution, type ServedEntry, setUp } from "./facet.ts"; +import { contributionOf } from "./members.ts"; import type { CloseFailed } from "./native.ts"; import { ChordObject, type Holder, type LocalContract, type RemoteContract, type Served } from "./object.ts"; import { type Placed, Placement } from "./placement.ts"; import { type Presence, References } from "./presence.ts"; import { Binding, type CheckedParams, type Held, type Reference, type Target } from "./reference.ts"; import { RevocationKeys, type RevokedBy } from "./revocation.ts"; +import { type RootOptions, RootScopes } from "./root.ts"; import { Route, type RouteOptions } from "./route.ts"; import { uuidv7 } from "./uuid.ts"; @@ -93,6 +108,7 @@ export type Position = { readonly kind: "on" | "onLive" | "native"; readonly pos */ export type HostError = | BatchError + | RegistryError | { readonly kind: "activation_failed"; readonly object: string; readonly failure: FailureCause } /** A live object failed at runtime (a `migrate` failing on access, a stored version newer than the code). */ | { readonly kind: "object_failed"; readonly object: string; readonly failure: FailureCause } @@ -163,7 +179,7 @@ export type GetOptions = { * A host whatever its principal type, as Chord's own parts use it: `ChordHost

` is invariant in `P` (`load` takes * facets typed for exactly `P`). */ -export type AnyHost = Omit; +export type AnyHost = Omit; /** A host. `P` is the app's principal type, `R` its revocation type. */ export interface Host

{ @@ -228,8 +244,21 @@ export interface Host

; /** - * Closes its links, then ends every object at once, then drains and closes them one by one, the last to go live - * first (reason `close`). Rejects with an `AggregateError` of the closes that threw. + * Registry `R`'s composed value as `R`'s version sees it, for code outside objects; waits for `load` first. Rejects + * when it has no value then (`unavailable/undeclared` without a provider, a first `compose` that threw, a version + * not served here: `contract_mismatch`); none can come before a reload. + */ + registry(R: Registry, context: Context): Promise>; + /** + * Runs `fn` with a call context for code outside objects (a Durable tool, a CLI): the host's principal (or + * `options.principal`), this host's incarnation, `context`'s telemetry and signal, `ctx.get`. When `fn` settles or + * `context` aborts, the context is sealed: its signal aborted, `ctx.get` refused `released`, every reference it took + * released; an abort rejects at once. `host.close` seals the open ones; after it, `root` rejects `released`. + */ + root(fn: (ctx: Ctx

) => Promise, context: Context, options?: RootOptions

): Promise; + /** + * Closes its links, seals its root scopes, then ends every object at once, then drains and closes them one by one, + * the last to go live first (reason `close`). Rejects with an `AggregateError` of the closes that threw. */ close(context: Context): Promise; } @@ -241,6 +270,9 @@ export function createHost

(options); } +/** No registry edges pending (a facet's contributions). */ +const NO_EDGES: readonly RegistryEdge[] = []; + /** No activation chain (Q5). */ const NO_CHAIN: readonly string[] = []; @@ -291,6 +323,9 @@ export class ChordHost

(); + readonly #roots = new RootScopes(this); /** The one heartbeat timer, running while the host has a link. */ #ticking: ReturnType | undefined; #state: "new" | "loading" | "loaded" | "closed" = "new"; @@ -321,7 +356,13 @@ export class ChordHost

{ + for (const contribution of contributions) contribution.record.add(contribution); + for (const record of this.#registries.values()) record.composeServed(); + }, context); } catch (error) { // The host closes as `close` would: its links too. Nothing was activated, so no close can fail. this.#closing ??= this.#close(); @@ -422,6 +463,35 @@ export class ChordHost

[]>; } + async registry(R: Registry, context: Context): Promise> { + this.batch.assertOutside("registry"); + const loading = this.pendingLoad; + if (loading !== undefined) await awaitWithContext(loading, context); + this.#assertOpen(); + const value = this.registryRecord(R.name).value(R.version); + const start = value.start(); + if (start.kind === "failed" && start.last === undefined) throw start.error; + // Its values are the composition as `R`'s version has it, `V`. + return value as RegistryValue; + } + + async root(fn: (ctx: Ctx

) => Promise, context: Context, options: RootOptions

= {}): Promise { + this.batch.assertOutside("root"); + this.#assertOpen(); + // The scope's ctx acts as `principal`, a `P`. + return this.#roots.run(fn as (ctx: Ctx) => Promise, options.principal ?? this.principal, context); + } + + /** The record of registry `name` on this host, made by its first use. */ + registryRecord(name: string): RegistryRecord { + let record = this.#registries.get(name); + if (record === undefined) { + record = new RegistryRecord(name, this.batch, (error) => this.report(error)); + this.#registries.set(name, record); + } + return record; + } + async close(_context: Context): Promise { this.batch.assertOutside("close"); this.#closing ??= this.#close(); @@ -782,6 +852,8 @@ export class ChordHost

}[]): void { + /** + * Runs every facet's setup, checking the plugin names (npm names, each once), and serves what they declared; returns + * their contributions, which enter the registries' tables once the setups ran. + */ + #setUp(facets: readonly { readonly name: string; readonly facet: Facet

}[]): RegistryContribution[] { const names = new Set(); + const made: RegistryContribution[] = []; for (const { name, facet } of facets) { if (!PLUGIN_NAME.test(name)) throw new Error(`"${name}" is not a valid plugin name`); if (names.has(name)) throw new Error(`plugin "${name}" is loaded twice`); names.add(name); - for (const entry of setUp(name, facet, (Type) => this.references(Type))) this.#serve(entry); + const { served, contributions } = setUp(name, facet, (Type) => this.references(Type)); + for (const entry of served) this.#serve(name, entry); + for (const contribution of contributions) made.push(this.#contribution(name, contribution)); } + return made; } - #serve({ contract, factory, options }: ServedEntry): void { + /** A facet's contribution, checked: in its registry's table while the host lives (its plugin stays loaded). */ + #contribution(plugin: string, { registry, value }: FacetContribution): RegistryContribution { + // A facet contributes no registry's value (it reads none): no edges of its own. + const record = this.registryRecord(registry.name); + return contributionOf(record, { plugin, type: undefined }, registry.version, value, NO_EDGES); + } + + #serve(plugin: string, entry: ServedEntry): void { + if (entry.kind === "registry") { + this.registryRecord(entry.registry.name).serve(entry.registry.version, entry.provider); + return; + } + const { contract, factory, options } = entry; if (this.#types.has(contract.name)) throw new Error(`${contract.name} is served twice`); // A client serves only local types (§3): a remote object is owned by a principal of the host serving it. if (contract.kind === "remote" && this.principal === undefined) { throw new Error(`${contract.name}: a host without a principal serves only local types`); } - const common = { provider: bindProvider(contract, {}), linger: options.linger ?? 0 }; + const common = { plugin, provider: bindProvider(contract, {}), linger: options.linger ?? 0 }; // The contract's kind decides the factory's signature and the contract's shape. const served: Served = contract.kind === "local" diff --git a/packages/chord/src/next/host/index.ts b/packages/chord/src/next/host/index.ts index 3d7611c5d..79b8a2634 100644 --- a/packages/chord/src/next/host/index.ts +++ b/packages/chord/src/next/host/index.ts @@ -1,10 +1,10 @@ // Hosts for apps: creating a host, facets, factories, natives, references and the ctx of method handlers, routines and -// their waiting helpers, revocation, presence and keep-alive counts. +// their waiting helpers, revocation, presence and keep-alive counts, root scopes. export type { ActivityCounts } from "./activity.ts"; export type { Factory, Self, StoreSelf } from "./attempt.ts"; export type { Ctx } from "./ctx.ts"; -export { defineFacet, type Facet, type FacetHost, type ServeOptions } from "./facet.ts"; +export { defineFacet, type Facet, type FacetHost, type FacetSelf, type ServeOptions } from "./facet.ts"; export type { FollowAttach, FollowAttachment, FollowObserver } from "./follow.ts"; export { createHost, @@ -20,6 +20,7 @@ export type { Presence } from "./presence.ts"; export type { RefState } from "./ref-state.ts"; export type { Held, Reference, RefIn } from "./reference.ts"; export type { RevokedBy } from "./revocation.ts"; +export type { RootOptions } from "./root.ts"; export type { RouteOptions } from "./route.ts"; export type { OnHandler, OnLiveHandler, OnNativeHandler, RoutineDeclarations } from "./routines.ts"; export { sleep, untilAborted } from "./sleep.ts"; diff --git a/packages/chord/src/next/host/members.ts b/packages/chord/src/next/host/members.ts index 7eaf1fc73..ed2222cd9 100644 --- a/packages/chord/src/next/host/members.ts +++ b/packages/chord/src/next/host/members.ts @@ -1,5 +1,6 @@ // The member check (Q14): an implementation is one variant with exactly that variant's members, methods being -// functions and state members JSON watchables; and the resolution of every derive's declared sources. +// functions and state members JSON watchables; and the resolution of what Chord follows behind every derive's declared +// sources (JSON and native) and every contribution to a registry (slice 8). // // The rule for watches across attempts (slice 6a §1.2, §1.4): references and other objects' derives follow a local // object's state member through its member source (`member-source.ts`), and a key of a keyed member they hold (`at`, @@ -17,7 +18,15 @@ import type { Context } from "../../context/index.ts"; import type { Contract } from "../contracts/index.ts"; import { currentVariants, LOCAL, takesNoArgs, type VariantData } from "../contracts/internal.ts"; import type { KeyedSlot } from "../documents/internal.ts"; -import type { Derive, DeriveSource } from "../state/internal.ts"; +import { + type ContributionContent, + type Contributor, + RegistryContribution, + type RegistryEdge, + type RegistryRecord, + RegistryValueSource, +} from "../registries/internal.ts"; +import { type Derive, type DeriveInput, type DeriveSource, NativeSource } from "../state/internal.ts"; import { type Binding, resolveKeyed, resolveWatchable } from "./reference.ts"; /** A checked implementation, as Chord calls and follows it. */ @@ -114,11 +123,35 @@ export function attachDerives( derives.forEach((derive, position) => { const member = exposed.get(derive); const name = member ?? `derive #${position + 1}`; - const resolve = (source: unknown, index: number): DeriveSource => { - const resolved = resolveWatchable(source); - if (resolved === undefined) throw new Error(`${name}: source ${index} is not a JSON watchable`); + const resolve = (source: unknown, index: number): DeriveInput => { + const resolved = resolveWatchable(source) ?? (source instanceof NativeSource ? source : undefined); + if (resolved === undefined) throw new Error(`${name}: source ${index} is not a Chord watchable`); return resolved; }; derive.attach(resolve, report(member)); }); } + +/** + * A contribution of `contributor` at `version` to `record`, as Chord follows it: a Chord watchable (a source, a + * constant, a document handle, a reference member, a native source) is followed, anything else is a plain value, + * whatever methods it has (registries carry natives). Refused (`invalid/cycle`) when it is a registry's value that + * closes a cycle of registries, through the tables and the contributor's `pending` edges. + */ +export function contributionOf( + record: RegistryRecord, + contributor: Contributor, + version: number, + value: unknown, + pending: readonly RegistryEdge[], +): RegistryContribution { + if (value instanceof RegistryValueSource) record.assertNoCycle(value, pending); + return new RegistryContribution(record, contributor, version, contentOf(value)); +} + +/** What Chord follows behind a contribution: its JSON or native source when it is a Chord watchable. */ +function contentOf(value: unknown): ContributionContent { + if (value instanceof NativeSource) return { kind: "native", source: value }; + const source = resolveWatchable(value); + return source === undefined ? { kind: "plain", value } : { kind: "json", source }; +} diff --git a/packages/chord/src/next/host/object.ts b/packages/chord/src/next/host/object.ts index f422a58fb..91e0bd2b8 100644 --- a/packages/chord/src/next/host/object.ts +++ b/packages/chord/src/next/host/object.ts @@ -2,7 +2,8 @@ // and job runs, its member sources, slice 6a §1.2), its holders, retries after failures (backoff, dependency recovery // R18, `ref.retry()`), a live object's failure (slice 6a §1.4), the waits and cycle checks of activations (Q5, its // instance id and the activation chain it was created for), lingering after its last reference (§3, D8), the job runs -// that keep it alive (§15), and its ending, revocation included (§4, R11). +// that keep it alive (§15), its contributions to registries while live and its wake signals (`retryOn`, slice 8), and +// its ending, revocation included (§4, R11). import { awaitWithContext, type Context } from "../../context/index.ts"; import type { Contract, ParamsOf } from "../contracts/index.ts"; @@ -23,6 +24,7 @@ import type { KeyedBinding } from "./members.ts"; import type { CloseFailed, NativeCloseReason } from "./native.ts"; import { type Ended, type Failed, LIVE, LOADING, type RefState } from "./ref-state.ts"; import type { Target } from "./reference.ts"; +import { type RetrySource, RetryWakes } from "./retry-on.ts"; import { uuidv7 } from "./uuid.ts"; /** A `local` contract, as a served type has it. */ @@ -48,6 +50,8 @@ export type Served = }); type ServedType = { + /** The plugin whose facet served it: its objects contribute to registries by it. */ + readonly plugin: string; readonly contract: C; readonly provider: Provider; /** How long a live object stays live after its last reference, in milliseconds (`serve`'s `linger`, 0 by default). */ @@ -146,6 +150,8 @@ export class ChordObject implements Target { readonly #keys: KeyedMembers; /** The one timer storing its jobs' held state changes by their earliest deadline (D8), while one is held. */ #flush: { readonly deadline: number; readonly timer: ReturnType } | undefined; + /** Its wake signals (`retryOn`), the last attempt's declaration; made by the first. */ + #wakes: RetryWakes | undefined; constructor( host: AnyHost, @@ -264,6 +270,17 @@ export class ChordObject implements Target { this.attempt(); } + /** + * Follows `sources` as its wake signals instead of the last attempt's (`self.retryOn`, right after a factory + * returned): a new value retries it while failed, as `ref.retry()` does; one while it activates retries it at once + * if that attempt fails; one while live does nothing. + */ + retryOn(sources: readonly RetrySource[]): void { + if (this.#wakes === undefined && sources.length === 0) return; + this.#wakes ??= new RetryWakes(); + this.#wakes.replace(sources, () => this.#wake()); + } + /** Calls a method of the live object for `caller`. */ async call(method: string, args: unknown, context: Context, caller: Caller): Promise { return this.#live().call(method, args, context, caller); @@ -383,6 +400,7 @@ export class ChordObject implements Target { this.#lastFailure = error; this.#outcome = settlement(); this.#outcome.reject(error); + this.#withdraw(attempt); attempt.seal(error); // Members following another object's sources freeze now: no commit of this object changes them. this.#freeze(attempt, "foreign"); @@ -530,10 +548,18 @@ export class ChordObject implements Target { #wentLive(attempt: Attempt): void { this.#woken = undefined; + if (this.#wakes !== undefined) this.#wakes.during = undefined; this.#state = { kind: "live", attempt }; this.#exposed = attempt; if (this.#members.size > 0 || !this.#keys.empty) this.host.batch.within(() => this.#rebind(attempt), this.#context); + const { contributions } = attempt; + // In the registries' tables before anyone sees it live (a getter released now reads them recomposed). + if (contributions.length > 0) { + this.host.batch.within(() => { + for (const contribution of contributions) contribution.record.add(contribution); + }, this.#context); + } this.#lastFailure = undefined; this.host.wentLive(this); this.#outcome.resolve(); @@ -551,6 +577,8 @@ export class ChordObject implements Target { #failed(attempt: Attempt, cause: FailureCause, holding: Attempt | undefined): void { const woken = this.#woken === attempt; this.#woken = undefined; + const wokenBy = this.#wakes?.during === attempt; + if (this.#wakes !== undefined) this.#wakes.during = undefined; // What its activation loaded for the pinned keys goes; they stay frozen. this.#keys.discard(); if (cause.kind === "transient") this.host.backoff.failed(this.identity); @@ -573,6 +601,44 @@ export class ChordObject implements Target { this.#idle(); // A dependency went live while this attempt activated: the wake it missed retries it now (R18). if (holding !== undefined && woken) this.dependencyLive(); + // A wake signal arrived while it activated: it retries it now. + else if (wokenBy) this.retry(); + } + + /** + * A wake signal's new value (inside its batch): retries a failed object (after the batch: a factory may not run in + * one), remembered while it activates, ignored while live and when its own contributions' leaving made it. + */ + #wake(): void { + const state = this.#state; + // Called by its wakes only. + const wakes = this.#wakes!; + if (state.kind === "activating") { + wakes.during = state.attempt; + return; + } + if (state.kind !== "failed" || wakes.leaving) return; + queueMicrotask(() => this.retry()); + } + + /** + * The live attempt `attempt` leaves the registries' tables (it fails or ends), inside a batch; a composition this + * leaving makes wakes no wake signal of this object. + */ + #withdraw(attempt: Attempt): void { + const { contributions } = attempt; + if (contributions.length === 0) return; + const { batch } = this.host; + const wakes = this.#wakes; + batch.within(() => { + if (wakes !== undefined) { + wakes.leaving = true; + batch.flushing(() => { + wakes.leaving = false; + }); + } + for (const contribution of contributions) contribution.record.remove(contribution); + }, this.#context); } /** The member source of `member`, made bound to `exposed`'s member by its first use. */ @@ -637,6 +703,7 @@ export class ChordObject implements Target { this.#flush = undefined; this.documents.seal(); this.#stopLingering(); + this.#wakes?.stop(); this.host.forget(this); this.#outcome.reject(error); // In a batch: the derives over its members are marked, so none keeps a cached value. @@ -656,6 +723,7 @@ export class ChordObject implements Target { const { holding } = state; return holding === undefined ? { kind: "end", previous } : { kind: "release", attempt: holding, previous }; } + if (state.kind === "live") this.#withdraw(state.attempt); state.attempt.seal(error); return { kind: "close", attempt: state.attempt, previous }; } diff --git a/packages/chord/src/next/host/ref-state.ts b/packages/chord/src/next/host/ref-state.ts index 75f9fa42e..9e5720405 100644 --- a/packages/chord/src/next/host/ref-state.ts +++ b/packages/chord/src/next/host/ref-state.ts @@ -1,9 +1,10 @@ // The state of a reference (§7): loading, live, failed or ended; and `ref.state`, the native watchable that follows // it. -import type { Context } from "../../context/index.ts"; -import { NativeWatch, type NativeWatchable } from "../../watch.ts"; +import { BACKGROUND_CONTEXT, type Context } from "../../context/index.ts"; +import { NativeWatch } from "../../watch.ts"; import type { ChordError, FailureCause } from "../data/index.ts"; +import { type Batch, NativeSource } from "../state/internal.ts"; /** * Where a reference stands. `loading`: its object activates or retries, it waits for `load` or a parent link, or its @@ -29,27 +30,21 @@ export const LOADING: RefState = { kind: "loading" }; export const LIVE: RefState = { kind: "live" }; -const IGNORE = (): void => {}; - /** - * `ref.state`: its value is the reference's state, read through `read`; each watch gets every new state, in its own - * context, and ends `ended` when the reference is released (the end is the notification). + * `ref.state`, a native source: its value is the reference's state, read through `read`; each watch gets every new + * state, in its own context (not one call per batch, as a followed source's), and ends `ended` when the reference is + * released (the end is the notification). Derives reading it are marked in a batch of its own when it changes. */ -export class ReferenceState implements NativeWatchable { +export class ReferenceState extends NativeSource { readonly #read: () => RefState; - /** Where a listener's throw goes (the host); dropped on release, so a released reference keeps no host. */ - #report: (error: unknown) => void; - /** Throws `invalid/reentrant` inside the host's batch (a watch taken in a derive compute); dropped on release. */ - #assertOutside: () => void; /** The watches with the contexts they were taken in, which their frames carry. */ readonly #watches = new Map, Context>(); /** The state the watches have: a change is announced only when the state is another object. */ #last: RefState; - constructor(read: () => RefState, report: (error: unknown) => void, assertOutside: () => void) { + constructor(read: () => RefState, batch: Batch | undefined) { + super(batch); this.#read = read; - this.#report = report; - this.#assertOutside = assertOutside; this.#last = read(); } @@ -57,35 +52,57 @@ export class ReferenceState implements NativeWatchable { return this.#read(); } + /** Every state in order: a watch sees each step it went through. */ async watch(context: Context): Promise> { - this.#assertOutside(); const state = this.#read(); - const watch: NativeWatch = new NativeWatch(state, () => this.#watches.delete(watch), this.#report); - // An ended reference has nothing more to announce. - if (state.kind === "ended") { - watch.end({ kind: "ended" }); - return watch; + const batch = this.batch; + // An ended reference has nothing more to announce, and keeps no host to report to. + if (batch === undefined || state.kind === "ended") { + const ended = new NativeWatch(state, IGNORE, IGNORE); + ended.end({ kind: "ended" }); + return ended; } + batch.assertOutside("watch"); + const watch = new NativeWatch(state, () => this.#watches.delete(watch), batch.reportListener); this.#watches.set(watch, context); if (context.abortSignal !== undefined) watch.observeAbort(context.abortSignal); return watch; } - /** The reference's state may have changed: every watch gets it when it is another state. */ - changed(): void { + /** The reference's state may have changed: every watch gets it when it is another state, derives are marked. */ + moved(): void { const state = this.#read(); if (state === this.#last) return; this.#last = state; for (const [watch, context] of this.#watches) watch.advance(state, context); + if (this.dependents.size > 0) this.#mark(); } - /** The reference ended (`ended`, the reference's state): every watch ends `ended`. */ - end(ended: Ended): void { + /** The reference ended (`ended`, the reference's state): every watch ends `ended`, derives are marked a last time. */ + released(ended: Ended): void { // An ended reference keeps nothing of its object: not the last failure's error, not the host. this.#last = ended; - this.#report = IGNORE; - this.#assertOutside = IGNORE; // Ending detaches, which deletes the current entry (safe while iterating) and runs no listener. for (const watch of this.#watches.keys()) watch.end({ kind: "ended" }); + const batch = this.batch; + if (batch === undefined || this.dependents.size === 0) { + this.end(); + return; + } + // Ended where the marks run: a release while a batch's flushes send marks in the next commit, after this. + batch.within(() => { + this.changed(); + this.end(); + }, BACKGROUND_CONTEXT); + } + + /** + * Marks the derives reading it, in the batch running now or one of its own (apart from `moved`: a closure there + * would allocate on every state change). A reference has no context of its own: the background context. + */ + #mark(): void { + this.batch?.within(() => this.changed(), BACKGROUND_CONTEXT); } } + +const IGNORE = (): void => {}; diff --git a/packages/chord/src/next/host/reference.ts b/packages/chord/src/next/host/reference.ts index 4b9ed1ff8..65da7ab33 100644 --- a/packages/chord/src/next/host/reference.ts +++ b/packages/chord/src/next/host/reference.ts @@ -136,8 +136,6 @@ const WAIT: Decision = { kind: "wait" }; const DONE: Decision = { kind: "done" }; -const IGNORE = (): void => {}; - /** A wait for a state: checked at every change, rejected with the reference's end. */ type Waiter = { readonly check: () => void; readonly end: (error: ChordError) => void }; @@ -257,7 +255,7 @@ export class Binding implements Holder { * rejects them with the cause; waits for a state decide. */ moved(): void { - this.#refState?.changed(); + this.#refState?.moved(); const state = this.refState; if (state.kind === "live") { this.#parked?.flush( @@ -430,7 +428,7 @@ export class Binding implements Holder { this.#parked?.reject(error); this.#parked = undefined; for (const watch of this.#watches) watch.end({ kind: "ended" }); - this.#refState?.end(ended); + this.#refState?.released(ended); // Each removes itself; promise reactions run later. for (const waiter of this.#waiters) waiter.end(error); this.#ended?.resolve(); @@ -495,13 +493,14 @@ export class Binding implements Holder { #watchableState(): ReferenceState { if (this.#refState !== undefined) return this.#refState; const state = this.#state; - // A released reference's state never changes: its watches end at once, with no listener to report. Bound, not a - // closure: `read` would share the closure's scope and keep the batch after release. - const [report, assertOutside] = - state.kind === "released" - ? [IGNORE, IGNORE] - : [state.batch.reportListener, state.batch.assertOutside.bind(state.batch, "watch")]; - this.#refState = new ReferenceState(() => this.refState, report, assertOutside); + // A released reference's state never changes: it is made ended, keeping no batch. + if (state.kind === "released") { + const made = new ReferenceState(() => this.refState, undefined); + made.released(state.state); + this.#refState = made; + return made; + } + this.#refState = new ReferenceState(() => this.refState, state.batch); return this.#refState; } diff --git a/packages/chord/src/next/host/retry-on.ts b/packages/chord/src/next/host/retry-on.ts new file mode 100644 index 000000000..84f997a11 --- /dev/null +++ b/packages/chord/src/next/host/retry-on.ts @@ -0,0 +1,55 @@ +// Wake signals (slice 8 §3, D2): `self.retryOn(...sources)`. The object follows each declared source while it lives, +// replaced by each attempt's declaration; a new value of a source wakes it (the object decides what a wake does: a +// retry while failed, remembered while it activates, nothing while live). Failures of a source wake nothing. + +import { RegistryValueSource } from "../registries/internal.ts"; +import { Constant, type NativeObserver } from "../state/internal.ts"; + +/** What `retryOn` takes: sources that live independently of the attempt (a registry value; a constant never wakes). */ +export type RetrySource = RegistryValueSource | Constant; + +/** + * Checks a `retryOn` source (declaration policy: TypeScript cannot tell a reference's member or this attempt's + * `writable` from a registry value or a constant). + */ +export function retrySource(source: unknown, label: string): RetrySource { + if (source instanceof RegistryValueSource || source instanceof Constant) return source; + throw new Error(`${label}: self.retryOn takes registry values and constants (sources that outlive the attempt)`); +} + +/** + * The wake signals of one object: its subscriptions (one set at a time, the last attempt's declaration) and what its + * object decides a wake by. + */ +export class RetryWakes { + /** The attempt a wake arrived during, while it activates: if it fails, it is retried at once. */ + during: object | undefined; + /** + * Set from its object's contributions leaving until that batch delivered: a composition its own leaving made does + * not wake it (an object failing live would retry itself without backoff). + */ + leaving = false; + #subscribed: { readonly source: RegistryValueSource; readonly observer: NativeObserver }[] = []; + + /** Follows `sources` instead of the previous set; `wake` runs inside the batch of each new value (it only records). */ + replace(sources: readonly RetrySource[], wake: () => void): void { + this.stop(); + const observer: NativeObserver = { + advance: (call) => { + if (call.kind === "value") wake(); + }, + end: () => {}, + }; + for (const source of sources) { + if (!(source instanceof RegistryValueSource)) continue; + source.subscribe(observer); + this.#subscribed.push({ source, observer }); + } + } + + /** Stops following (the object ended, or a new declaration replaces the set). */ + stop(): void { + for (const { source, observer } of this.#subscribed) source.unsubscribe(observer); + this.#subscribed = []; + } +} diff --git a/packages/chord/src/next/host/root.ts b/packages/chord/src/next/host/root.ts new file mode 100644 index 000000000..3ebdf3d4f --- /dev/null +++ b/packages/chord/src/next/host/root.ts @@ -0,0 +1,76 @@ +// Root scopes (slice 8 §4, D3, design §8): `host.root(fn, context, options?)`, a call context for code outside objects +// (a Durable tool, a CLI), sealed when `fn` settles, when `context` aborts and when the host closes: its signal aborted, +// its `ctx.get` refused `released`, every reference it took released. + +import { awaitWithContext, type Context } from "../../context/index.ts"; +import { ChordError, type Principal } from "../data/index.ts"; +import { CallContext, type Ctx, withLinkedAbortSignal } from "./ctx.ts"; +import type { AnyHost } from "./host.ts"; + +/** What `host.root` takes besides `fn` and the context. */ +export type RootOptions

= { + /** Whom the scope acts as: the host's principal by default; a trusted same-host caller's choice (base code). */ + readonly principal?: P; +}; + +/** An open root scope: its seal, and the rejection its caller gets when the host closes. */ +type OpenScope = { readonly seal: (reason: unknown) => void; readonly closed: PromiseWithResolvers }; + +/** The open root scopes of a host, each sealed once; `close` seals the rest and rejects their callers. */ +export class RootScopes { + readonly #host: AnyHost; + /** Each open scope: how it is sealed with a reason, and its caller's rejection when the host closes. */ + readonly #open = new Set(); + + constructor(host: AnyHost) { + this.#host = host; + } + + /** + * Runs `fn` with a call context like a handler's (`principal`, this host's incarnation, `context`'s telemetry, a + * signal linked to `context`'s), sealed when `fn` settles, `context` aborts or the host closes. Resolves as `fn` + * does; rejects at once when `context` aborts or the host closes (an uncooperative `fn` runs on and fails at its + * next use of Chord), or when `context` was aborted already (`fn` does not run). + */ + run(fn: (ctx: Ctx) => Promise, principal: Principal | undefined, context: Context): Promise { + context.abortSignal?.throwIfAborted(); + const host = this.#host; + const controller = new AbortController(); + const linked = withLinkedAbortSignal(controller.signal, context); + const scope = linked.context; + const ctx = new CallContext(scope, { principal, incarnation: host.incarnation }, host); + const open: OpenScope = { + seal: (reason) => { + if (!this.#open.delete(open)) return; + scope.abortSignal?.removeEventListener("abort", aborted); + // References first: a `get` still waiting gives up with `released`, not with the abort's reason. + ctx.settle(); + // While still linked, so the scope's own signal (a caller's context has one) aborts too. + controller.abort(reason); + linked.release(); + }, + closed: Promise.withResolvers(), + }; + const aborted = (): void => open.seal(scope.abortSignal?.reason); + this.#open.add(open); + scope.abortSignal?.addEventListener("abort", aborted, { once: true }); + // Plugin code: a synchronous throw becomes the rejection. + const running = (async () => fn(ctx))(); + const settled = (): void => open.seal(SETTLED); + running.then(settled, settled); + // A method's caller gets `released` when its object ends, whatever the handler still does: so does this one. One + // wait on the caller's signal around both, so the close removes its listener too. + return awaitWithContext(Promise.race([running, open.closed.promise]), context); + } + + /** The host closes: every open scope is sealed with `reason`, its caller rejected with it. */ + close(reason: unknown): void { + for (const open of [...this.#open]) { + open.seal(reason); + open.closed.reject(reason); + } + } +} + +/** The abort reason of a scope whose `fn` settled. */ +const SETTLED = new ChordError({ code: "released", reason: "settled", message: "host.root: its fn settled" }); diff --git a/packages/chord/src/next/host/routines.ts b/packages/chord/src/next/host/routines.ts index ea924ec7b..837551b9e 100644 --- a/packages/chord/src/next/host/routines.ts +++ b/packages/chord/src/next/host/routines.ts @@ -14,7 +14,7 @@ import { type WatchEnd, } from "../../watch.ts"; import type { Principal } from "../data/index.ts"; -import { Source } from "../state/internal.ts"; +import { NativeSource, Source } from "../state/internal.ts"; import { nextDelay } from "./backoff.ts"; import { CallContext, type Caller, type Ctx } from "./ctx.ts"; import type { AnyHost, Position } from "./host.ts"; @@ -94,6 +94,9 @@ type Called = { readonly kind: "done" } | { readonly kind: "failed"; readonly er const DONE: Called = { kind: "done" }; +/** What `#failing` says of a source failed now. */ +const FAILING = { kind: "failed" } as const; + /** One run of an `onLive` that lasted this long resets its backoff. */ const LONG_RUN = 60_000; @@ -267,10 +270,14 @@ class OnRoutine { } /** - * Whether the source is a Chord source failed now (`failed`), or one with no value yet (`no_value`, with the source to - * follow); `undefined` otherwise (natives and Durable's views tell their failures through their watches). + * Whether the source is a Chord source failed now (`failed`), or a JSON one with no value yet (`no_value`, with the + * source to follow); `undefined` otherwise (Durable's views tell their failures through their watches). */ #failing(): { readonly kind: "failed" } | { readonly kind: "no_value"; readonly source: Source } | undefined { + const declared = this.#declaration.source; + // A native source opens its watch at its last good value while failed (a registry whose `compose` threw): its + // first call is the failure, not that stale value. + if (declared instanceof NativeSource) return declared.start().kind === "failed" ? FAILING : undefined; let source: unknown; try { source = resolveWatchable(this.#declaration.source); diff --git a/packages/chord/src/next/index.ts b/packages/chord/src/next/index.ts index 8e759f345..1055a2cf6 100644 --- a/packages/chord/src/next/index.ts +++ b/packages/chord/src/next/index.ts @@ -7,6 +7,7 @@ export * from "./documents/index.ts"; export * from "./host/index.ts"; export * from "./jobs/index.ts"; export * from "./link/index.ts"; +export * from "./registries/index.ts"; export * from "./state/index.ts"; /** @internal The store's surface for adapters (slice 6b §2.1). */ export { type DecodedRow, OwnerTable, StoreOwnerTables } from "./store/adapter.ts"; diff --git a/packages/chord/src/next/registries/README.md b/packages/chord/src/next/registries/README.md new file mode 100644 index 000000000..070c98877 --- /dev/null +++ b/packages/chord/src/next/registries/README.md @@ -0,0 +1,81 @@ +# registries + +Registries (design §17, slice 8): an extension point between plugins. The defining plugin shares a token and serves +it with `compose`; facets and host-lifetime objects contribute; readers follow the composed value. + +## For apps (`index.ts`) + +```ts +import { defineRegistry, keyedEntries } from "./index.ts"; + +// Shared by every plugin, as a contract: identity and types only. +export const Http = defineRegistry("chord.http", { version: 2 }); + +defineFacet((host, self) => { + host.serve(Http, { // the defining plugin: the one provider + compose: (table, { report }) => routerOf(keyedEntries(table, (c) => c.routes, report)), + accepts: { 1: { contribution: (old: HttpV1) => upgrade(old), value: (router) => routerV1(router) } }, + }); + self.contribute(Http, { routes: { "GET /notes": exportNotes } }); // while this plugin is loaded +}); +host.serve(Mcp, (_, self) => { // local, empty params: may contribute (R21) + const ext = self.writable(mcpExtension([])); + self.contribute(Extensions, ext); // from live until it fails, ends or restarts + self.retryOn(self.registry(Models)); // a new model wakes it while failed (rare changes) + return {}; +}); +host.serve(Server, (_, self) => { + const router = self.registry(Http); // a dependency; `.value` at use + return { request: async (req, ctx) => router.value.route(req, ctx) }; +}); +``` + +- `defineRegistry(name, { version })`: a token. `name` follows the type names' rule (`[a-z0-9@/._~-]`), + `version` is the version this plugin is built against. `C` is what contributors give, `V` what readers get. +- `host.serve(R, { compose, accepts? })` in the defining plugin's facet (`host/`). `compose(table, { report })` is pure + and synchronous and always returns a value (a fallback built in); `report({ message, plugins })` reports a conflict + it resolved, as `registry_conflict`, once per episode (a conflict reported by the last good composition is not + reported again). It runs at the end of `load`'s setups (every served registry once, an empty table too), then lazily: + a change of the table marks it, and it composes when read or, while someone watches, observes or waits for it, in + the batch that changed the table (after the commit's adoptions); an unchanged table (every entry the same) composes + nothing. A `compose` that throws is the owner's bug: `registry_failed` once per episode, watches get one `failed` + call, `.value` keeps the last good composition (before any, it throws the error); the next good composition is a + value call, also when its value is the same. +- The table (`RegistryTable`): one `RegistryEntry { plugin, type?, version, value }` per live contributor, sorted by + plugin name, then type (a facet's entry first): never load order. A contribution of an older version is adapted with + `accepts[version].contribution` once per contribution value; a newer one, or an older one without an adapter, is left + out and reported `contribution_refused` once. A Chord watchable contribution (a writable, a derive, a document, a + reference member, another registry's value) is followed: its change marks the table; one that fails or ends (a + registry whose `compose` threw too, though its readers keep its last good value), or whose adapter throws, is left + out and reported `contribution_failed` once per episode, and is back when it heals. Anything else is a plain value, + whatever methods it has (`undefined` included). +- Readers: `self.registry(R)` (a declared dependency, `host/`) and `await host.registry(R, ctx)` give a + `RegistryValue`: a native watchable (`.value`, `watch`) with `observe(observer, ctx)`, a synchronous observer of + its compositions (Durable's shape, for `registryReader`). A reader whose token is older than the provider's gets the + composition mapped back with `accepts[version].value`, once per composition, only while read or followed; a mapping + that throws fails that version only (`registry_failed`, its last good mapping kept). A reader without a mapping, or + newer than the provider, gets `contract_mismatch` (its object fails `broken`: only a reload helps). The mapping is + kept only while a reader or follower is there. Without a provider a registry has no value + (`unavailable/undeclared`); its contributions wait in the table. +- `keyedEntries(table, entriesOf, report)`: a map of every entry's keyed items, the first by plugin name keeping a key, + a duplicate reported through `compose`'s `report`; `orderedEntries(table, itemsOf, orderOf?)`: every entry's items in + table order, or by `orderOf`. Both keep each item's `plugin` (and `type`): a contribution is another plugin's code, so + a use site catches its throws and rejections, reports the plugin, and goes on. +- Registries carry natives: an entry lives exactly as long as its contributor. A last good composition kept after a + throwing `compose` may still name a contributor that left: use sites look up at each use and keep nothing. + +## For Chord's layers (`internal.ts`) + +- `RegistryRecord`: one per registry and host (`ChordHost.registryRecord(name)`, made by its first use): the provider + (`serve`), the contributions in table order (`add`, `remove`, inside a batch), the last table and composition + (`start()`: composed now when dirty; a read while composing is a cycle, `invalid/cycle`), the episode reports, the + direct edges between registries whose values are contributions in its table (counted, kept while those + contributions are in it; `assertNoCycle` refuses a cycle at `contribute`, through those edges and the + contributing attempt's own earlier declarations), and its values by reader version + (`value(version)`); `close()` at the host's close. +- `RegistryContribution` (`contribution.ts`): one contributor's contribution, a `Dependent` of its source; + `entry(version, provider)` is its table entry (the same object while its value is), or `undefined` when left out. +- `RegistryValueSource`: a `FollowedSource` per reader version; `read(waiter)` registers an activation's object + (`RegistryWaiter`), woken with `dependencyLive()` (on a microtask) when the value gets its first good value; readers + waiting for one count as followers, so a change composes for them. At the host's close it keeps its last start and + lets the record (and the host) go. diff --git a/packages/chord/src/next/registries/contribution.ts b/packages/chord/src/next/registries/contribution.ts new file mode 100644 index 000000000..7572a40a7 --- /dev/null +++ b/packages/chord/src/next/registries/contribution.ts @@ -0,0 +1,158 @@ +// One contributor's contribution to a registry (slice 8 §1): its table entry, made once per value (an older version's +// adapted once per value), followed when it is a Chord watchable, left out and reported when it fails or is refused. + +import { type Dependent, type DeriveSource, handedOut, type NativeSource, Source } from "../state/internal.ts"; +import type { RegistryRecord } from "./record.ts"; +import type { RegistryEntry, RegistryProvider } from "./registry.ts"; +import { RegistryValueSource } from "./value.ts"; + +/** Who contributes: a facet (by plugin) or a host-lifetime object (by plugin and type). */ +export type Contributor = { readonly plugin: string; readonly type: string | undefined }; + +/** + * What a contribution is: a plain value, a Chord JSON watchable (a source or a constant), or a native source; the + * watchables followed for their value. + */ +export type ContributionContent = + | { readonly kind: "plain"; readonly value: unknown } + | { readonly kind: "json"; readonly source: DeriveSource } + | { readonly kind: "native"; readonly source: NativeSource }; + +/** + * One contributor's contribution to a registry: a plain value, or a Chord source it follows as a dependent (a change + * marks the registry's table). Its table entry is made once per value (an older version's adapted once per value); + * a failure or refusal leaves it out, reported once per episode. + */ +export class RegistryContribution implements Dependent { + /** The registry it contributes to. */ + readonly record: RegistryRecord; + readonly contributor: Contributor; + /** The registry version its contributor was built against. */ + readonly version: number; + readonly #content: ContributionContent; + /** The last adaptation of an older contribution: the value it was made from and its result. */ + #adapted: { readonly from: unknown; readonly value: unknown } | undefined; + #entry: RegistryEntry | undefined; + /** Left out since its last entry: failing (reported once), or refused (reported once). */ + #out: "none" | "failed" | "refused" = "none"; + + constructor(record: RegistryRecord, contributor: Contributor, version: number, content: ContributionContent) { + this.record = record; + this.contributor = contributor; + this.version = version; + this.#content = content; + } + + /** Its source changed (inside a batch): the registry's table is marked. */ + invalidate(): void { + this.record.contributionChanged(); + } + + /** A retirement of its source's document changes nothing beyond the value. */ + sourceRetired(): void {} + + /** Starts following its source, if it has one. */ + follow(): void { + const source = this.#followed(); + source?.dependents.add(this); + } + + /** Stops following its source. */ + unfollow(): void { + this.#followed()?.dependents.delete(this); + } + + /** The registry whose value it is, when it is one (its edge in the cycle check, while it is in the table). */ + get reads(): RegistryRecord | undefined { + const content = this.#content; + return content.kind === "native" && content.source instanceof RegistryValueSource + ? content.source.record + : undefined; + } + + /** Its source when it changes (a constant never does). */ + #followed(): Source | NativeSource | undefined { + const content = this.#content; + if (content.kind === "native") return content.source; + return content.kind === "json" && content.source instanceof Source ? content.source : undefined; + } + + /** + * Its entry in a table of the provider at `version`: the same object while its value is the same; `undefined` when + * left out (a refused version, a failing source, a throwing adapter). + */ + entry(version: number, provider: RegistryProvider): RegistryEntry | undefined { + const adapter = this.version < version ? provider.accepts?.[this.version]?.contribution : undefined; + if (this.version > version || (this.version < version && adapter === undefined)) { + if (this.#out !== "refused") this.#refused(); + return undefined; + } + let value: unknown; + try { + const raw = this.#raw(); + value = adapter === undefined ? raw : this.#adapt(raw, adapter); + } catch (error) { + if (this.#out !== "failed") this.#failed(error); + return undefined; + } + this.#out = "none"; + const entry = this.#entry; + // `undefined` is a value: the entry, not its field, says whether there was one. + if (entry !== undefined && entry.value === value) return entry; + const { plugin, type } = this.contributor; + this.#entry = { plugin, ...(type === undefined ? {} : { type }), version: this.version, value }; + return this.#entry; + } + + /** + * The contributed value now: a JSON source's value throws while it is failed or ended; a native source that failed + * (a registry whose `compose` threw) keeps a last good value for its readers, but is left out here as failed. + */ + #raw(): unknown { + const content = this.#content; + switch (content.kind) { + case "plain": + return content.value; + case "json": + return content.source.value; + case "native": + return content.source.current(); + } + } + + #adapt(raw: unknown, adapter: (old: never) => unknown): unknown { + const adapted = this.#adapted; + if (adapted !== undefined && adapted.from === raw) return adapted.value; + const value = this.record.adapt(() => adapter(raw as never)); + this.#adapted = { from: raw, value }; + return value; + } + + #refused(): void { + this.#out = "refused"; + const { plugin, type } = this.contributor; + const registry = this.record.name; + const where = type === undefined ? {} : { type }; + this.record.reportContribution({ + kind: "contribution_refused", + registry, + plugin, + ...where, + version: this.version, + }); + } + + #failed(error: unknown): void { + this.#out = "failed"; + const { plugin, type } = this.contributor; + const registry = this.record.name; + const where = type === undefined ? {} : { type }; + this.record.reportContribution({ + kind: "contribution_failed", + registry, + plugin, + ...where, + error: handedOut(error), + }); + } +} diff --git a/packages/chord/src/next/registries/entries.ts b/packages/chord/src/next/registries/entries.ts new file mode 100644 index 000000000..069363ec8 --- /dev/null +++ b/packages/chord/src/next/registries/entries.ts @@ -0,0 +1,54 @@ +// Table helpers for `compose` (slice 8 §1, Q8): a map by key (the first by plugin name wins, duplicates reported) and +// an ordered list; both keep each item's plugin, so a use site reports the contributor of an entry that throws. + +import type { ComposeContext, RegistryTable } from "./registry.ts"; + +/** An item taken from a table, with the contributor it came from. */ +export type TableItem = { readonly value: I; readonly plugin: string; readonly type?: string }; + +/** + * A map of every entry's keyed items (`entriesOf`, e.g. its routes), in table order: a key taken already is reported + * through `report` (`compose`'s) naming both plugins, and the first by plugin name keeps it. + */ +export function keyedEntries( + table: RegistryTable, + entriesOf: (value: C) => Readonly> | undefined, + report: ComposeContext["report"], +): ReadonlyMap> { + const map = new Map>(); + for (const entry of table) { + const items = entriesOf(entry.value); + if (items === undefined) continue; + for (const [key, value] of Object.entries(items)) { + const taken = map.get(key); + if (taken !== undefined) { + report({ message: `"${key}" is contributed twice`, plugins: [taken.plugin, entry.plugin] }); + continue; + } + map.set(key, item(value, entry.plugin, entry.type)); + } + } + return map; +} + +/** + * Every entry's items (`itemsOf`) in one list: in table order, or sorted by `orderOf` (a phase's index, for example), + * table order kept among equals. + */ +export function orderedEntries( + table: RegistryTable, + itemsOf: (value: C) => readonly I[] | undefined, + orderOf?: (item: I) => number, +): readonly TableItem[] { + const items: TableItem[] = []; + for (const entry of table) { + for (const value of itemsOf(entry.value) ?? []) items.push(item(value, entry.plugin, entry.type)); + } + // `sort` is stable: equal orders keep table order. + if (orderOf !== undefined) items.sort((a, b) => orderOf(a.value) - orderOf(b.value)); + return items; +} + +function item(value: I, plugin: string, type: string | undefined): TableItem { + return type === undefined ? { value, plugin } : { value, plugin, type }; +} diff --git a/packages/chord/src/next/registries/index.ts b/packages/chord/src/next/registries/index.ts new file mode 100644 index 000000000..8c9d8154c --- /dev/null +++ b/packages/chord/src/next/registries/index.ts @@ -0,0 +1,16 @@ +// Registries for apps: tokens, what a provider serves them with, the table `compose` gets and its helpers, and the +// composed value readers follow. + +export { keyedEntries, orderedEntries, type TableItem } from "./entries.ts"; +export { + type ComposeContext, + type Contribution, + defineRegistry, + type Registry, + type RegistryAccepts, + type RegistryConflict, + type RegistryEntry, + type RegistryProvider, + type RegistryTable, + type RegistryValue, +} from "./registry.ts"; diff --git a/packages/chord/src/next/registries/internal.ts b/packages/chord/src/next/registries/internal.ts new file mode 100644 index 000000000..1a6969089 --- /dev/null +++ b/packages/chord/src/next/registries/internal.ts @@ -0,0 +1,5 @@ +// Registries for Chord's layers: a host's record of each registry, its contributions and its values by version. + +export { type ContributionContent, type Contributor, RegistryContribution } from "./contribution.ts"; +export { type RegistryEdge, type RegistryError, RegistryRecord } from "./record.ts"; +export { RegistryValueSource, type RegistryWaiter } from "./value.ts"; diff --git a/packages/chord/src/next/registries/record.ts b/packages/chord/src/next/registries/record.ts new file mode 100644 index 000000000..f879132f9 --- /dev/null +++ b/packages/chord/src/next/registries/record.ts @@ -0,0 +1,301 @@ +// A registry on a host (slice 8 §1): its provider, its contributions in table order (`contribution.ts`), the table +// and its lazy composition (marked when the table changes, composed when read or followed, in the batch's +// deliveries), the reports of conflicts and failures (once per episode), and the values its readers follow by +// version. + +import { ChordError } from "../data/index.ts"; +import { invalid } from "../data/internal.ts"; +import { type Batch, handedOut, type NativeStart } from "../state/internal.ts"; +import type { RegistryContribution } from "./contribution.ts"; +import type { ComposeContext, RegistryConflict, RegistryEntry, RegistryProvider, RegistryTable } from "./registry.ts"; +import { RegistryValueSource } from "./value.ts"; + +/** What a registry reports to its host's `onError`, by kind; each once per episode. */ +export type RegistryError = + /** `compose` reported a conflict it resolved. */ + | { + readonly kind: "registry_conflict"; + readonly registry: string; + readonly message: string; + readonly plugins: readonly string[]; + } + /** `compose` (or an older version's `value` adapter) threw: the last good composition stays. */ + | { readonly kind: "registry_failed"; readonly registry: string; readonly error: unknown } + /** A watchable contribution failed or ended, or its `contribution` adapter threw: it is left out until it heals. */ + | { + readonly kind: "contribution_failed"; + readonly registry: string; + readonly plugin: string; + readonly type?: string; + readonly error: unknown; + } + /** A contribution of a version the provider does not accept (newer, or older without an adapter) is left out. */ + | { + readonly kind: "contribution_refused"; + readonly registry: string; + readonly plugin: string; + readonly type?: string; + readonly version: number; + }; + +/** One registry's value contributed to another: `from`'s table reads `to`. */ +export type RegistryEdge = { readonly from: RegistryRecord; readonly to: RegistryRecord }; + +/** The provider as served: the defining plugin's version, `compose` and adapters. */ +type Provider = { readonly version: number; readonly provider: RegistryProvider }; + +/** Where a composition stands: never composed, its last good value, and its failure while `compose` throws. */ +type Composition = { + readonly good: { readonly value: unknown } | undefined; + readonly failure: { readonly error: unknown } | undefined; +}; + +const NEVER: Composition = { good: undefined, failure: undefined }; + +/** One registry of a host. */ +export class RegistryRecord { + readonly name: string; + readonly #batch: Batch; + readonly #report: (error: RegistryError) => void; + #provider: Provider | undefined; + /** The contributions of live contributors, sorted by plugin, then type (a facet's first). */ + readonly #contributions: RegistryContribution[] = []; + /** The last table composed, kept so an unchanged table composes nothing. */ + #table: RegistryTable | undefined; + #dirty = true; + #composition: Composition = NEVER; + /** Set while its table is built and composed: a read then is a cycle of registries. */ + #composing = false; + /** The conflicts the last good composition reported, by key: a conflict is reported once per episode. */ + #conflicts = new Set(); + /** + * Registries whose values are contributions in this one's table, each with how many: a contribution that would + * close a cycle of them is refused. An edge lives as long as its contribution is in the table. + */ + readonly #reads = new Map(); + /** The values readers follow, by version; made by the first reader of a version. */ + readonly #values = new Map(); + + constructor(name: string, batch: Batch, report: (error: RegistryError) => void) { + this.name = name; + this.#batch = batch; + this.#report = report; + } + + /** The provider's version; `undefined` while none serves it. */ + get providedVersion(): number | undefined { + return this.#provider?.version; + } + + /** The value readers of `version` follow. */ + value(version: number): RegistryValueSource { + let value = this.#values.get(version); + if (value === undefined) { + value = new RegistryValueSource(this, version, this.#batch); + this.#values.set(version, value); + } + return value; + } + + /** Serves the registry with `provider` at `version` (the defining plugin's facet); a second serve is refused. */ + serve(version: number, provider: RegistryProvider): void { + if (this.#provider !== undefined) throw new Error(`registry ${this.name} is served twice`); + // Declaration policy: plugin code is bundled without type checks. + if (typeof provider?.compose !== "function") throw new Error(`registry ${this.name}: compose is a function`); + this.#provider = { version, provider }; + } + + /** The first composition, at the end of `load`'s setups, inside one batch for every served registry. */ + composeServed(): void { + if (this.#provider === undefined) return; + this.#invalidate(); + this.start(); + } + + /** + * Refuses a contribution that is `source`'s value when it would close a cycle (declaration policy, at + * `contribute`): through the registries in the tables now and the edges `pending` (the contributing attempt's + * earlier declarations, which enter the tables only once it is live). + */ + assertNoCycle(source: RegistryValueSource, pending: readonly RegistryEdge[]): void { + const { record } = source; + if (record === undefined) return; + if (record.#reaches(this, new Set(), pending)) { + throw invalid("cycle", `registries ${this.name} and ${record.name} read each other`); + } + } + + /** Adds a live contributor's contribution, inside a batch: the table changes. */ + add(contribution: RegistryContribution): void { + const at = this.#contributions.findIndex((other) => compareContributors(contribution, other) < 0); + this.#contributions.splice(at < 0 ? this.#contributions.length : at, 0, contribution); + const { reads } = contribution; + if (reads !== undefined) this.#reads.set(reads, (this.#reads.get(reads) ?? 0) + 1); + contribution.follow(); + this.#invalidate(); + } + + /** Removes an ending contributor's contribution, inside a batch: the table changes. */ + remove(contribution: RegistryContribution): void { + const at = this.#contributions.indexOf(contribution); + if (at < 0) return; + this.#contributions.splice(at, 1); + const { reads } = contribution; + if (reads !== undefined) { + const left = this.#reads.get(reads)! - 1; + if (left === 0) this.#reads.delete(reads); + else this.#reads.set(reads, left); + } + contribution.unfollow(); + this.#invalidate(); + } + + /** A followed contribution changed, inside a batch: the table is marked, composed when read or followed. */ + contributionChanged(): void { + this.#invalidate(); + } + + /** + * The current composition, composed now when the table changed: the value, or `compose`'s failure with the last + * good one; `unavailable/undeclared` while no provider serves it. A read while it composes is a cycle. + */ + start(): NativeStart { + if (this.#composing) throw invalid("cycle", `registry ${this.name} is read while it composes`); + if (this.#provider === undefined) { + const error = new ChordError({ + code: "unavailable", + reason: "undeclared", + message: `registry ${this.name}: no plugin serves it here`, + }); + return { kind: "failed", error, last: undefined }; + } + if (this.#dirty) this.#recompose(this.#provider); + const { good, failure } = this.#composition; + if (failure !== undefined) return { kind: "failed", error: failure.error, last: good }; + // A composition without a failure has a value. + return { kind: "value", value: good!.value }; + } + + /** The adapter mapping the composed value back for readers of `version`; `undefined` when there is none. */ + valueAdapter(version: number): ((current: never) => unknown) | undefined { + return this.#provider?.provider.accepts?.[version]?.value; + } + + /** Runs an adapter of the provider as a pure compute, counted. */ + adapt(adapter: () => T): T { + this.#batch.count("chord.registry.adapts"); + return this.#batch.compute(adapter); + } + + /** Reports a contribution left out (`contribution_failed`, `contribution_refused`; the caller keeps the episode). */ + reportContribution(error: RegistryError): void { + this.#report(error); + } + + /** Reports a throw of `compose` or of a value adapter (`registry_failed`; the caller keeps the episode). */ + failed(error: unknown): void { + this.#report({ kind: "registry_failed", registry: this.name, error: handedOut(error) }); + } + + /** + * The host closed: the contributions stop following their sources, the values end (a value kept by a reader still + * reads the last composition). + */ + close(): void { + for (const contribution of this.#contributions) contribution.unfollow(); + this.#contributions.length = 0; + this.#reads.clear(); + this.#table = undefined; + this.#dirty = false; + for (const value of this.#values.values()) value.close(); + this.#values.clear(); + } + + /** + * The table changed: every version's value is marked and listed (each lists itself once per batch); while it is + * not composed since an earlier change, only listed (its dependents are marked already). + */ + #invalidate(): void { + if (this.#dirty) { + for (const value of this.#values.values()) value.relisted(); + return; + } + this.#dirty = true; + for (const value of this.#values.values()) value.recomposed(); + } + + /** + * Builds the table and composes it, unless it is the last one composed, entry for entry (`compose` is pure). A + * throw is a failure episode: reported once, the last good composition kept; conflicts are reported once per + * episode, as their message and plugins. + */ + #recompose({ version, provider }: Provider): void { + this.#dirty = false; + this.#composing = true; + try { + const table = this.#build(version, provider); + if (table === this.#table && this.#composition !== NEVER) return; + this.#table = table; + const conflicts = new Map(); + const context: ComposeContext = { + report: (conflict) => + conflicts.set(`${conflict.message}\u0000${conflict.plugins.join("\u0000")}`, conflict), + }; + let value: unknown; + try { + this.#batch.count("chord.registry.composes"); + value = this.#batch.compute(() => provider.compose(table, context)); + } catch (error) { + if (this.#composition.failure === undefined) this.failed(error); + this.#composition = { good: this.#composition.good, failure: { error } }; + return; + } + this.#composition = { good: { value }, failure: undefined }; + for (const [key, conflict] of conflicts) { + if (this.#conflicts.has(key)) continue; + this.#report({ kind: "registry_conflict", registry: this.name, ...conflict }); + } + this.#conflicts = new Set(conflicts.keys()); + } finally { + this.#composing = false; + } + } + + /** The table of the live contributions now; the last one itself when every entry is the same object. */ + #build(version: number, provider: RegistryProvider): RegistryTable { + const previous = this.#table; + const table: RegistryEntry[] = []; + let same = previous !== undefined; + for (const contribution of this.#contributions) { + const entry = contribution.entry(version, provider); + if (entry === undefined) continue; + if (same && previous![table.length] !== entry) same = false; + table.push(entry); + } + return same && previous!.length === table.length ? previous! : table; + } + + /** Whether `target` is reached from here through the tables' edges and `pending`. */ + #reaches(target: RegistryRecord, visited: Set, pending: readonly RegistryEdge[]): boolean { + if (this === target) return true; + visited.add(this); + for (const next of this.#reads.keys()) { + if (!visited.has(next) && next.#reaches(target, visited, pending)) return true; + } + for (const { from, to } of pending) { + if (from === this && !visited.has(to) && to.#reaches(target, visited, pending)) return true; + } + return false; + } +} + +/** Table order: by plugin name, then type, a facet (no type) first; code point order, never load order. */ +function compareContributors(a: RegistryContribution, b: RegistryContribution): number { + const left = a.contributor; + const right = b.contributor; + if (left.plugin !== right.plugin) return left.plugin < right.plugin ? -1 : 1; + const leftType = left.type ?? ""; + const rightType = right.type ?? ""; + if (leftType === rightType) return 0; + return leftType < rightType ? -1 : 1; +} diff --git a/packages/chord/src/next/registries/registry.ts b/packages/chord/src/next/registries/registry.ts new file mode 100644 index 000000000..6b79109b8 --- /dev/null +++ b/packages/chord/src/next/registries/registry.ts @@ -0,0 +1,89 @@ +// Registry tokens (design §17, slice 8 D1): `defineRegistry`, shared by every plugin as a contract is (identity and +// types only), and what the defining plugin's facet serves it with (`compose`, `accepts`); the table `compose` gets and +// the composed value readers follow. + +import type { Context } from "../../context/index.ts"; +import type { NativeWatchable, Watchable } from "../../watch.ts"; +import { checkName } from "../contracts/names.ts"; +import type { NativeObservation, NativeObserver } from "../state/index.ts"; + +/** + * A registry token: its name (as type names, `[a-z0-9@/._~-]`), the version the bundling plugin was built against, the + * contribution type `C` and the composed value type `V`. Shared: every contributor and reader bundles its own copy. + */ +export type Registry = { + readonly kind: "registry"; + readonly name: string; + readonly version: number; + /** Type only: `C` in, `V` out. */ + readonly "~types"?: (contribution: C) => V; +}; + +/** One contribution in a registry's table: its plugin, its object type for an object contributor, its version. */ +export type RegistryEntry = { + readonly plugin: string; + /** The contributing object's type; absent for a facet. */ + readonly type?: string; + /** The registry version its contributor was built against (an older one adapted to `C`). */ + readonly version: number; + readonly value: C; +}; + +/** A registry's table: one entry per live contributor, sorted by plugin name, then type (a facet's first). */ +export type RegistryTable = readonly RegistryEntry[]; + +/** A conflict `compose` resolved with its fallback: reported as `registry_conflict`, once per conflict episode. */ +export type RegistryConflict = { readonly message: string; readonly plugins: readonly string[] }; + +/** What `compose` gets besides the table. */ +export type ComposeContext = { + /** Reports a conflict `compose` resolved (the first by plugin name wins, for example). */ + report(conflict: RegistryConflict): void; +}; + +/** + * Adapters of older versions, by version, the provider's code: `contribution` maps an older contribution to `C` (once + * per contribution value), `value` maps the composed value back for a reader of that version (once per composition). + * A plugin annotates the old contribution's type (`(old: HttpV1) => upgrade(old)`). + */ +export type RegistryAccepts = { + readonly [version: number]: { + readonly contribution?: (old: never) => C; + readonly value?: (current: V) => unknown; + }; +}; + +/** What the defining plugin's facet serves a registry with (`host.serve(R, provider)`). */ +export type RegistryProvider = { + /** + * Composes the table: pure and synchronous, always a value (a fallback built in), conflicts reported. Runs when the + * table changed and someone reads or follows the value; a throw is the owner's bug (`registry_failed`, the last + * good composition stays). + */ + readonly compose: (table: RegistryTable, context: ComposeContext) => V; + readonly accepts?: RegistryAccepts; +}; + +/** What a contributor contributes: a plain value, or a Chord watchable of one (followed: its change recomposes). */ +export type Contribution = C | Watchable | NativeWatchable; + +/** + * A registry's composed value as a reader of one version sees it (`self.registry`, `host.registry`): a native + * watchable; `.value` is the last good composition (a later throwing `compose` keeps it), a watch gets each new one. + */ +export interface RegistryValue extends NativeWatchable { + /** + * A synchronous observer of its compositions (as Durable's `observeXXX`): `advance` runs inside the batch that + * composed, so it only records; an attached observer counts as watching, so the composition runs for it. + */ + observe(observer: NativeObserver, context: Context): NativeObservation; +} + +/** Declares a registry token: `name` and the `version` this plugin is built against. */ +export function defineRegistry(name: string, options: { readonly version: number }): Registry { + checkName(name, "registry"); + const { version } = options; + // Declaration policy: JavaScript callers reach this check. + if (!Number.isSafeInteger(version) || version < 1) throw new Error(`${name}.version: expected a positive integer`); + return { kind: "registry", name, version }; +} diff --git a/packages/chord/src/next/registries/value.ts b/packages/chord/src/next/registries/value.ts new file mode 100644 index 000000000..7b3b8e1db --- /dev/null +++ b/packages/chord/src/next/registries/value.ts @@ -0,0 +1,177 @@ +// Registry values (slice 8 §1 "Reading", S2, S3): a registry's composed value as the readers of one version see it, a +// native source. The provider's version reads the composition itself; an older one reads it mapped back with +// `accepts[old].value`, once per composition, kept only while someone reads or follows it; a version the provider +// cannot serve fails `contract_mismatch`. Readers declared by attempts are woken (R18) when the value gets its first +// good value. + +import { ChordError } from "../data/index.ts"; +import { type Batch, FollowedSource, type NativeStart } from "../state/internal.ts"; +import type { RegistryRecord } from "./record.ts"; + +/** An object waiting for a registry's value (`self.registry`): retried at once when its first value arrives (R18). */ +export interface RegistryWaiter { + dependencyLive(): void; +} + +/** The last good mapping for an older version, and the composition it was made from. */ +type Mapped = { readonly from: unknown; readonly value: unknown }; + +/** A failed mapping of composition `from`, until another composition maps. */ +type MapFailure = { readonly from: unknown; readonly error: unknown }; + +/** + * Where a value's registry stands for it: on the host, or closed with the host (its last start kept, the record let + * go, so a value an app keeps keeps no closed host). + */ +type Registered = + | { readonly kind: "open"; readonly record: RegistryRecord } + | { readonly kind: "closed"; readonly last: NativeStart }; + +/** One version's view of a registry's composed value. */ +export class RegistryValueSource extends FollowedSource { + readonly name: string; + readonly version: number; + #registered: Registered; + /** Readers declared by attempts, each a token of its own (one object may read twice), until released. */ + readonly #readers = new Set<{ readonly waiter: RegistryWaiter }>(); + /** Whether it had a good value: the first one wakes the readers. */ + #valued = false; + #mapped: Mapped | undefined; + #mapFailure: MapFailure | undefined; + + constructor(record: RegistryRecord, version: number, batch: Batch) { + super(batch); + this.name = record.name; + this.version = version; + this.#registered = { kind: "open", record }; + } + + /** Its registry's record while the host is open. */ + get record(): RegistryRecord | undefined { + const registered = this.#registered; + return registered.kind === "open" ? registered.record : undefined; + } + + /** The composition (or its mapping) at its last good value; throws the failure when there is none. */ + get value(): unknown { + const start = this.start(); + if (start.kind === "value") return start.value; + if (start.last === undefined) throw start.error; + return start.last.value; + } + + /** Its composition (mapped for an older version), composed now if its table changed; a first value wakes readers. */ + override start(): NativeStart { + const registered = this.#registered; + if (registered.kind === "closed") return registered.last; + const start = this.#start(registered.record); + if (!this.#valued && (start.kind === "value" || start.last !== undefined)) { + this.#valued = true; + // Not inside the batch that composed: a retry runs a factory, which may not run in a batch. + for (const { waiter } of this.#readers) queueMicrotask(() => waiter.dependencyLive()); + } + return start; + } + + /** Its composition (or mapping) now; throws while it is failed, though `.value` keeps the last good one. */ + override current(): unknown { + const start = this.start(); + if (start.kind === "failed") throw start.error; + return start.value; + } + + /** Followed while someone watches or observes it, and while readers wait for its first value. */ + override get watched(): boolean { + return super.watched || (this.#readers.size > 0 && !this.#valued); + } + + /** Registers `waiter` (an activation's declared dependency) until the returned release. */ + read(waiter: RegistryWaiter): () => void { + const token = { waiter }; + this.#readers.add(token); + return () => { + this.#readers.delete(token); + if (this.#readers.size === 0 && !super.watched) this.#dropMapping(); + }; + } + + /** The registry's table changed (inside a batch): its dependents are marked, its followers get the next frame. */ + recomposed(): void { + this.changed(); + } + + /** + * The registry's table changed again while not composed since (inside a batch): its dependents are marked already + * (none read it meanwhile, or it would be composed); its followers get the next frame. Marking stops here, so a + * cycle of registries through their values ends. + */ + relisted(): void { + this.list(); + } + + /** The host closed: its watches and observers end, its readers and mapping go; it keeps its last start only. */ + close(): void { + let last: NativeStart; + try { + last = this.start(); + } catch (error) { + last = { kind: "failed", error, last: undefined }; + } + this.#registered = { kind: "closed", last }; + this.#readers.clear(); + this.#dropMapping(); + this.end(); + } + + /** Nobody follows it: without readers, its mapping goes (computed again on the next read). */ + protected override unfollowed(): void { + if (this.#readers.size === 0) this.#dropMapping(); + } + + #dropMapping(): void { + this.#mapped = undefined; + this.#mapFailure = undefined; + } + + #start(record: RegistryRecord): NativeStart { + const provided = record.providedVersion; + if (provided === undefined || provided === this.version) return record.start(); + const map = provided > this.version ? record.valueAdapter(this.version) : undefined; + if (map === undefined) { + const message = `registry ${this.name}: version ${this.version} is not served here (version ${provided} is)`; + return { kind: "failed", error: new ChordError({ code: "contract_mismatch", message }), last: undefined }; + } + return this.#mappedStart(record, record.start(), map); + } + + /** + * The composition `current` mapped back with `map`, once per composition; a throwing mapping fails this version + * only (reported once per episode), keeping its last good mapping; a composition mapped before heals it. + */ + #mappedStart( + record: RegistryRecord, + current: NativeStart, + map: (current: never) => unknown, + ): NativeStart { + if (current.kind === "failed" && current.last === undefined) return current; + const from = current.kind === "value" ? current.value : current.last!.value; + // `undefined` is a composition too: the records, not their fields, say whether there is one. + const mapped = this.#mapped; + const failure = this.#mapFailure; + if (mapped !== undefined && mapped.from === from) { + this.#mapFailure = undefined; + } else if (failure === undefined || failure.from !== from) { + try { + this.#mapped = { from, value: record.adapt(() => map(from as never)) }; + this.#mapFailure = undefined; + } catch (error) { + if (failure === undefined) record.failed(error); + this.#mapFailure = { from, error }; + } + } + const last = this.#mapped === undefined ? undefined : { value: this.#mapped.value }; + if (this.#mapFailure !== undefined) return { kind: "failed", error: this.#mapFailure.error, last }; + if (current.kind === "failed") return { kind: "failed", error: current.error, last }; + return { kind: "value", value: last!.value }; + } +} diff --git a/packages/chord/src/next/state/README.md b/packages/chord/src/next/state/README.md index b386d167b..c10600a71 100644 --- a/packages/chord/src/next/state/README.md +++ b/packages/chord/src/next/state/README.md @@ -1,6 +1,7 @@ # state -Watchables, derives and the batch: how values change, are computed from each other and reach their watches. +Watchables (JSON and native), derives, writables and the batch: how values change, are computed from each other and +reach their watches. ## For apps (`index.ts`) @@ -25,6 +26,17 @@ watch.start(async (call, ctx) => { `[["r", value]]`). Values are JSON. A watch ends only for good (its owner ended, `stop()`, an aborted context, a throwing listener). - `constant(value)`: a watchable that never changes; a state member or a derive source. +- Native sources (`NativeWatchable`, W4): in-process values compared by identity, never ops, never state members (a + factory returning one fails `broken`, Q14): `self.writable(initial)`, a registry's composed value + (`self.registry`, `registries/`), and `ref.state`. They are derive sources (`self.derive([writable, doc], …)` takes + both kinds; the result is JSON) and `on` sources. A watch gets one call per batch that changed the value, each new + value in order; a failure (a registry whose `compose` threw) is one `failed` call per episode, the watch stays open. + `observe(observer, ctx)` attaches a synchronous observer (Durable's shape: `advance(call, ctx)` inside the batch, + record only; `end(end)`), which counts as watching. `ref.state` gives every state to its watches, not one per batch. +- `Writable` (`self.writable(initial)`): `set(value)`; the same value (by identity) notifies nobody and costs one + comparison; inside a batch it is marked there, outside it opens its own; inside a derive compute or a `compose` it + throws `invalid/reentrant`. It belongs to its attempt: `set` after the attempt's end throws `released`, its watches + end `ended`, its value stays readable. - `mapShared(f)`: maps an array, calling `f` once per object element (by identity) and reusing its result while the element stays the same object, so a derive over a list keeps unchanged parts identical and its diff cheap. @@ -55,8 +67,10 @@ Rules: the flushes listed with `flushing(flush)` during it (a link sends one state message per batch, W1); `within(adopt, ctx)` adopts a change inside the commit running now, or as a commit of its own, or, while a commit's flushes send, as the next commit (presence and keep-alive counts: a departure found while sending lands there); - `run(work)` sets the reentrancy flag (nested); `assertOutside(operation)`; `count(counter)` counts derive work on - the host's metrics (`chord.derive.computes`, `chord.derive.diffs`). + `run(work)` sets the reentrancy flag (nested); `compute(work)` also marks a pure compute (derives, `compose`), in + which `assertNotComputing(operation)` refuses changes; `assertOutside(operation)`; `count(counter)` counts derive and + registry work on the host's metrics (`chord.derive.computes`, `chord.derive.diffs`, `chord.registry.composes`, + `chord.registry.adapts`). `changed(source)` takes any `Delivered` (JSON or native source). - `Source`: a changing value on a host (a singleton document, a derive, a replica) with its watches, subscribers and dependents (`Dependent`: derives, and member sources passing changes on to theirs). Subclasses call `changed()` when a commit changes them and implement `deliver(ctx)` (`publish` a value, `failWatches` a recoverable failure, @@ -67,4 +81,13 @@ Rules: - `Derive`: declared with its raw sources; `attach(resolve, report)` resolves them at activation (and drops the raw list; `report` gets a failure of its compute, once per episode), `detach(error)` when its object ends (later reads throw `error` for good), `invalidate()` from its sources (a failed derive too: it recomputes on its next read). -- `Constant`: the class behind `constant()`. `DeriveSource` is what a derive reads: a `Source` or a `Constant`. +- `Constant`: the class behind `constant()`. `DeriveSource` is a JSON watchable as Chord follows it (a state member): + a `Source` or a `Constant`; `DeriveInput` is what a derive reads: those and `NativeSource`s. +- `NativeSource`: the base of native sources, what derives and registry tables read: `dependents`, `start()` + (`NativeStart`: a value, or a failure with its last good value), `changed()` (inside a batch: marks the dependents) + and `end()` (drops them and the batch). `FollowedSource` adds followers through the batch: `openWatch`, `observe` + (its abort listener removed at the end), `subscribe(observer)` (Chord's own followers, also without a value: + `retryOn`), `deliver(ctx)` (the new value when its followers have another, a failure once per episode; nothing is + kept once nobody follows). `WritableSource` is the writable, its owner (`WritableOwner`) the attempt's context and + seal check, let go when the attempt ends; `ref.state` (`host/`) is a plain `NativeSource` giving each state to its + own watches. diff --git a/packages/chord/src/next/state/batch.ts b/packages/chord/src/next/state/batch.ts index 66dbd5d5a..38eeb0901 100644 --- a/packages/chord/src/next/state/batch.ts +++ b/packages/chord/src/next/state/batch.ts @@ -4,13 +4,18 @@ import type { TelemetryMetrics } from "@earendil-works/pi-telemetry"; import type { Context } from "../../context/index.ts"; import { invalid } from "../data/internal.ts"; -import type { Source } from "./watchable.ts"; /** What a batch reports: a watch listener that threw, or a frame delivery that threw (a derive's value is no JSON). */ export type BatchError = | { readonly kind: "listener_error"; readonly error: unknown } | { readonly kind: "delivery_failed"; readonly error: unknown }; +/** What a commit lists to get its frame delivered: a watched source it changed (JSON or native). */ +export interface Delivered { + /** Delivers this batch's frame to its watches and subscribers; called once per batch that listed it. */ + deliver(context: Context): void; +} + /** A host's batch: the reentrancy flag, the watched sources a commit changed, and where background errors go. */ export class Batch { /** Reports a throwing watch listener's error (to the host's `onError`); what watches get as their `report`. */ @@ -19,8 +24,10 @@ export class Batch { readonly #metrics: TelemetryMetrics | undefined; /** Set while adopting, delivering or computing a derive; one flag per host. */ #running = false; + /** Set while a pure compute runs (a derive's, a registry's `compose`): nothing may change a source then. */ + #computing = false; /** The watched sources the current commit changed, each listed once by its `changed`. */ - #changed: Source[] = []; + #changed: Delivered[] = []; /** What runs once the current commit's frames were delivered, each listed once by its `flushing`. */ #flushes: ((context: Context) => void)[] = []; /** Set while the current commit's flushes run (its links send). */ @@ -46,6 +53,28 @@ export class Batch { if (this.#running) throw invalid("reentrant", `${operation} inside a commit or a derive compute`); } + /** + * Throws `invalid/reentrant` inside a pure compute (a derive's, a registry's `compose`): `operation` would change a + * source while one is computed from it. Changes inside an adoption are fine (they land in its batch). + */ + assertNotComputing(operation: string): void { + if (this.#computing) throw invalid("reentrant", `${operation} inside a derive compute or a compose`); + } + + /** Runs the pure compute `work` (`run`, and changes refused: `assertNotComputing`); nests. */ + compute(work: () => T): T { + const running = this.#running; + const computing = this.#computing; + this.#running = true; + this.#computing = true; + try { + return work(); + } finally { + this.#running = running; + this.#computing = computing; + } + } + /** Runs `work` with reentrant calls refused; nests (a derive compute inside a commit). */ run(work: () => T): T { const outer = this.#running; @@ -129,7 +158,7 @@ export class Batch { } /** Lists a watched source the current commit changed. */ - changed(source: Source): void { + changed(source: Delivered): void { this.#changed.push(source); } @@ -138,8 +167,12 @@ export class Batch { this.#flushes.push(flush); } - /** Counts one unit of derive work on the host's metrics (a lazy read has no context to count through). */ - count(counter: "chord.derive.computes" | "chord.derive.diffs"): void { + /** + * Counts one unit of derive or registry work on the host's metrics (a lazy read has no context to count through). + */ + count( + counter: "chord.derive.computes" | "chord.derive.diffs" | "chord.registry.composes" | "chord.registry.adapts", + ): void { this.#metrics?.counter(counter).add(1); } } diff --git a/packages/chord/src/next/state/derive.ts b/packages/chord/src/next/state/derive.ts index 6bb86f617..df364f10d 100644 --- a/packages/chord/src/next/state/derive.ts +++ b/packages/chord/src/next/state/derive.ts @@ -5,11 +5,15 @@ import type { Context } from "../../context/index.ts"; import { diffRevisions, type Op } from "../../delta/index.ts"; import type { JsonValue } from "../../json.ts"; import type { Batch } from "./batch.ts"; +import { NativeSource } from "./native-source.ts"; import { type Constant, type Dependent, handedOut, Source, type WatchStart } from "./watchable.ts"; -/** What a derive reads: a source it depends on, or a constant. */ +/** A JSON watchable as Chord follows it (a state member, a JSON derive source): a source, or a constant. */ export type DeriveSource = Source | Constant; +/** What a derive reads: a JSON source or constant, or a native source (a `writable`, a registry value, `ref.state`). */ +export type DeriveInput = DeriveSource | NativeSource; + /** * Where a derive stands. `dirty`: a source changed since the last compute (or there was none). A failed derive (its * compute threw) throws its error until a source it reads changes or is re-bound, then recomputes (self-healing, design @@ -34,10 +38,10 @@ const NO_VALUE: unique symbol = Symbol("no value"); * it. */ export class Derive extends Source implements Dependent { - readonly #compute: (values: readonly JsonValue[]) => JsonValue; + readonly #compute: (values: readonly unknown[]) => JsonValue; /** The sources as declared, until `attach` resolves them. */ #declared: readonly unknown[] | undefined; - #sources: readonly DeriveSource[] = []; + #sources: readonly DeriveInput[] = []; #state: DeriveState = { kind: "dirty" }; /** The value this derive's watches have, kept from its marking to its frame in the batch. */ #framed: JsonValue | typeof NO_VALUE | undefined; @@ -49,7 +53,7 @@ export class Derive extends Source implements Dependent { /** Its last good value, which a watch opened while it is failed starts at (`NO_VALUE` before the first). */ #good: JsonValue | typeof NO_VALUE = NO_VALUE; - constructor(sources: readonly unknown[], compute: (values: readonly JsonValue[]) => JsonValue, batch: Batch) { + constructor(sources: readonly unknown[], compute: (values: readonly unknown[]) => JsonValue, batch: Batch) { super(batch); this.#declared = sources; this.#compute = compute; @@ -77,16 +81,20 @@ export class Derive extends Source implements Dependent { * `report` gets a throw of its compute, once per failure episode (a value ends it); a failing source's error is its * source's to report. */ - attach(resolve: (source: unknown, index: number) => DeriveSource, report: (error: unknown) => void): void { + attach(resolve: (source: unknown, index: number) => DeriveInput, report: (error: unknown) => void): void { this.#report = report; this.#sources = this.#declared!.map(resolve); this.#declared = undefined; - for (const source of this.#sources) if (source instanceof Source) source.dependents.add(this); + for (const source of this.#sources) { + if (source instanceof Source || source instanceof NativeSource) source.dependents.add(this); + } } /** Its object ended: no source keeps it, its watches end `ended`, reads throw `error` (the object's). */ detach(error: Error): void { - for (const source of this.#sources) if (source instanceof Source) source.dependents.delete(this); + for (const source of this.#sources) { + if (source instanceof Source || source instanceof NativeSource) source.dependents.delete(this); + } this.#sources = []; this.#declared = undefined; this.#state = { kind: "detached", error }; @@ -146,7 +154,7 @@ export class Derive extends Source implements Dependent { } #recompute(): JsonValue { - return this.batch.run(() => { + return this.batch.compute(() => { try { this.batch.count("chord.derive.computes"); // A failing source throws its own error here, so a derive over a failed derive fails with the same error. diff --git a/packages/chord/src/next/state/index.ts b/packages/chord/src/next/state/index.ts index 8fbd9ad1e..b6ffb24db 100644 --- a/packages/chord/src/next/state/index.ts +++ b/packages/chord/src/next/state/index.ts @@ -1,5 +1,7 @@ -// State for apps: watchables, constants, and the derive helper `mapShared`. +// State for apps: watchables, constants, writables, observers of native sources, and the derive helper `mapShared`. export type { Watchable } from "../../watch.ts"; export { mapShared } from "./derive.ts"; +export type { NativeObservation, NativeObserver } from "./native-source.ts"; export { constant } from "./watchable.ts"; +export type { Writable } from "./writable.ts"; diff --git a/packages/chord/src/next/state/internal.ts b/packages/chord/src/next/state/internal.ts index 5ecd3641f..f4c0475cc 100644 --- a/packages/chord/src/next/state/internal.ts +++ b/packages/chord/src/next/state/internal.ts @@ -1,5 +1,14 @@ -// State for Chord's layers: the batch, sources, derives and constants behind the app's watchables. +// State for Chord's layers: the batch, sources (JSON and native), derives, constants and writables behind the +// app's watchables. -export { Batch, type BatchError } from "./batch.ts"; -export { Derive, type DeriveSource } from "./derive.ts"; +export { Batch, type BatchError, type Delivered } from "./batch.ts"; +export { Derive, type DeriveInput, type DeriveSource } from "./derive.ts"; +export { + FollowedSource, + type NativeObservation, + type NativeObserver, + NativeSource, + type NativeStart, +} from "./native-source.ts"; export { Constant, type Dependent, handedOut, Source, type Subscriber, type WatchStart } from "./watchable.ts"; +export { type WritableOwner, WritableSource } from "./writable.ts"; diff --git a/packages/chord/src/next/state/native-source.ts b/packages/chord/src/next/state/native-source.ts new file mode 100644 index 000000000..9d6308801 --- /dev/null +++ b/packages/chord/src/next/state/native-source.ts @@ -0,0 +1,291 @@ +// Native sources (W4, slice 8 §2): in-process values that change, compared by identity (a `writable`, a registry's +// composed value, `ref.state`). Derives and registry tables depend on them and the batch marks them; the followed +// ones (writables, registry values) give their watches and observers one call per batch that changed them. + +import type { Context } from "../../context/index.ts"; +import { + NativeWatch, + type NativeWatchable, + type NativeWatchCall, + type NativeWatchHandle, + type WatchEnd, +} from "../../watch.ts"; +import type { Batch } from "./batch.ts"; +import { type Dependent, handedOut } from "./watchable.ts"; + +/** + * A synchronous observer of a native source (Durable's observer shape): `advance` is called inside the batch that + * changed the source, so it only records; `end` once, when the source ended (`ended`) or the observer's context was + * aborted (`aborted`). + */ +export interface NativeObserver { + /** The source's call of this batch: its new value, or a failure it may recover from. */ + advance(call: NativeWatchCall, context: Context): void; + /** The observer is detached for good with `end`. */ + end(end: WatchEnd): void; +} + +/** What `observe` gives: the value the observer starts at, and how to detach it (Durable's `Observation`). */ +export type NativeObservation = { readonly value: T; detach(): void }; + +/** + * What a new watch of a native source starts at: its value, or while it is failed, the failure and its last good value + * (`last` undefined when it never had one: a watch cannot start then). + */ +export type NativeStart = + | { readonly kind: "value"; readonly value: T } + | { readonly kind: "failed"; readonly error: unknown; readonly last: { readonly value: T } | undefined }; + +/** + * A native source as derives and registry tables read it: its value (`start()` while it may fail), the dependents a + * change marks (`changed()`, inside a batch), and the host's batch until it ended. How its watches get their calls is + * the subclass's (`FollowedSource` for writables and registry values; `ref.state` gives each state to its watches). + */ +export abstract class NativeSource implements NativeWatchable { + /** Derives and registry tables reading this source; each removes itself when it stops. */ + readonly dependents = new Set(); + /** The host's batch; dropped when the source ended, so an ended source keeps no host. */ + #batch: Batch | undefined; + + /** `batch` undefined: made ended (a released reference's state). */ + constructor(batch: Batch | undefined) { + this.#batch = batch; + } + + abstract get value(): T; + + abstract watch(context: Context): Promise>; + + /** What a new watch starts at; a source that can fail and recover tells its failure. */ + start(): NativeStart { + return { kind: "value", value: this.value }; + } + + /** + * Its value now as a contribution reads it: throws its failure while it is failed, where `.value` keeps the last good + * value for readers (a registry whose `compose` threw). + */ + current(): T { + return this.value; + } + + /** The host's batch, `undefined` once the source ended. */ + protected get batch(): Batch | undefined { + return this.#batch; + } + + /** Records a change inside the current batch: every dependent marked. */ + protected changed(): void { + for (const dependent of this.dependents) dependent.invalidate(); + } + + /** Ends the source for good: its dependents dropped, the batch let go. */ + protected end(): void { + this.#batch = undefined; + this.dependents.clear(); + } +} + +/** + * What the watches and observers were last given: a value, a failure (one call per failure episode), or nothing yet + * (a source with no value, followed for its first one). + */ +type Shown = { readonly kind: "value"; readonly value: T } | { readonly kind: "failed" } | { readonly kind: "none" }; + +const ENDED: WatchEnd = { kind: "ended" }; + +const ABORTED: WatchEnd = { kind: "aborted" }; + +const FAILED: Shown = { kind: "failed" }; + +const NONE = { kind: "none" } as const; + +const NOTHING = (): void => {}; + +/** + * A native source its followers get through the batch (a writable, a registry value): a change lists it for one frame + * when someone follows it; the frame is its value then, delivered only when it is another value (identity) than the + * one its followers have, or after a failure, whose call comes once per episode. + */ +export abstract class FollowedSource extends NativeSource { + readonly #watches = new Set>(); + /** Synchronous observers and Chord's own subscribers (`retryOn`), fed inside the batch, each with its cleanup. */ + readonly #observers = new Map, () => void>(); + /** What the followers have; `undefined` while nobody follows (so no old value is kept). */ + #shown: Shown | undefined; + /** Listed for a frame in the current batch. */ + #listed = false; + + /** Whether a watch or an observer follows it (a registry value also while readers wait for its first value). */ + get watched(): boolean { + return this.#watches.size > 0 || this.#observers.size > 0; + } + + async watch(context: Context): Promise> { + return this.openWatch(context); + } + + /** + * `watch` without the Promise. On a failed source it opens at the last good value and its first call is the + * failure; with no value to start at it throws the failure. An ended source gives a watch that has ended. + */ + openWatch(context: Context): NativeWatch { + const batch = this.batch; + if (batch === undefined) { + const ended = new NativeWatch(this.value, NOTHING, NOTHING); + ended.end(ENDED); + return ended; + } + batch.assertOutside("watch"); + const start = this.start(); + if (start.kind === "failed" && start.last === undefined) throw start.error; + const value = start.kind === "value" ? start.value : start.last!.value; + this.#follow(start); + const watch: NativeWatch = new NativeWatch(value, () => this.#unfollow(watch), batch.reportListener); + this.#watches.add(watch); + if (context.abortSignal !== undefined) watch.observeAbort(context.abortSignal); + if (start.kind === "failed") watch.fail(handedOut(start.error), context); + return watch; + } + + /** + * Attaches a synchronous observer at the current value (Durable's `observeXXX`); it counts as following, so a lazy + * value is computed in the batch that changed it. Throws the failure when there is no value to start at; aborting + * `context` detaches it (`aborted`); the source's end detaches it too (`ended`), its abort listener removed. + */ + observe(observer: NativeObserver, context: Context): NativeObservation { + this.batch?.assertOutside("observe"); + const start = this.start(); + if (start.kind === "failed" && start.last === undefined) throw start.error; + const value = start.kind === "value" ? start.value : start.last!.value; + if (this.batch === undefined) { + observer.end(ENDED); + return { value, detach: NOTHING }; + } + const signal = context.abortSignal; + if (signal?.aborted) { + observer.end(ABORTED); + return { value, detach: NOTHING }; + } + const aborted = (): void => { + if (!this.#observers.has(observer)) return; + this.unsubscribe(observer); + observer.end(ABORTED); + }; + const cleanup = signal === undefined ? NOTHING : () => signal.removeEventListener("abort", aborted); + signal?.addEventListener("abort", aborted, { once: true }); + this.#follow(start); + this.#observers.set(observer, cleanup); + return { value, detach: () => this.unsubscribe(observer) }; + } + + /** + * Feeds `observer` every call from now on, inside each batch, also while the source has no value (Chord's own + * followers: `retryOn`). Nothing is delivered for the current value. + */ + subscribe(observer: NativeObserver): void { + if (this.batch === undefined) return; + this.#follow(this.#startOrNone()); + this.#observers.set(observer, NOTHING); + } + + unsubscribe(observer: NativeObserver): void { + const cleanup = this.#observers.get(observer); + if (cleanup === undefined) return; + this.#observers.delete(observer); + cleanup(); + this.#settleFollowers(); + } + + /** Delivers this batch's frame: the new value when the followers have another, or a failure once per episode. */ + deliver(context: Context): void { + this.#listed = false; + const shown = this.#shown; + if (shown === undefined) return; + this.#deliver(shown, context); + // Readers waiting for a first value pulled it through this delivery: nothing is kept for them. + this.#settleFollowers(); + } + + /** + * Records a change inside the current batch: every dependent marked, and the source listed for a frame when someone + * follows it (or `watched` says so). + */ + protected override changed(): void { + super.changed(); + this.list(); + } + + /** + * Lists the source for a frame in the current batch when someone follows it (or `watched` says so), its dependents + * left as they are (marked already). + */ + protected list(): void { + if (this.#listed || !this.watched || this.batch === undefined) return; + this.#listed = true; + // Only readers waiting for a first value follow it without a follower's record: they have none. Not computed + // here: marking runs inside adoptions, the value is pulled in the delivery. + this.#shown ??= NONE; + this.batch.changed(this); + } + + /** Nothing follows it any more (no watch, no observer): a subclass drops what it kept for followers. */ + protected unfollowed(): void {} + + /** Ends the source for good: watches and observers end `ended` (their abort listeners removed). */ + protected override end(): void { + super.end(); + this.#shown = undefined; + // Ending detaches, which deletes the current entry (safe while iterating) and runs no listener. + for (const watch of this.#watches) watch.end(ENDED); + const observers = [...this.#observers]; + this.#observers.clear(); + for (const [observer, cleanup] of observers) { + cleanup(); + observer.end(ENDED); + } + } + + #deliver(shown: Shown, context: Context): void { + const start = this.#startOrNone(); + if (start.kind === "failed") { + // A failure reaches followers that had a value: those without one wait for the first. + if (shown.kind !== "value") return; + this.#shown = FAILED; + const error = handedOut(start.error); + for (const watch of this.#watches) watch.fail(error, context); + for (const observer of [...this.#observers.keys()]) observer.advance({ kind: "failed", error }, context); + return; + } + if (start.kind !== "value" || (shown.kind === "value" && shown.value === start.value)) return; + const { value } = start; + this.#shown = start; + for (const watch of this.#watches) watch.advance(value, context); + // A copy: an observer may detach itself in its call. + for (const observer of [...this.#observers.keys()]) observer.advance({ kind: "value", value }, context); + } + + /** Its start, `none` while it has no value. */ + #startOrNone(): NativeStart | { readonly kind: "none" } { + const start = this.start(); + return start.kind === "failed" && start.last === undefined ? NONE : start; + } + + /** A follower comes: the first one records what followers have (a later one opened at the same current value). */ + #follow(start: NativeStart | { readonly kind: "none" }): void { + if (this.#shown !== undefined) return; + this.#shown = start.kind === "failed" ? FAILED : start; + } + + #unfollow(watch: NativeWatch): void { + this.#watches.delete(watch); + this.#settleFollowers(); + } + + /** No follower left: nothing is shown any more, so no old value is kept. */ + #settleFollowers(): void { + if (this.#watches.size > 0 || this.#observers.size > 0) return; + this.#shown = undefined; + this.unfollowed(); + } +} diff --git a/packages/chord/src/next/state/writable.ts b/packages/chord/src/next/state/writable.ts new file mode 100644 index 000000000..e8760b409 --- /dev/null +++ b/packages/chord/src/next/state/writable.ts @@ -0,0 +1,78 @@ +// Writables (slice 8 §2, design §17): `self.writable(initial)`, an in-process value its attempt sets, identity +// compared; a contribution, a derive source and an `on` source, never a state member. + +import type { Context } from "../../context/index.ts"; +import type { NativeWatchable } from "../../watch.ts"; +import type { Batch } from "./batch.ts"; +import { FollowedSource } from "./native-source.ts"; + +/** An in-process value that changes when its owner sets it; `set` of the same value notifies nobody. */ +export interface Writable extends NativeWatchable { + /** + * Sets the value: a new value (by identity) is a new frame for its watches and marks the derives and registry + * tables reading it, in the batch running now or in one of its own. Throws `released` once its attempt ended, and + * `invalid/reentrant` inside a derive compute or a `compose`. + */ + set(value: T): void; +} + +/** Who owns a writable: the context its own batches run in, and the check that its attempt still runs. */ +export type WritableOwner = { + readonly context: Context; + /** Throws `released` once the attempt that made it was sealed. */ + assertUsable(): void; +}; + +/** Whom a writable belongs to: its running attempt, or nothing once that attempt ended (its error kept). */ +type Ownership = + | { readonly kind: "owned"; readonly owner: WritableOwner } + | { readonly kind: "ended"; readonly error: Error }; + +/** A writable: its value, set by its owner's code, delivered through the host's batch. */ +export class WritableSource extends FollowedSource implements Writable { + #value: T; + #ownership: Ownership; + + constructor(initial: T, owner: WritableOwner, batch: Batch) { + super(batch); + this.#value = initial; + this.#ownership = { kind: "owned", owner }; + } + + get value(): T { + return this.#value; + } + + set(value: T): void { + const ownership = this.#ownership; + if (ownership.kind === "ended") throw ownership.error; + ownership.owner.assertUsable(); + // The cost guard: setting the value it has costs one comparison past the seal check. + if (value === this.#value) return; + const batch = this.batch!; + batch.assertNotComputing("writable.set"); + this.#change(value, batch, ownership.owner.context); + } + + /** + * Adopts `value` in the batch running now, or in one of its own. Apart from `set`: a closure there would make every + * `set` allocate its context, the same value's too. + */ + #change(value: T, batch: Batch, context: Context): void { + batch.within(() => { + this.#value = value; + this.changed(); + }, context); + } + + /** + * Its attempt ended with `error`: its watches end `ended`, `set` throws `error`, its value stays readable; it keeps + * neither its attempt nor the host. + */ + detach(error: Error): void { + // Kept by `set`: its stack is made text now, so it holds none of the frames' objects. + void error.stack; + this.#ownership = { kind: "ended", error }; + this.end(); + } +} diff --git a/packages/chord/test/next/bench/calls.ts b/packages/chord/test/next/bench/calls.ts new file mode 100644 index 000000000..d8e5dc0f7 --- /dev/null +++ b/packages/chord/test/next/bench/calls.ts @@ -0,0 +1,90 @@ +// Benchmark of the paths slice 8 touched under every object (no registries involved): a local method call (its ctx), +// a commit through a watched derive (the batch, a derive's compute) and activation churn (an object with params made, +// live and ended per get: its attempt's declarations). Median µs per operation over bursts, and heap allocated per +// operation (growth plus what the collections during the burst freed). Runs on HEAD as it is (copy it there). +// Run by hand from packages/chord (not in CI), pinned for comparisons: +// taskset -c 6 node --conditions=source --experimental-strip-types --expose-gc test/next/bench/calls.ts +import { performance } from "node:perf_hooks"; +import { GCProfiler } from "node:v8"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { int, local, method, obj } from "../../../src/next/contracts/index.ts"; +import { defineState } from "../../../src/next/documents/index.ts"; +import { createHost, defineFacet } from "../../../src/next/host/index.ts"; + +const BURSTS = 9; + +const gc = (globalThis as { gc?: () => void }).gc!; + +const Counter = defineState("bench.counter", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 0 }) }); +const Store = local("bench.store", { + version: 1, + params: {}, + members: { doubled: int(), bump: method(), set: method(int()) }, +}); +const Item = local("bench.item", { version: 1, params: { id: int() }, members: { n: int() } }); + +const host = createHost({ onError: (error) => console.log("onError", error.kind) }); +await host.load( + [ + { + name: "@bench/calls", + facet: defineFacet((h) => { + h.serve(Store, (_, self) => { + const counter = self.state(Counter); + return { + doubled: self.derive([counter], ([c]) => c.n * 2), + bump: async () => null, + set: async (n, ctx) => { + await counter.change((draft) => { + draft.n = n; + }, ctx); + return null; + }, + }; + }); + h.serve(Item, ({ id }, self) => ({ n: self.derive([self.get(Store, {}).doubled], ([d]) => d + id) })); + }), + }, + ], + context, +); + +const store = await host.get(Store, {}, context); +const watch = await store.doubled.watch(context); +watch.start(async () => {}); + +/** Median µs per operation over the bursts (after one warm-up burst), and heap bytes allocated per operation. */ +async function measure(runs: number, operation: (i: number) => Promise): Promise { + const times: number[] = []; + let bytes = 0; + let counted = 0; + for (let burst = -1; burst < BURSTS; burst++) { + gc(); + const before = process.memoryUsage().heapUsed; + const profiler = new GCProfiler(); + profiler.start(); + const started = performance.now(); + for (let i = 0; i < runs; i++) await operation(i); + const elapsed = performance.now() - started; + const after = process.memoryUsage().heapUsed; + let freed = 0; + for (const { beforeGC, afterGC } of profiler.stop().statistics) { + freed += beforeGC.heapStatistics.usedHeapSize - afterGC.heapStatistics.usedHeapSize; + } + if (burst < 0) continue; + times.push((elapsed * 1000) / runs); + bytes += after - before + freed; + counted += runs; + } + times.sort((a, b) => a - b); + return `${times[Math.floor(times.length / 2)]!.toFixed(2)} µs, ${Math.round(bytes / counted)} B allocated`; +} + +console.log("local method call:", await measure(20_000, () => store.bump(context))); +console.log("commit through a watched derive:", await measure(10_000, (i) => store.set(i + 1, context))); +console.log( + "activation churn (get, live, release):", + await measure(2_000, async (i) => (await host.get(Item, { id: i }, context)).release()), +); +await watch.stop(); +await host.close(context); diff --git a/packages/chord/test/next/bench/registries.ts b/packages/chord/test/next/bench/registries.ts new file mode 100644 index 000000000..5eb75504b --- /dev/null +++ b/packages/chord/test/next/bench/registries.ts @@ -0,0 +1,108 @@ +// Benchmark of slice 8's registries and writables: a `writable.set` delivered to one watch, a set of the same value +// (one comparison), and one contribution of N (10, 100) changed while the registry is watched (one table build and one +// compose per set, over N entries). Median µs per operation over bursts, heap allocated per operation, and the +// registry's compose count per set (the cost guard: one). +// Run by hand from packages/chord (not in CI), pinned for comparisons: +// taskset -c 6 node --conditions=source --experimental-strip-types --expose-gc test/next/bench/registries.ts +import { performance } from "node:perf_hooks"; +import { GCProfiler } from "node:v8"; +import { InMemoryTelemetryContext } from "@earendil-works/pi-telemetry"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { int, local } from "../../../src/next/contracts/index.ts"; +import { createHost, defineFacet } from "../../../src/next/host/index.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; +import type { Writable } from "../../../src/next/state/index.ts"; + +const BURSTS = 9; + +const gc = (globalThis as { gc?: () => void }).gc!; + +/** Median µs per operation over the bursts (after one warm-up burst), and heap bytes allocated per operation. */ +async function measure(runs: number, operation: (i: number) => void): Promise { + const times: number[] = []; + let bytes = 0; + let counted = 0; + for (let burst = -1; burst < BURSTS; burst++) { + gc(); + const before = process.memoryUsage().heapUsed; + const profiler = new GCProfiler(); + profiler.start(); + const started = performance.now(); + for (let i = 0; i < runs; i++) operation(i); + // The watches' listeners run on microtasks: their work counts too. + await new Promise((resolve) => setImmediate(resolve)); + const elapsed = performance.now() - started; + const after = process.memoryUsage().heapUsed; + let freed = 0; + for (const { beforeGC, afterGC } of profiler.stop().statistics) { + freed += beforeGC.heapStatistics.usedHeapSize - afterGC.heapStatistics.usedHeapSize; + } + if (burst < 0) continue; + times.push((elapsed * 1000) / runs); + bytes += after - before + freed; + counted += runs; + } + times.sort((a, b) => a - b); + return `${times[Math.floor(times.length / 2)]!.toFixed(3)} µs, ${Math.round(bytes / counted)} B allocated`; +} + +const Sum = defineRegistry("bench.sum", { version: 1 }); + +/** A host with `contributors` host-lifetime objects each contributing a writable number to `Sum`, watched. */ +async function registryOf(contributors: number) { + const telemetry = new InMemoryTelemetryContext(); + const writables: Writable[] = []; + const types = Array.from({ length: contributors }, (_, i) => + local(`bench.contributor-${i}`, { version: 1, params: {}, members: { n: int() } }), + ); + const host = createHost({ telemetry, onError: (error) => console.log("onError", error.kind) }); + const facet = defineFacet((h) => { + h.serve(Sum, { compose: (table) => table.reduce((sum, { value }) => sum + value, 0) }); + for (const Type of types) { + h.serve(Type, (_, self) => { + const value = self.writable(0); + writables.push(value); + self.contribute(Sum, value); + return { n: self.derive([value], ([n]) => n) }; + }); + } + }); + await host.load([{ name: "@bench/registries", facet }], context); + const sum = await host.registry(Sum, context); + const watch = await sum.watch(context); + watch.start(async () => {}); + const composes = (): number => + telemetry + .getMetrics() + .filter(({ name }) => name === "chord.registry.composes") + .reduce((n, { value }) => n + value, 0); + return { host, writables, watch, composes }; +} + +const one = await registryOf(1); +const writable = one.writables[0]!; +const watch = await writable.watch(context); +watch.start(async () => {}); +console.log("writable.set, one watch:", await measure(20_000, (i) => writable.set(i + 1))); +// The listeners' backlogs dropped first: what follows measures the set alone. +await watch.stop(); +await one.watch.stop(); +writable.set(-1); +console.log("writable.set of the same value:", await measure(1_000_000, () => writable.set(-1))); +console.log("an empty operation (the loop's own cost):", await measure(1_000_000, () => {})); +await one.host.close(context); + +for (const contributors of [10, 100]) { + const { host, writables, watch, composes } = await registryOf(contributors); + const before = composes(); + const target = writables[Math.floor(contributors / 2)]!; + const sets = 5_000; + const result = await measure(sets, (i) => target.set(i + 1)); + console.log( + `one of ${contributors} contributions set, registry watched:`, + result, + `${((composes() - before) / (sets * (BURSTS + 1))).toFixed(2)} composes per set`, + ); + await watch.stop(); + await host.close(context); +} diff --git a/packages/chord/test/next/data/errors.test.ts b/packages/chord/test/next/data/errors.test.ts index 65dee086a..c5d955a99 100644 --- a/packages/chord/test/next/data/errors.test.ts +++ b/packages/chord/test/next/data/errors.test.ts @@ -25,7 +25,9 @@ describe("ChordError", () => { const error = new ChordError({ code: "too_large", reason: "part", message: "m" }) as ChordError; if (error.code === "invalid") expectTypeOf(error.reason).toEqualTypeOf>(); if (error.code === "unavailable") - expectTypeOf(error.reason).toEqualTypeOf<"locked" | "park_timeout" | "busy" | "storage_failed" | undefined>(); + expectTypeOf(error.reason).toEqualTypeOf< + "locked" | "park_timeout" | "busy" | "storage_failed" | "undeclared" | undefined + >(); if (error.code === "gone") expectTypeOf(error.reason).toEqualTypeOf(); expect(error).toBeInstanceOf(Error); expect(error.name).toBe("ChordError"); @@ -83,7 +85,7 @@ describe("protocol table", () => { not_ready: [], released: ["ended", "closed", "settled", "version", "discarded"], too_large: ["part"], - unavailable: ["locked", "park_timeout", "busy", "storage_failed"], + unavailable: ["locked", "park_timeout", "busy", "storage_failed", "undeclared"], aborted: [], }, }; diff --git a/packages/chord/test/next/examples/29-registries.ts b/packages/chord/test/next/examples/29-registries.ts new file mode 100644 index 000000000..462bd20b0 --- /dev/null +++ b/packages/chord/test/next/examples/29-registries.ts @@ -0,0 +1,115 @@ +// Registries: an extension point between plugins. The defining plugin shares a token (`defineRegistry`: a name, a +// version, the contribution and value types) and serves it with `compose`, a pure function from the table of +// contributions (sorted by plugin name, never load order) to one value, conflicts resolved by a fallback and reported +// (`registry_conflict`). Facets and host-lifetime objects contribute one value each: a plain value, or a watchable +// (`self.writable`) whose changes recompose. Readers follow the composed value (`self.registry`, a dependency like +// `self.get`; `host.registry` outside objects) and guard each contribution where they call it. +// Run from packages/chord: +// node --conditions=source --experimental-strip-types test/next/examples/29-registries.ts +import { BACKGROUND_CONTEXT } from "../../../src/context/index.ts"; +import { local, method, obj, str } from "../../../src/next/contracts/index.ts"; +import { createHost, defineFacet } from "../../../src/next/host/index.ts"; +import { defineRegistry, keyedEntries, type TableItem } from "../../../src/next/registries/index.ts"; + +const context = BACKGROUND_CONTEXT; + +// 1. The token, shared by every plugin (bundled like a contract): routes in, a router out. +type Handler = (path: string) => string; +type Routes = { readonly routes: Readonly> }; +type Router = ReadonlyMap>; +const Http = defineRegistry("example.http", { version: 1 }); + +/** The HTTP server: it reads the router at each request and guards the contribution it calls. */ +const Server = local("example.server", { + version: 1, + params: {}, + members: { request: method(obj({ path: str() }), str()) }, +}); + +/** A plugin whose routes are known only at runtime: it contributes a writable it sets. */ +const Mounts = local("example.mounts", { version: 1, params: {}, members: { mount: method(obj({ path: str() })) } }); + +// 2. The defining plugin serves the registry: the first route by plugin name wins, a duplicate is reported. +const httpPlugin = defineFacet((host) => { + host.serve(Http, { compose: (table, { report }) => keyedEntries(table, (entry) => entry.routes, report) }); + host.serve(Server, (_, self) => { + const router = self.registry(Http); + return { + request: async ({ path }) => { + const route = router.value.get(path); + if (route === undefined) return "404"; + // A contribution is another plugin's code: catch, report its plugin, go on. + try { + return route.value(path); + } catch (error) { + console.log(`route ${path} of ${route.plugin} threw:`, (error as Error).message); + return "500"; + } + }, + }; + }); +}); + +// 3. Two plugins contribute from their facets: plain values, in the table while the plugin is loaded. +const notesPlugin = defineFacet((_, self) => { + self.contribute(Http, { routes: { "/notes": () => "notes", "/broken": () => JSON.parse("{") } }); +}); +const searchPlugin = defineFacet((_, self) => { + self.contribute(Http, { routes: { "/search": () => "results", "/notes": () => "search's notes" } }); +}); + +// 4. A host-lifetime object contributes a writable: its changes recompose the router. +const mountsPlugin = defineFacet((host) => { + host.serve(Mounts, (_, self) => { + const routes = self.writable({ routes: {} }); + self.contribute(Http, routes); + return { + mount: async ({ path }) => { + routes.set({ routes: { ...routes.value.routes, [path]: () => `mounted ${path}` } }); + return null; + }, + }; + }); +}); + +const host = createHost({ + onError: (error) => { + if (error.kind === "registry_conflict") console.log("conflict:", error.message, error.plugins); + else console.log("onError:", error.kind); + }, +}); + +// 5. A reader before `load` waits: the provider comes with `load`, which composes every registry before any object. +const early = host.registry(Http, context); +await host.load( + [ + { name: "@example/http", facet: httpPlugin }, + { name: "@example/notes", facet: notesPlugin }, + { name: "@example/search", facet: searchPlugin }, + { name: "@example/mounts", facet: mountsPlugin }, + ], + context, +); +const router = await early; +console.log( + "routes:", + [...router.value].map(([path, { plugin }]) => `${path} (${plugin})`), +); + +// 6. Requests: a duplicate went to the first plugin by name; a throwing contribution is guarded at the use site. +const server = await host.get(Server, {}, context); +console.log("/notes ->", await server.request({ path: "/notes" }, context)); +console.log("/broken ->", await server.request({ path: "/broken" }, context)); + +// 7. A watch of the composed value gets each new router; mounting recomposes it. +const watch = await router.watch(context); +watch.start(async (call) => { + if (call.kind === "value") console.log("router now has:", [...call.value.keys()]); +}); +const mounts = await host.get(Mounts, {}, context); +await mounts.mount({ path: "/files" }, context); +await new Promise((resolve) => setTimeout(resolve, 0)); +console.log("/files ->", await server.request({ path: "/files" }, context)); + +await watch.stop(); +await host.close(context); diff --git a/packages/chord/test/next/examples/30-host-root.ts b/packages/chord/test/next/examples/30-host-root.ts new file mode 100644 index 000000000..885ab4414 --- /dev/null +++ b/packages/chord/test/next/examples/30-host-root.ts @@ -0,0 +1,85 @@ +// `host.root(fn, context)`: a call scope for code outside objects (a Durable tool, a CLI command). `fn` gets a ctx like a +// method handler's: the host's principal (or one a trusted caller names), this host's incarnation, the context's +// telemetry and signal, and `ctx.get` for references. When `fn` settles, or the context aborts, or the host closes, the +// scope is sealed: its signal aborts, its references are released, and a late `ctx.get` is refused `released`; so code +// that ignores its signal fails at its next use of Chord. (p3 wraps it as `chord(fn, context)` for Durable's tools.) +// Run from packages/chord: +// node --conditions=source --experimental-strip-types test/next/examples/30-host-root.ts +import { BACKGROUND_CONTEXT, withCancel } from "../../../src/context/index.ts"; +import { arr, local, method, obj, str } from "../../../src/next/contracts/index.ts"; +import { defineState } from "../../../src/next/documents/index.ts"; +import { type Ctx, createHost, defineFacet, sleep } from "../../../src/next/host/index.ts"; + +const context = BACKGROUND_CONTEXT; + +type Principal = { readonly kind: "server" } | { readonly kind: "owner"; readonly name: string }; + +const Notes = local("example.notes", { + version: 1, + params: { session: str() }, + members: { list: arr(str()), add: method(obj({ text: str() })) }, +}); + +const List = defineState("example.list", { + schema: obj({ list: arr(str()) }), + version: 1, + initial: () => ({ list: [] }), +}); + +const facet = defineFacet((host) => { + host.serve(Notes, (_, self) => { + const notes = self.state(List); + return { + list: self.derive([notes], ([n]) => n.list), + add: async ({ text }, ctx) => { + await notes.change((draft) => { + draft.list.push(`${text} (by ${ctx.principal?.kind})`); + }, ctx); + return null; + }, + }; + }); +}); + +const host = createHost({ principal: { kind: "server" }, onError: (error) => console.log(error.kind) }); + +// 1. A scope made before `load` resolved: its `ctx.get` parks until the host is loaded. +const early = host.root(async (ctx) => (await ctx.get(Notes, { session: "s1" })).list.value, context); +await host.load([{ name: "@example/notes", facet }], context); +console.log("before load, read after it:", await early); + +// 2. A tool's call: get, call, return plain data. The reference is released when fn settles. +const addNote = (text: string) => + host.root(async (ctx) => { + const notes = await ctx.get(Notes, { session: "s1" }); + await notes.add({ text }, ctx); + return notes.list.value; + }, context); +console.log("after a tool call:", await addNote("first")); + +// 3. Base code acting for an authenticated user names the principal. +const owner: Principal = { kind: "owner", name: "ada" }; +console.log("as the owner:", await host.root(async (ctx) => ctx.principal, context, { principal: owner })); + +// 4. A kept ctx is sealed once fn settled. +let kept: Ctx | undefined; +await host.root(async (ctx) => { + kept = ctx; +}, context); +console.log("kept ctx aborted:", kept!.abortSignal?.aborted); +await kept!.get(Notes, { session: "s1" }).catch((error: Error) => console.log("kept ctx.get:", error.message)); + +// 5. Cancelling the context rejects at once and seals the scope; an fn that ignores its signal fails at its next +// use of Chord. +const { context: cancellable, cancel } = withCancel(context); +const stubborn = host.root(async (ctx) => { + await new Promise((resolve) => setTimeout(resolve, 20)); + await ctx.get(Notes, { session: "s1" }); +}, cancellable); +cancel(new Error("user pressed stop")); +await stubborn.catch((error: Error) => console.log("cancelled:", error.message)); + +// 6. Closing the host seals the scopes still open and rejects their callers at once: an fn sleeping on its ctx ends. +const sleeping = host.root((ctx) => sleep(60_000, ctx), context).catch((error: Error) => `closed: ${error.message}`); +await host.close(context); +console.log(await sleeping); diff --git a/packages/chord/test/next/host/attempt.test.ts b/packages/chord/test/next/host/attempt.test.ts index 867502320..4066748e7 100644 --- a/packages/chord/test/next/host/attempt.test.ts +++ b/packages/chord/test/next/host/attempt.test.ts @@ -1,5 +1,5 @@ import { createHook } from "node:async_hooks"; -import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { afterEach, beforeEach, describe, expect, expectTypeOf, it, vi } from "vitest"; import { type Context, BACKGROUND_CONTEXT as context, withCancel } from "../../../src/context/index.ts"; import { arr, int, local, method, obj, opt, remote, str } from "../../../src/next/contracts/index.ts"; import { broken, isChordError } from "../../../src/next/data/index.ts"; @@ -12,6 +12,7 @@ import { type Self, type StoreSelf, } from "../../../src/next/host/index.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; import { constant } from "../../../src/next/state/index.ts"; import type { StagedWrite } from "../../../src/next/store/step.ts"; import type { ChordStore } from "../../../src/next/store/store.ts"; @@ -425,6 +426,36 @@ describe("declarations", () => { () => {}, ), ).toThrow("self.open is valid only in the factory"); + const Names = defineRegistry("app.names", { version: 1 }); + expect(() => later!.registry(Names)).toThrow("self.registry is valid only in the factory"); + expect(() => later!.writable(1)).toThrow("self.writable is valid only in the factory"); + expect(() => later!.retryOn()).toThrow("self.retryOn is valid only in the factory"); + }); + + it("contribute only in host-lifetime objects (R21): `Self` has no `contribute`, a declaration error at runtime", async () => { + expectTypeOf().not.toHaveProperty("contribute"); + expectTypeOf().toHaveProperty("contribute"); + const Names = defineRegistry("app.names", { version: 1 }); + const Store = local("app.store", { version: 1, params: {}, members: {} }); + const { host } = await loaded((h) => { + h.serve(Names, { compose: (table) => table.length }); + h.serve(Page, (_, self) => { + (self as unknown as StoreSelf).contribute(Names, "page"); + return { label: constant(""), slow: async () => null }; + }); + h.serve(Store, (_, self) => { + self.contribute(Names, "a"); + self.contribute(Names, "b"); + return {}; + }); + }); + await expect(host.get(Page, { id: 1 }, context)).rejects.toThrow( + "app.page(1): self.contribute is valid only in a local type with empty params", + ); + await expect(host.get(Store, {}, context)).rejects.toThrow( + "app.store: registry app.names is contributed to twice", + ); + await host.close(context); }); it("refuse a document definition declared twice", async () => { diff --git a/packages/chord/test/next/host/ctx.test.ts b/packages/chord/test/next/host/ctx.test.ts index 732ed2d3a..df259c227 100644 --- a/packages/chord/test/next/host/ctx.test.ts +++ b/packages/chord/test/next/host/ctx.test.ts @@ -61,7 +61,7 @@ describe("ctx", () => { expect(await rejection(seen.ctx!.get(Item, { id: 1 }))).toEqual({ code: "released", reason: "settled", - message: "app.item: ctx.get after its handler settled", + message: "app.item: ctx.get after its call settled", }); }); @@ -103,7 +103,7 @@ describe("ctx", () => { expect(await seen.late).toMatchObject({ code: "released", reason: "settled", - message: "app.item: ctx.get after its handler settled", + message: "app.item: ctx.get after its call settled", }); await flushed(); // The late reference's object ended with it. @@ -130,7 +130,7 @@ describe("ctx", () => { expect(await pending).toMatchObject({ code: "released", reason: "settled", - message: "app.far: ctx.get after its handler settled", + message: "app.far: ctx.get after its call settled", }); }); diff --git a/packages/chord/test/next/host/facet.test.ts b/packages/chord/test/next/host/facet.test.ts index 54fe1113d..0dd05e818 100644 --- a/packages/chord/test/next/host/facet.test.ts +++ b/packages/chord/test/next/host/facet.test.ts @@ -1,6 +1,7 @@ import { describe, expect, expectTypeOf, it } from "vitest"; import { int, local } from "../../../src/next/contracts/index.ts"; -import { defineFacet, type Facet, type FacetHost, type Host } from "../../../src/next/host/index.ts"; +import { defineFacet, type Facet, type FacetHost, type FacetSelf, type Host } from "../../../src/next/host/index.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; import { constant } from "../../../src/next/state/index.ts"; import { loaded } from "../support.ts"; @@ -17,13 +18,34 @@ describe("facets", () => { ); }); - it("only declare: a setup takes the host alone, which has no documents, routines or natives", () => { - expectTypeOf["setup"]>>().toEqualTypeOf<[FacetHost<{ kind: "owner" }>]>(); + it("only declare: a setup takes the host and a self that only contributes; no documents, routines or natives", () => { + expectTypeOf["setup"]>>().toEqualTypeOf< + [FacetHost<{ kind: "owner" }>, FacetSelf] + >(); + expectTypeOf().toEqualTypeOf<"contribute">(); expectTypeOf().not.toHaveProperty("state"); expectTypeOf().not.toHaveProperty("on"); expectTypeOf().not.toHaveProperty("onLive"); }); + it("contribute at most one value per registry, only while their setup runs", async () => { + const Names = defineRegistry("app.names", { version: 1 }); + await expect( + loaded((_, self) => { + self.contribute(Names, "a"); + self.contribute(Names, "b"); + }), + ).rejects.toThrow("@test/app: registry app.names is contributed to twice"); + let later: FacetSelf | undefined; + await loaded((host, self) => { + host.serve(Names, { compose: (table) => table.length }); + later = self; + }); + expect(() => later!.contribute(Names, "late")).toThrow( + "@test/app: self.contribute is valid only in the facet's setup", + ); + }); + it("are typed by exactly the app's principal type", () => { type App = { kind: "owner" } | { kind: "guest"; session: string }; const facet = defineFacet(() => {}); diff --git a/packages/chord/test/next/host/members.test.ts b/packages/chord/test/next/host/members.test.ts index 2efc060e0..9c5e40793 100644 --- a/packages/chord/test/next/host/members.test.ts +++ b/packages/chord/test/next/host/members.test.ts @@ -22,10 +22,10 @@ describe("the member check (Q14)", () => { }), ); await expect(host.get(Named, { n: 1 }, context)).rejects.toThrow( - "app.named(1): label: source 1 is not a JSON watchable", + "app.named(1): label: source 1 is not a Chord watchable", ); await expect(host.get(Named, { n: 2 }, context)).rejects.toThrow( - "app.named(2): derive #1: source 0 is not a JSON watchable", + "app.named(2): derive #1: source 0 is not a Chord watchable", ); }); }); diff --git a/packages/chord/test/next/host/retry-on.test.ts b/packages/chord/test/next/host/retry-on.test.ts new file mode 100644 index 000000000..5677b6c28 --- /dev/null +++ b/packages/chord/test/next/host/retry-on.test.ts @@ -0,0 +1,204 @@ +import { describe, expect, it } from "vitest"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { int, local, method } from "../../../src/next/contracts/index.ts"; +import { broken, ChordError } from "../../../src/next/data/index.ts"; +import type { FacetHost, Self, StoreSelf } from "../../../src/next/host/index.ts"; +import { ChordObject } from "../../../src/next/host/object.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; +import { constant } from "../../../src/next/state/index.ts"; +import { expectCollected, failLive, flushed, instancesOf, loaded } from "../support.ts"; + +const objects = instancesOf(ChordObject, "attempt"); + +/** The models registry: the number of models contributed; it throws when told to. */ +const Models = defineRegistry("app.models", { version: 1 }); + +const Runtime = local("app.runtime", { version: 1, params: {}, members: { set: method(int()) } }); + +/** Serves `Models` (throwing for a negative sum) and `Runtime`, contributing a writable count of models. */ +function serveModels(h: FacetHost): void { + h.serve(Models, { + compose: (table) => { + const sum = table.reduce((n, { value }) => n + value, 0); + if (sum < 0) throw new Error("negative"); + return sum; + }, + }); + h.serve(Runtime, (_, self) => { + const count = self.writable(0); + self.contribute(Models, count); + return { + set: async (n: number) => { + count.set(n); + return null; + }, + }; + }); +} + +const User = local("app.user", { version: 1, params: { id: int() }, members: {} }); + +/** + * Serves `User`, whose open reads the models first and fails broken when there were none (after `wait`, if given); + * `opens` counts its attempts. + */ +function serveUser(h: FacetHost, opens: { n: number }, wait?: () => Promise): void { + h.serve(User, (_, self) => { + const models = self.registry(Models); + self.retryOn(models); + self.open( + async () => { + const seen = models.value; + opens.n++; + await wait?.(); + if (seen === 0) throw broken(new ChordError({ code: "unavailable", message: "no model" })); + }, + () => {}, + ); + return {}; + }); +} + +describe("retryOn", () => { + it("retries a failed object (a broken open) when a source gets a new value: the watches were installed before the open", async () => { + const opens = { n: 0 }; + const { host } = await loaded((h) => { + serveModels(h); + serveUser(h, opens); + }); + const user = host.hold(User, { id: 1 }, context); + await flushed(); + expect(user.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } }); + await (await host.get(Runtime, {}, context)).set(1, context); + await flushed(); + expect(user.state.value).toEqual({ kind: "live" }); + expect(opens.n).toBe(2); + await host.close(context); + }); + + it("wakes on value calls only: a throwing compose wakes nothing, its next good composition does", async () => { + const opens = { n: 0 }; + const { host } = await loaded((h) => { + serveModels(h); + serveUser(h, opens); + }); + const user = host.hold(User, { id: 1 }, context); + await flushed(); + const runtime = await host.get(Runtime, {}, context); + await runtime.set(-1, context); + await flushed(); + expect(opens.n).toBe(1); + await runtime.set(2, context); + await flushed(); + expect(opens.n).toBe(2); + expect(user.state.value).toEqual({ kind: "live" }); + await host.close(context); + }); + + it("remembers a wake while it activates: that attempt failing is retried at once", async () => { + const opens = { n: 0 }; + const gate = Promise.withResolvers(); + const { host } = await loaded((h) => { + serveModels(h); + serveUser(h, opens, async () => { + if (opens.n === 1) await gate.promise; + }); + }); + const runtime = await host.get(Runtime, {}, context); + const user = host.hold(User, { id: 1 }, context); + await flushed(); + // The first attempt read no model and waits; a model arrives meanwhile; then that attempt fails broken. + await runtime.set(1, context); + expect(user.state.value).toEqual({ kind: "loading" }); + gate.resolve(); + await flushed(); + expect(opens.n).toBe(2); + expect(user.state.value).toEqual({ kind: "live" }); + await host.close(context); + }); + + it("ignores wakes while live", async () => { + const opens = { n: 0 }; + const { host } = await loaded((h) => { + serveModels(h); + h.serve(User, (_, self) => { + self.retryOn(self.registry(Models)); + self.open( + async () => void opens.n++, + () => {}, + ); + return {}; + }); + }); + const user = host.hold(User, { id: 1 }, context); + await flushed(); + await (await host.get(Runtime, {}, context)).set(3, context); + await flushed(); + expect(opens.n).toBe(1); + expect(user.state.value).toEqual({ kind: "live" }); + await host.close(context); + }); + + it("refuses sources that end with an attempt: a reference's member, its own writable, `ref.state`", async () => { + const Other = local("app.other", { version: 1, params: {}, members: { n: int() } }); + const Picky = local("app.picky", { version: 1, params: { which: int() }, members: {} }); + const { host } = await loaded((h) => { + h.serve(Other, () => ({ n: constant(1) })); + h.serve(Picky, ({ which }, self) => { + const other = self.get(Other, {}); + const sources = [other.n, self.writable(1), other.state, constant(1)]; + self.retryOn(sources[which]!); + return {}; + }); + }); + for (const which of [0, 1, 2]) { + await expect(host.get(Picky, { which }, context)).rejects.toThrow( + `app.picky(${which}): self.retryOn takes registry values and constants (sources that outlive the attempt)`, + ); + } + (await host.get(Picky, { which: 3 }, context)).release(); + await host.close(context); + }); + + it("are the object's: a failed object that ends releases them, and is collected while the registry lives", async () => { + const opens = { n: 0 }; + const { host } = await loaded((h) => { + serveModels(h); + serveUser(h, opens); + }); + const refs = await (async () => { + const user = host.hold(User, { id: 1 }, context); + await flushed(); + user.release(); + await flushed(); + return objects.filter((ref) => ref.deref()?.label === "app.user(1)"); + })(); + expect(refs.length).toBeGreaterThan(0); + await expectCollected(...refs); + await host.close(context); + }); + + it("does not wake an object by its own contributions leaving (it failed live: no retry without backoff)", async () => { + const Contributor = local("app.contributor", { version: 1, params: {}, members: {} }); + let contributor: StoreSelf | undefined; + const { host } = await loaded((h) => { + serveModels(h); + h.serve(Contributor, (_, self) => { + contributor = self; + self.contribute(Models, 1); + self.retryOn(self.registry(Models)); + return {}; + }); + }); + const models = await host.registry(Models, context); + const watched: number[] = []; + (await models.watch(context)).start(async (call) => void (call.kind === "value" && watched.push(call.value))); + failLive(objects, contributor as Self, new ChordError({ code: "unavailable", message: "boom" })); + await flushed(); + const reference = host.hold(Contributor, {}, context); + await flushed(); + expect(reference.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } }); + expect(watched).toEqual([0]); + await host.close(context); + }); +}); diff --git a/packages/chord/test/next/host/root.test.ts b/packages/chord/test/next/host/root.test.ts new file mode 100644 index 000000000..76c1c2b7e --- /dev/null +++ b/packages/chord/test/next/host/root.test.ts @@ -0,0 +1,175 @@ +import { getEventListeners } from "node:events"; +import { describe, expect, it } from "vitest"; +import { BACKGROUND_CONTEXT as context, withAbortSignal, withCancel } from "../../../src/context/index.ts"; +import { int, local, method } from "../../../src/next/contracts/index.ts"; +import { type Ctx, createHost, defineFacet, sleep } from "../../../src/next/host/index.ts"; +import { constant } from "../../../src/next/state/index.ts"; +import { expectCollected, flushed, loaded, rejection, type TestPrincipal } from "../support.ts"; + +const Info = local("app.info", { version: 1, params: { id: int() }, members: { id: int(), wait: method() } }); + +/** Serves `Info`; its `wait` waits until the caller's context aborts. */ +function serveInfo(h: Parameters[0]>[0]): void { + h.serve(Info, ({ id }) => ({ + id: constant(id), + wait: async (ctx) => { + await sleep(60_000, ctx); + return null; + }, + })); +} + +describe("host.root", () => { + it("gives fn a call context: the host's principal (or the given one), this host's incarnation, `ctx.get`", async () => { + const host = createHost({ principal: { kind: "server" }, onError: () => {} }); + await host.load([{ name: "@app/info", facet: defineFacet(serveInfo) }], context); + const seen = await host.root(async (ctx) => { + const info = await ctx.get(Info, { id: 7 }); + return { principal: ctx.principal, incarnation: ctx.incarnation, id: info.id.value }; + }, context); + expect(seen).toMatchObject({ principal: { kind: "server" }, id: 7 }); + const owner = { kind: "owner", access: "local" } as const; + expect(await host.root(async (ctx) => ctx.principal, context, { principal: owner })).toEqual(owner); + await host.close(context); + }); + + it("seals the context when fn settles: its references released, its signal aborted, `ctx.get` refused `released`", async () => { + const { host } = await loaded(serveInfo); + let kept: Ctx | undefined; + const info = await host.root(async (ctx) => { + kept = ctx; + return ctx.get(Info, { id: 1 }); + }, context); + expect(info.state.value).toMatchObject({ kind: "ended", error: { code: "released" } }); + expect(kept!.abortSignal?.aborted).toBe(true); + expect(await rejection(kept!.get(Info, { id: 1 }))).toMatchObject({ code: "released", reason: "settled" }); + // A throwing fn seals too, and rejects with its error. + await expect( + host.root(async () => { + throw new Error("fn failed"); + }, context), + ).rejects.toThrow("fn failed"); + await host.close(context); + }); + + it("an abort rejects at once and seals: a running call through its reference rejects, a later get is `released`", async () => { + const { host } = await loaded(serveInfo); + const { context: cancellable, cancel } = withCancel(context); + let kept: Ctx | undefined; + let call: Promise | undefined; + const running = host.root(async (ctx) => { + kept = ctx; + const info = await ctx.get(Info, { id: 1 }); + call = info.wait(context); + // Uncooperative: it waits on nothing of Chord's. + await new Promise(() => {}); + }, cancellable); + await flushed(); + cancel(new Error("stop")); + expect(await rejection(running)).toEqual(new Error("stop")); + expect(await rejection(call!)).toMatchObject({ code: "released" }); + expect(await rejection(kept!.get(Info, { id: 2 }))).toMatchObject({ code: "released", reason: "settled" }); + await host.close(context); + }); + + it("an aborted context rejects before fn runs; a synchronous throw is a rejection", async () => { + const { host } = await loaded(serveInfo); + const { context: aborted, cancel } = withCancel(context); + cancel(new Error("gone")); + let ran = false; + const fn = async (): Promise => { + ran = true; + }; + await expect(host.root(fn, aborted)).rejects.toThrow("gone"); + expect(ran).toBe(false); + await expect( + host.root( + (() => { + throw new Error("sync"); + }) as never, + context, + ), + ).rejects.toThrow("sync"); + await host.close(context); + }); + + it("a `ctx.get` before `load` resolved parks until it did", async () => { + const host = createHost({ onError: () => {} }); + const result = host.root(async (ctx) => (await ctx.get(Info, { id: 3 })).id.value, context); + await flushed(); + await host.load([{ name: "@app/info", facet: defineFacet(serveInfo) }], context); + expect(await result).toBe(3); + await host.close(context); + }); + + it("`host.close` seals every open scope (an fn sleeping on its ctx ends); after it, `root` rejects `released`", async () => { + const { host } = await loaded(serveInfo); + const sleeping = rejection(host.root((ctx) => sleep(60_000, ctx), context)); + await flushed(); + await host.close(context); + expect(await sleeping).toMatchObject({ code: "released", reason: "closed" }); + expect(await rejection(host.root(async () => 1, context))).toMatchObject({ code: "released", reason: "closed" }); + }); + + it("aborts the scope's own signal when fn settles and at `host.close`, the caller's context having one (Fable, Opus, Sol)", async () => { + const { host } = await loaded(serveInfo); + const settled = withCancel(context); + let kept: Ctx | undefined; + await host.root(async (ctx) => { + kept = ctx; + }, settled.context); + expect(kept!.abortSignal?.aborted).toBe(true); + expect(settled.context.abortSignal?.aborted).toBe(false); + const closing = withCancel(context); + let slept: "running" | "ended" = "running"; + const sleeping = host.root(async (ctx) => { + try { + await sleep(60_000, ctx); + } finally { + slept = "ended"; + } + }, closing.context); + const outcome = rejection(sleeping); + await flushed(); + await host.close(context); + await flushed(); + expect(slept).toBe("ended"); + expect(closing.context.abortSignal?.aborted).toBe(false); + expect(await outcome).toMatchObject({ code: "released", reason: "closed" }); + }); + + it("`host.close` rejects the caller at once, whatever an uncooperative fn still does (Sol)", async () => { + const { host } = await loaded(serveInfo); + const running = host.root(() => new Promise(() => {}), context); + const outcome = running.then( + () => "resolved", + (error: unknown) => error, + ); + await host.close(context); + expect(await outcome).toMatchObject({ code: "released", reason: "closed" }); + }); + + it("leaves no listener on the caller's signal once `host.close` rejected an fn that never settles (Astra)", async () => { + const controller = new AbortController(); + const host = createHost({ onError: () => {} }); + const running = host.root(async () => new Promise(() => {}), withAbortSignal(controller.signal, context)); + const rejected = running.catch((error: unknown) => error); + await host.close(context); + expect(await rejected).toMatchObject({ code: "released", reason: "closed" }); + expect(getEventListeners(controller.signal, "abort")).toHaveLength(0); + }); + + it("keeps nothing once sealed: its scope and references are collected while the caller's signal lives on", async () => { + const { host } = await loaded(serveInfo); + const { context: long } = withCancel(context); + const refs = await (async () => { + const made: WeakRef[] = []; + await host.root(async (ctx) => { + made.push(new WeakRef(ctx), new WeakRef(await ctx.get(Info, { id: 1 }))); + }, long); + return made; + })(); + await expectCollected(...refs); + await host.close(context); + }); +}); diff --git a/packages/chord/test/next/registries/entries.test.ts b/packages/chord/test/next/registries/entries.test.ts new file mode 100644 index 000000000..88bd61003 --- /dev/null +++ b/packages/chord/test/next/registries/entries.test.ts @@ -0,0 +1,49 @@ +import { describe, expect, it } from "vitest"; +import { + keyedEntries, + orderedEntries, + type RegistryConflict, + type RegistryTable, +} from "../../../src/next/registries/index.ts"; + +const table: RegistryTable<{ readonly routes?: Readonly>; readonly items?: readonly string[] }> = + [ + { plugin: "@a/one", version: 1, value: { routes: { "/x": "one", "/y": "one" }, items: ["late", "first"] } }, + { plugin: "@b/two", type: "b.store", version: 1, value: { routes: { "/x": "two" }, items: ["second"] } }, + { plugin: "@c/three", version: 1, value: {} }, + ]; + +describe("keyedEntries", () => { + it("maps every entry's keys with their plugin; a duplicate is reported and the first by plugin name keeps it", () => { + const conflicts: RegistryConflict[] = []; + const map = keyedEntries( + table, + (value) => value.routes, + (conflict) => conflicts.push(conflict), + ); + expect([...map]).toEqual([ + ["/x", { value: "one", plugin: "@a/one" }], + ["/y", { value: "one", plugin: "@a/one" }], + ]); + expect(conflicts).toEqual([{ message: '"/x" is contributed twice', plugins: ["@a/one", "@b/two"] }]); + }); +}); + +describe("orderedEntries", () => { + it("lists every entry's items in table order, with their plugin and type", () => { + expect(orderedEntries(table, (value) => value.items)).toEqual([ + { value: "late", plugin: "@a/one" }, + { value: "first", plugin: "@a/one" }, + { value: "second", plugin: "@b/two", type: "b.store" }, + ]); + }); + + it("sorts by `orderOf`, table order kept among equals", () => { + const order = (item: string) => (item === "first" ? 0 : 1); + expect(orderedEntries(table, (value) => value.items, order).map(({ value }) => value)).toEqual([ + "first", + "late", + "second", + ]); + }); +}); diff --git a/packages/chord/test/next/registries/record.test.ts b/packages/chord/test/next/registries/record.test.ts new file mode 100644 index 000000000..c25b27fcf --- /dev/null +++ b/packages/chord/test/next/registries/record.test.ts @@ -0,0 +1,649 @@ +import { describe, expect, it } from "vitest"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { bool, int, local, method, obj } from "../../../src/next/contracts/index.ts"; +import { defineState } from "../../../src/next/documents/index.ts"; +import type { HostError } from "../../../src/next/host/index.ts"; +import { ChordObject } from "../../../src/next/host/object.ts"; +import { + defineRegistry, + keyedEntries, + type Registry, + type RegistryTable, + type TableItem, +} from "../../../src/next/registries/index.ts"; +import type { NativeWatchCall } from "../../../src/watch.ts"; +import { + delivered, + expectCollected, + failures, + flushed, + instancesOf, + loadedPlugins, + metered, + storeRestartable, +} from "../support.ts"; + +const objects = instancesOf(ChordObject, "attempt"); + +/** A registry whose composition is its table as `plugin:type=value` strings. */ +const Names = defineRegistry("app.names", { version: 1 }); + +const listed = (table: RegistryTable): readonly string[] => + table.map(({ plugin, type, value }) => `${plugin}:${type ?? ""}=${value}`); + +/** A host-lifetime object contributing a writable it sets with `set`. */ +const Setter = local("app.setter", { version: 1, params: {}, members: { set: method(int()) } }); + +/** Serves `Setter` contributing a writable of `R`'s contribution type, `initial` first, `set(n)` makes `of(n)`. */ +function setter(R: Registry, initial: C, of: (n: number) => C) { + return (host: Parameters[0][string]>[0]) => + host.serve(Setter, (_, self) => { + const value = self.writable(initial); + self.contribute(R, value); + return { + set: async (n: number) => { + value.set(of(n)); + return null; + }, + }; + }); +} + +function kinds(errors: readonly HostError[]): string[] { + return errors.map((error) => error.kind); +} + +describe("the table", () => { + it("is sorted by plugin name, then type, a facet first: never load order", async () => { + const { host } = await loadedPlugins({ + "@z/late": (_, self) => self.contribute(Names, "z"), + "@a/early": (host, self) => { + self.contribute(Names, "a"); + setter(Names, "store", String)(host); + }, + "@m/provider": (host) => host.serve(Names, { compose: listed }), + }); + const value = await host.registry(Names, context); + expect(value.value).toEqual(["@a/early:=a", "@a/early:app.setter=store", "@z/late:=z"]); + await host.close(context); + }); + + it("adapts an older contribution once per value; a newer one, or an older one without an adapter, is left out and reported once", async () => { + const V3 = defineRegistry("app.versions", { version: 3 }); + const V2 = defineRegistry<{ old: string }, unknown>("app.versions", { version: 2 }); + const V1 = defineRegistry("app.versions", { version: 1 }); + const V4 = defineRegistry("app.versions", { version: 4 }); + const adapted: string[] = []; + const { host, errors } = await loadedPlugins({ + "@app/provider": (host) => + host.serve(V3, { + compose: listed, + accepts: { + 2: { + contribution: (old: { old: string }) => { + adapted.push(old.old); + return `adapted ${old.old}`; + }, + }, + }, + }), + "@app/one": (_, self) => self.contribute(V1, "one"), + "@app/two": setter(V2, { old: "first" }, (n) => ({ old: `${n}` })), + "@app/three": (_, self) => self.contribute(V3, "three"), + "@app/four": (_, self) => self.contribute(V4, "four"), + }); + const value = await host.registry(V3, context); + expect(value.value).toEqual(["@app/three:=three", "@app/two:app.setter=adapted first"]); + expect(errors).toEqual([ + { kind: "contribution_refused", registry: "app.versions", plugin: "@app/four", version: 4 }, + { kind: "contribution_refused", registry: "app.versions", plugin: "@app/one", version: 1 }, + ]); + const setters = await host.get(Setter, {}, context); + await setters.set(5, context); + expect(value.value).toEqual(["@app/three:=three", "@app/two:app.setter=adapted 5"]); + // Each value adapted once, though the table was built again; no refusal reported twice. + expect(adapted).toEqual(["first", "5"]); + expect(kinds(errors)).toEqual(["contribution_refused", "contribution_refused"]); + await host.close(context); + }); + + it("takes a contribution declared before its provider's setup into the first composition", async () => { + let composed: readonly string[] | undefined; + const { host } = await loadedPlugins({ + "@app/a": (_, self) => self.contribute(Names, "first"), + "@app/provider": (host) => + host.serve(Names, { + compose: (table) => { + composed = listed(table); + return composed; + }, + }), + }); + expect(composed).toEqual(["@app/a:=first"]); + await host.close(context); + }); +}); + +describe("compose", () => { + it("composes every served registry once at the end of load's setups, an empty table too, before any object activates", async () => { + const { telemetry, counts } = await metered(); + const Other = defineRegistry("app.other", { version: 1 }); + const Reader = local("app.reader", { version: 1, params: {}, members: { seen: int() } }); + let seen: readonly string[] | undefined; + const { host } = await loadedPlugins( + { + "@app/provider": (host, self) => { + host.serve(Names, { compose: listed }); + host.serve(Other, { compose: (table) => table.length }); + self.contribute(Names, "a"); + host.serve(Reader, (_, self) => { + const names = self.registry(Names); + self.open( + async () => { + seen = names.value; + }, + () => {}, + ); + return { seen: self.derive([self.registry(Other)], ([n]) => n) }; + }); + }, + }, + { telemetry }, + ); + expect(seen).toEqual(["@app/provider:=a"]); + expect((await host.get(Reader, {}, context)).seen.value).toBe(0); + expect(counts()["chord.registry.composes"]).toBe(2); + await host.close(context); + }); + + it("is lazy: a change marks the table, a read composes it once; no compose for an unchanged table", async () => { + const { telemetry, counts } = await metered(); + const Flags = defineRegistry("app.flags", { version: 1 }); + const Doc = defineState("app.flag", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 1 }) }); + const Flagger = local("app.flagger", { version: 1, params: {}, members: { set: method(int()) } }); + const { host } = await loadedPlugins( + { + "@app/flags": (host) => { + host.serve(Flags, { compose: (table) => table.filter(({ value }) => value).length }); + host.serve(Flagger, (_, self) => { + const doc = self.state(Doc); + self.contribute( + Flags, + self.derive([doc], ([d]) => d.n > 0), + ); + return { + set: async (n: number, ctx) => { + await doc.change((d) => { + d.n = n; + }, ctx); + return null; + }, + }; + }); + }, + }, + { telemetry }, + ); + const flags = await host.registry(Flags, context); + const flagger = await host.get(Flagger, {}, context); + expect(flags.value).toBe(1); + const base = counts()["chord.registry.composes"]!; + expect(flags.value).toBe(1); + expect(counts()["chord.registry.composes"]).toBe(base); + // The derive changes, its value does not: the table stays, nothing composes. + await flagger.set(2, context); + await flagger.set(3, context); + expect(flags.value).toBe(1); + expect(counts()["chord.registry.composes"]).toBe(base); + // Nobody reads it: nothing composes until a read. + await flagger.set(-1, context); + await flagger.set(-2, context); + expect(counts()["chord.registry.composes"]).toBe(base); + expect(flags.value).toBe(0); + expect(counts()["chord.registry.composes"]).toBe(base + 1); + await host.close(context); + }); + + it("composes a watched registry in the batch of the commit that changed a contribution, once per batch", async () => { + const { telemetry, counts } = await metered(); + const Doc = defineState("app.names", { + schema: obj({ a: int(), b: int() }), + version: 1, + initial: () => ({ a: 0, b: 0 }), + }); + const Two = local("app.two", { version: 1, params: {}, members: { both: method(int()), sum: int() } }); + const Sum = defineRegistry<{ readonly a: number; readonly b: number }, number>("app.sum", { version: 1 }); + const { host } = await loadedPlugins( + { + "@app/sum": (host) => { + host.serve(Sum, { compose: (table) => table.reduce((n, { value }) => n + value.a + value.b, 0) }); + host.serve(Two, (_, self) => { + const doc = self.state(Doc); + self.contribute(Sum, doc); + return { + sum: self.derive([self.registry(Sum)], ([n]) => n), + both: async (n: number, ctx) => { + await self.tx(async (tx) => { + const d = await tx.draft(doc); + d.a = n; + d.b = n; + }, ctx); + return null; + }, + }; + }); + }, + }, + { telemetry }, + ); + const two = await host.get(Two, {}, context); + const sums: number[] = []; + const watch = await two.sum.watch(context); + watch.start(async (call) => { + if (call.kind === "value") sums.push(call.value); + }); + const base = counts()["chord.registry.composes"]!; + await two.both(2, context); + await delivered(); + // One commit, both fields: one compose in its batch, one frame of the derive over it. + expect(sums).toEqual([4]); + expect(counts()["chord.registry.composes"]).toBe(base + 1); + await host.close(context); + }); + + it("reports a conflict once per episode", async () => { + const Routes = defineRegistry>, ReadonlyMap>>( + "app.routes", + { version: 1 }, + ); + const { host, errors } = await loadedPlugins({ + "@app/a": (_, self) => self.contribute(Routes, { "/x": "a" }), + "@app/b": setter(Routes, { "/x": "b" } as Readonly>, (n) => ({ + "/x": "b", + [`/${n}`]: "b", + })), + "@app/http": (host) => + host.serve(Routes, { compose: (table, { report }) => keyedEntries(table, (c) => c, report) }), + }); + const routes = await host.registry(Routes, context); + expect(routes.value.get("/x")).toEqual({ value: "a", plugin: "@app/a" }); + const setters = await host.get(Setter, {}, context); + await setters.set(1, context); + void routes.value; + await setters.set(2, context); + void routes.value; + expect(errors).toEqual([ + { + kind: "registry_conflict", + registry: "app.routes", + message: '"/x" is contributed twice', + plugins: ["@app/a", "@app/b"], + }, + ]); + await host.close(context); + }); + + it("a throwing compose: reported once, a failed call, `.value` keeps the last good composition; the next good one heals", async () => { + const Count = defineRegistry("app.count", { version: 1 }); + const { host, errors } = await loadedPlugins({ + "@app/count": (host) => { + host.serve(Count, { + compose: (table) => { + const sum = table.reduce((n, { value }) => n + value, 0); + if (sum < 0) throw new Error(`negative: ${sum}`); + return sum; + }, + }); + setter(Count, 1, (n) => n)(host); + }, + }); + const count = await host.registry(Count, context); + const calls: NativeWatchCall[] = []; + const watch = await count.watch(context); + watch.start(async (call) => void calls.push(call)); + const setters = await host.get(Setter, {}, context); + await setters.set(-1, context); + await setters.set(-2, context); + await delivered(); + expect(count.value).toBe(1); + expect(errors).toEqual([{ kind: "registry_failed", registry: "app.count", error: new Error("negative: -1") }]); + await setters.set(1, context); + await delivered(); + // The heal is a value call, also when its value is the last good one. + expect(calls).toEqual([ + { kind: "failed", error: new Error("negative: -1") }, + { kind: "value", value: 1 }, + ]); + await setters.set(3, context); + await delivered(); + expect(calls.at(-1)).toEqual({ kind: "value", value: 3 }); + await host.close(context); + }); + + it("before any good composition, `.value` and `host.registry` throw the error", async () => { + const Broken = defineRegistry("app.broken", { version: 1 }); + const { host, errors } = await loadedPlugins({ + "@app/broken": (host) => + host.serve(Broken, { + compose: () => { + throw new Error("bug"); + }, + }), + }); + await expect(host.registry(Broken, context)).rejects.toThrow("bug"); + expect(kinds(errors)).toEqual(["registry_failed"]); + await host.close(context); + }); +}); + +describe("contributions", () => { + it("a watchable contribution that fails is left out and reported once, and is back once it heals", async () => { + const Doc = defineState("app.n", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 1 }) }); + const Failing = local("app.failing", { version: 1, params: {}, members: { set: method(int()) } }); + const { host, errors } = await loadedPlugins({ + "@app/names": (host, self) => { + host.serve(Names, { compose: listed }); + self.contribute(Names, "facet"); + host.serve(Failing, (_, self) => { + const doc = self.state(Doc); + self.contribute( + Names, + self.derive([doc], ([d]) => { + if (d.n < 0) throw new Error("negative"); + return `n${d.n}`; + }), + ); + return { + set: async (n: number, ctx) => { + await doc.change((d) => { + d.n = n; + }, ctx); + return null; + }, + }; + }); + }, + }); + const names = await host.registry(Names, context); + const failing = await host.get(Failing, {}, context); + expect(names.value).toEqual(["@app/names:=facet", "@app/names:app.failing=n1"]); + await failing.set(-1, context); + expect(names.value).toEqual(["@app/names:=facet"]); + await failing.set(-2, context); + expect(names.value).toEqual(["@app/names:=facet"]); + await failing.set(2, context); + expect(names.value).toEqual(["@app/names:=facet", "@app/names:app.failing=n2"]); + expect(errors.filter(({ kind }) => kind === "contribution_failed")).toEqual([ + { + kind: "contribution_failed", + registry: "app.names", + plugin: "@app/names", + type: "app.failing", + error: new Error("negative"), + }, + ]); + await host.close(context); + }); + + it("a contribution adapter that throws leaves that contribution out (`contribution_failed`)", async () => { + const V2 = defineRegistry("app.adapted", { version: 2 }); + const V1 = defineRegistry("app.adapted", { version: 1 }); + const { host, errors } = await loadedPlugins({ + "@app/provider": (host, self) => { + host.serve(V2, { + compose: listed, + accepts: { + 1: { + contribution: () => { + throw new Error("cannot adapt"); + }, + }, + }, + }); + self.contribute(V2, "current"); + }, + "@app/old": (_, self) => self.contribute(V1, "old"), + }); + expect((await host.registry(V2, context)).value).toEqual(["@app/provider:=current"]); + expect(errors).toEqual([ + { kind: "contribution_failed", registry: "app.adapted", plugin: "@app/old", error: new Error("cannot adapt") }, + ]); + await host.close(context); + }); + + it("refuses a contribution closing a cycle with the registries' tables; one through a derive is cut and reported", async () => { + const A = defineRegistry("app.a", { version: 1 }); + const B = defineRegistry("app.b", { version: 1 }); + const Direct = local("app.direct", { version: 1, params: {}, members: {} }); + const Late = local("app.late", { version: 1, params: {}, members: {} }); + const Derived = local("app.derived", { version: 1, params: {}, members: {} }); + const Back = local("app.back", { version: 1, params: {}, members: {} }); + let lateAttempts = 0; + const { host, errors } = await loadedPlugins({ + "@app/cycles": (host) => { + host.serve(A, { compose: (table) => table.length }); + host.serve(B, { compose: (table) => table.length }); + host.serve(Direct, (_, self) => { + self.contribute(A, self.registry(B)); + return {}; + }); + // Its first attempt is discarded (the factory throws), then, once `Direct` is live, it closes a cycle. + host.serve(Late, (_, self) => { + self.contribute(B, self.registry(A)); + if (++lateAttempts === 1) throw new Error("first attempt"); + return {}; + }); + host.serve(Derived, (_, self) => { + self.contribute( + A, + self.derive([self.registry(B)], ([n]) => n as number), + ); + return {}; + }); + }, + "@app/back": (host) => + host.serve(Back, (_, self) => { + self.contribute( + B, + self.derive([self.registry(A)], ([n]) => n as number), + ); + return {}; + }), + }); + const late = host.hold(Late, {}, context); + late.retry(); + await flushed(); + expect(late.state.value).toMatchObject({ + kind: "failed", + cause: { kind: "broken", error: { code: "invalid", reason: "cycle" } }, + }); + // A reads B's derive, which reads A while A composes: that contribution is cut, A composes without the cycle. + expect((await host.registry(A, context)).value).toBe(2); + expect(errors.find((error) => error.kind === "contribution_failed")).toMatchObject({ + registry: "app.b", + plugin: "@app/back", + error: { code: "invalid", reason: "cycle" }, + }); + late.release(); + await host.close(context); + }); + + it("refuses a cycle declared within one factory: broken, invalid/cycle (Sol)", async () => { + const A = defineRegistry("app.fix-a", { version: 1 }); + const B = defineRegistry("app.fix-b", { version: 1 }); + const Both = local("app.direct-cycle", { version: 1, params: {}, members: {} }); + const { host, errors } = await loadedPlugins({ + "@app/cycle": (h) => { + h.serve(A, { compose: (table) => table.length }); + h.serve(B, { compose: (table) => table.length }); + h.serve(Both, (_, self) => { + self.contribute(A, self.registry(B)); + self.contribute(B, self.registry(A)); + return {}; + }); + }, + }); + expect( + errors.find((error) => error.kind === "activation_failed" && error.object === "app.direct-cycle"), + ).toMatchObject({ failure: { kind: "broken", error: { code: "invalid", reason: "cycle" } } }); + await host.close(context); + }); + + it("leaves no cycle edge from a contribution that never entered a table (Sol)", async () => { + const A = defineRegistry("app.edge-a", { version: 1 }); + const B = defineRegistry("app.edge-b", { version: 1 }); + const Failed = local("app.failed", { version: 1, params: {}, members: {} }); + const Good = local("app.good", { version: 1, params: {}, members: {} }); + const { host, errors } = await loadedPlugins({ + "@app/a": (h) => { + h.serve(A, { compose: (table) => table.length }); + h.serve(B, { compose: (table) => table.length }); + h.serve(Failed, (_, self) => { + self.contribute(A, self.registry(B)); + throw new Error("factory discarded"); + }); + h.serve(Good, (_, self) => { + self.contribute(B, self.registry(A)); + return {}; + }); + }, + }); + expect(failures(errors)).toEqual(["app.failed: broken"]); + await host.close(context); + }); + + it("follows only Chord watchables: anything else is a plain value, whatever methods it has (Sol)", async () => { + const Natives = defineRegistry<{ watch(): string; readonly value: string }, readonly string[]>("app.natives", { + version: 1, + }); + const native = { value: "a handle", watch: () => "a native operation" }; + const { host } = await loadedPlugins({ + "@app/x": (h, self) => { + h.serve(Natives, { compose: (table) => table.map(({ value }) => value.watch()) }); + self.contribute(Natives, native); + }, + }); + expect((await host.registry(Natives, context)).value).toEqual(["a native operation"]); + await host.close(context); + }); + + it("keeps an `undefined` contribution in the table (Sol, Astra)", async () => { + const Nothing = defineRegistry("app.undefined", { version: 1 }); + const { host } = await loadedPlugins({ + "@app/a": (h, self) => { + h.serve(Nothing, { compose: (table) => table.length }); + self.contribute(Nothing, undefined); + }, + }); + expect((await host.registry(Nothing, context)).value).toBe(1); + await host.close(context); + }); + + it("leaves out a registry's value contributed to another while that registry's compose throws (Sol, Astra)", async () => { + const Source = defineRegistry("app.source", { version: 1 }); + const Target = defineRegistry("app.target", { version: 1 }); + const Bridge = local("app.bridge", { version: 1, params: {}, members: {} }); + const { host, errors } = await loadedPlugins({ + "@app/a": (h) => { + h.serve(Source, { + compose: (table) => { + const n = table.reduce((sum, { value }) => sum + value, 0); + if (n < 0) throw new Error("negative"); + return n; + }, + }); + h.serve(Target, { compose: (table) => table.map(({ value }) => value) }); + setter(Source, 1, (n) => n)(h); + h.serve(Bridge, (_, self) => { + self.contribute(Target, self.registry(Source)); + return {}; + }); + }, + }); + const target = await host.registry(Target, context); + expect(target.value).toEqual([1]); + const setters = await host.get(Setter, {}, context); + await setters.set(-1, context); + // The source's readers keep its last good value; as a contribution it is failed: left out, reported. + expect((await host.registry(Source, context)).value).toBe(1); + expect(target.value).toEqual([]); + expect(errors.find((error) => error.kind === "contribution_failed")).toMatchObject({ + registry: "app.target", + plugin: "@app/a", + type: "app.bridge", + error: new Error("negative"), + }); + await setters.set(3, context); + expect(target.value).toEqual([3]); + await host.close(context); + }); + + it("a followed contribution whose runtime ended its observer stays with its last value (R4: the source keeps it)", async () => { + let end: (() => void) | undefined; + const Follower = local("app.follower", { version: 1, params: {}, members: {} }); + const { host, errors } = await loadedPlugins({ + "@app/a": (h) => { + h.serve(Names, { compose: listed }); + h.serve(Follower, (_, self) => { + self.contribute( + Names, + self.follow(async (observer) => { + end = () => observer.end({ kind: "ended" }); + return { value: "followed", detach: () => {} }; + }), + ); + return {}; + }); + }, + }); + const names = await host.registry(Names, context); + expect(names.value).toEqual(["@app/a:app.follower=followed"]); + end!(); + await flushed(); + expect(names.value).toEqual(["@app/a:app.follower=followed"]); + expect(errors.filter(({ kind }) => kind === "contribution_failed")).toEqual([]); + await host.close(context); + }); + + it("an object's contribution is in the table from live until it fails, again once retried, until it ends", async () => { + const { host, fail, retry } = await storeRestartable( + objects, + (attempt, self) => self.contribute(Names, `attempt ${attempt}`), + { serve: (host) => host.serve(Names, { compose: listed }) }, + ); + const names = await host.registry(Names, context); + expect(names.value).toEqual(["@test/app:app.store=attempt 1"]); + fail(); + expect(names.value).toEqual([]); + await retry(); + expect(names.value).toEqual(["@test/app:app.store=attempt 2"]); + await host.close(context); + // Closed: nothing composes any more; a value a reader keeps reads the last composition. + expect(names.value).toEqual(["@test/app:app.store=attempt 2"]); + }); +}); + +describe("the table's memory", () => { + it("keeps nothing of a closed host: records, tables and followed contributions go with it", async () => { + const Doc = defineState("app.kept", { schema: obj({ on: bool() }), version: 1, initial: () => ({ on: true }) }); + const Kept = local("app.kept", { version: 1, params: {}, members: {} }); + const refs = await (async () => { + const { host } = await loadedPlugins({ + "@app/kept": (host) => { + host.serve(Names, { compose: listed }); + host.serve(Kept, (_, self) => { + self.contribute( + Names, + self.derive([self.state(Doc)], ([d]) => `${d.on}`), + ); + return {}; + }); + }, + }); + const names = new WeakRef(await host.registry(Names, context)); + await host.close(context); + return [names, new WeakRef(host)]; + })(); + await expectCollected(...refs); + }); +}); diff --git a/packages/chord/test/next/registries/registry.test.ts b/packages/chord/test/next/registries/registry.test.ts new file mode 100644 index 000000000..61db2cd14 --- /dev/null +++ b/packages/chord/test/next/registries/registry.test.ts @@ -0,0 +1,67 @@ +import { describe, expect, expectTypeOf, it } from "vitest"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { createHost, defineFacet } from "../../../src/next/host/index.ts"; +import { defineRegistry, type Registry, type RegistryValue } from "../../../src/next/registries/index.ts"; +import { loadedPlugins } from "../support.ts"; + +describe("tokens", () => { + it("carry a name, a version and the types only; names and versions are checked at declaration", () => { + const Http = defineRegistry<{ readonly path: string }, readonly string[]>("chord.http", { version: 2 }); + expect(Http).toEqual({ kind: "registry", name: "chord.http", version: 2 }); + expectTypeOf(Http).toEqualTypeOf>(); + expect(() => defineRegistry("Chord.Http", { version: 1 })).toThrow('registry "Chord.Http": expected lowercase'); + expect(() => defineRegistry("chord.http", { version: 0 })).toThrow( + "chord.http.version: expected a positive integer", + ); + }); +}); + +describe("providers", () => { + const Names = defineRegistry("app.names", { version: 1 }); + + it("one per host: a second serve is refused at load", async () => { + const host = createHost({ onError: () => {} }); + const serve = defineFacet((h) => h.serve(Names, { compose: (table) => table.length })); + await expect( + host.load( + [ + { name: "@app/a", facet: serve }, + { name: "@app/b", facet: serve }, + ], + context, + ), + ).rejects.toThrow("registry app.names is served twice"); + }); + + it("a provider without compose is refused at load (plugin code is not type-checked)", async () => { + await expect(loadedPlugins({ "@app/a": (h) => h.serve(Names, {} as never) })).rejects.toThrow( + "registry app.names: compose is a function", + ); + }); + + it("a registry whose provider comes with `load` heals a `host.registry` made before it", async () => { + const host = createHost({ onError: () => {} }); + const waiting = host.registry(Names, context); + await host.load( + [{ name: "@app/a", facet: defineFacet((h) => h.serve(Names, { compose: (t) => t.length })) }], + context, + ); + const value = await waiting; + expectTypeOf(value).toEqualTypeOf>(); + expect(value.value).toBe(0); + await host.close(context); + }); + + it("are served only while the setup runs", async () => { + let later: Parameters[0]>[0] | undefined; + const { host } = await loadedPlugins({ + "@app/a": (h) => { + later = h; + }, + }); + expect(() => later!.serve(Names, { compose: (t) => t.length })).toThrow( + "@app/a: host.serve is valid only in the facet's setup", + ); + await host.close(context); + }); +}); diff --git a/packages/chord/test/next/registries/value.test.ts b/packages/chord/test/next/registries/value.test.ts new file mode 100644 index 000000000..e23722948 --- /dev/null +++ b/packages/chord/test/next/registries/value.test.ts @@ -0,0 +1,398 @@ +import { getEventListeners } from "node:events"; +import { describe, expect, it } from "vitest"; +import { BACKGROUND_CONTEXT as context, withAbortSignal, withCancel } from "../../../src/context/index.ts"; +import { int, local, method, str } from "../../../src/next/contracts/index.ts"; +import type { defineFacet } from "../../../src/next/host/index.ts"; +import { ChordObject } from "../../../src/next/host/object.ts"; +import { defineRegistry, type Registry } from "../../../src/next/registries/index.ts"; +import type { NativeWatchCall, WatchEnd } from "../../../src/watch.ts"; +import { + expectCollected, + failures, + flushed, + instancesOf, + loadedPlugins, + metered, + rejection, + settledHeap, +} from "../support.ts"; + +const objects = instancesOf(ChordObject, "attempt"); + +/** The provider's version 2: contributions are numbers, the value their sum. */ +const Sum = defineRegistry("app.sum", { version: 2 }); + +/** A reader built against version 1: its value was a plain number. */ +const SumV1 = defineRegistry("app.sum", { version: 1 }); + +/** A reader built against a version the provider does not know yet. */ +const SumV3 = defineRegistry("app.sum", { version: 3 }); + +const Setter = local("app.setter", { version: 1, params: {}, members: { set: method(int()) } }); + +/** Serves `Setter`, contributing a writable number to `Sum`. */ +function setter(host: Parameters[0]>[0], initial = 1): void { + host.serve(Setter, (_, self) => { + const value = self.writable(initial); + self.contribute(Sum, value); + return { + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); +} + +const total = (table: readonly { readonly value: number }[]) => ({ + total: table.reduce((n, { value }) => n + value, 0), +}); + +describe("older readers (S2)", () => { + it("get the composition mapped back with accepts[old].value, once per composition", async () => { + const { telemetry, counts } = await metered(); + const mapped: number[] = []; + const { host } = await loadedPlugins( + { + "@app/sum": (host) => { + host.serve(Sum, { + compose: total, + accepts: { + 1: { + value: (current) => { + mapped.push(current.total); + return current.total; + }, + }, + }, + }); + setter(host); + }, + }, + { telemetry }, + ); + const old = await host.registry(SumV1, context); + const current = await host.registry(Sum, context); + expect(old.value).toBe(1); + expect(old.value).toBe(1); + expect(current.value).toEqual({ total: 1 }); + await (await host.get(Setter, {}, context)).set(4, context); + expect(old.value).toBe(4); + expect(mapped).toEqual([1, 4]); + expect(counts()["chord.registry.adapts"]).toBe(2); + await host.close(context); + }); + + it("without a value mapping, and for a reader newer than the provider, fail `contract_mismatch`: broken, reported", async () => { + const Newer = local("app.newer", { version: 1, params: {}, members: { total: int() } }); + const Older = local("app.older", { version: 1, params: {}, members: { total: int() } }); + const { host, errors } = await loadedPlugins({ + "@app/sum": (host) => { + host.serve(Sum, { compose: total }); + host.serve(Newer, (_, self) => ({ total: self.derive([self.registry(SumV3)], ([v]) => v.total) })); + host.serve(Older, (_, self) => ({ total: self.derive([self.registry(SumV1)], ([v]) => v) })); + }, + }); + expect(await rejection(host.registry(SumV1, context))).toMatchObject({ code: "contract_mismatch" }); + expect(await rejection(host.registry(SumV3, context))).toMatchObject({ code: "contract_mismatch" }); + // Only a reload helps: no backoff, the mismatch itself the cause. + expect(failures(errors)).toEqual(["app.newer: broken", "app.older: broken"]); + expect(errors[0]).toMatchObject({ failure: { error: { code: "contract_mismatch" } } }); + await host.close(context); + }); + + it("maps an `undefined` composition (Sol, Astra)", async () => { + const Nothing = defineRegistry("app.nothing", { version: 2 }); + const NothingV1 = defineRegistry("app.nothing", { version: 1 }); + const { host } = await loadedPlugins({ + "@app/a": (h) => h.serve(Nothing, { compose: () => undefined, accepts: { 1: { value: () => "mapped" } } }), + }); + expect((await host.registry(NothingV1, context)).value).toBe("mapped"); + await host.close(context); + }); + + it("a failed mapping heals when the composition is the one mapped before (Astra)", async () => { + const Count = defineRegistry("app.count", { version: 2 }); + const CountV1 = defineRegistry("app.count", { version: 1 }); + const Counter = local("app.counter", { version: 1, params: {}, members: { set: method(int()) } }); + const { host } = await loadedPlugins({ + "@app/count": (h) => { + h.serve(Count, { + compose: (table) => table.reduce((n, { value }) => n + value, 0), + accepts: { + 1: { + value: (n) => { + if (n < 0) throw new Error("negative"); + return n; + }, + }, + }, + }); + h.serve(Counter, (_, self) => { + const value = self.writable(1); + self.contribute(Count, value); + return { + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); + }, + }); + const old = await host.registry(CountV1, context); + const calls: NativeWatchCall[] = []; + old.observe({ advance: (call) => calls.push(call), end: () => {} }, context); + const counter = await host.get(Counter, {}, context); + await counter.set(-1, context); + // Back to the composition mapped before (the same number): the mapping heals. + await counter.set(1, context); + expect(calls).toEqual([ + { kind: "failed", error: new Error("negative") }, + { kind: "value", value: 1 }, + ]); + await host.close(context); + }); +}); + +describe("readers (S3)", () => { + const Reader = local("app.reader", { version: 1, params: {}, members: { total: int() } }); + const serveReader = ( + host: Parameters[0]>[0], + R: Registry = Sum, + ) => host.serve(Reader, (_, self) => ({ total: self.derive([self.registry(R)], ([v]) => v.total) })); + + it("activate once the registry has a value; a reader of a registry nobody serves fails at once, `load` waits for nothing", async () => { + const started = Date.now(); + const { host, errors } = await loadedPlugins({ "@app/reader": (host) => serveReader(host) }); + expect(Date.now() - started).toBeLessThan(1_000); + expect(errors).toMatchObject([ + { + kind: "activation_failed", + object: "app.reader", + failure: { + kind: "transient", + error: { + code: "unavailable", + message: "app.reader: registry app.sum has no value", + cause: { code: "unavailable", reason: "undeclared" }, + }, + }, + }, + ]); + expect(await rejection(host.registry(Sum, context))).toMatchObject({ code: "unavailable", reason: "undeclared" }); + await host.close(context); + }); + + it("a reader failed on a first `compose` that threw is retried at once by the first good composition (R18)", async () => { + const { host, errors } = await loadedPlugins({ + "@app/sum": (host) => { + host.serve(Sum, { + compose: (table) => { + const sum = total(table); + if (sum.total === 0) throw new Error("empty"); + return sum; + }, + }); + serveReader(host); + }, + "@app/setter": (host) => setter(host, 0), + }); + expect(failures(errors)).toEqual(["app.reader: transient"]); + const setters = await host.get(Setter, {}, context); + await setters.set(3, context); + // The reader's backoff is a second at least: the composition retried it. + const reader = await host.get(Reader, {}, context, { readyTimeout: 500 }); + expect(reader.total.value).toBe(3); + await host.close(context); + }); + + it("once live, a later throwing `compose` changes nothing for the reader: `.value` keeps the last good one", async () => { + const { host } = await loadedPlugins({ + "@app/sum": (host) => { + host.serve(Sum, { + compose: (table) => { + const sum = total(table); + if (sum.total < 0) throw new Error("negative"); + return sum; + }, + }); + serveReader(host); + setter(host, 2); + }, + }); + const reader = await host.get(Reader, {}, context); + const setters = await host.get(Setter, {}, context); + expect(reader.total.value).toBe(2); + await setters.set(-1, context); + expect(reader.total.value).toBe(2); + expect(reader.state.value).toEqual({ kind: "live" }); + await host.close(context); + }); +}); + +describe("readers' memory", () => { + it("an ended reader is collected while the registry lives: its derive and its read leave the value", async () => { + const Reader = local("app.reader", { version: 1, params: { id: int() }, members: { total: int() } }); + const { host } = await loadedPlugins({ + "@app/sum": (host) => { + host.serve(Sum, { compose: total, accepts: { 1: { value: ({ total }) => total } } }); + host.serve(Reader, (_, self) => ({ + total: self.derive([self.registry(Sum), self.registry(SumV1)], ([v, old]) => v.total + old), + })); + setter(host); + }, + }); + const refs = await (async () => { + const reader = await host.get(Reader, { id: 1 }, context); + expect(reader.total.value).toBe(2); + reader.release(); + await new Promise((resolve) => setTimeout(resolve, 0)); + return objects.filter((ref) => ref.deref()?.label === "app.reader(1)"); + })(); + expect(refs.length).toBeGreaterThan(0); + await (await host.get(Setter, {}, context)).set(5, context); + await expectCollected(...refs); + await host.close(context); + }); +}); + +describe("values' memory", () => { + it("drops an older version's mapping once its last reader ended (Sol)", async () => { + const New = defineRegistry("app.version-gc", { version: 2 }); + const Old = defineRegistry("app.version-gc", { + version: 1, + }); + const Reader = local("app.old-reader", { version: 1, params: { id: int() }, members: { n: int() } }); + let mapped: WeakRef | undefined; + const { host } = await loadedPlugins({ + "@app/a": (h) => { + h.serve(New, { + compose: () => 1, + accepts: { + 1: { + value: (n) => { + const value = { n, payload: new Uint8Array(1_000_000) }; + mapped = new WeakRef(value); + return value; + }, + }, + }, + }); + h.serve(Reader, (_, self) => ({ n: self.derive([self.registry(Old)], ([v]) => v.n) })); + }, + }); + const reader = await host.get(Reader, { id: 1 }, context); + expect(reader.n.value).toBe(1); + reader.release(); + await flushed(); + await expectCollected(mapped!); + await host.close(context); + }); + + it("keeps no old composition once its readers stopped waiting and nobody follows it (Opus)", async () => { + const Big = defineRegistry("app.big", { version: 1 }); + let first = true; + const made: WeakRef[] = []; + const Waiting = local("app.waiting", { version: 1, params: {}, members: {} }); + const Counter = local("app.counter", { version: 1, params: {}, members: { set: method(int()) } }); + const { host } = await loadedPlugins({ + "@app/a": (h) => { + h.serve(Big, { + compose: (table) => { + if (first) throw new Error("first"); + const value = { payload: new Array(1000).fill(table.length) }; + made.push(new WeakRef(value)); + return value; + }, + }); + h.serve(Counter, (_, self) => { + const value = self.writable(0); + self.contribute(Big, value); + return { + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); + h.serve(Waiting, (_, self) => { + void self.registry(Big); + return {}; + }); + }, + }); + const counter = await host.get(Counter, {}, context); + first = false; + // The waiting reader's composition (pulled in the batch of this set) wakes it. + await counter.set(1, context); + await flushed(); + await counter.set(2, context); + void (await host.registry(Big, context)).value; + await settledHeap(); + // Only the current composition is alive. + expect(made.map((ref) => ref.deref() !== undefined)).toEqual([false, true]); + await host.close(context); + }); + + it("a value an app keeps after `host.close` keeps no closed host, and reads its last composition (Sol, Astra)", async () => { + const Kept = defineRegistry("app.kept", { version: 1 }); + const signal = new AbortController(); + const { weak, value } = await (async () => { + const { host } = await loadedPlugins({ "@app/a": (h) => h.serve(Kept, { compose: () => 7 }) }); + const value = await host.registry(Kept, context); + value.observe({ advance: () => {}, end: () => {} }, withAbortSignal(signal.signal, context)); + expect(getEventListeners(signal.signal, "abort")).toHaveLength(1); + await host.close(context); + // The observation ended with the host: its abort listener went with it. + expect(getEventListeners(signal.signal, "abort")).toHaveLength(0); + return { weak: new WeakRef(host), value }; + })(); + await expectCollected(weak); + expect(value.value).toBe(7); + }); +}); + +describe("observe", () => { + it("calls a synchronous observer inside the batch that composed; it counts as watching", async () => { + const Labels = defineRegistry("app.labels", { version: 1 }); + const Label = local("app.label", { version: 1, params: {}, members: { set: method(str()) } }); + const { host } = await loadedPlugins({ + "@app/labels": (host) => { + host.serve(Labels, { compose: (table) => table.map(({ value }) => value).join(",") }); + host.serve(Label, (_, self) => { + const label = self.writable("a"); + self.contribute(Labels, label); + return { + set: async (text: string) => { + label.set(text); + return null; + }, + }; + }); + }, + }); + const labels = await host.registry(Labels, context); + const calls: (NativeWatchCall | WatchEnd)[] = []; + const { context: cancellable, cancel } = withCancel(context); + const observation = labels.observe( + { advance: (call) => calls.push(call), end: (end) => calls.push(end) }, + cancellable, + ); + expect(observation.value).toBe("a"); + const label = await host.get(Label, {}, context); + await label.set("b", context); + // Already called: synchronously, in the set's batch. + expect(calls).toEqual([{ kind: "value", value: "b" }]); + cancel(); + await label.set("c", context); + expect(calls).toEqual([{ kind: "value", value: "b" }, { kind: "aborted" }]); + const again = labels.observe({ advance: (call) => calls.push(call), end: (end) => calls.push(end) }, context); + again.detach(); + await label.set("d", context); + expect(calls).toHaveLength(2); + labels.observe({ advance: () => {}, end: (end) => calls.push(end) }, context); + await host.close(context); + expect(calls.at(-1)).toEqual({ kind: "ended" }); + }); +}); diff --git a/packages/chord/test/next/state/native-source.test.ts b/packages/chord/test/next/state/native-source.test.ts new file mode 100644 index 000000000..cb31e8770 --- /dev/null +++ b/packages/chord/test/next/state/native-source.test.ts @@ -0,0 +1,152 @@ +import { describe, expect, expectTypeOf, it } from "vitest"; +import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts"; +import { int, local, method, str } from "../../../src/next/contracts/index.ts"; +import { ChordError } from "../../../src/next/data/index.ts"; +import type { Self } from "../../../src/next/host/index.ts"; +import { ChordObject } from "../../../src/next/host/object.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; +import { constant } from "../../../src/next/state/index.ts"; +import type { NativeWatchCall, Watchable } from "../../../src/watch.ts"; +import { delivered, failLive, flushed, instancesOf, loaded, rejection } from "../support.ts"; + +const objects = instancesOf(ChordObject, "attempt"); + +const Count = defineRegistry("app.count", { version: 1 }); + +const Setter = local("app.setter", { version: 1, params: {}, members: { set: method(int()) } }); + +/** Serves `Count` (a sum that throws when negative) and `Setter`, which contributes a writable number. */ +function serveCount(h: Parameters[0]>[0]): void { + h.serve(Count, { + compose: (table) => { + const sum = table.reduce((n, { value }) => n + value, 0); + if (sum < 0) throw new Error(`negative: ${sum}`); + return sum; + }, + }); + h.serve(Setter, (_, self) => { + const value = self.writable(1); + self.contribute(Count, value); + return { + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); +} + +describe("native sources as derive sources", () => { + it("a derive reads JSON and native sources alike, marked when a native one changes", async () => { + const Mixed = local("app.mixed", { version: 1, params: {}, members: { label: str(), name: method(str()) } }); + const { host } = await loaded((h) => { + serveCount(h); + h.serve(Mixed, (_, self) => { + const name = self.writable("a"); + return { + label: self.derive([name, self.registry(Count)], ([n, count]) => `${n}:${count}`), + name: async (text: string) => { + name.set(text); + return null; + }, + }; + }); + }); + const mixed = await host.get(Mixed, {}, context); + await (await host.get(Setter, {}, context)).set(2, context); + expect(mixed.label.value).toBe("a:2"); + await mixed.name("b", context); + expect(mixed.label.value).toBe("b:2"); + await host.close(context); + }); + + it("`ref.state` is a derive source: a state member can show a dependency's reachability", async () => { + const Flaky = local("app.flaky", { version: 1, params: { id: int() }, members: {} }); + const Watcher = local("app.watcher", { version: 1, params: {}, members: { flaky: str() } }); + let flakySelf: Self | undefined; + const { host } = await loaded((h) => { + h.serve(Flaky, (_, self) => { + flakySelf = self; + return {}; + }); + h.serve(Watcher, (_, self) => { + const flaky = self.get(Flaky, { id: 1 }); + return { flaky: self.derive([flaky.state], ([state]) => state.kind) }; + }); + }); + const watcher = await host.get(Watcher, {}, context); + const kinds: string[] = []; + (await watcher.flaky.watch(context)).start( + async (call) => void (call.kind === "value" && kinds.push(call.value)), + ); + expect(watcher.flaky.value).toBe("live"); + failLive(objects, flakySelf!, new ChordError({ code: "unavailable", message: "down" })); + expect(watcher.flaky.value).toBe("failed"); + await delivered(); + expect(kinds).toEqual(["failed"]); + await host.close(context); + }); + + it("types a derive's sources: watchables of either kind, never a native's handle (Fable)", () => { + const declare = (self: Self): void => { + const handle = self.open( + async () => 1, + () => {}, + ); + // @ts-expect-error a native handle has a `value` but is no watchable + self.derive([handle], ([n]) => n); + expectTypeOf(self.derive([self.writable(1), constant("a")], ([n, s]) => s.length + n)).toEqualTypeOf< + Watchable + >(); + }; + expect(declare).toBeTypeOf("function"); + }); + + it("refuses a native source as a state member (Q14)", async () => { + const Bad = local("app.bad", { version: 1, params: {}, members: { n: int() } }); + const { host } = await loaded((h) => h.serve(Bad, (_, self) => ({ n: self.writable(1) as never }))); + expect(await rejection(host.get(Bad, {}, context))).toMatchObject({ + message: "app.bad: n: a state member is a JSON watchable", + }); + await host.close(context); + }); +}); + +describe("watching a native source that fails", () => { + it("a watch taken while it is failed opens at its last good value and gets the failure first", async () => { + const { host } = await loaded(serveCount); + const count = await host.registry(Count, context); + await (await host.get(Setter, {}, context)).set(-1, context); + const calls: NativeWatchCall[] = []; + const watch = await count.watch(context); + expect(watch.value).toBe(1); + watch.start(async (call) => void calls.push(call)); + await delivered(); + expect(calls).toEqual([{ kind: "failed", error: new Error("negative: -1") }]); + await host.close(context); + }); + + it("an `on` routine started while its source is failed waits for its next good value, never calls with the stale one", async () => { + const Follower = local("app.follower", { version: 1, params: { id: int() }, members: {} }); + const seen: number[] = []; + const { host } = await loaded((h) => { + serveCount(h); + h.serve(Follower, (_, self) => { + self.on(self.registry(Count), ({ value }) => void seen.push(value)); + return {}; + }); + }); + const setters = await host.get(Setter, {}, context); + await setters.set(-1, context); + // Read: the failure is composed before the follower starts. + void (await host.registry(Count, context)).value; + const follower = await host.get(Follower, { id: 1 }, context); + await flushed(); + expect(seen).toEqual([]); + await setters.set(4, context); + await delivered(); + expect(seen).toEqual([4]); + follower.release(); + await host.close(context); + }); +}); diff --git a/packages/chord/test/next/state/writable.test.ts b/packages/chord/test/next/state/writable.test.ts new file mode 100644 index 000000000..30b27309f --- /dev/null +++ b/packages/chord/test/next/state/writable.test.ts @@ -0,0 +1,210 @@ +import { getEventListeners } from "node:events"; +import { describe, expect, it } from "vitest"; +import { BACKGROUND_CONTEXT as context, withAbortSignal } from "../../../src/context/index.ts"; +import { int, local, method } from "../../../src/next/contracts/index.ts"; +import type { ChordHost } from "../../../src/next/host/host.ts"; +import type { Self } from "../../../src/next/host/index.ts"; +import { ChordObject } from "../../../src/next/host/object.ts"; +import { defineRegistry } from "../../../src/next/registries/index.ts"; +import type { Writable } from "../../../src/next/state/index.ts"; +import type { WritableSource } from "../../../src/next/state/internal.ts"; +import type { NativeWatchCall, WatchEnd } from "../../../src/watch.ts"; +import { delivered, expectCollected, instancesOf, loaded, metered, rejection, restartable } from "../support.ts"; + +const objects = instancesOf(ChordObject, "attempt"); + +const Counter = local("app.counter", { version: 1, params: {}, members: { doubled: int(), set: method(int()) } }); + +/** Serves `Counter` with a writable number, its double as a state member; the writable is handed out to the test. */ +async function counter(options: Parameters[1] = {}) { + let writable: Writable | undefined; + let self: Self | undefined; + const { host, errors } = await loaded((h) => { + h.serve(Counter, (_, made) => { + self = made; + const value = made.writable(1); + writable = value; + return { + doubled: made.derive([value], ([n]) => n * 2), + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); + }, options); + const reference = await host.get(Counter, {}, context); + return { host, errors, reference, writable: writable!, self: self! }; +} + +describe("writable", () => { + it("a new value (by identity) is a new frame; the same value notifies nobody and costs one comparison", async () => { + const { telemetry, counts } = await metered(); + const { host, reference, writable } = await counter({ telemetry }); + const calls: NativeWatchCall[] = []; + (await writable.watch(context)).start(async (call) => void calls.push(call)); + const doubled: number[] = []; + (await reference.doubled.watch(context)).start( + async (call) => void (call.kind === "value" && doubled.push(call.value)), + ); + const computes = counts()["chord.derive.computes"]; + writable.set(1); + await delivered(); + expect(calls).toEqual([]); + expect(counts()["chord.derive.computes"]).toBe(computes); + writable.set(2); + writable.set(3); + await delivered(); + expect(calls).toEqual([ + { kind: "value", value: 2 }, + { kind: "value", value: 3 }, + ]); + // A JSON watch folds what came while it waited: the derive's last value. + expect(doubled).toEqual([6]); + expect(writable.value).toBe(3); + await host.close(context); + }); + + it("inside a batch, sets mark it there: one frame with the last value, the derives over it computed once", async () => { + const { telemetry, counts } = await metered(); + const { host, reference, writable } = await counter({ telemetry }); + const calls: NativeWatchCall[] = []; + (await writable.watch(context)).start(async (call) => void calls.push(call)); + const doubled: number[] = []; + (await reference.doubled.watch(context)).start( + async (call) => void (call.kind === "value" && doubled.push(call.value)), + ); + const computes = counts()["chord.derive.computes"]!; + (host as unknown as ChordHost).batch.commit(() => { + writable.set(5); + writable.set(6); + }, context); + await delivered(); + expect(calls).toEqual([{ kind: "value", value: 6 }]); + expect(doubled).toEqual([12]); + expect(counts()["chord.derive.computes"]).toBe(computes + 1); + await host.close(context); + }); + + it("refuses a set inside a derive compute or a compose (`invalid/reentrant`)", async () => { + const Echo = defineRegistry("app.echo", { version: 1 }); + let inner: Writable | undefined; + const Bad = local("app.bad", { version: 1, params: {}, members: { bad: int() } }); + const Good = local("app.good", { version: 1, params: {}, members: {} }); + const { host, errors } = await loaded((h) => { + h.serve(Echo, { + compose: (table) => { + inner?.set(table.length + 100); + return table.length; + }, + }); + h.serve(Good, (_, self) => { + inner = self.writable(0); + self.contribute(Echo, inner); + return {}; + }); + h.serve(Bad, (_, self) => { + const own = self.writable(0); + return { + bad: self.derive([own], ([n]) => { + own.set(n + 1); + return n; + }), + }; + }); + }); + expect((await host.registry(Echo, context)).value).toBe(0); + expect(errors.find((error) => error.kind === "registry_failed")).toMatchObject({ + error: { code: "invalid", reason: "reentrant" }, + }); + expect(errors.find((error) => error.kind === "activation_failed")).toMatchObject({ + object: "app.bad", + failure: { kind: "broken", error: { code: "invalid", reason: "reentrant" } }, + }); + await host.close(context); + }); + + it("is the attempt's: `set` after its end throws `released`, its watches end `ended`, its value stays readable", async () => { + let writable: Writable | undefined; + const { host, restart } = await restartable(objects, (attempt, self) => { + if (attempt === 1) writable = self.writable(1); + }); + const watch = await writable!.watch(context); + await restart(); + expect(await watch.closed).toEqual({ kind: "ended" } satisfies WatchEnd); + expect(() => writable!.set(2)).toThrow("app.restartable(1): ended"); + expect(writable!.value).toBe(1); + await host.close(context); + }); + + it("is a contribution, a derive source and an `on` source", async () => { + const Values = defineRegistry("app.values", { version: 1 }); + const Source = local("app.source", { version: 1, params: {}, members: { set: method(int()), seen: int() } }); + const seen: number[] = []; + const { host } = await loaded((h) => { + h.serve(Values, { compose: (table) => table.map(({ value }) => value) }); + h.serve(Source, (_, self) => { + const value = self.writable(1); + self.contribute(Values, value); + self.on(value, ({ value: n }) => void seen.push(n)); + return { + seen: self.derive([value], ([n]) => n), + set: async (n: number) => { + value.set(n); + return null; + }, + }; + }); + }); + const source = await host.get(Source, {}, context); + const values = await host.registry(Values, context); + await source.set(2, context); + await delivered(); + expect(values.value).toEqual([2]); + expect(source.seen.value).toBe(2); + expect(seen).toEqual([1, 2]); + await host.close(context); + }); + + it("is collected with its attempt", async () => { + const refs = await (async () => { + const made: WeakRef[] = []; + const { host, restart } = await restartable(objects, (attempt, self) => { + if (attempt === 1) made.push(new WeakRef(self.writable({ big: new Array(1000).fill(0) }))); + }); + await restart(); + await host.close(context); + return made; + })(); + await expectCollected(...refs); + }); + + it("a set of the same value after its attempt ended throws `released` too (Sol, Astra)", async () => { + const { host, writable } = await counter(); + await host.close(context); + expect(() => writable.set(1)).toThrow("app.counter: ended"); + }); + + it("a writable an app keeps after `host.close` keeps no closed host; its observers' abort listeners go (Sol)", async () => { + const signal = new AbortController(); + const { weak, writable } = await (async () => { + const { host, writable } = await counter(); + (writable as WritableSource).observe( + { advance: () => {}, end: () => {} }, + withAbortSignal(signal.signal, context), + ); + expect(getEventListeners(signal.signal, "abort")).toHaveLength(1); + await host.close(context); + expect(getEventListeners(signal.signal, "abort")).toHaveLength(0); + return { weak: new WeakRef(host), writable }; + })(); + await expectCollected(weak); + expect(writable.value).toBe(1); + }); + + it("a late callback's set after the host closed throws `released`", async () => { + const { host, writable } = await counter(); + await host.close(context); + expect(await rejection(Promise.resolve().then(() => writable.set(9)))).toMatchObject({ code: "released" }); + }); +}); diff --git a/packages/chord/test/next/support.ts b/packages/chord/test/next/support.ts index 428dea36f..9796d534f 100644 --- a/packages/chord/test/next/support.ts +++ b/packages/chord/test/next/support.ts @@ -35,6 +35,7 @@ import { createHost, defineFacet, type FacetHost, + type FacetSelf, type Host, type HostError, type RefState, @@ -231,7 +232,7 @@ async function restartableOf void, + setup: (host: FacetHost, self: FacetSelf) => void, options: { readonly telemetry?: InMemoryTelemetryContext; readonly context?: Context; @@ -247,6 +248,21 @@ export async function loaded( return { host, errors }; } +/** + * A host with background errors collected, loaded with one facet per plugin, in the order given (`setups` by plugin + * name). + */ +export async function loadedPlugins( + setups: Readonly void>>, + options: { readonly telemetry?: InMemoryTelemetryContext; readonly readyTimeout?: number } = {}, +): Promise<{ readonly host: Host; readonly errors: HostError[] }> { + const errors: HostError[] = []; + const host = createHost({ ...options, onError: (error) => errors.push(error) }); + const facets = Object.entries(setups).map(([name, setup]) => ({ name, facet: defineFacet(setup) })); + await host.load(facets, BACKGROUND_CONTEXT); + return { host, errors }; +} + /** A store on a fresh SQLite `:memory:` database, as Chord's layers see it. */ export async function memoryStore(options?: { readonly outbox?: boolean }): Promise { const db = await openNodeSqliteDatabase(":memory:"); diff --git a/packages/durable/CHANGELOG.md b/packages/durable/CHANGELOG.md index 56763148d..92430f3e4 100644 --- a/packages/durable/CHANGELOG.md +++ b/packages/durable/CHANGELOG.md @@ -98,6 +98,8 @@ - `RunFailureReason` (`pi.live.runFailure.reason`) gains `nothing_to_answer` and `turn_limit`, also an `unanswered` submission's `reason`: exhaustive switches over it need cases ([#11043](https://github.com/earendil-works/pi/issues/11043)) - `pi.live.run` has a required `turns` field, the run's tool rounds so far; code that writes `run` itself sets it (`turns: 0`). `pi.live` is version 3: its `migrate` adds `turns: 0` to a stored run without it, version 1 or a version 2 an earlier unreleased build wrote, whose run in flight otherwise failed its next tool round with "Value contains a non-finite number" ([#11043](https://github.com/earendil-works/pi/issues/11043)) +- A Session's close ends synchronous observers (`observeDoc()`, `observeConversation()`, `observeTaskGraph()`, `observeCommits()`) `failed` with the error "session closed" instead of `ended`, so an adapter following a Session (Chord's `self.follow`) fails and follows the next one; `ended` stays for an observed owner that ended (a task document's task), and watches still end `ended`. Before: `end: (end) => { if (end.kind === "ended") done(); }`. After: `end: (end) => { if (end.kind === "ended" || end.kind === "failed") done(); }`. + ### Added - `StoreStorage` (`@earendil-works/pi-durable/storage/store`): Durable's `Storage` on Chord v2's store. `StoreStorage.open(store, owner, context)` keeps a Session's records as declared tables of the store under `owner` and its documents in the store's documents engine, takes its ids from the store's one allocator, and commits each batch as one write of the store's commit step, adopted in the batch of the host the store serves, so a Chord object and a Durable Session commit through one queue and one outbox. It passes the storage conformance suite; `close()` leaves the store open (the app owns it). @@ -115,6 +117,8 @@ - `HookApi.entry()` and `entries()`, and `TaskRuntime.entries()`: a hook or task reads the conversation's raw history, which compaction never changes, so a range bounded by entry IDs reads the same before and after a compaction and on a rerun ([#11141](https://github.com/earendil-works/pi/issues/11141)) - Run-end tasks: a `GenerationHooks.afterRun(run, api)` hook returns `{ tasks }`, built with the typed `runEndTask(task, input)`, after a run answered (a final answer or a `terminate` round; not when an `onYield` continuation is taken, not after a failure or an abort). The answer is committed and the run's inputs settle `done` first, so `Submission.wait()` resolves at the answer; the generation then creates the tasks as its children and waits for them, keeping `pi.live.run` (with `run.afterRun`) and the conversation busy, so idle waits include them, Stop aborts them, and a crash resumes them. Input queued meanwhile, steers included, is placed when they are terminal and starts the next run; a task admits a model-visible write or a fix-run follow-up in its own commit with `submit(tx, draft)`. A crash before the tasks exist asks the hooks again; a broken request (an uninstalled or built-in task, a non-JSON input, a throwing `initial()`) is reported and skipped. `run` names the run's entries: `first` (`pi.live.run.first`, carried into `runFailure.first` so a continue covers a failed run's history), `answer`, and `last` ([#11141](https://github.com/earendil-works/pi/issues/11141)) +- `registryReader(value, context)`: a `RegistryReader` over a Chord v2 registry value whose composition is a registry snapshot (`RegistryValue`): `snapshot()` reads the current composition, `subscribe` observes its compositions synchronously inside Chord's batch, so a Harness opened in a Chord object takes its extensions from a Chord registry. + ### Changed - Document, conversation view, and task graph watches use Chord's watch handles (`Watch`). diff --git a/packages/durable/docs/spec.md b/packages/durable/docs/spec.md index 4f3795e33..385f8e948 100644 --- a/packages/durable/docs/spec.md +++ b/packages/durable/docs/spec.md @@ -5431,9 +5431,11 @@ advances without a commit, when a registry change or a failed migration changes a task's `blocked` reason (section 9.4): that call runs on the Session line, outside any `adopt`, with a context of its own. A document observer follows its address as a document watch does, and `observeDoc()` gives `undefined` for an -absent one. `end` comes once: when the Session closes (`ended`) or fails -(`failed` with the error), or, for a task document, right after the retirement change of -the commit that makes its task terminal, inside that commit's `adopt` (`ended`). +absent one. `end` comes once: when the Session closes (`failed` with the error +"session closed": an adapter following the Session, such as Chord's `self.follow`, +fails and follows the next Session; watches end `ended`) or fails (`failed` with +the error), or, for a task document, right after the retirement change of the +commit that makes its task terminal, inside that commit's `adopt` (`ended`). `detach()` is idempotent, and nothing follows it, not even the rest of the commit underway; a detached observer is referenced by nothing of the Session. `context` governs only the attachment: cancelling it while attaching rejects diff --git a/packages/durable/src/harness/registry-reader.ts b/packages/durable/src/harness/registry-reader.ts new file mode 100644 index 000000000..97f8a44bd --- /dev/null +++ b/packages/durable/src/harness/registry-reader.ts @@ -0,0 +1,34 @@ +// A Harness's registry over a Chord registry (Chord v2 slice 8 §5): the composed value of p3's extensions registry, +// read as Durable's `RegistryReader`. + +import type { Context } from "@earendil-works/chord/context"; +import type { RegistryValue } from "@earendil-works/chord/next"; +import type { RegistryReader, RegistrySnapshot, ToolRegistration } from "./types.ts"; + +/** + * A `RegistryReader` over a Chord registry value whose composition is a registry snapshot: `snapshot()` reads the + * current composition (the last good one while `compose` fails), `subscribe` observes its compositions, its listener + * called synchronously inside the Chord batch that composed (so it only wakes the scheduler; a Chord call there + * throws `invalid/reentrant`). `context` is the observation's: aborting it detaches every listener. + */ +export function registryReader( + value: RegistryValue>, + context: Context, +): RegistryReader { + return { + snapshot: () => value.value, + subscribe: (listener) => { + const observation = value.observe( + { + advance: (call) => { + // A failed composition keeps the last snapshot: nothing to reconsider. + if (call.kind === "value") listener(); + }, + end: () => {}, + }, + context, + ); + return () => observation.detach(); + }, + }; +} diff --git a/packages/durable/src/harness/task-graph.ts b/packages/durable/src/harness/task-graph.ts index 2e0b78875..943eba33a 100644 --- a/packages/durable/src/harness/task-graph.ts +++ b/packages/durable/src/harness/task-graph.ts @@ -1,16 +1,10 @@ -import { - type Context, - type JsonValue, - Watch, - type WatchCall, - type WatchEnd, - type WatchHandle, -} from "@earendil-works/chord"; +import { type Context, type JsonValue, Watch, type WatchCall, type WatchHandle } from "@earendil-works/chord"; import { BACKGROUND_CONTEXT, withoutAbortSignal } from "@earendil-works/chord/context"; import { applyImmutable, type Op } from "@earendil-works/chord/delta"; import { + closeEndOf, + closeEnds, contained, - ENDED, guardObserver, type ObservedCommit, observation, @@ -110,11 +104,10 @@ export class TaskGraphView { blocking.subscribeBlocked(() => this.#refreshBlocked()); session.subscribeClose(() => { this.#closed = true; - const failure = this.#session.failure; - const end: WatchEnd = failure === undefined ? ENDED : { kind: "failed", error: failure.error }; + const ends = closeEnds(this.#session.failure); for (const observer of [...(this.#mount?.observers ?? [])]) { contained( - () => observer.end(end), + () => observer.end(closeEndOf(observer, ends)), (error) => session.report(error), ); } diff --git a/packages/durable/src/harness/view.ts b/packages/durable/src/harness/view.ts index d2999c47f..72cd9c890 100644 --- a/packages/durable/src/harness/view.ts +++ b/packages/durable/src/harness/view.ts @@ -10,8 +10,9 @@ import { withoutAbortSignal } from "@earendil-works/chord/context"; import { applyImmutable, type NonEmptyPath, type Op, type Path } from "@earendil-works/chord/delta"; import { idFromNumber } from "../ids.ts"; import { + closeEndOf, + closeEnds, contained, - ENDED, guardObserver, type ObservedCommit, observation, @@ -158,12 +159,11 @@ export class ConversationViews { }); session.subscribeClose(() => { this.#closed = true; - const failure = this.#session.failure; - const end: WatchEnd = failure === undefined ? ENDED : { kind: "failed", error: failure.error }; + const ends = closeEnds(this.#session.failure); for (const mount of this.#mounts.values()) for (const windowed of [...mount.observers]) { contained( - () => windowed.observer.end(end), + () => windowed.observer.end(closeEndOf(windowed.observer, ends)), (error) => this.report(error), ); } diff --git a/packages/durable/src/index.ts b/packages/durable/src/index.ts index 4b4671129..59068ad12 100644 --- a/packages/durable/src/index.ts +++ b/packages/durable/src/index.ts @@ -54,6 +54,7 @@ export { } from "./harness/live.ts"; export { ProviderDoc, type ProviderState } from "./harness/provider.ts"; export { createRegistry } from "./harness/registry.ts"; +export { registryReader } from "./harness/registry-reader.ts"; export type { BlockedReason } from "./harness/scheduler.ts"; export type { TaskGraph, TaskGraphNode, TaskGraphState, TaskGraphWatch } from "./harness/task-graph.ts"; export { DURABLE_TELEMETRY_SCHEMA } from "./harness/telemetry.ts"; diff --git a/packages/durable/src/session/session.ts b/packages/durable/src/session/session.ts index 5f5df8778..a6bb44556 100644 --- a/packages/durable/src/session/session.ts +++ b/packages/durable/src/session/session.ts @@ -612,8 +612,8 @@ export class SessionImpl implements Session { const observers = [...this.#commitListeners]; this.#commitListeners.clear(); // After the last publication: the commits admitted before the seal have settled, held changes are stored. - const end: WatchEnd = - this.#failure === undefined ? ENDED : { kind: "failed", error: this.#failure.error }; + // Commit observers are synchronous observers, all of them. + const end = closeEnds(this.#failure).observer; for (const observer of observers) observer.end(end); this.#internalListeners.clear(); this.#documents.clear(); @@ -1269,7 +1269,7 @@ export class SessionImpl implements Session { }); unsubscribeClose = this.subscribeClose(() => { detach(); - observer.end(this.#failure === undefined ? ENDED : { kind: "failed", error: this.#failure.error }); + observer.end(closeEndOf(observer, closeEnds(this.#failure))); }); return observer; } @@ -1419,6 +1419,28 @@ function toError(error: unknown): Error { /** The end of an observer whose source closed. */ export const ENDED: WatchEnd = { kind: "ended" }; +/** + * How a Session's close ends what follows it (decision "Slice 8 S1"), made once per close: an app's watch `ended`; a + * synchronous observer (an adapter following the Session, which may open it again) `failed` with "session closed", so + * it follows the next Session; `ended` stays for an observed owner that ended (a task document's task). A failed + * Session ends both `failed` with its error. + */ +export function closeEnds(failure: { readonly error: unknown } | undefined): { + readonly watch: WatchEnd; + readonly observer: WatchEnd; +} { + if (failure !== undefined) { + const failed: WatchEnd = { kind: "failed", error: failure.error }; + return { watch: failed, observer: failed }; + } + return { watch: ENDED, observer: { kind: "failed", error: new Error("session closed") } }; +} + +/** The end `ends` give `follower`: a watch's, or an observer's. */ +export function closeEndOf(follower: unknown, ends: ReturnType): WatchEnd { + return follower instanceof Watch ? ends.watch : ends.observer; +} + /** * The `Observation` of a guarded observer attached at `value`. Built outside the attaching closure, so a detached * observation keeps nothing but `value` and the spent guard alive. diff --git a/packages/durable/src/types.ts b/packages/durable/src/types.ts index 4f17a3711..c2aa21688 100644 --- a/packages/durable/src/types.ts +++ b/packages/durable/src/types.ts @@ -1101,8 +1101,9 @@ export type DocumentWatch = WatchHandle>; * may come before the attaching promise resolves; their ops apply from `Observation.value`. `advance` must only record * the call, synchronously and quickly, never await, call the Session, or throw. A throw is reported as * `listener_error`, detaches the observer, and ends it `listener_error` with the error. `end` is called once: `ended` - * when the Session closes or the observed scope's owner ends (a task document's task, after its retirement change, - * inside that commit's `adopt`), `failed` with the error that failed the Session; never after `detach()`. + * when the observed scope's owner ends (a task document's task, after its retirement change, inside that commit's + * `adopt`), `failed` with "session closed" when the Session closes (an adapter follows the next Session) or with the + * error that failed the Session; never after `detach()`. */ export type Observer = { advance(call: WatchCall, context: Context): void; @@ -1111,7 +1112,7 @@ export type Observer = { /** * A synchronous observer of stored commits (`Session.observeCommits()`): `advance` gets each `CommitPublication`, `end` - * the Session's close. + * the Session's close (`failed` with "session closed", or with the error that failed the Session). */ export type CommitObserver = { advance(publication: CommitPublication, context: Context): void; diff --git a/packages/durable/test/examples/37-harness-registry.ts b/packages/durable/test/examples/37-harness-registry.ts new file mode 100644 index 000000000..404a61a0d --- /dev/null +++ b/packages/durable/test/examples/37-harness-registry.ts @@ -0,0 +1,172 @@ +// A Durable harness in Chord objects, p3's pattern (Chord v2 slice 8): the object that opens Durable on the host's store +// takes its extensions from a Chord registry (`registryReader`), so any plugin extends the harness by contributing an +// extension; and it contributes its Harness to a second registry, through which another object follows its views +// (`self.follow`). When that Harness's Session closes, the follower fails and follows the next Harness, woken by its +// contribution (`self.retryOn`). +// Run from packages/durable: +// node --conditions=source --experimental-strip-types test/examples/37-harness-registry.ts + +import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context"; +import { + arr, + createHost, + defineFacet, + defineRegistry, + local, + method, + type Native, + openStore, + type RegistryTable, + sqliteBackend, + str, +} from "@earendil-works/chord/next"; +import { openNodeSqliteDatabase } from "@earendil-works/chord/sqlite/node"; +import { createModels } from "@earendil-works/pi-ai"; +import { + createRegistry, + defineExtension, + defineTask, + type Extension, + Harness, + type RegistrySnapshot, + ROOT_CONVERSATION_ID, + registryReader, +} from "../../src/index.ts"; +import { StoreStorage } from "../../src/storage/store/index.ts"; + +const context = BACKGROUND_CONTEXT; + +// 1. The core's registries: extensions in, Durable's registry snapshot out; and the Harness, for other objects. +const Extensions = defineRegistry("example.extensions", { version: 1 }); +const Harnesses = defineRegistry, Harness | undefined>("example.harnesses", { version: 1 }); + +/** Builds Durable's snapshot without throwing: an extension Durable refuses (a task name taken) is reported, left out. */ +function snapshotOf( + table: RegistryTable, + report: (conflict: { message: string; plugins: string[] }) => void, +) { + const registry = createRegistry(); + for (const { value, plugin } of table) { + try { + registry.install(value); + } catch (error) { + report({ message: (error as Error).message, plugins: [plugin] }); + } + } + return registry.snapshot(); +} + +const Core = local("example.harness", { + version: 1, + params: {}, + members: { greet: method(undefined, str()), close: method() }, +}); +const Transcript = local("example.transcript", { version: 1, params: {}, members: { kinds: arr(str()) } }); + +// 2. The core facet: the harness object opens Durable on the store with its extensions read from the registry. +const store = await openStore(sqliteBackend(await openNodeSqliteDatabase(":memory:")), context); +const core = defineFacet((host) => { + host.serve(Extensions, { compose: (table, { report }) => snapshotOf(table, report) }); + host.serve(Harnesses, { compose: (table) => table.at(-1)?.value.value }); + host.serve(Core, (_, self) => { + // A declared dependency: the object activates (and opens Durable) once the registry has a value. + const extensions = self.registry(Extensions); + const harness = self.open( + async (open) => { + const opened = await Harness.open( + await StoreStorage.open(store, Core.name, open), + { models: createModels(), registry: registryReader(extensions, open) }, + open, + ); + await opened.root(open); + return opened; + }, + (h, _reason, close) => h.close(close), + ); + self.contribute(Harnesses, harness); + // Following its own task graph, the object fails with its Session (S1), and its retry opens the next Harness. + self.follow((observer, ctx) => harness.value.observeTaskGraph(observer, ctx)); + return { + greet: async (ctx) => { + // The composed snapshot, read at use: is the task's extension installed? + if (extensions.value.task(Greet.definition.name) === undefined) return "no greeter installed"; + const root = await harness.value.conversation(ROOT_CONVERSATION_ID, ctx); + const id = await root!.commit( + (tx) => tx.createTask(Greet, undefined, { ownership: { kind: "conversation" } }), + ctx, + ); + harness.value.resume(); + return JSON.stringify((await harness.value.waitForTask(id, ctx)).state.outcome); + }, + close: async (ctx) => { + await harness.value.close(ctx); + return null; + }, + }; + }); + // 3. Another object follows the harness's conversation view through the registry. + host.serve(Transcript, (_, self) => { + const harness = self.registry(Harnesses); + self.retryOn(harness); + const view = self.follow(async (observer, ctx) => + harness.value?.observeConversation(ROOT_CONVERSATION_ID, observer, ctx), + ); + return { kinds: self.derive([view], ([v]) => v.entries.map((entry) => entry.kind)) }; + }); +}); + +// 4. Two plugins contribute extensions with the same task name, which Durable refuses: the first by plugin name +// (copycat) is installed, the other reported. +const Greet = defineTask({ + name: "example.greet", + version: 1, + initial: () => ({ phase: "run" }), + phases: { + run: async (_task, runtime, taskContext) => { + await runtime.commit(async (tx) => { + await tx.appendEntry(ROOT_CONVERSATION_ID, { kind: "example.greeted", data: {} }); + return { status: "terminal", outcome: { status: "completed", result: "hello" } }; + }, taskContext); + }, + }, + abort: async () => {}, +}); +const greeter = defineFacet((_, self) => + self.contribute(Extensions, defineExtension({ name: "greeter", tasks: [Greet] })), +); +const copycat = defineFacet((_, self) => + self.contribute(Extensions, defineExtension({ name: "copycat", tasks: [Greet] })), +); + +const host = createHost({ + store, + onError: (error) => console.log("onError:", error.kind, error.kind === "registry_conflict" ? error.message : ""), +}); +await host.load( + [ + { name: "@example/core", facet: core }, + { name: "@example/greeter", facet: greeter }, + { name: "@example/copycat", facet: copycat }, + ], + context, +); + +// 5. The task comes from a contributed extension; the transcript object sees its entry. (It failed once at start, before +// the harness object contributed its Harness: `activation_failed`, then the contribution woke it.) +const harness = host.hold(Core, {}, context); +const transcript = host.hold(Transcript, {}, context); +console.log("greet:", await harness.greet(context)); +await new Promise((resolve) => setTimeout(resolve, 10)); +console.log("transcript:", transcript.kinds.value); + +// 6. The Session closes: both objects fail; a retry of the harness object opens the next Harness (on the same store: +// the transcript is still there), whose contribution wakes the follower. +await harness.close(context).catch((error: Error) => console.log("close:", error.message)); +await new Promise((resolve) => setTimeout(resolve, 10)); +console.log("transcript after the close:", transcript.state.value.kind); +harness.retry(); +await new Promise((resolve) => setTimeout(resolve, 50)); +console.log("transcript after the retry:", transcript.state.value.kind, transcript.kinds.value); + +await host.close(context); +await store.close(context); diff --git a/packages/durable/test/harness-observers.test.ts b/packages/durable/test/harness-observers.test.ts index fdd6d7787..bbd3db12e 100644 --- a/packages/durable/test/harness-observers.test.ts +++ b/packages/durable/test/harness-observers.test.ts @@ -124,8 +124,11 @@ describe("observeConversation and observeTaskGraph", () => { await root.commit((tx) => tx.appendEntry(root.id, { kind: "note", data: {} }), context); expect(view.calls).toHaveLength(1); await watch.stop(); + const graphWatch = await harness.watchTaskGraph(context); await harness.close(context); - expect(graph.calls).toEqual([{ end: { kind: "ended" } }]); + // S1: a Session close fails an observer (it may follow the next Session); a watch ends `ended`. + expect(graph.calls).toEqual([{ end: { kind: "failed", error: new Error("session closed") } }]); + expect(await graphWatch.closed).toEqual({ kind: "ended" }); }); it("end with the error that failed the Harness", async () => { diff --git a/packages/durable/test/harness-registry-reader.test.ts b/packages/durable/test/harness-registry-reader.test.ts new file mode 100644 index 000000000..b2a82da35 --- /dev/null +++ b/packages/durable/test/harness-registry-reader.test.ts @@ -0,0 +1,198 @@ +import { BACKGROUND_CONTEXT as context } from "@earendil-works/chord/context"; +import { + arr, + createHost, + defineFacet, + defineRegistry, + type FacetHost, + local, + method, + type Native, + type RegistryTable, + str, +} from "@earendil-works/chord/next"; +import { createModels } from "@earendil-works/pi-ai/models"; +import { describe, expect, it } from "vitest"; +import { + createRegistry, + defineExtension, + defineTask, + type Extension, + Harness, + MemoryStorage, + type RegistrySnapshot, + ROOT_CONVERSATION_ID, + registryReader, +} from "../src/index.ts"; + +/** p3's pattern: the extensions registry composes Durable's snapshot, leaving out an extension Durable refuses. */ +const Extensions = defineRegistry("app.harness.extensions", { version: 1 }); + +function compose(table: RegistryTable, report: (conflict: { message: string; plugins: string[] }) => void) { + const registry = createRegistry(); + for (const { value, plugin } of table) { + try { + registry.install(value); + } catch (error) { + report({ message: (error as Error).message, plugins: [plugin] }); + } + } + return registry.snapshot(); +} + +const Done = defineTask({ + name: "app.done", + version: 1, + initial: () => ({ phase: "run" }), + phases: { + run: async (_task, runtime, taskContext) => { + await runtime.commit( + () => ({ status: "terminal", outcome: { status: "completed", result: null } }), + taskContext, + ); + }, + }, + abort: async () => {}, +}); + +/** A host-lifetime object contributing an extension it replaces with `install` (one with the task `app.done`). */ +const Plugin = local("app.plugin", { version: 1, params: {}, members: { install: method() } }); + +function serve(host: FacetHost): void { + host.serve(Extensions, { compose: (table, { report }) => compose(table, report) }); + host.serve(Plugin, (_, self) => { + const extension = self.writable(defineExtension({ name: "app.plugin" })); + self.contribute(Extensions, extension); + return { + install: async () => { + extension.set(defineExtension({ name: "app.plugin", tasks: [Done] })); + return null; + }, + }; + }); +} + +describe("registryReader", () => { + it("reads the composed snapshot; a listener is called synchronously, inside the batch that composed", async () => { + const host = createHost({ onError: () => {} }); + await host.load([{ name: "@app/core", facet: defineFacet(serve) }], context); + const value = await host.registry(Extensions, context); + const reader = registryReader(value, context); + expect(reader.snapshot().extension("app.plugin")?.tasks).toBeUndefined(); + const calls: string[] = []; + const unsubscribe = reader.subscribe(() => { + // Inside Chord's batch: a Chord call here is refused, so a listener only wakes its scheduler. + try { + value.observe({ advance: () => {}, end: () => {} }, context); + calls.push("outside"); + } catch (error) { + calls.push((error as { reason?: string }).reason ?? "other"); + } + }); + const plugin = await host.get(Plugin, {}, context); + await plugin.install(context); + expect(calls).toEqual(["reentrant"]); + expect(reader.snapshot().task("app.done")).toBe(Done); + unsubscribe(); + await host.close(context); + }); + + it("wakes a Harness's scheduler: a task blocked on its missing definition runs once a contribution brings it", async () => { + const host = createHost({ onError: () => {} }); + await host.load([{ name: "@app/core", facet: defineFacet(serve) }], context); + const harness = await Harness.open( + new MemoryStorage(), + { models: createModels(), registry: registryReader(await host.registry(Extensions, context), context) }, + context, + ); + harness.resume(); + const root = await harness.root(context); + const id = await root.commit( + (tx) => tx.createTask(Done, undefined, { ownership: { kind: "conversation" } }), + context, + ); + const done = harness.waitForTask(id, context); + await new Promise((resolve) => setTimeout(resolve, 10)); + await (await host.get(Plugin, {}, context)).install(context); + expect((await done).state.outcome).toEqual({ status: "completed", result: null }); + await harness.close(context); + await host.close(context); + }); +}); + +/** p3's pattern for Durable's views from other objects: the object that opens Durable contributes its Harness. */ +const Harnesses = defineRegistry, Harness | undefined>("app.harnesses", { version: 1 }); + +/** Opens a Harness (a new one per attempt), contributes it, and follows its task graph: its Session closing fails it. */ +const Durable = local("app.durable", { version: 1, params: {}, members: { close: method(), write: method(str()) } }); + +/** Follows the root conversation's entry kinds through the contributed Harness, from another object. */ +const Viewer = local("app.viewer", { version: 1, params: {}, members: { kinds: arr(str()) } }); + +function serveViews(host: FacetHost): void { + host.serve(Harnesses, { compose: (table) => table.at(-1)?.value.value }); + host.serve(Durable, (_, self) => { + const harness = self.open( + async (open) => { + const opened = await Harness.open( + new MemoryStorage(), + { models: createModels(), registry: createRegistry() }, + open, + ); + await opened.root(open); + return opened; + }, + (h, _reason, close) => h.close(close), + ); + self.contribute(Harnesses, harness); + self.follow((observer, ctx) => harness.value.observeTaskGraph(observer, ctx)); + return { + close: async (ctx) => { + await harness.value.close(ctx); + return null; + }, + write: async (kind, ctx) => { + const root = await harness.value.conversation(ROOT_CONVERSATION_ID, ctx); + await root!.submit({ type: "write", entry: { kind } }, ctx); + return null; + }, + }; + }); + host.serve(Viewer, (_, self) => { + const harness = self.registry(Harnesses); + self.retryOn(harness); + const view = self.follow(async (observer, ctx) => + harness.value === undefined + ? undefined + : harness.value.observeConversation(ROOT_CONVERSATION_ID, observer, ctx), + ); + return { kinds: self.derive([view], ([v]) => v.entries.map((entry) => entry.kind)) }; + }); +} + +describe("Durable's views from other objects (slice 8 §5, S1)", () => { + it("a follower of a contributed Harness fails when that Harness's Session closes, and follows the next one", async () => { + const host = createHost({ onError: () => {} }); + await host.load([{ name: "@app/core", facet: defineFacet(serveViews) }], context); + const durable = host.hold(Durable, {}, context); + const viewer = host.hold(Viewer, {}, context); + await new Promise((resolve) => setTimeout(resolve, 10)); + await durable.write("app.first", context); + await new Promise((resolve) => setTimeout(resolve, 10)); + expect(viewer.kinds.value).toContain("app.first"); + // The Session closes: both followers fail `transient` (S1); a retry opens the next Harness, whose contribution + // wakes the viewer (`retryOn`), which follows it. + // The call is cut off by its object's failure: the Session's close fails the object that opened it. + await expect(durable.close(context)).rejects.toThrow("session closed"); + await new Promise((resolve) => setTimeout(resolve, 10)); + expect(viewer.state.value).toMatchObject({ kind: "failed", cause: { kind: "transient" } }); + durable.retry(); + await new Promise((resolve) => setTimeout(resolve, 50)); + expect(viewer.state.value).toEqual({ kind: "live" }); + expect(viewer.kinds.value).not.toContain("app.first"); + await durable.write("app.second", context); + await new Promise((resolve) => setTimeout(resolve, 10)); + expect(viewer.kinds.value).toContain("app.second"); + await host.close(context); + }); +}); diff --git a/packages/durable/test/session-observers.test.ts b/packages/durable/test/session-observers.test.ts index 0dcbc324a..a22731cef 100644 --- a/packages/durable/test/session-observers.test.ts +++ b/packages/durable/test/session-observers.test.ts @@ -173,13 +173,31 @@ describe("observeDoc", () => { expect(calls).toEqual([]); }); - it("ends once when the Session closes", async () => { + it("ends once when the Session closes, `failed` (S1: an adapter follows the next Session), a watch `ended`", async () => { const { storage, session } = open(); await add(session); const { calls, observer } = recorder(storage); await session.observeDoc(SessionCounter, observer, context); + const watch = (await session.watchDoc(SessionCounter, context))!; await session.close(context); - expect(calls).toEqual([{ end: { kind: "ended" } }]); + expect(calls).toEqual([{ end: { kind: "failed", error: new Error("session closed") } }]); + expect(await watch.closed).toEqual({ kind: "ended" }); + }); + + it("ends each close's observers with an error of that close, not one shared by every Session", async () => { + const errors: unknown[] = []; + for (let i = 0; i < 2; i++) { + const { session } = open(); + await add(session); + await session.observeDoc( + SessionCounter, + { advance: () => {}, end: (end) => errors.push(end.kind === "failed" ? end.error : end) }, + context, + ); + await session.close(context); + } + expect(errors).toEqual([new Error("session closed"), new Error("session closed")]); + expect(errors[0]).not.toBe(errors[1]); }); it("ends with the error that failed the Session", async () => { @@ -368,14 +386,18 @@ describe("observeCommits", () => { return { calls, observer }; } - it("gets every stored commit, then `ended` once the Session closed, after its last publication", async () => { + it("gets every stored commit, then `failed` once the Session closed (S1), after its last publication", async () => { const { session } = open(); const { calls, observer } = commits(); session.observeCommits(observer); await add(session); await add(session); await session.close(context); - expect(calls).toEqual([{ seq: expect.any(Number) }, { seq: expect.any(Number) }, { end: { kind: "ended" } }]); + expect(calls).toEqual([ + { seq: expect.any(Number) }, + { seq: expect.any(Number) }, + { end: { kind: "failed", error: new Error("session closed") } }, + ]); }); it("ends `failed` with the error that failed the Session", async () => { diff --git a/scripts/check-entry-graphs.mjs b/scripts/check-entry-graphs.mjs index b3b033a73..388eb0012 100644 --- a/scripts/check-entry-graphs.mjs +++ b/scripts/check-entry-graphs.mjs @@ -67,8 +67,9 @@ const BUDGETS = { // The Harness's request identity reaches pi-ai's utils/identity.ts and its version constant. Partial commits // parse streamed tool-call arguments with pi-ai's utils/streaming-json.ts (and its json-parse.ts) to append what // argument strings grew by (harness/partial-changes.ts). Chord's watches fold pending ops with Delta's - // pending queue (delta/pending.ts). - maxFiles: 77, + // pending queue (delta/pending.ts). `registryReader` (harness/registry-reader.ts) reads a Chord v2 registry + // value through type-only imports. + maxFiles: 78, forbid: [ "packages/ai/src/index.ts", "packages/ai/src/utils/typebox-helpers.ts",