sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

refactor(chord): simplification audit of slices 6a-7b

Browse source 
The owner's audit decisions (decisions.md, "Simplification audit"): cut the reload-only
installing state (staging is synchronous) and the static registry cycle edges (the
runtime guard stays); retry propagation by a visited set and a hop budget (retry.hops)
instead of operation ids and through lists; a pure swap sends holds a fresh bound; one
rule after a live failure (every member followed until the admitted commits published);
parked_long for any long park, naming the first blocker in declaration order with its
state; one seal token per attempt. Keys follow the old instance during a restart's drain
like members. src/next: about 255 code lines fewer.
This commit is contained in:
Mario Zechner 2026-10-30 13:30:53 +00:00
parent be15fe3c17
commit 50af8c1601
51 changed files with 1123 additions and 946 deletions
Download patch file Download diff file Expand all files Collapse all files

30
packages/chord/docs/next/benchmarks.md
View file

@ -850,3 +850,33 @@ The small call, measured apart (ten bursts of 1,000 after 3,000 warm-up calls, t
29.5-30.6 → 27.6-27.9 µs and text 25.1-25.9 → 27.5-28.3 µs; with the young generation pinned, binary 29.7-30.7 →
30.3-30.7 µs and text 23.6-23.7 → 23.9-24.1 µs. The stock text gap is the young generation's sizing, as for commit to
frame above (pinned, the two match); the snapshot before the final review's fixes measures the same.
## Simplification audit (2026-10-30)
HEAD `4b535d57e` → the audit build (A1, A2, A5, A9-A12), copies at paths of equal length, CPU 6, alternating.
`calls.ts`, three runs: local call 2.70-2.73 → 2.69-2.73 µs (6,660 B both), watched derive commit 10.41-10.75 →
10.42-10.74 µs (14,899 B both), activation churn 59.0-60.9 → 59.1-61.4 µs (41,473-41,482 → 41,478 B). Retained per live
object with one dependency (10,000 kept) 6,745-6,754 → 6,745-6,752 B; a first cut of the declaration-ordered
dependency list (A11) kept it on live attempts, +225 B, and is now let go after the dependency check. `links.ts` with
the young generation pinned: commit to frame, hold to live and retained per hold equal within run spread (176.4 →
176.2-176.6 KiB per hold).
**Text-codec calls in isolation** (Sol's probe: a remote object over one memory link, 5,000 warm-up calls, ten bursts
of 1,000 small calls, then 200 warm-up and 30 bursts of 10 calls with a 1 MiB argument; binary first, then text; the
heap collected before each burst and the collections in it recorded: none in any burst), seven alternating pairs,
median of the pairs' medians:
| Case | stock | young generation pinned |
| --- | --- | --- |
| binary small | 26.53 → 26.37 µs | 25.72 → 25.90 µs |
| text small | 19.31 → 20.95 µs (+8.5%) | 19.11 → 19.93 µs (+4.3%) |
| binary 1 MiB | 985 → 993 µs | 539 → 412 µs (bimodal: 187-607 µs per run) |
| text 1 MiB | 958 → 963 µs | 273 → 284 µs (221-352 µs per run) |
Heap per call is the same (text small 15,171 → 15,160-15,202 B). The text small gap is not per-call work: the codec
alone (encode, decode and the message check of a call and a result message, 20,000 per burst) measures the same in both
trees (text call 1.55 µs and 52 B, result 1.52 µs and 171 B), deoptimizations and inlining decisions are the same, and
the gap follows the shape of the text decoder's `fromFields`, not what runs: the build with HEAD's `text.ts` matches HEAD
(19.0-19.2 µs), the one change in it that matters is the never-executed `T` (retry) case, and writing that case with an
object spread again (`...(f.length >= 2 ? { hops } : …)`) restores HEAD's 19.1 µs while other edits of the same size do
not. With text measured first the gap is 22.4-22.5 → 23.0-23.3 µs (+2-3%). V8 code generation, recorded as noise.

75
packages/chord/docs/next/decisions.md
View file

@ -2269,6 +2269,81 @@ validation and copies the v2 kernel added around Delta.
descriptions on the wire (the `describe` message kind of R1 and the per-link description cache go away), R3 is moot,
the coverage check across fingerprints of one version is gone.
## Simplification audit (2026-10-30)
The owner asked, after slice 7b's seven fix rounds, how many other mechanisms serve a rare case at a high cost. Audits
by Opus and Sol (`/tmp/audit-{opus,sol}.md`) and this session; each item decided by the owner with its code and case.
- **A1, `installing` is cut (owner, 2026-10-30: "cut"):** model §1's state for a hold or request that does not fit the
running code while a reload stages: staging is synchronous (`stagePlugins`, `recordFactory`), so no message or caller
can arrive while it runs (the only test reached it from a factory calling back into the host). The `Serving` kind, the
host's map, `Hold.start`'s branch and the test go. During the app's bundling before `host.reload` the old code
answers a mismatch `failed { waiting }` until the commit wakes the request (model §5's accepted gap). Supersedes
model §1's `installing` row and design §21's installing sentences.
- **A2, static registry cycle edges are cut (owner, 2026-10-30: "yeah cut"):** the edges a contribution of another
registry's value recorded (`RegistryEdge`, `#reads` counts, `assertNoCycle`, `#reaches`, an attempt's pending edges,
staging's edges) go; the runtime guard stays (a registry read while it composes throws `invalid/cycle`; the
contribution that read it is left out and reported `contribution_failed`). A ring of registries is caught at its first
composition instead of at `self.contribute`: the contributing object goes live, the rest of the registry works.
Supersedes slice 8's "a cycle of registries reading each other refused when the second one serves or contributes".
- **A3, routine calls keep their ops (owner, 2026-10-30: "keep"):** `self.on` handlers keep `{ value, ops, retired }`
with the routine's own folding (`PendingOps`, the basis after a throw).
- **A4, cross-host activation chains stay (owner, 2026-10-30: "yep keep"):** they catch the common, sequential form of a
cross-host dependency ring (one request whose activation creates the other side) with an immediate `invalid/cycle`
naming the path; `parked_long` stays the slow net for the rare simultaneous race.
- **A5, retry propagation by a hop budget (owner, 2026-10-30: "b if correct"):** `ref.retry()` still passes on from a
parked object to its failed dependencies (model §3.7), but its dedupe is simpler: on one host one retry's walk keeps a
visited set (a diamond retries a shared dependency once); across links the `retry` message carries a hop budget (an
integer, 16 from the first host, one less per host, dropped at 0) instead of the operation id (uuidv7), the objects'
last-8 remembered ids and the `through` list. Bounded effects, both harmless: a diamond across hosts may send a shared
dependency two retries (the second meets it activating and is ignored, or starts one more attempt if the first
already failed); a waiting ring across hosts (the undetected simultaneous cycle) passes the retry round until the budget
runs out, starting no attempt (parked objects only pass it on). Instance ids stay (activation chains, A4).
- **A6, staging factories against a recording `self` stay (owner, 2026-10-30: "keep"):** a throwing new factory keeps
rejecting the whole reload (the host stays on its old code), and each object's fate is decided from its recorded
declarations before the commit.
- **A7, registry last-good with provenance stays (owner, 2026-10-30: "a").**
- **A8, deferred job-state persistence keeps its held ops (owner, 2026-10-30: "keep").**
- **A9, a pure swap sends holds a fresh `bound` (owner, 2026-10-30: "simplify"):** a hold over a link whose object
swaps in place gets a fresh `bound` (one snapshot of its members), the path a restart, a variant change and a version
change already take, instead of a re-subscription with `[["r", value]]` per changed member (`Hold.#rebound`, its
constant-to-source and failed-member cases, `currentOf`). A tab's watchers may get one frame for an unchanged member;
nothing is lost or stale. Supersedes design §21/§22's per-member `r` rule for holds after a swap.
- **A10, one rule after a live failure (owner, 2026-10-30: "simplify"):** every member of a failed live instance keeps
being followed (references and holds) until the commits it admitted published (`documents.settled()`), then all stop:
the restarts' rule. The "own" against "foreign" split (`#follows === "own"`, `Attempt.owns` and its mapping-derive
case, `Hold.#stopForeign`) goes; a re-exposed member of another object may show one more change in that window.
Refines 6a's build-review rule "members following another object's sources freeze at once".
- **A11, `parked_long` widened (owner, 2026-10-30: "widen"):** reported once per parking episode when an attempt has been
parked longer than its `readyTimeout`, whatever its blocker (the first in declaration order named, with its state);
the remote-loading condition, `heldOnLink` and the clock restart at a link go. Also covers a cross-host ring one of
whose members waits on a failed dependency. Supersedes decision 6's narrow condition.
- **A12, the routine fence token folds into the seal (owner, 2026-10-30: "fold"):** one seal token per attempt; after a
swap the old routine calls get `closeTimeout` to finish, then the whole old attempt is sealed (its method calls had
the same bound). `RoutineContext.fence` and the token choice per transaction go. Supersedes slice 7b decision 4's
separate token.
- **A13, the table version fence stays (owner, 2026-10-30: "keep"):** an old call stuck past a restart's drain never
issues SQL against a migrated table (which would stop the store and close the host).
- **A14, key pins keep their hand-over between instances (owner, 2026-10-30: "keep").**
- **A15, keyed members across objects stay, with their versioning (owner, 2026-10-30: "keep essential"):** they fill
the gap P3 creates (a view with params following one entry of a persisted store live); the alternatives (a singleton
map, a method) are weaker.
Summary: cut A1 (`installing`), A2 (static registry cycle edges); simplified A5 (retry hop budget), A9 (a fresh `bound`
after a swap), A10 (one rule after a live failure), A11 (`parked_long` widened), A12 (one seal token); kept A3, A4, A6,
A7, A8, A13, A14, A15.
## Portability boundaries in the kernel
From `research/chord-on-durable-objects.md`, adopted for the Chord v2 kernel because they cost little and help tests:

2
packages/chord/docs/next/dependencies/model.md
View file

@ -1,6 +1,6 @@
# Dependency resolution in Chord v2: one model (decided, 2026-10-29)
Status: decided (owner, 2026-10-29); **built for slice 7a** (§1-4 and §6-11 without what installs and reloads add:
Status: decided (owner, 2026-10-29); **built for slice 7a** (audit A1, 2026-10-30: `installing` is cut, staging being synchronous; A5: retry dedupe by a visited set and a hop budget; A11: `parked_long` for any long park) (§1-4 and §6-11 without what installs and reloads add:
§1's `installing`, §4's install-end and reload wakes, §3.9's reload handover and §5's removal, versions and rollout
are slice 7b's; folded into `design.md` §3, §5, §6, §7, §17, §21's start barrier, §22; `slice-7a.md`). The draft
history below is kept as it was. Sources: the owner's direction

61
packages/chord/docs/next/design.md
View file

@ -250,7 +250,7 @@ and closed by the app with `store.close(ctx)` after `host.close`; the host never
background, as a tagged `HostError`: `listener_error` (a watch listener threw), `delivery_failed` (a frame delivery
threw), `activation_failed { object, failure }`, `close_failed { object, native, error }`, `close_timeout { object,
native }`, `drain_timeout { object }`, `protocol_error { error }`, `resolve_failed { address, error }`; later routine
errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), the registries' `registry_conflict { registry, message, plugins }`, `registry_failed { registry, error }`, `contribution_failed { registry, plugin, type?, error }` and `contribution_refused { registry, plugin, type?, version }` (each once per episode, slice 8), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6), `parked_long { object, waitingFor, since }` (an attempt parked longer than its `readyTimeout` on a remote attempt that is only `loading`, once per parking episode: a diagnostic of a ring across hosts started at the same moment, slice 7a, §6) and `storage_failed { error }`
errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), the registries' `registry_conflict { registry, message, plugins }`, `registry_failed { registry, error }`, `contribution_failed { registry, plugin, type?, error }` and `contribution_refused { registry, plugin, type?, version }` (each once per episode, slice 8), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6), `parked_long { object, waitingFor, state, since }` (an attempt parked longer than its `readyTimeout`, whatever its blocker, once per parking episode: a diagnostic of a ring across hosts started at the same moment or a long wait, slice 7a, audit A11, §6) and `storage_failed { error }`
(slice 6a), which every app must handle: Chord never ends the process,
the app exits with its own code for its supervisor or reopens.
@ -639,8 +639,9 @@ host.serve(Notes, ({ session }, principal, self) =>
- **A live object can fail** (owner, 2026-10-26): a runtime failure of the owner (a `migrate` failing on access, a
stored version newer than the code) seals the live attempt and makes the object `failed` (`broken`): calls reject
with the cause, state members keep their last values, natives close, documents stay on the object; recovery by
reload or `ref.retry()`; live dependents stay live. The members freeze only after the commits admitted before the
seal adopted and published once, and a retry's factory waits for that (2026-10-27, slice 6a build reviews).
reload or `ref.retry()`; live dependents stay live. Its references keep following every member (its own and
re-exposed ones) until the commits admitted before the seal adopted and published once, then take their last values
(audit A10, 2026-10-30: one rule, as restarts), and a retry's factory waits for that.
- **Parked on its dependencies** (dependency model §3, slice 7a; supersedes R18's dependency deadline and backoff): a
dependency is what an attempt declares and needs before it goes live: `self.get` (local or remote), `self.registry`,
keyed pins (`at(key)` on another object's keyed member); `self.follow` is the attempt's own step, not a dependency.
@ -653,7 +654,7 @@ host.serve(Notes, ({ session }, principal, self) =>
| one failed with `invalid/cycle` (a dependent off the ring) or `forbidden` (an `authorize` that refused it: authority, M§3.6) | `broken` with that error: nothing heals it but `ref.retry()` or a reload |
| one `ended` (revoked, its host closed) | `broken` with that error: the request cannot come back |
**No timer runs while parked:** no dependency deadline, no backoff; the attempt's `readyTimeout` deadline is paused
**Nothing times out while parked:** no dependency deadline, no backoff (only `parked_long`'s diagnostic report, below); the attempt's `readyTimeout` deadline is paused
(as eager migrations pause it) and resumes with the budget left; the identity backoff stays for the object's own
`transient` failures only. **Wakes:** the parked attempt checks again at every change of a dependency (live,
loading, failed, a new cause compared by value, ended, and its first outcome, below), coalesced per object after the
@ -662,10 +663,10 @@ host.serve(Notes, ({ session }, principal, self) =>
check is the boundary:** once all are live (and the pinned keys loaded), the own steps run; a dependency that leaves
`live` afterwards changes nothing (R18 after live: the object goes live; its calls through that reference meet its
state). **Retry passes on:** `ref.retry()` (or a `retry` message) on an object parked `failed { waiting }` passes on
to its failed dependencies, once per retry operation: the operation (a uuidv7) travels in the `retry` message
(`operation`) with the instance ids of the objects it passed through (`through`, added hop by hop, at most 32), and
an object already in `through` drops it, so a ring across hosts stops after one lap however many retries run at
once; an object also remembers the last few ids it handled, so a diamond retries it once. A `broken` or
to its failed dependencies (audit A5, 2026-10-30): on one host one retry's walk keeps a visited set, so a diamond
retries a shared dependency once; across links the `retry` message carries a hop budget (`hops`, 16 from the first
host, one less per host, dropped at 0), so a waiting ring across hosts passes it round until the budget runs out
(starting no attempt) and a diamond across hosts may retry a shared dependency twice (harmless). A `broken` or
`transient` one starts an attempt, a parked one passes it on, a request waiting unbound in an index is left as it is
(an event wakes it), a refused hold over a link re-runs its start. A `retryOn` wake while parked does nothing. When
a parked object's last reference goes, it ends at once (it is not live: no `linger`) and releases its requests.
@ -688,10 +689,8 @@ host.serve(Notes, ({ session }, principal, self) =>
object is created, so a cycle entering an object already activating is not detected. Example: A on the server and B
on a worker start at the same moment, A declares B and B declares A; the holds carry `[A]` and `[B]`, neither finds
the other, and without a dependency deadline both stay parked. It is made visible: an attempt parked longer than its
`readyTimeout` on a remote attempt that is itself only `loading` (its hold on a link; the clock and `since` start
when the hold gets on a link and stop while the path has no link)
is reported once per parking episode as
`parked_long { object, waitingFor, since }`; nothing fails or retries. A cycle is a plugin bug; **the guide's four
`readyTimeout`, whatever its blocker (audit A11, 2026-10-30), is reported once per parking episode as
`parked_long { object, waitingFor, state, since }` (`waitingFor` the blocker's label, `state` `loading` or `failed`); nothing fails or retries. A cycle is a plugin bug; **the guide's four
rules** for plugin authors and agents (host README): (1) a factory declares only what the object needs, never a type
that needs it back; (2) back-references and occasional lookups go into methods and routines (`ctx.get`); (3) data
flows up through state and registries, not mutual `self.get`; (4) host-lifetime objects on the server never declare
@ -816,8 +815,7 @@ location are dropped.
last values readable while its hold is failed. Nothing outlives its object instance and nothing is handed between
objects.
- **References across a reload** (slice 7b §6): a reference is never ended by a code change. Its object swapped in
place or restarted: it follows the new instance (above; a hold over a link is re-subscribed or gets a fresh `bound`,
§22). Its version no longer accepted, or a member of it refused (`contract_mismatch`): it leaves the object and waits
place or restarted: it follows the new instance (above; a hold over a link gets a fresh `bound`, §22). Its version no longer accepted, or a member of it refused (`contract_mismatch`): it leaves the object and waits
unbound (`failed { waiting }`) with its last values; the object and its other holders stay. Its type removed: it
waits unbound `not_served` with its last values; a publication that serves the name again binds it to the next
object (a new object: its in-memory documents start over from `initial()`, persisted ones load again). On a removal
@ -845,13 +843,12 @@ location are dropped.
| --- | --- | --- |
| link drop | park until reconnect | readable (`loading`) |
| object loading or restarting | park until live | readable (`loading`) |
| rollout: the provider host installing a revision the hold does not fit (slice 7b) | park until installed, bounded | readable (`loading`) |
| unbound (not served, another version) or parked on a failed dependency | reject at once with the cause | `failed { waiting }` (slice 7a) |
| object `failed` | reject at once with the cause | `failed` with cause and `retryAt` |
- A failed reference stays alive: while held, Chord retries a `transient` failure per the backoff and wakes a
`waiting` one at the event it waits for; `ref.retry()` tries at once (a Retry button; over a link the `retry
{ hold }` message, which also re-runs a refused hold's start: the host address, binding; design audit 2026-10-27;
{ hold, hops }` message, which also re-runs a refused hold's start: the host address, binding; design audit 2026-10-27;
on a parked object it passes on to the failed dependencies, §6); success goes `loading`, then `live`. A provider
tells a pending hold's consumer each failure (`failed`, with the rest of its backoff window for a transient one),
each new cause of an object parked on a failed dependency, and the next attempt (`loading`).
@ -1405,8 +1402,8 @@ self.onLive(async (ctx) => { // set up, wait
have finished (bounded; a stuck call is reported and the new routines start anyway): a handover per object, so
routines need no names. Example it prevents: an old handler midway through `restart(W1)` and a new one seeing W1
still pending and restarting it again. The bound is `closeTimeout` (owner, 2026-10-24); calls do not wait for the
handover, only the new routines do (slice 7b). The old routine calls' transactions carry one token, sealed when
the new routines start, so a stuck old call past the handover is fenced on its own (`drain_timeout` reported).
handover, only the new routines do (slice 7b). A stuck old call past the handover is reported `drain_timeout` and
the whole old attempt is sealed then (audit A12: one seal token per attempt; its method calls had the same bound).
- **`on(source, fn)`:** any watchable (a document, a derive, a reference member, a registry value, `ref.state`). The
first call gets the current value (`[["r", value]]`, W2b), then one call per change; calls never overlap (S8);
changes made during a call arrive folded in the next (newest value, folded ops); a running call is not aborted by a
@ -1692,6 +1689,9 @@ const router = self.registry(Http); // a declared dependency; read rou
Tools, sections, hooks, wraps and tasks reach the harness as one Durable `Extension` per plugin contributed to
p3's core registry; `compose` builds Durable's registry snapshot (Durable applies selection and wraps, and pins a
snapshot per task phase). Skills and other lists work the same way.
- **A contribution may be another registry's value** (it is followed: its change recomposes). A ring of registries
reading each other is caught when one composes (audit A2): a registry read while it composes throws `invalid/cycle`,
the contribution that read it is left out and reported `contribution_failed`, the rest of the registry works.
- **Removing a contribution:** there is no remove call (`contribute` is declarative, in the factory). An entry leaves
the table when its contributor ends, or after a reload whose new code no longer contributes. A live contributor that
has nothing to offer for now sets its readable to the registry's empty value (an extension without tools, empty
@ -1844,16 +1844,15 @@ type ReloadReport = {
the documents, tables and jobs it declares with their versions, its contributions and requests, runs the checks an
attempt makes without effects (declared twice; persisted data outside a stable identity; contributions outside a
host-lifetime object; a document or job at an older version, or at its version with another schema), and hands out
inert handles. Staging is synchronous. **Installing** (model §1): while it stages, a request or arriving hold that
the running code does not serve waits `installing` (`loading`) and is evaluated again when the reload ends, by its
commit or its rejection.
inert handles. Staging is synchronous, so no request or arriving hold meets it half done (audit A1: model §1's
`installing` is cut).
- **The commit** (one batch): the served types become the staged ones (an unchanged plugin's entries stay as they
are); a removed plugin's registry provider goes (its readers wait `not_served`), a reloaded one's replaces the old
(`RegistryRecord.serve` replaces), the reloaded plugins' facet contributions are replaced by plugin (object
contributions follow their objects), every changed registry composes once; a removed type's presence list goes.
After the batch, each object takes its fate, host-lifetime objects of new types start (no barrier), and the
publication wakes the requests indexed under the names the reload serves (`UnboundIndex.wake(names)`, per name),
refused holds of those names re-run their start, and the requests that waited `installing` are evaluated again.
refused holds of those names re-run their start.
- **Fates** (decided in staging; objects of plugins not in the reload are untouched):
| The object | Fate |
@ -1883,7 +1882,7 @@ type ReloadReport = {
contributions are replaced in one batch; `self.writable` starts at its new initial value (decision 2); `retryOn`
sources are re-declared. **Routines hand over per object:** the old attempt's routines are aborted; the new ones
start once the old routine calls finished, bounded by `closeTimeout` (a stuck one is reported `drain_timeout` and
fenced: the old routine calls' transactions carry one token, sealed when the new routines start); calls never wait.
the whole old attempt is sealed then, audit A12: one seal token per attempt); calls never wait.
- **Natives are not kept across reloads by name:** natives have no line, transactions or schema, so old and new code
would interleave unchecked. In practice a restart costs little (the Durable harness is base code, terminal PTYs live
in their own process, model runtimes, sandboxes and the search index rebuild in milliseconds); reloading the plugin
@ -1910,13 +1909,13 @@ type ReloadReport = {
### Rollout across hosts
- **The rollout is the app's** (decision "Slice 7b's open points" (3)): it calls `host.reload` on each host in order,
server, then workers, then clients; Chord guarantees per host (staging, commit, `installing`, rollback). The app kit's
server, then workers, then clients; Chord guarantees per host (staging, commit, rollback). The app kit's
dev component (slice 7c) is such an app. An install never waits for activations (a reverse-order rollout would
deadlock).
- **Each host knows only its own install** (model §5): while it stages, holds that do not fit its running code wait
`installing` (`loading`), evaluated again at its end; a host not installing yet answers a mismatch `failed
{ waiting, version }` until its install commits and wakes it (a blip of at most one install). What a consumer on
another host sees follows §7: kept, re-subscribed or freshly bound while its version is accepted; `failed
- **Each host knows only its own install** (model §5): until its reload commits (while the app bundles, before
`host.reload`), a hold that does not fit its running code is answered `failed { waiting, version }` and the commit
wakes it (a blip of at most one install; staging itself is synchronous, audit A1). What a consumer on
another host sees follows §7: kept and freshly bound while its version is accepted; `failed
{ waiting }` (`version`, `not_served`) otherwise, bound again by the install that serves it; a consumer whose own
code moves to a new version releases its old request and makes a new one.
- **The app's loader keeps its own duties:** the desired code set, a digest over the facet bundles' content hashes (an
@ -1967,7 +1966,7 @@ validation, no clone of snapshots or ops.
| `hold` | to provider | `origin?`, `hold`, `type`, `version`, `fingerprint`, `params` (part), `address`, `principal?`, `incarnation?`, `chain?` (activation chain, Q5), `limit?` (decoded size limit, 4b) |
| `revoke` | parent to child (route link) | the revocation value (4b) |
| `release` | to provider | `origin?`, `hold` |
| `retry` | to provider | `origin?`, `hold`, `operation?` (the retry it belongs to, a uuidv7), `through?` (with `operation`: the instance ids of the objects it passed through, at most 32; an object in it drops the retry, so a ring stops after one lap); relays forward both (slice 7a fix rounds 1, 2) |
| `retry` | to provider | `origin?`, `hold`, `hops` (the retry's hop budget, 1-16: 16 from the first host, one less for each host it passes on from, none sent at 0; audit A5); relays forward it unchanged |
| `call` | to provider | `origin?`, `call`, `hold`, `method` (index), `args?` (part; absent: no arguments), `segments?` |
| `abort` | to provider | `origin?`, `call` (named `cancel` until 2026-10-27) |
| `ack` | to provider | `origin?`; one per applied `bound`/`state` message |
@ -2060,8 +2059,8 @@ validation, no clone of snapshots or ops.
and `state` messages already sent until their `ack` or the link's loss. A reload sends it when the object's type is
removed (`not_served`), its version is no longer accepted (`version`), or a swap parks on a failed dependency; a
restart or a parked swap still loading sends `loading`; both are followed by a fresh `bound` when the object is live
again (applied as a re-binding). A swap in place keeps the hold bound and sends one `[["r", value]]` per member whose
value changed (a fresh `bound` when the variant changed, or the hold's version became an older one).
again (applied as a re-binding). A swap in place keeps the hold bound and sends it a fresh `bound` once the new
instance is live (audit A9, 2026-10-30: one rebind path; a tab's watchers may get a frame for an unchanged member).
---

2
packages/chord/docs/next/slice-8.md
View file

@ -59,7 +59,7 @@ await host.root(async (ctx) => (await ctx.get(Notes, { session })).add(note, ctx
when the table is built, an older one without an adapter or a newer one is left out and reported (`HostError`
`contribution_refused { registry, plugin, version }`).
- **Contributions:** a plain value, or a Chord watchable (a `writable`, a derive, a document member; another
registry's value too, a cycle of registries reading each other refused when the second one serves or contributes), followed as a dependent of the table (its change recomposes), recognised by Chord's own mark; anything
registry's value too, a cycle of registries reading each other refused when the second one serves or contributes (superseded by audit A2: caught at the first composition by the runtime guard, reported `contribution_failed`)), followed as a dependent of the table (its change recomposes), recognised by Chord's own mark; anything
else is a plain value, whatever methods it has (a harness, a conversation handle or a runtime is contributed as a plain
value, not followed; build review 2026-10-27, replacing "a foreign watchable is refused"). The table's entry is `{ plugin, type?, version, value }` (`type` for
an object contributor; served types carry their plugin's name).

2
packages/chord/docs/next/slices.md
View file

@ -134,7 +134,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 18 | platform bindings as core objects | doc (8) |
| 19 | `host.references` (presence) | 4b |
| 20 | flows kept by their owner, the `steps` helper, the Questions service | doc (5); deferred decisions #9, #11 |
| 21 | `host.reload`: staging, atomic commit, pure swaps, native restarts, removal and re-serving, `installing` | 7b |
| 21 | `host.reload`: staging, atomic commit, pure swaps, native restarts, removal and re-serving | 7b |
| 21 | loading plugins (loader D), dev loop; the rollout across hosts is the app's (decision "Slice 7b's open points" (3)) | 7c |
| 22 | message format, header schema, unknown ids, one message per commit per link, consumers apply ops, plugin errors | 3a |
| 22 | relays, size limits per part, compression at the origin, adapted bindings get diffs | 4a (relays), 4b (limits, compression), 7b |

3
packages/chord/src/next/documents/address-source.ts
View file

@ -7,14 +7,11 @@ import type { Context } from "../../context/index.ts";
import type { Op, Prepared } from "../../delta/index.ts";
import { Source } from "../state/internal.ts";
import type { JsonObject } from "./definition.ts";
import type { Documents } from "./documents.ts";
const NO_OPS: readonly Op[] = [];
/** A source following one document address; `value` is the document's, or `initial()` while it is absent. */
export abstract class AddressSource extends Source {
/** The documents of the object whose document this is (an attempt asks whether a member is its own). */
abstract readonly owner: Documents;
/** The ops of the current batch's frame, and whether the document was retired in it. */
#ops = NO_OPS;
#retired = false;

2
packages/chord/src/next/documents/keyed-slot.ts
View file

@ -382,7 +382,6 @@ export class KeyedSlot implements DocumentSlot, PersistedSlot {
* document's value, `initial()` while it is absent; retirements arrive with the `retired` flag.
*/
export class KeySource extends AddressSource {
readonly owner: Documents;
readonly #slot: KeyedSlot;
readonly key: string;
/** Its pins: at 0 the slot forgets it. */
@ -393,7 +392,6 @@ export class KeySource extends AddressSource {
constructor(slot: KeyedSlot, key: string, initial: JsonObject, ready: boolean) {
super(slot.owner.batch);
this.owner = slot.owner;
this.#slot = slot;
this.key = key;
this.#initial = initial;

39
packages/chord/src/next/host/README.md
View file

@ -185,8 +185,7 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
way. A dependency failing with `invalid/cycle` makes its dependent broken with that error (it would wait on a ring
that never heals). The gap (owner, 2026-10-29: no probe): a chain is fixed when its object is created, so a ring
across hosts whose objects start at the same moment is not detected, and without a dependency deadline it stays
parked; an attempt parked longer than its `readyTimeout` on a remote attempt that is only `loading` (its hold on a
link: the clock and `since` run only while it is, so a late link is not reported at once) is reported once per
parked; an attempt parked longer than its `readyTimeout`, whatever its blocker (audit A11), is reported once per
parking episode (`parked_long`), nothing fails. A cycle is a plugin bug: see the authoring rules below.
- `onError(error: HostError)` gets everything that fails in the background, by `kind` (`failure` where the value is a
`FailureCause`, `error` otherwise; a position is `{ kind: "on" | "onLive" | "native", position }`, from 1):
@ -201,8 +200,9 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
`error`: a routine call threw, or its watch could not be taken; a source's failure calls never reach a routine), `job_failed` (`object`, `job`,
`id`, `error`: a run threw, not after an abort), `job_finish_failed` (the same: a `finish` threw; it is retried) and
`job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end),
`parked_long` (`object`, `waitingFor`: the remote blocker's label, `since`: when it parked; once per parking episode,
slice 7a), and
`parked_long` (`object`, `waitingFor`: the first blocker's label in declaration order, a reference's or `registry
<name>`, `state`: `loading` or `failed`, `since`: when it parked; once per parking episode, its pinned keys' loading
included, slice 7a, audit A11), and
the registries' (slice 8, each once per episode) `registry_conflict` (`registry`, `message`, `plugins`: a conflict
`compose` resolved), `registry_failed` (`registry`, `error`: `compose` or an older version's `value` adapter threw),
`contribution_failed` (`registry`, `plugin`, `type?`, `error`: a watchable contribution failed or ended, or its
@ -214,9 +214,10 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
empty params: `StoreSelf.contribute`; any other factory throws, R21) from `live` until it fails, ends or restarts,
entering before anyone sees it live and leaving at its seal. One value per registry each; a Chord watchable (a
source, a constant, a document handle, a reference member, a writable, a registry value) is followed through the
batch, anything else is a plain value whatever methods it has (registries carry natives); a registry value closing
a cycle of registries (through the tables and the factory's own earlier contributions) is refused `invalid/cycle`
at `contribute` (an edge lives as long as its contribution is in a table). Every served registry composes once at the end of `load`'s setups, before any object activates.
batch, anything else is a plain value whatever methods it has (registries carry natives); a ring of registries
reading each other is caught when one composes (audit A2): the registry read while it composes throws
`invalid/cycle`, the contribution that read it is left out and reported `contribution_failed`, the rest works. Every
served registry composes once at the end of `load`'s setups, before any object activates.
`self.registry(R)` is a declared dependency: without a usable value at its version (no provider:
`unavailable/undeclared`; a first `compose` or the version mapping that threw; a version the provider does not serve:
`contract_mismatch`) the attempt parks `failed { waiting }` naming the registry, and every change of whether the value
@ -413,13 +414,16 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
own document or derive. When the reference's target changes (a new instance after a reload, retry or restart; a new
object after removal and re-serving; another variant) it moves them there and sends one frame (a diff from the last
value, whole after a failure). While it points at nothing it keeps its last values: no call while loading, one
`failed` call while failed; a member the new variant lacks fails `invalid/member`. The keys a reference holds on a
keyed member (`at(key)`, `watch(key)`) are its own (`reference-keys.ts`): pinned again on each new live instance,
their last value kept until that pin loads; a dependent's factory pin loads before the dependent goes live. Older
consumers read through the instance's shared mapping for their version (`adapted.ts`).
`failed` call while failed; a member the new variant lacks fails `invalid/member`. A failed live instance's members
(its own and re-exposed ones) are followed until the commits it admitted published, then all hold (audit A10), as a
restart's drained instance's. The keys a reference holds on a keyed member (`at(key)`, `watch(key)`) are its own
(`reference-keys.ts`): they follow the instance they are pinned on as long as the object does (members' rule), are
pinned again on each new live instance, their last value kept until that pin loads; a dependent's factory pin loads
before the dependent goes live. Older consumers read through the instance's shared mapping for their version
(`adapted.ts`).
- `ref.retry()`: a failed object (broken too) gets an attempt now, its backoff window skipped; every reference to it
goes `loading`. One parked `failed { waiting }` passes it on (above). Ignored while an attempt runs otherwise, once
live and once released; over a link it is a `retry` message carrying its operation.
live and once released; over a link it is a `retry` message carrying its hop budget (`hops`).
Rules:
@ -447,10 +451,10 @@ Rules:
deadline paused); the factory never runs again; every change of a dependency (its state, a new cause by value, its
first outcome) makes it check again once after the batch that made it; once all are live it goes on to its own
steps, and a dependency leaving `live` after that changes nothing. `ref.retry()` on a parked `failed { waiting }`
object passes on to its failed dependencies once per retry operation (a uuidv7 that travels in the `retry`
message with the instance ids of the objects it passed through: one already in that list drops it, so a ring across
hosts stops after one lap whatever the number of concurrent retries, and the last few ids an object handled
shortcut a diamond: a `broken` or
object passes on to its failed dependencies (audit A5): on one host one retry's walk keeps a visited set (a diamond
retries a shared dependency once); across links the `retry` message carries a hop budget (`hops`: 16 from the first
host, one less per host, never sent at 0), so a waiting ring across hosts passes it round until the budget runs out,
starting no attempt, and a diamond across hosts may retry a shared dependency twice (harmless): a `broken` or
`transient` one starts an attempt, a parked one passes it on, a request waiting in an index is left alone, a
refused hold over a link re-runs its start); a `retryOn` wake does nothing while parked. A parked object whose last reference goes ends at once.
- **Authoring rules against cycles** (dependency model §7; a cycle is a plugin bug, and a ring across hosts started at
@ -536,7 +540,8 @@ their number; `published`, `index`/`unindex` (the unbound index), `closing`, `li
`ChordObject` (`state`, `retry`) and `Holder` (what holds an object: a binding, or a hold over a link: `moved()` when
the object's `state` changed, which a new holder also reads itself; `end(error)`), `Binding` (`moved`, `end`),
`CheckedParams`, `Caller` (principal and incarnation of a reference's maker or a hold's origin), `RefState` with `LOADING`, `LIVE`,
`RetryOperation` (one `ref.retry()` passing on), `Serving` (what a hold placed here binds to, or why it waits) and
`RetryWalk` and `retryWalk` (one retry's walk on this host, with its hop budget), `Serving` (what a hold placed here
binds to, or why it waits) and
`UnboundRequest` (what the unbound index holds). Also on `ChordHost` for `remote/`: `revoke` (a parent's `revoke` message), `compression`, `count(kind,
delta)` (the keep-alive counts), `cycle(chain, id, closing)` (the `invalid/cycle` error naming a chain by labels),
`unplaced` (the path of remote references failed for a part past the limits) and `pathOf(Type, params, principal)`

16
packages/chord/src/next/host/adapted.ts
View file

@ -65,11 +65,8 @@ export class AdaptedMembers {
readonly #instance: AdaptedInstance;
/** The members read so far: a mapping derive, or the instance's member itself (unchanged). */
readonly #members = new Map<string, DeriveSource>();
/**
* The mapping derives it made, with the instance's members each maps (an unchanged member is the instance's own: it
* never detaches that one).
*/
readonly #derives = new Map<Derive, readonly DeriveSource[]>();
/** The mapping derives it made (an unchanged member is the instance's own: it never detaches that one). */
readonly #derives = new Set<Derive>();
/** The keys held so far, by the instance's key document. */
#keys: Map<string, Map<DeriveSource, MappedKey>> | undefined;
@ -143,14 +140,9 @@ export class AdaptedMembers {
return this.#keyed(member);
}
/** The instance's members mapping derive `source` maps; `undefined` for a source it did not make. */
mapped(source: DeriveSource): readonly DeriveSource[] | undefined {
return source instanceof Derive ? this.#derives.get(source) : undefined;
}
/** Its instance ended, or the version's last reference left: its derives go (their watches end with `error`). */
detach(error: Error): void {
for (const derive of this.#derives.keys()) derive.detach(error);
for (const derive of this.#derives) derive.detach(error);
this.#derives.clear();
this.#members.clear();
for (const byDocument of this.#keys?.values() ?? []) {
@ -201,7 +193,7 @@ export class AdaptedMembers {
(values) => map(Object.fromEntries(names.map((name, index) => [name, values[index]]))),
batch,
);
this.#derives.set(derive, sources);
this.#derives.add(derive);
// The sources are the instance's members already.
derive.attach(
(source) => source as DeriveSource,

140
packages/chord/src/next/host/attempt.ts
View file

@ -14,7 +14,6 @@ import { ChordError, type FailureCause, type Principal } from "../data/index.ts"
import { asChordError, checkArgs, checkParams, invalid, openFailure, type Value } from "../data/internal.ts";
import type { HandleOf, JsonObject, StateDefinition, Tx } from "../documents/index.ts";
import {
AddressSource,
type AnyDefinition,
type KeyDeclarer,
type KeyedSlot,
@ -24,12 +23,7 @@ import {
import type { JobDefinition, JobHandle, JobImplementation } from "../jobs/index.ts";
import { type AnyJobImplementation, type CutReason, Job } from "../jobs/internal.ts";
import type { Contribution, Registry, RegistryValue } from "../registries/index.ts";
import {
type RegistryContribution,
type RegistryEdge,
RegistryValueSource,
type RegistryWaiter,
} from "../registries/internal.ts";
import type { RegistryContribution, RegistryWaiter } from "../registries/internal.ts";
import type { Writable } from "../state/index.ts";
import { Derive, type DeriveSource, WritableSource } from "../state/internal.ts";
import type { Columns } from "../store/columns.ts";
@ -39,7 +33,7 @@ import type { TableDefinition } from "../store/table.ts";
import { type Table, TableHandle } from "../store/table-handle.ts";
import { prepareTables } from "../store/table-migration.ts";
import { AdaptedMembers } from "./adapted.ts";
import { CallContext, type Caller, type Ctx, RoutineContext, withLinkedAbortSignal } from "./ctx.ts";
import { CallContext, type Caller, type Ctx, withLinkedAbortSignal } from "./ctx.ts";
import { Deadline } from "./deadline.ts";
import { checkDependencies, type Dependency, ParkedLong, parkedCause, waitingOn } from "./dependencies.ts";
import { type FollowAttach, FollowSource } from "./follow.ts";
@ -63,7 +57,7 @@ import {
} from "./native.ts";
import type { ChordObject, Served } from "./object.ts";
import type { Failed } from "./ref-state.ts";
import type { Binding, CheckedParams, Reference, RetryOperation } from "./reference.ts";
import type { Binding, CheckedParams, Reference, RetryWalk } from "./reference.ts";
import { type RetrySource, retrySource } from "./retry-on.ts";
import {
type OnHandler,
@ -160,13 +154,13 @@ export interface StoreSelf<P extends Principal = Principal> extends Self<P> {
contribute<C>(R: Registry<C, unknown>, value: Contribution<C>): void;
}
/** A key of a keyed member pinned on one instance, loaded: its source (mapped when older), and the release (once). */
export type PinnedKey = { readonly source: DeriveSource; readonly release: () => void };
/**
* A factory: synchronous and declarative, run once per attempt. A remote one also gets the principal in its object's
* identity and returns one variant, `{ variant, members }`. Methods get a `Ctx` for the app's principal type `P`.
*/
/** A key of a keyed member pinned on one instance, loaded: its source (mapped when older), and the release (once). */
export type PinnedKey = { readonly source: DeriveSource; readonly release: () => void };
export type Factory<C extends Contract, P extends Principal = Principal> = ShapeOf<C>["kind"] extends "local"
? (params: ParamsOf<C>, self: keyof ParamsOf<C> extends never ? StoreSelf<P> : Self<P>) => Implementation<C, Ctx<P>>
: (params: ParamsOf<C>, principal: P, self: Self<P>) => Implementation<C, Ctx<P>>;
@ -177,12 +171,8 @@ const NONE: readonly never[] = [];
/** Running work of one kind (calls, routine calls, job invocations), with the bookkeeping each runs once. */
type Work = Map<Promise<unknown>, (() => void) | undefined>;
/**
* Its routines once started: aborted when they stop (its seal, a reload's handover or restart), their running calls,
* and the token of their transactions, sealed with the attempt and when a swap's new routines start (a stuck old
* routine call is fenced on its own, slice 7b §5).
*/
type RoutineRun = { readonly abort: AbortController; readonly calls: Work; readonly token: SealToken };
/** Its routines once started: aborted when they stop (its seal, a reload's handover or restart), and their running calls. */
type RoutineRun = { readonly abort: AbortController; readonly calls: Work };
/**
* What a reload did to an attempt: `started` (a reload started it), `handedOver` (a swap: its jobs are the new
@ -208,8 +198,6 @@ function cleanUp(work: Work | undefined): void {
/** An attempt's registry reads, contributions and wake signals, and its writables (slice 8). */
type RegistryDeclarations = {
/** The registry values it declared (`self.registry`): dependencies, as its references are. */
readonly registries: RegistryValueSource[];
/** Releases its registry reads (with its references): a read keeps an older version's mapping. */
readonly reads: (() => void)[];
/** Its wake signals (`self.retryOn`), handed to its object right after the factory. */
@ -279,6 +267,12 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
/** References this attempt declared with `self.get` (params checked), bound during activation. */
readonly #references: { readonly binding: Binding; readonly contract: Contract; readonly params: CheckedParams }[] =
[];
/**
* Its dependencies (`self.get` references and `self.registry` values) in declaration order: the parked cause and
* `parked_long` name the first that blocks. Made by the first; let go after the dependency check (a live attempt
* keeps none).
*/
#dependencies: Dependency[] | undefined;
readonly #natives: NativeSlot[] = [];
/**
* Running calls' handlers, routine calls and job runs, each with its bookkeeping (call counts, `ctx.get`
@ -386,6 +380,8 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
const checked = checkParams(Type, params, this.#opening);
const binding = host.reference(Type, this.#object, this.#caller(), {});
this.#references.push({ binding, contract: Type, params: checked });
this.#dependencies ??= [];
this.#dependencies.push({ kind: "reference", binding });
// The binding was built from `Type`'s members.
return binding.reference as Reference<C>;
}
@ -393,7 +389,8 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
registry<C, V>(R: Registry<C, V>): RegistryValue<V> {
this.#assertDeclaring("registry");
const value = this.#object.host.registryRecord(R.name).value(R.version);
this.#declarations().registries.push(value);
this.#dependencies ??= [];
this.#dependencies.push({ kind: "registry", value });
// Its values are the composition as `R`'s version has it, `V`.
return value as RegistryValue<V>;
}
@ -419,7 +416,7 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
contribute<C>(R: Registry<C, unknown>, value: Contribution<C>): void {
this.#assertDeclaring("contribute");
const { served, label, host } = this.#object;
const { served, label } = this.#object;
// R21: the types allow it only to host-lifetime objects; plugin code is bundled without type checks.
if (served.kind !== "local" || !served.hostLifetime) {
throw new Error(`${label}: self.contribute is valid only in a local type with empty params`);
@ -428,9 +425,6 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
if (declared.some(({ registry }) => registry.name === R.name)) {
throw new Error(`${label}: registry ${R.name} is contributed to twice`);
}
// Refused at `contribute` (declaration policy), also a cycle within this factory's own declarations; the edges
// live with the contributions in the tables, from live on.
if (value instanceof RegistryValueSource) host.registryRecord(R.name).assertNoCycle(value, this.#pendingEdges());
declared.push({ registry: R as Registry<unknown, unknown>, value });
}
@ -454,10 +448,7 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
}
tx<R>(fn: (tx: Tx) => R | Promise<R>, context: Context): Promise<R> {
// A routine call's transactions carry its routines' token: a swap's handover fences the stuck ones on their own.
const token =
context instanceof RoutineContext && context.fence === this.#routineRun?.token ? context.fence : this.#token;
return this.#object.documents.tx(fn, token, context);
return this.#object.documents.tx(fn, this.#token, context);
}
follow<T>(attach: FollowAttach<T>): Watchable<T> {
@ -586,18 +577,17 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
else void previous.handOverRoutines().then(() => this.#startRoutines());
}
/** Starts its routines, unless it was sealed meanwhile; their calls' transactions carry its routines' token. */
/** Starts its routines, unless it was sealed meanwhile. */
#startRoutines(): void {
if (this.#abandoned !== undefined || this.#reloaded?.routinesStopped === true) return;
const { host, label } = this.#object;
const run: RoutineRun = { abort: new AbortController(), calls: new Map(), token: new SealToken() };
const run: RoutineRun = { abort: new AbortController(), calls: new Map() };
this.#routineRun = run;
this.#routines.start({
label,
host,
caller: this.#caller(),
context: withAbortSignal(run.abort.signal, this.#context),
fence: run.token,
admit: (running, cleanup) => this.#admit(run.calls, running, cleanup),
});
}
@ -627,16 +617,16 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
/**
* The new attempt's routines may start: once this attempt's routine calls finished, bounded by `closeTimeout` (a
* stuck one is reported `drain_timeout` and fenced: its transactions are refused from now on).
* stuck one is reported `drain_timeout`; the whole attempt is sealed by then: the swap's retirement drains its
* calls and routine calls on the same bound from the swap's start, then seals it, so a stuck call's later commit
* is refused `released`).
*/
async handOverRoutines(): Promise<void> {
const { host, label } = this.#object;
const run = this.#routineRun;
const calls = run?.calls ?? NO_WORK;
const calls = this.#routineRun?.calls ?? NO_WORK;
if ((await drained(calls.keys(), calls.size, host.timeouts.close)) === "timeout") {
host.report({ kind: "drain_timeout", object: label });
}
if (run !== undefined) this.#object.documents.fence(run.token);
}
/**
@ -794,24 +784,6 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
return this.#members!.keyed;
}
/**
* Whether `source` (a state member) changes only through this object's own commits: one of this attempt's derives,
* one of the object's documents, or an older version's mapping of members it owns. Another object's reference member
* or document is foreign.
*/
owns(source: DeriveSource): boolean {
if (source instanceof Derive) {
if (this.#derives.includes(source)) return true;
for (const adapted of this.#adapted?.values() ?? []) {
const mapped = adapted.mapped(source);
// An older version's mapping follows the admitted commits of the members it maps (Astra, G1).
if (mapped !== undefined) return mapped.every((member) => this.owns(member));
}
return false;
}
return source instanceof AddressSource && source.owner === this.#object.documents;
}
/**
* State member `member` as a consumer of `version` follows it: its own member at its version, else its shared mapping
* for that version (`AdaptedMembers`); `invalid/member` outside the variant, `contract_mismatch` for a version or
@ -904,7 +876,6 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
this.abandon({ kind: "ended", error });
const { documents } = this.#object;
documents.fence(this.#token);
if (this.#routineRun !== undefined) documents.fence(this.#routineRun.token);
for (const job of this.#jobs.values()) job.seal(error, runs);
}
@ -970,7 +941,7 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
/** The activation steps; each catches its own failures with their kind (the table in slice 2's spec). */
async #activate(deadline: Deadline): Promise<Activation> {
let implementation: unknown;
const dependencies: Dependency[] = [];
let dependencies: readonly Dependency[] = NONE;
try {
implementation = this.#declare();
// Right after the factory, before anything can fail: a failed attempt keeps its wake signals.
@ -986,18 +957,18 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
const { host } = this.#object;
for (const { binding, contract, params } of this.#references) {
host.bind(binding, contract, params, this.#opening);
dependencies.push({ kind: "reference", binding });
}
// Once all are bound: a dependency made here may have bound back to this object meanwhile.
for (const dependency of dependencies) {
const target = dependency.kind === "reference" ? dependency.binding.target : undefined;
for (const { binding } of this.#references) {
const { target } = binding;
if (target !== undefined) this.#object.assertNoCycle(target, this.#opening);
}
const declarations = this.#registryDeclarations;
for (const value of declarations?.registries ?? NONE) {
// Needed only up to the dependency check: a live attempt keeps none.
dependencies = this.#dependencies ?? NONE;
this.#dependencies = undefined;
for (const dependency of dependencies) {
// Read from here until its references are released: an older version's mapping is kept meanwhile.
declarations?.reads.push(value.read());
dependencies.push({ kind: "registry", value });
if (dependency.kind === "registry") this.#declarations().reads.push(dependency.value.read());
}
} catch (error) {
return this.#broken(error);
@ -1038,7 +1009,7 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
if (check.kind === "broken")
return this.#abandoned ?? { kind: "failed", cause: { kind: "broken", error: check.error } };
if (check.kind === "live") {
const pinned = await this.#pinKeys();
const pinned = await this.#pinKeys(long);
if (this.#abandoned !== undefined) return this.#abandoned;
// Changed while the keys loaded: checked again.
if (pinned === undefined && !this.#moved) {
@ -1047,11 +1018,10 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
return undefined;
}
if (pinned !== undefined) object.parked(this, pinned, true);
long.stop();
} else {
object.parked(this, check.shown, check.firstPass);
this.#settleCheck(parkedCause(check));
long.waitingOn(check.remoteLoading);
long.blockedBy(check.blocker);
}
if (!(await this.#nextChange())) return this.#abandoned!;
}
@ -1082,16 +1052,21 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
}
/**
* Loads the keys the factory pinned on its dependencies' keyed members (`at(key)`); `undefined` once all are, else
* what the parked attempt shows for the first that failed to load (`waiting` on its reference).
* Loads the keys the factory pinned on its dependencies' keyed members (`at(key)`), each reference's named to `long`
* as the blocker while they load (`failed` when they failed); `undefined` once all are, else what the parked attempt
* shows for the first that failed to load (`waiting` on its reference).
*/
async #pinKeys(): Promise<Failed | undefined> {
async #pinKeys(long: ParkedLong): Promise<Failed | undefined> {
for (const { binding } of this.#references) {
const { label } = binding;
try {
const pinning = binding.pinKeys(this.#opening);
if (pinning !== undefined) await pinning;
if (pinning === undefined) continue;
long.blockedBy({ label, state: "loading" });
await pinning;
} catch (error) {
return waitingOn(binding.label, asChordError(error, binding.label));
long.blockedBy({ label, state: "failed" });
return waitingOn(label, asChordError(error, label));
}
}
return undefined;
@ -1121,13 +1096,13 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
/**
* `ref.retry()` on its parked object (`failed { waiting }`): passed on to every failed dependency (a broken or
* transient one retries, a parked one passes it on, a request waiting in an index is left alone), as `operation`;
* then it checks again (a key that failed to load is loaded again).
* transient one retries, a parked one passes it on, a request waiting in an index is left alone), as a step of
* `walk`; then it checks again (a key that failed to load is loaded again).
*/
retryDependencies(operation: RetryOperation): void {
retryDependencies(walk: RetryWalk): void {
for (const dependency of this.#parked ?? NONE) {
if (dependency.kind === "reference" && dependency.binding.refState.kind === "failed") {
dependency.binding.retry(operation);
dependency.binding.retry(walk);
}
}
this.dependencyMoved();
@ -1182,9 +1157,8 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
#resolveContributions({ declared }: RegistryDeclarations): readonly RegistryContribution[] {
const { served, host } = this.#object;
const contributor = { plugin: served.plugin, type: served.contract.name };
const pending = this.#pendingEdges();
return declared.map(({ registry, value }) =>
contributionOf(host.registryRecord(registry.name), contributor, registry.version, value, pending),
contributionOf(host.registryRecord(registry.name), contributor, registry.version, value),
);
}
@ -1265,20 +1239,8 @@ export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
return this.#abandoned ?? { kind: "failed", cause };
}
/** The registry edges its declared contributions would add (another registry's value contributed). */
#pendingEdges(): RegistryEdge[] {
const edges: RegistryEdge[] = [];
const { host } = this.#object;
for (const { registry, value } of this.#registryDeclarations?.declared ?? NONE) {
const to = value instanceof RegistryValueSource ? value.record : undefined;
if (to !== undefined) edges.push({ from: host.registryRecord(registry.name), to });
}
return edges;
}
#declarations(): RegistryDeclarations {
this.#registryDeclarations ??= {
registries: [],
reads: [],
retrySources: [],
writables: [],

16
packages/chord/src/next/host/ctx.ts
View file

@ -1,10 +1,9 @@
// The `ctx` a method handler gets (§8): its call's context, who called, the calling host's incarnation, and
// call-scoped references; a routine call's ctx and its fence (slice 7b); and the abort signal of a call's context, linked to long-lived sources until it settles.
// call-scoped references; and the abort signal of a call's context, linked to long-lived sources until it settles.
import { type Context, type ContextKey, withAbortSignal, withoutAbortSignal } from "../../context/index.ts";
import type { Contract, ParamsOf } from "../contracts/index.ts";
import { ChordError, type Principal } from "../data/index.ts";
import type { SealToken } from "../documents/internal.ts";
import type { AnyHost, GetOptions } from "./host.ts";
import type { Binding, Reference } from "./reference.ts";
@ -93,19 +92,6 @@ export class CallContext implements Ctx {
}
}
/**
* The ctx of one routine call: a handler's ctx whose transactions carry its routines' token (`fence`), so a stuck
* routine call past a swap's handover is fenced on its own (slice 7b §5).
*/
export class RoutineContext extends CallContext {
readonly fence: SealToken;
constructor(context: Context, caller: Caller, host: AnyHost, fence: SealToken) {
super(context, caller, host);
this.fence = fence;
}
}
function settled(Type: Contract): ChordError {
return new ChordError({
code: "released",

94
packages/chord/src/next/host/dependencies.ts
View file

@ -1,8 +1,8 @@
// Parked attempts (dependency model §3, §7): the dependency check, what an attempt's declared dependencies (`self.get`
// references and `self.registry` values) say at one moment: all live, or what the parked attempt shows while one is not
// (`loading`, `failed { waiting }` naming the nearest blocker and the root cause, or `broken` for a cycle, authority or
// an ended dependency), whether this host's first pass can count it (no blocker here still without its first outcome, decision
// (a)), and the remote blocker that is only loading; and the `parked_long` report a long wait on that blocker makes.
// an ended dependency), whether this host's first pass can count it (no blocker here still without its first outcome,
// decision (a)), and its first blocker; and the `parked_long` report of a long parking episode.
import { ChordError, type FailureCause, isChordError } from "../data/index.ts";
import { asChordError, waitingFor } from "../data/internal.ts";
@ -16,12 +16,15 @@ export type Dependency =
| { readonly kind: "reference"; readonly binding: Binding }
| { readonly kind: "registry"; readonly value: RegistryValueSource };
/** A blocker as `parked_long` names it: a reference's label or `registry <name>`, and whether it loads or failed. */
export type Blocker = { readonly label: string; readonly state: "loading" | "failed" };
/**
* What the dependencies say. `live`: every one is live (hydrated: a reference's members have values). `parked`: what
* the attempt shows (`loading`, or `failed { waiting }` naming its first failed blocker), whether its first pass is
* over (`firstPass`: no blocker, loading or failed, is an object here still without its first outcome), and a remote
* blocker whose hold is on a link and only loading (its label), for `parked_long`. `broken`: one failed for a cycle
* (`invalid/cycle`), for authority (`forbidden`) or ended: it never comes back by itself.
* over (`firstPass`: no blocker, loading or failed, is an object here still without its first outcome), and its first
* blocker in declaration order (`parked_long` names it). `broken`: one failed for a cycle (`invalid/cycle`), for
* authority (`forbidden`) or ended: it never comes back by itself.
*/
export type Check =
| { readonly kind: "live" }
@ -29,7 +32,7 @@ export type Check =
readonly kind: "parked";
readonly shown: RefState;
readonly firstPass: boolean;
readonly remoteLoading: string | undefined;
readonly blocker: Blocker;
/** The first blocker in declaration order that is only loading (a reload's report names it). */
readonly loadingOn: string | undefined;
}
@ -41,14 +44,16 @@ const LIVE: Check = { kind: "live" };
export function checkDependencies(dependencies: readonly Dependency[]): Check {
let failed: Failed | undefined;
let firstPass = true;
let remoteLoading: string | undefined;
let blocker: Blocker | undefined;
let loadingOn: string | undefined;
for (const dependency of dependencies) {
if (dependency.kind === "registry") {
const { value } = dependency;
const start = value.start();
if (start.kind === "value" || start.last !== undefined) continue;
failed ??= waitingOn(`registry ${value.name}`, asChordError(start.error, `registry ${value.name}`));
const label = `registry ${value.name}`;
failed ??= waitingOn(label, asChordError(start.error, label));
blocker ??= { label, state: "failed" };
continue;
}
const { binding } = dependency;
@ -62,20 +67,19 @@ export function checkDependencies(dependencies: readonly Dependency[]): Check {
const { error } = state.cause;
if (healsNever(error)) return { kind: "broken", error };
failed ??= waitingOn(binding.label, error);
blocker ??= { label: binding.label, state: "failed" };
// Parked `waiting` on something here still in its first attempt: no first outcome yet (fix round 1).
if (binding.target?.blocksFirstPass === true) firstPass = false;
continue;
}
case "loading": {
case "loading":
loadingOn ??= binding.label;
const target = binding.target;
if (target?.blocksFirstPass === true) firstPass = false;
if (target?.heldOnLink === true) remoteLoading ??= target.label;
}
blocker ??= { label: binding.label, state: "loading" };
if (binding.target?.blocksFirstPass === true) firstPass = false;
}
}
if (failed === undefined && loadingOn === undefined) return LIVE;
return { kind: "parked", shown: failed ?? LOADING, firstPass, remoteLoading, loadingOn };
if (blocker === undefined) return LIVE;
return { kind: "parked", shown: failed ?? LOADING, firstPass, blocker, loadingOn };
}
/**
@ -104,52 +108,40 @@ export function waitingOn(label: string, error: ChordError): Failed {
}
/**
* The `parked_long` report of one parking episode (dependency model §7): once, when the attempt has waited
* `readyTimeout` on a remote blocker that is only loading with its hold on a link (a ring across hosts started at the
* same moment stays parked so; nothing fails or retries). The clock, and the report's `since`, start when such a
* blocker is there and stop when none is (a link that drops, a path still without one): only the time a remote
* attempt answered for the wait counts. Its one timer runs only meanwhile.
* The `parked_long` report of one parking episode (dependency model §7, audit A11): once, when the attempt has been
* parked longer than `readyTimeout`, whatever its blocker (a ring across hosts started at the same moment stays parked
* so, and so does a long wait; nothing fails or retries). The clock and the report's `since` start when the episode
* does (its pinned keys' loading included); the report names the blocker of the latest check. Its one timer runs until
* the report or the episode's end (`stop`).
*/
export class ParkedLong {
readonly #object: ChordObject;
/** When the attempt began to wait on a remote blocker that is only loading on a link, while it does. */
#since: number | undefined;
#timer: ReturnType<typeof setTimeout> | undefined;
/** The remote blocker it would name now. */
#blocker: string | undefined;
#reported = false;
readonly #since = Date.now();
readonly #timer: ReturnType<typeof setTimeout>;
/** The blocker the latest check found. */
#blocker: Blocker | undefined;
constructor(object: ChordObject) {
this.#object = object;
this.#timer = setTimeout(() => {
const blocker = this.#blocker;
if (blocker === undefined) return;
const { label, state } = blocker;
object.host.report({
kind: "parked_long",
object: object.label,
waitingFor: label,
state,
since: this.#since,
});
}, object.host.timeouts.ready);
}
/** What the check found: `blocker`, a remote attempt that is only loading on a link, or none. */
waitingOn(blocker: string | undefined): void {
/** What the latest check found blocking. */
blockedBy(blocker: Blocker): void {
this.#blocker = blocker;
if (blocker === undefined) {
this.stop();
return;
}
if (this.#reported || this.#timer !== undefined) return;
this.#since ??= Date.now();
const due = this.#since + this.#object.host.timeouts.ready - Date.now();
this.#timer = setTimeout(() => this.#report(), Math.max(0, due));
}
/** The episode ended, or no remote blocker is loading on a link now: the clock and its timer stop. */
/** The episode ended: no report from now on. */
stop(): void {
clearTimeout(this.#timer);
this.#timer = undefined;
this.#since = undefined;
}
#report(): void {
this.#timer = undefined;
const waitingFor = this.#blocker;
const since = this.#since;
if (this.#reported || waitingFor === undefined || since === undefined) return;
this.#reported = true;
const object = this.#object;
object.host.report({ kind: "parked_long", object: object.label, waitingFor, since });
}
}

126
packages/chord/src/next/host/host.ts
View file

@ -30,12 +30,7 @@ import { binaryCodec, type Codec, type Compression, type Link, type PartLimits }
import { linkLimits } from "../link/internal.ts";
import type { Registry, RegistryValue } from "../registries/index.ts";
import type { RegistryValueSource } from "../registries/internal.ts";
import {
type RegistryContribution,
type RegistryEdge,
type RegistryError,
RegistryRecord,
} from "../registries/internal.ts";
import { type RegistryContribution, type RegistryError, RegistryRecord } from "../registries/internal.ts";
import {
type HoldRequest,
HostLink,
@ -158,10 +153,18 @@ export type HostError =
/** A blocked run was dropped without `finish`: aborted, or its object ended. */
| { readonly kind: "job_dropped"; readonly object: string; readonly job: string; readonly id: string }
/**
* An attempt parked `readyTimeout` (since `since`, `Date.now()` time) while `waitingFor`, a remote attempt, is only
* loading: once per parking episode, a diagnostic of a ring across hosts started at the same moment (nothing fails).
* An attempt parked longer than `readyTimeout` (since `since`, `Date.now()` time), whatever its blocker: once per
* parking episode, naming the first blocker in declaration order (`waitingFor`, a reference's label or `registry
* <name>`) and its `state`; a diagnostic of a ring across hosts started at the same moment or a long wait (nothing
* fails).
*/
| { readonly kind: "parked_long"; readonly object: string; readonly waitingFor: string; readonly since: number };
| {
readonly kind: "parked_long";
readonly object: string;
readonly waitingFor: string;
readonly state: "loading" | "failed";
readonly since: number;
};
/**
* What `accept` and `connect` take: the principal of the host at the other end, as this host stamps it; `trusted` skips
@ -321,9 +324,7 @@ export type ReloadedObject = { readonly object: string } & (
*/
export type Serving =
| { readonly kind: "object"; readonly object: ChordObject }
| { readonly kind: "waiting"; readonly cause: FailureCause }
/** The host is installing a reload and the hold does not fit its running code: `loading` until the reload ends. */
| { readonly kind: "installing" };
| { readonly kind: "waiting"; readonly cause: FailureCause };
/** A published plugin: its facet (an unchanged reload passes the same object), its registries and contributions. */
type LoadedPlugin<P extends Principal> = {
@ -359,12 +360,6 @@ export function createHost<P extends Principal = Principal, R extends JsonValue
return new ChordHost<P, R>(options);
}
/** No registry edges pending (a facet's contributions). */
const NO_EDGES: readonly RegistryEdge[] = [];
/** A hold for a type or version the running code does not serve, while a reload installs. */
const INSTALLING: Serving = { kind: "installing" };
/** No activation chain (Q5). */
const NO_CHAIN: readonly string[] = [];
@ -395,11 +390,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
readonly #plugins = new Map<string, LoadedPlugin<P>>();
/** The reloads, one after another (slice 7b §1): the last one's end, which the next one waits for. */
#reloading: Promise<unknown> = Promise.resolve();
/**
* While a reload stages (slice 7b §2, model §1): requests that do not fit the running code wait here `installing`
* (`loading`) by name, re-evaluated when the reload ends, by its commit or its rejection; `undefined` otherwise.
*/
#installing: Map<UnboundRequest, string> | undefined;
/** By identity, in the order they went live (the others where they were created). */
readonly #objects = new Map<string, ChordObject>();
/** Teardowns still running, by identity: the next instance of an identity waits for its previous ones. */
@ -499,10 +489,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
if (context.abortSignal?.aborted) throw reloadAborted();
const plan = this.#stageReload(facets, context);
// Aborted while it staged (a setup may abort it synchronously): rejected as a staging error, nothing changed.
if (context.abortSignal?.aborted) {
this.#installed();
throw reloadAborted();
}
if (context.abortSignal?.aborted) throw reloadAborted();
const started = this.#commit(plan, context);
const objects = await this.#report(started, context);
const { added, changed, removed } = plan;
@ -510,40 +497,33 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
}
/**
* Staging (slice 7b §2): the host is installing meanwhile; every setup runs into a staged table (`stagePlugins`),
* every object of a changed or removed plugin whose type is still served has its new factory recorded, which
* decides its fate. A throw rejects the reload with nothing changed (the requests that waited `installing` are
* evaluated again against the running code).
* Staging (slice 7b §2), synchronous (no request or arriving hold meets it half done): every setup runs into a
* staged table (`stagePlugins`), every object of a changed or removed plugin whose type is still served has its new
* factory recorded, which decides its fate. A throw rejects the reload with nothing changed.
*/
#stageReload(facets: readonly PluginFacet<P>[], context: Context): ReloadPlan<P> {
this.#installing = new Map();
try {
const staged = stagePlugins(facets, this.#stagingHost());
const byName = facetsByName(facets);
const added: string[] = [];
const changed: string[] = [];
const removed = [...this.#plugins.keys()].filter((name) => !byName.has(name));
for (const [name, facet] of byName) {
const loaded = this.#plugins.get(name);
if (loaded === undefined) added.push(name);
else if (loaded.facet !== facet) changed.push(name);
}
const reloaded = new Set([...changed, ...removed]);
const fates: PlannedFate[] = [];
for (const object of this.#objects.values()) {
if (!reloaded.has(object.served.plugin)) continue;
const served = staged.types.get(object.served.contract.name);
if (served === undefined || served.kind !== object.served.kind) {
fates.push({ kind: "end", object });
continue;
}
fates.push({ kind: object.reloadFate(recordFactory(object, served, context)), object, served });
}
return { staged, facets: byName, added, changed, removed, fates };
} catch (error) {
this.#installed();
throw error;
const staged = stagePlugins(facets, this.#stagingHost());
const byName = facetsByName(facets);
const added: string[] = [];
const changed: string[] = [];
const removed = [...this.#plugins.keys()].filter((name) => !byName.has(name));
for (const [name, facet] of byName) {
const loaded = this.#plugins.get(name);
if (loaded === undefined) added.push(name);
else if (loaded.facet !== facet) changed.push(name);
}
const reloaded = new Set([...changed, ...removed]);
const fates: PlannedFate[] = [];
for (const object of this.#objects.values()) {
if (!reloaded.has(object.served.plugin)) continue;
const served = staged.types.get(object.served.contract.name);
if (served === undefined || served.kind !== object.served.kind) {
fates.push({ kind: "end", object });
continue;
}
fates.push({ kind: object.reloadFate(recordFactory(object, served, context)), object, served });
}
return { staged, facets: byName, added, changed, removed, fates };
}
/**
@ -551,7 +531,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
* they were), the removed plugins' registry providers go and the reloaded ones' are replaced, their facet
* contributions replaced by plugin, every changed registry composed once; a removed type's presence list goes.
* Then each object takes its fate, host-lifetime objects of new types start, and the publication wakes the
* requests waiting for the names the reload serves, and those that waited `installing`.
* requests waiting for the names the reload serves.
*/
#commit(plan: ReloadPlan<P>, context: Context): readonly StartedFate[] {
const { staged, added, changed, removed } = plan;
@ -585,7 +565,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
if (type.kind === "local" && type.hostLifetime) this.#object(type, EMPTY_PARAMS, undefined, context);
}
this.#unbound.wake(names);
this.#installed();
return started;
}
@ -626,14 +605,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
}
}
/** The reload ends (committed or rejected): requests that waited `installing` are evaluated again. */
#installed(): void {
const installing = this.#installing;
this.#installing = undefined;
// Those the commit's publication woke already are not woken again.
for (const [request, name] of installing ?? []) if (this.#unbound.delete(name, request)) request.wake();
}
/** This host as staging sees it: whom it acts as, its address and its presence lists. */
#stagingHost(): StagingHost {
// Setups get presence lists of this host's `P`; staging passes them on as they are.
@ -803,15 +774,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
this.#unbound.add(name, request);
}
/**
* `request` (a hold) waits `installing` under `name` (the type it asks for) until the running reload ends, by its
* commit or its rejection (model §1, §5).
*/
installing(name: string, request: UnboundRequest): void {
this.#installing?.set(request, name);
this.#unbound.add(name, request);
}
/**
* `binding`, whose object left it (its type removed, its version no longer bound: slice 7b §6), waits unbound with
* `cause` under its type's name; a publication places it again (`params` checked as it was, under `context`).
@ -916,10 +878,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
unboundCause(served, Type.kind, Type.name, Type.version, Type.fingerprint) ??
this.#misplaced(Type, served!, params.params);
if (unbound !== undefined) {
const place = (): void => this.#place(binding, Type, params, context);
// Installing: it waits `loading` for the reload's end, which may serve it.
if (this.#installing !== undefined) this.#installing.set(binding, Type.name);
this.#wait(binding, Type.name, this.#installing === undefined ? unbound : undefined, place);
this.#wait(binding, Type.name, unbound, () => this.#place(binding, Type, params, context));
return undefined;
}
// Checked above: served here at the caller's version.
@ -944,8 +903,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
const { type, version, fingerprint, chain = NO_CHAIN } = request;
const served = this.#types.get(type);
const unbound = unboundCause(served, "remote", type, version, fingerprint);
if (unbound !== undefined)
return this.#installing === undefined ? { kind: "waiting", cause: unbound } : INSTALLING;
if (unbound !== undefined) return { kind: "waiting", cause: unbound };
// Checked above: a remote type served here.
const remote = served as Extract<Served, { readonly kind: "remote" }>;
const checked = checkParams(remote.contract, params, context);
@ -1239,13 +1197,11 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
registries.push(registry);
}
const contributions = (staged.contributions.get(name) ?? []).map(({ registry, value }) =>
// A facet contributes no registry's value (it reads none): no edges of its own.
contributionOf(
this.registryRecord(registry.name),
{ plugin: name, type: undefined },
registry.version,
value,
NO_EDGES,
),
);
for (const contribution of contributions) contribution.record.add(contribution);

4
packages/chord/src/next/host/internal.ts
View file

@ -12,8 +12,8 @@ export {
type Binding,
type CheckedParams,
type Followed,
type RetryOperation,
retryOperation,
type RetryWalk,
retryWalk,
type Target,
} from "./reference.ts";
export type { UnboundRequest } from "./unbound.ts";

9
packages/chord/src/next/host/members.ts
View file

@ -19,9 +19,7 @@ import {
type ContributionContent,
type Contributor,
RegistryContribution,
type RegistryEdge,
type RegistryRecord,
RegistryValueSource,
} from "../registries/internal.ts";
import { type Derive, type DeriveInput, type DeriveSource, NativeSource } from "../state/internal.ts";
import { type Binding, resolveKeyed, resolveWatchable } from "./reference.ts";
@ -131,18 +129,15 @@ export function attachDerives(
/**
* A contribution of `contributor` at `version` to `record`, as Chord follows it: a Chord watchable (a source, a
* constant, a document handle, a reference member, a native source) is followed, anything else is a plain value,
* whatever methods it has (registries carry natives). Refused (`invalid/cycle`) when it is a registry's value that
* closes a cycle of registries, through the tables and the contributor's `pending` edges.
* constant, a document handle, a reference member, a native source: another registry's value too) is followed,
* anything else is a plain value, whatever methods it has (registries carry natives).
*/
export function contributionOf(
record: RegistryRecord,
contributor: Contributor,
version: number,
value: unknown,
pending: readonly RegistryEdge[],
): RegistryContribution {
if (value instanceof RegistryValueSource) record.assertNoCycle(value, pending);
return new RegistryContribution(record, contributor, version, contentOf(value));
}

86
packages/chord/src/next/host/object.ts
View file

@ -28,7 +28,7 @@ import type { Dependency } from "./dependencies.ts";
import type { AnyHost } from "./host.ts";
import type { CloseFailed, NativeCloseReason } from "./native.ts";
import { type Ended, type Failed, LIVE, LOADING, type RefState } from "./ref-state.ts";
import { type CheckedParams, type Followed, type RetryOperation, retryOperation, type Target } from "./reference.ts";
import { type CheckedParams, type Followed, type RetryWalk, retryWalk, type Target } from "./reference.ts";
import { type RetrySource, RetryWakes } from "./retry-on.ts";
import { migratesTables, type Recorded } from "./staging.ts";
import { uuidv7 } from "./uuid.ts";
@ -150,9 +150,6 @@ const CLOSE: NativeCloseReason = { kind: "close" };
const RELOAD: NativeCloseReason = { kind: "reload" };
/** How many retry ids an object remembers: a diamond's shortcut only (a ring is stopped by a retry's `through`). */
const RETRIES_KEPT = 8;
/** No dependencies waited for: the shared empty list. */
const NO_DEPENDENCIES: readonly Dependency[] = [];
@ -202,11 +199,6 @@ export class ChordObject implements Target {
*/
#firstOutcome: PromiseWithResolvers<void> | undefined;
#hasFirstOutcome = false;
/**
* The ids of the last retry operations it handled (at most `RETRIES_KEPT`, the newest last; made by the first): a
* retry reaching it twice through a diamond is handled once.
*/
#retried: string[] | undefined;
/** The error of the last failed attempt, or the last cause a parked attempt showed, until one goes live. */
#lastFailure: ChordError | undefined;
/** Ends the object when its type's `linger` passed after its last reference; cleared by a new one. */
@ -220,11 +212,11 @@ export class ChordObject implements Target {
*/
#exposed: Attempt | undefined;
/**
* Whether references still follow `#exposed`, which may still change: `all` while it is live, drains (a restart) or
* retires (a swap's old instance until its successor is live or it retired); `own` after its live failure until the
* commits admitted before it published (its own members only, 6a's freeze); `none` once it stopped changing.
* Whether references still follow `#exposed`, which may still change: while it is live, drains (a restart), retires
* (a swap's old instance until its successor is live or it retired), or failed live until the commits admitted
* before it published (every member, its own and re-exposed ones: audit A10); not once it stopped changing.
*/
#follows: "all" | "own" | "none" = "none";
#following = false;
/** The one timer storing its jobs' held state changes by their earliest deadline (D8), while one is held. */
#flush: { readonly deadline: number; readonly timer: ReturnType<typeof setTimeout> } | undefined;
/** Its wake signals (`retryOn`), the last attempt's declaration; made by the first. */
@ -289,11 +281,6 @@ export class ChordObject implements Target {
return !this.#hasFirstOutcome;
}
/** An object here: it is not held on a link. */
get heldOnLink(): false {
return false;
}
/**
* The error of its last failure while none went live since: a failed attempt's, or the cause its parked attempt
* showed last (kept when the dependency falls back to `loading`, so a getter's deadline names it).
@ -349,21 +336,16 @@ export class ChordObject implements Target {
}
/**
* `ref.retry()` and a `retry` message, as `operation` (handled once): an attempt now when failed (a broken one too)
* or waiting, its backoff window skipped; passed on to the failed dependencies while parked `failed { waiting }`,
* with this object added to the objects it passed through (at most `CHAIN`: a retry stops past 32 hops); ignored
* otherwise (an attempt runs, or it is live), and when it passed through this object already (a ring) or was handled
* here already (a diamond).
* `ref.retry()` and a `retry` message, as a step of `walk` (handled once per walk: a diamond reaches it twice): an
* attempt now when failed (a broken one too) or waiting, its backoff window skipped; passed on to the failed
* dependencies while parked `failed { waiting }`; ignored otherwise (an attempt runs, or it is live).
*/
retry(operation: RetryOperation = retryOperation()): void {
const instance = this.#instance;
if (instance !== undefined && operation.through.includes(instance)) return;
if (this.#handled(operation.id)) return;
retry(walk: RetryWalk = retryWalk()): void {
if (walk.visited.has(this)) return;
walk.visited.add(this);
const state = this.#state;
if (state.kind === "activating") {
const { through } = operation;
if (state.shown.kind !== "failed" || instance === undefined || through.length >= CHAIN) return;
state.attempt.retryDependencies({ id: operation.id, through: [...through, instance] });
if (state.shown.kind === "failed") state.attempt.retryDependencies(walk);
return;
}
if (state.kind !== "waiting" && state.kind !== "failed") return;
@ -394,7 +376,7 @@ export class ChordObject implements Target {
/**
* Whether a reload's pure swap runs in place: the new attempt neither parked nor failed yet. Holds stay bound
* meanwhile (their calls wait at the gate) and re-subscribe once it is live.
* meanwhile (their calls wait at the gate) and get a fresh `bound` once it is live.
*/
get swapsInPlace(): boolean {
const state = this.#state;
@ -414,8 +396,8 @@ export class ChordObject implements Target {
/**
* What a reference of `version` follows for state member `member` now (decision "References are the only
* indirection"): the shown instance's member (its mapping at an older version) while that instance is live, while a
* swap brings its successor in place, and for its own members after a live failure until its admitted commits
* published; else nothing: `failed` with the cause's error, or `loading`, with that member to take a last value from.
* swap brings its successor in place, and after a live failure until its admitted commits published; else nothing:
* `failed` with the cause's error, or `loading`, with that member to take a last value from.
*/
followed(member: string, version: number): Followed {
const exposed = this.#exposed;
@ -430,11 +412,7 @@ export class ChordObject implements Target {
if (failure.code !== "contract_mismatch") lacking = failure;
}
}
const follows = this.#follows;
if (
exposed !== undefined &&
(follows === "all" || (follows === "own" && shown !== undefined && exposed.owns(shown)))
) {
if (exposed !== undefined && this.#following) {
if (shown !== undefined) return { kind: "live", source: shown, instance: exposed };
if (lacking !== undefined) return { kind: "lacking", error: lacking };
return { kind: "loading", last: undefined, instance: undefined };
@ -455,7 +433,7 @@ export class ChordObject implements Target {
* attached to); `undefined` once it stopped changing.
*/
get attachable(): Attempt | undefined {
return this.#follows === "none" ? undefined : this.#exposed;
return this.#following ? this.#exposed : undefined;
}
/** The live instance, which references pin their keys on (a new one: they pin again); `undefined` when not live. */
@ -582,10 +560,9 @@ export class ChordObject implements Target {
this.#lastFailure = error;
this.#withdraw(attempt);
attempt.seal(error);
// Members following another object's sources stop now: no commit of this object changes them.
this.#follows = "own";
// The commits admitted before the seal adopt and publish once (frames included); then the other members stop
// too. A retry's factory and hydration wait for both (its ending), so no old commit lands in a new attempt.
// Its references follow every member on until the commits admitted before the seal adopted and published once
// (frames included), then stop. A retry's factory and hydration wait for both (its ending), so no old commit
// lands in a new attempt.
const frozen = this.documents.settled().then(() => this.#stopFollowing(attempt));
const closing = Promise.all([attempt.close(END), frozen]).then(([failures]) => {
attempt.detach();
@ -921,7 +898,7 @@ export class ChordObject implements Target {
const unbound = this.#unbindUnfit(attempt);
this.#state = { kind: "live", attempt };
this.#exposed = attempt;
this.#follows = "all";
this.#following = true;
const { contributions } = attempt;
// The old attempt's contributions leave in the batch the new ones enter (slice 7b §5), so no reader sees a gap.
const leaving = swap !== undefined && !swap.parked ? swap.old.contributions : NO_CONTRIBUTIONS;
@ -934,7 +911,7 @@ export class ChordObject implements Target {
}
this.#lastFailure = undefined;
this.host.wentLive(this);
// Its references move to the new instance (one frame each); holds re-subscribe.
// Its references move to the new instance (one frame each); holds get a fresh `bound`.
this.#notify();
this.#settleFirstOutcome();
swap?.done.resolve();
@ -1091,8 +1068,8 @@ export class ChordObject implements Target {
* object lets it go (a reference made later never attached to it: it takes nothing from it).
*/
#stopFollowing(attempt: Attempt): void {
if (this.#exposed !== attempt || this.#follows === "none") return;
this.#follows = "none";
if (this.#exposed !== attempt || !this.#following) return;
this.#following = false;
this.#notify();
// Its references took their values (`#notify` retargets them at once): a failed object keeps no ended instance.
this.#exposed = undefined;
@ -1137,7 +1114,7 @@ export class ChordObject implements Target {
// An object of the start barrier's cohort that ends before its first outcome settles it as ended.
this.#settleFirstOutcome();
// Sealed: its instance stops changing; its references take their last values from it.
this.#follows = "none";
this.#following = false;
const holders = [...this.#holders];
this.#holders.clear();
this.host.unreferenced(this, holders, this.#context);
@ -1241,19 +1218,6 @@ export class ChordObject implements Target {
throw new ChordError({ code: "not_ready", message: `${this.label}: not live yet` });
}
/**
* Whether the retry operation `id` reached it before (among the last few: the diamond's shortcut; a ring is stopped
* by `through`); records it otherwise.
*/
#handled(id: string): boolean {
this.#retried ??= [];
const retried = this.#retried;
if (retried.includes(id)) return true;
if (retried.length === RETRIES_KEPT) retried.shift();
retried.push(id);
return false;
}
/** The settlement of its first outcome (once): dependents parked on it check their first pass again. */
#settleFirstOutcome(): void {
if (this.#hasFirstOutcome) return;

13
packages/chord/src/next/host/reference-keys.ts
View file

@ -1,8 +1,10 @@
// The keys a reference holds (slice 6a §12; decision "References are the only indirection" (4)): its key watches and
// its factory's pins (`at(key)`) of its object's keyed members, one source per member and key shared by them, pinned on
// the object's live instance (loaded, mapped to the reference's version). When the reference moves to a new instance
// or a new object it pins the key there again, keeping the last value until that pin loads, then one frame; while it
// points at nothing the key keeps its last value (one `failed` call when its target failed, none while it loads).
// the object's live instance (loaded, mapped to the reference's version). While the object still follows the instance a
// key is pinned on (a swap's old instance, a restart's drain, a live failure's admitted commits) the key follows it on,
// as members do. When the reference moves to a new instance or a new object it pins the key there again, keeping the
// last value until that pin loads, then one frame; while it points at nothing the key keeps its last value (one
// `failed` call when its target failed, none while it loads).
import { awaitWithContext, BACKGROUND_CONTEXT, type Context } from "../../context/index.ts";
import { ChordError } from "../data/index.ts";
@ -70,11 +72,13 @@ export class ReferenceKeys {
/**
* Its reference's target changed, inside a batch: a key pins again on a new live instance (last value meanwhile);
* while the target is not live it keeps its last value (one `failed` call for a failure) and its pin goes.
* while the target is not live it follows the instance it is pinned on as long as the object does (a commit admitted
* there still shows), then keeps its last value (one `failed` call for a failure) and its pin goes.
*/
follow(context: Context): void {
const target = this.#holder.target();
const instance = target?.instance;
const attachable = target?.attachable;
for (const byKey of this.#entries.values()) {
for (const entry of byKey.values()) {
if (instance !== undefined) {
@ -82,6 +86,7 @@ export class ReferenceKeys {
if (entry.instance !== instance) this.#pin(entry).catch(() => {});
continue;
}
if (attachable !== undefined && entry.instance === attachable) continue;
const state = target?.state;
if (state?.kind === "failed") entry.source.fail(state.cause.error, context);
else entry.source.hold();

38
packages/chord/src/next/host/reference.ts
View file

@ -18,6 +18,7 @@ import {
} from "../contracts/internal.ts";
import { ChordError, type FailureCause, type Principal } from "../data/index.ts";
import { Keyed, type KeyedSlot, type SealToken, Singleton } from "../documents/internal.ts";
import { RETRY_HOPS } from "../link/internal.ts";
import { type Batch, Constant, type DeriveSource, Source } from "../state/internal.ts";
import type { PinnedKey } from "./attempt.ts";
import { type Caller, withLinkedAbortSignal } from "./ctx.ts";
@ -28,7 +29,6 @@ import { type Ended, type Failed, LOADING, ReferenceState, type RefState } from
import { ReferenceKeys } from "./reference-keys.ts";
import { ReferenceSource } from "./reference-source.ts";
import type { UnboundRequest } from "./unbound.ts";
import { uuidv7 } from "./uuid.ts";
/**
* A reference: a local contract's members, or for a remote contract the members of the variant its object chose,
@ -81,11 +81,6 @@ export interface Target {
readonly lastFailure: ChordError | undefined;
/** Its instance id while it activates on this host (Q5); `undefined` otherwise and over a link. */
readonly activationId: string | undefined;
/**
* Whether it is held on a link now: a remote attempt answers for it there (`parked_long` counts it while it is
* loading); not while its path has no link, and never for an object here.
*/
readonly heldOnLink: boolean;
/**
* Whether it holds up this host's start barrier: an object here without its first outcome (decision (a)); a remote
* object never does.
@ -120,10 +115,10 @@ export interface Target {
*/
pinKey(member: string, key: string, version: number, context: Context): Promise<PinnedKey>;
/**
* `ref.retry()` (`operation` once per retry, deduping diamonds): an attempt now if failed; an object parked on failed
* dependencies passes it on; ignored otherwise.
* `ref.retry()`, as one step of `walk`: an attempt now if failed; an object parked on failed dependencies passes it
* on; ignored otherwise.
*/
retry(operation: RetryOperation): void;
retry(walk: RetryWalk): void;
/**
* The chain of activation waits from this object to `target`, both included (Q5); `undefined` when there is none.
* `visited` keeps a search over shared dependencies linear in the waits.
@ -152,16 +147,15 @@ export type Followed =
};
/**
* One `ref.retry()` passing on through parked objects, across links too (the `retry` message's `operation` and
* `through`): its id (a uuidv7) and the instance ids of the objects it passed through, added hop by hop (at most
* `CHAIN`). An object in `through` drops it, so a ring of hosts stops after one lap however many retries run at once;
* an object remembers the last few ids it handled too, so a diamond retries it once.
* One retry's walk through parked objects on this host (dependency model §3.7, audit A5): the objects it reached (one
* reached again, through a diamond, ignores it) and the hop budget a `retry` message to another host carries (none is
* sent at 0, so a waiting ring of hosts stops).
*/
export type RetryOperation = { readonly id: string; readonly through: string[] };
export type RetryWalk = { readonly visited: Set<ChordObject>; readonly hops: number };
/** A new retry operation, through nothing yet (`ref.retry()`, a `retry` message that names none). */
export function retryOperation(): RetryOperation {
return { id: uuidv7(), through: [] };
/** A walk starting here: `ref.retry()`'s, with the first host's budget, or a `retry` message's, with one hop less. */
export function retryWalk(hops = RETRY_HOPS): RetryWalk {
return { visited: new Set(), hops };
}
/** A reference's params as L1 checked them, with their identity key. */
@ -632,14 +626,14 @@ export class Binding implements Holder, UnboundRequest {
}
/**
* `ref.retry()`, as `operation` (one per `ref.retry()`, passed on through parked objects): the object is asked for
* an attempt now (ignored unless failed; a parked one passes it on); a refused reference is placed again; one
* waiting in the index is left as it is (an event wakes it).
* `ref.retry()`, as a step of `walk` (a new one for `ref.retry()`, passed on through parked objects): the object is
* asked for an attempt now (ignored unless failed; a parked one passes it on); a refused reference is placed again;
* one waiting in the index is left as it is (an event wakes it).
*/
retry(operation: RetryOperation = retryOperation()): void {
retry(walk: RetryWalk = retryWalk()): void {
const state = this.#state;
if (state.kind === "bound") {
state.target.retry(operation);
state.target.retry(walk);
return;
}
if (state.kind !== "refused") return;

7
packages/chord/src/next/host/routines.ts
View file

@ -14,10 +14,9 @@ import {
type WatchEnd,
} from "../../watch.ts";
import type { Principal } from "../data/index.ts";
import type { SealToken } from "../documents/internal.ts";
import { NativeSource, Source } from "../state/internal.ts";
import { nextDelay } from "./backoff.ts";
import { type Caller, type Ctx, RoutineContext } from "./ctx.ts";
import { CallContext, type Caller, type Ctx } from "./ctx.ts";
import type { AnyHost, Position } from "./host.ts";
import { resolveWatchable } from "./reference.ts";
import { sleep } from "./sleep.ts";
@ -31,8 +30,6 @@ export type RoutineOwner = {
readonly host: AnyHost;
readonly caller: Caller;
readonly context: Context;
/** The token its calls' transactions carry (a swap's handover fences it, slice 7b §5). */
readonly fence: SealToken;
/**
* Adds a running call to the drain; `cleanup` (its bookkeeping) runs once, when it settles or when the attempt ends
* with it still running.
@ -163,7 +160,7 @@ async function live(owner: RoutineOwner, { at, fn }: Extract<Declaration, { kind
* never rejects.
*/
function call(owner: RoutineOwner, invoke: (ctx: Ctx) => void | Promise<void>): Promise<Called> {
const ctx = new RoutineContext(owner.context, owner.caller, owner.host, owner.fence);
const ctx = new CallContext(owner.context, owner.caller, owner.host);
const settled = Promise.withResolvers<Called>();
owner.admit(settled.promise, () => ctx.settle());
const finish = (called: Called): void => {

20
packages/chord/src/next/host/staging.ts
View file

@ -13,7 +13,7 @@ import { bindProvider, checkParams } from "../data/internal.ts";
import type { AnyDefinition } from "../documents/internal.ts";
import type { JobDefinition } from "../jobs/index.ts";
import type { Registry, RegistryProvider } from "../registries/index.ts";
import { checkProvider, type RegistryEdge, RegistryValueSource } from "../registries/internal.ts";
import { checkProvider, type RegistryValueSource } from "../registries/internal.ts";
import { tableSqlName } from "../store/sql.ts";
import type { ChordStore } from "../store/store.ts";
import type { TableDefinition } from "../store/table.ts";
@ -134,10 +134,9 @@ export type Recorded = {
* Runs `served`'s factory for `object` (its params and principal) against a recording `self` and returns what it
* declared. Every declaration check an attempt makes runs, without effects (declared twice; persisted data in a type
* without a stable identity, on a host without a store, or claimed by another owner; contributions outside
* host-lifetime objects, twice, or closing a cycle of registries; `retryOn` sources that do not outlive the attempt;
* params; a document or job declared at an older version than the object's, or at its version with another schema);
* a throw is the factory's. Handles it hands out are inert, except registry values (the host's own, which `retryOn`
* and contributions check by kind).
* host-lifetime objects, or twice; `retryOn` sources that do not outlive the attempt; params; a document or job
* declared at an older version than the object's, or at its version with another schema); a throw is the factory's.
* Handles it hands out are inert, except registry values (the host's own, which `retryOn` checks by kind).
*/
export function recordFactory(object: ChordObject, served: Served, context: Context): Recorded {
const recording = new RecordingSelf(object, served, context);
@ -181,8 +180,6 @@ class RecordingSelf {
readonly #tables = new Map<string, TableDefinition>();
readonly #jobs = new Map<string, JobDefinition<unknown, unknown, unknown>>();
readonly #contributions = new Map<string, Registry<unknown, unknown>>();
/** The registry edges its contributions of other registries' values add (the cycle check of a real attempt). */
readonly #edges: RegistryEdge[] = [];
readonly #requests: string[] = [];
constructor(object: ChordObject, served: Served, context: Context) {
@ -239,7 +236,7 @@ class RecordingSelf {
registry(R: Registry<unknown, unknown>): RegistryValueSource {
this.#assertDeclaring("registry");
this.#requests.push(R.name);
// The host's own value (an attempt declares the same one): `retryOn` and contributions check its kind.
// The host's own value (an attempt declares the same one): `retryOn` checks its kind.
return this.#object.host.registryRecord(R.name).value(R.version);
}
@ -253,7 +250,7 @@ class RecordingSelf {
for (const source of sources) retrySource(source, this.#object.label);
}
contribute(R: Registry<unknown, unknown>, value: unknown): void {
contribute(R: Registry<unknown, unknown>): void {
this.#assertDeclaring("contribute");
const served = this.#served;
const { label } = this.#object;
@ -262,11 +259,6 @@ class RecordingSelf {
throw new Error(`${label}: self.contribute is valid only in a local type with empty params`);
}
if (this.#contributions.has(R.name)) throw new Error(`${label}: registry ${R.name} is contributed to twice`);
const record = this.#object.host.registryRecord(R.name);
if (value instanceof RegistryValueSource) {
record.assertNoCycle(value, this.#edges);
if (value.record !== undefined) this.#edges.push({ from: record, to: value.record });
}
this.#contributions.set(R.name, R);
}

7
packages/chord/src/next/host/unbound.ts
View file

@ -31,12 +31,11 @@ export class UnboundIndex {
requests.add(request);
}
/** `request` ended or binds: it leaves the index; whether it was there. */
delete(name: string, request: UnboundRequest): boolean {
/** `request` ended or binds: it leaves the index. */
delete(name: string, request: UnboundRequest): void {
const requests = this.#byName.get(name);
if (requests === undefined || !requests.delete(request)) return false;
if (requests === undefined || !requests.delete(request)) return;
if (requests.size === 0) this.#byName.delete(name);
return true;
}
/** The host's publication: every request indexed (by any name) runs its check again, out of the index first. */

12
packages/chord/src/next/link/README.md
View file

@ -57,8 +57,8 @@ tab.connect(webSocketLink(new WebSocket(url)), { principal: { kind: "server" } }
as a count and 16 raw bytes per instance id, its `limit` as a varint. No outer length.
- `textCodec`, for debugging: one JSON tuple per message with a short kind tag and positional fields (`["R", hold]`,
`["c", call, hold, method, args?]`, `["b", [[hold, variant, [[member, value]…], [[member, error]…]?]…]]`, `["s",
[[hold, [[member, ops]…], [[member, error]…]?]…]]`, `["f", hold, "t" | "b", error, retryIn?]`, `["T", hold]` for
`retry`, `["l", hold]` for `loading`, `["a"]` ack, `["C"]` closed, `["p"]` ping, `["P"]` pong, …; a message with an origin wrapped as
[[hold, [[member, ops]…], [[member, error]…]?]…]]`, `["f", hold, "t" | "b", error, retryIn?]`, `["T", hold, hops]`
for `retry`, `["l", hold]` for `loading`, `["a"]` ack, `["C"]` closed, `["p"]` ping, `["P"]` pong, …; a message with an origin wrapped as
`["@", origin, tuple]`); parts are
spliced into the tuple as their JSON text and re-serialized at decode (one parse per part per hop, accepted for a
debugging codec); a payload marked as carrying bytes ends with its segments' lengths, the segments following the
@ -83,9 +83,11 @@ tab.connect(webSocketLink(new WebSocket(url)), { principal: { kind: "server" } }
its origin), `closed { origin }` (a relay's downstream origin ended: its holds and calls go), `ping {}`, `pong {}`,
`revoke { revocation }` (from the parent, applied with the child's own `revokedBy`; no origin), `failed { hold, cause,
retryIn? }` (`retryIn`: the milliseconds left of the provider's backoff window, absent for a broken failure),
`loading { hold }` (the hold's object, or a relay's upstream, starts over), `retry { hold }` (the consumer asks for an
attempt now), `result { call, value?, segments? }`, `error`, `gone`. `error`, `gone` and a `failed` cause carry the
wire error format (`WireError`: a ChordError's JSON, or a plugin's code and message). `args` absent is a method without arguments,
`loading { hold }` (the hold's object, or a relay's upstream, starts over), `retry { hold, hops }` (the consumer
asks for an attempt now; `hops`, 1 to 16, is its budget: the first host sends 16, the host there passes it on to its
remote dependencies with one less, never at 0; relays forward it as it is), `result { call, value?, segments? }`,
`error`, `gone`. `error`, `gone` and a `failed` cause carry the wire error format (`WireError`: a ChordError's JSON,
or a plugin's code and message). `args` absent is a method without arguments,
`value` absent is `null`; `segments` present marks a payload with byte segments (its method's schema has a `bytes()`
leaf, `Snapshot.bytes`). `member`, `method` and `variant` are indexes into the hold's bound version's names, each
variant's sorted (`variantNames`, `variantOrder` in contracts). Every kind but `hello`, `revoke`, `ping` and `pong` may carry

26
packages/chord/src/next/link/binary.ts
View file

@ -50,11 +50,6 @@ const PRESENT = 1;
const SEGMENTS = 2;
/** A retry's optional fields, one bit each: its operation (16 raw bytes), then the ids it passed through (a list). */
const OPERATION = 1;
const THROUGH = 2;
/** A hold's optional fields, one bit each. */
const PRINCIPAL = 1;
@ -131,17 +126,10 @@ function encodeFields(message: Message, writer: Writer): void {
case "loading":
writer.varint(message.hold);
return;
case "retry": {
const { operation, through } = message;
case "retry":
writer.varint(message.hold);
writer.byte((operation === undefined ? 0 : OPERATION) | (through === undefined ? 0 : THROUGH));
if (operation !== undefined) writer.raw(hex(operation.replaceAll("-", ""), 16));
if (through !== undefined) {
writer.varint(through.length);
for (const instance of through) writer.raw(hex(instance.replaceAll("-", ""), 16));
}
writer.varint(message.hops);
return;
}
case "call":
writer.varint(message.call);
writer.varint(message.hold);
@ -242,14 +230,8 @@ function decodeFields(kind: (typeof KINDS)[number], reader: Reader): Message {
case "release":
case "loading":
return { kind, hold: reader.varint() };
case "retry": {
const hold = reader.varint();
const flags = reader.byte();
if (flags > (OPERATION | THROUGH)) throw new Error("binary: an unknown retry flag");
const operation = flags & OPERATION ? { operation: uuid(reader.raw(16)) } : {};
const through = flags & THROUGH ? { through: reader.list(() => uuid(reader.raw(16))) } : {};
return { kind, hold, ...operation, ...through };
}
case "retry":
return { kind, hold: reader.varint(), hops: reader.varint() };
case "call": {
const [call, hold, method] = [reader.varint(), reader.varint(), reader.varint()];
return { kind, call, hold, method, ...reader.payload("args") };

1
packages/chord/src/next/link/internal.ts
View file

@ -16,6 +16,7 @@ export {
NO_RETIRED,
type OriginMessage,
type ProviderMessage,
RETRY_HOPS,
type StateEntry,
type WireAddress,
} from "./message.ts";

17
packages/chord/src/next/link/message.ts
View file

@ -50,6 +50,9 @@ const incarnation = str({ pattern: "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f
/** The longest activation chain a hold carries (Q5): a longer one keeps its last entries. */
export const CHAIN = 32;
/** A `retry`'s hop budget as the first host sends it (dependency model §3.7, audit A5): one less per host after it. */
export const RETRY_HOPS = 16;
/** The kinds of a `failed` message's cause, in their codes' order (the binary codec's byte, the text codec's letter). */
export const CAUSE_KINDS: ["transient", "broken", "waiting"] = ["transient", "broken", "waiting"];
@ -78,16 +81,10 @@ const MESSAGE = union([
obj({ kind: lit("release"), origin: opt(origin), hold: id }),
obj({ kind: lit("call"), origin: opt(origin), call: id, hold: id, method: index, args: opt(part), segments }),
obj({ kind: lit("abort"), origin: opt(origin), call: id }),
// Asks the provider for an attempt of a failed hold's object now (`ref.retry()`); `operation` names the one retry it
// belongs to (a uuidv7) and `through` the instance ids of the objects it passed through (with `operation` only), so
// an object it reaches twice (a diamond, a ring) handles it once. Relays forward both.
obj({
kind: lit("retry"),
origin: opt(origin),
hold: id,
operation: opt(incarnation),
through: opt(Type.Array(incarnation, { maxItems: CHAIN })),
}),
// Asks the provider for an attempt of a failed hold's object now (`ref.retry()`); `hops` is its budget: the host
// there passes it on to its remote dependencies with one less, never at 0, so a waiting ring of hosts stops. Relays
// forward it as it is.
obj({ kind: lit("retry"), origin: opt(origin), hold: id, hops: int({ min: 1, max: RETRY_HOPS }) }),
// Binds several holds at once (first bindings, re-holds), applied in one batch.
obj({
kind: lit("bound"),

14
packages/chord/src/next/link/text.ts
View file

@ -105,10 +105,7 @@ function ownTuple(message: Message): string {
case "revoke":
return json(["V", message.revocation]);
case "retry":
// `through` travels only with an `operation` (Chord sends both).
if (message.operation === undefined) return json(["T", message.hold]);
if (message.through === undefined) return json(["T", message.hold, message.operation]);
return json(["T", message.hold, message.operation, message.through]);
return json(["T", message.hold, message.hops]);
case "loading":
return json(["l", message.hold]);
case "ack":
@ -281,13 +278,8 @@ function fromFields(fields: unknown[], rest: Uint8Array): Message {
arity(1);
return { kind: "revoke", revocation: f[0] };
case "T":
arity(1, 2, 3);
return {
kind: "retry",
hold: field(0),
...(f.length >= 2 ? { operation: field<string>(1) } : {}),
...(f.length === 3 ? { through: field<string[]>(2) } : {}),
};
arity(2);
return { kind: "retry", hold: field(0), hops: field(1) };
case "l":
arity(1);
return { kind: "loading", hold: field(0) };

8
packages/chord/src/next/registries/README.md
View file

@ -72,11 +72,9 @@ host.serve(Server, (_, self) => {
- `RegistryRecord`: one per registry and host (`ChordHost.registryRecord(name)`, made by its first use): the provider
(`serve`), the contributions in table order (`add`, `remove`, inside a batch), the last table and composition
(`start()`: composed now when dirty; a read while composing is a cycle, `invalid/cycle`), the episode reports, the
direct edges between registries whose values are contributions in its table (counted, kept while those
contributions are in it; `assertNoCycle` refuses a cycle at `contribute`, through those edges and the
contributing attempt's own earlier declarations), and its values by reader version
(`value(version)`); `close()` at the host's close.
(`start()`: composed now when dirty; a read while composing is a cycle, `invalid/cycle`: the contribution that read
it is left out, so a ring of registries is caught at its first composition, audit A2), the episode reports, and its
values by reader version (`value(version)`); `close()` at the host's close.
- `RegistryContribution` (`contribution.ts`): one contributor's contribution, a `Dependent` of its source;
`entry(version, provider)` is its table entry (the same object while its value is), or `undefined` when left out.
- `RegistryValueSource`: a `FollowedSource` per reader version; `read()` registers a reader (an attempt that declared

9
packages/chord/src/next/registries/contribution.ts
View file

@ -4,7 +4,6 @@
import { type Dependent, type DeriveSource, handedOut, type NativeSource, Source } from "../state/internal.ts";
import type { RegistryRecord } from "./record.ts";
import type { RegistryEntry, RegistryProvider } from "./registry.ts";
import { RegistryValueSource } from "./value.ts";
/** Who contributes: a facet (by plugin) or a host-lifetime object (by plugin and type). */
export type Contributor = { readonly plugin: string; readonly type: string | undefined };
@ -62,14 +61,6 @@ export class RegistryContribution implements Dependent {
this.#followed()?.dependents.delete(this);
}
/** The registry whose value it is, when it is one (its edge in the cycle check, while it is in the table). */
get reads(): RegistryRecord | undefined {
const content = this.#content;
return content.kind === "native" && content.source instanceof RegistryValueSource
? content.source.record
: undefined;
}
/** Its source when it changes (a constant never does). */
#followed(): Source | NativeSource | undefined {
const content = this.#content;

2
packages/chord/src/next/registries/internal.ts
View file

@ -1,5 +1,5 @@
// Registries for Chord's layers: a host's record of each registry, its contributions and its values by version.
export { type ContributionContent, type Contributor, RegistryContribution } from "./contribution.ts";
export { checkProvider, type RegistryEdge, type RegistryError, RegistryRecord } from "./record.ts";
export { checkProvider, type RegistryError, RegistryRecord } from "./record.ts";
export { RegistryValueSource, type RegistryWaiter } from "./value.ts";

43
packages/chord/src/next/registries/record.ts
View file

@ -38,9 +38,6 @@ export type RegistryError =
readonly version: number;
};
/** One registry's value contributed to another: `from`'s table reads `to`. */
export type RegistryEdge = { readonly from: RegistryRecord; readonly to: RegistryRecord };
/** The provider as served: the defining plugin's version, `compose` and adapters. */
type Provider = { readonly version: number; readonly provider: RegistryProvider<unknown, unknown> };
@ -79,11 +76,6 @@ export class RegistryRecord {
#composing = false;
/** The conflicts the last good composition reported, by key: a conflict is reported once per episode. */
#conflicts = new Set<string>();
/**
* Registries whose values are contributions in this one's table, each with how many: a contribution that would
* close a cycle of them is refused. An edge lives as long as its contribution is in the table.
*/
readonly #reads = new Map<RegistryRecord, number>();
/** The values readers follow, by version; made by the first reader of a version. */
readonly #values = new Map<number, RegistryValueSource>();
@ -135,26 +127,11 @@ export class RegistryRecord {
this.start();
}
/**
* Refuses a contribution that is `source`'s value when it would close a cycle (declaration policy, at
* `contribute`): through the registries in the tables now and the edges `pending` (the contributing attempt's
* earlier declarations, which enter the tables only once it is live).
*/
assertNoCycle(source: RegistryValueSource, pending: readonly RegistryEdge[]): void {
const { record } = source;
if (record === undefined) return;
if (record.#reaches(this, new Set(), pending)) {
throw invalid("cycle", `registries ${this.name} and ${record.name} read each other`);
}
}
/** Adds a live contributor's contribution, inside a batch: the table changes. */
add(contribution: RegistryContribution): void {
const at = this.#contributions.findIndex((other) => compareContributors(contribution, other) < 0);
// A new list per change (contributors come and go rarely): a composition keeps the one it was built from.
this.#contributions = this.#contributions.toSpliced(at < 0 ? this.#contributions.length : at, 0, contribution);
const { reads } = contribution;
if (reads !== undefined) this.#reads.set(reads, (this.#reads.get(reads) ?? 0) + 1);
contribution.follow();
this.#invalidate();
}
@ -164,12 +141,6 @@ export class RegistryRecord {
const at = this.#contributions.indexOf(contribution);
if (at < 0) return;
this.#contributions = this.#contributions.toSpliced(at, 1);
const { reads } = contribution;
if (reads !== undefined) {
const left = this.#reads.get(reads)! - 1;
if (left === 0) this.#reads.delete(reads);
else this.#reads.set(reads, left);
}
contribution.unfollow();
this.#invalidate();
}
@ -242,7 +213,6 @@ export class RegistryRecord {
close(): void {
for (const contribution of this.#contributions) contribution.unfollow();
this.#contributions = [];
this.#reads.clear();
this.#table = undefined;
this.#dirty = false;
for (const value of this.#values.values()) value.close();
@ -317,19 +287,6 @@ export class RegistryRecord {
}
return same && previous!.length === table.length ? previous! : table;
}
/** Whether `target` is reached from here through the tables' edges and `pending`. */
#reaches(target: RegistryRecord, visited: Set<RegistryRecord>, pending: readonly RegistryEdge[]): boolean {
if (this === target) return true;
visited.add(this);
for (const next of this.#reads.keys()) {
if (!visited.has(next) && next.#reaches(target, visited, pending)) return true;
}
for (const { from, to } of pending) {
if (from === this && !visited.has(to) && to.#reaches(target, visited, pending)) return true;
}
return false;
}
}
/** Table order: by plugin name, then type, a facet (no type) first; code point order, never load order. */

6
packages/chord/src/next/registries/value.ts
View file

@ -73,12 +73,6 @@ export class RegistryValueSource extends FollowedSource<unknown> {
this.#registered = { kind: "open", record };
}
/** Its registry's record while the host is open. */
get record(): RegistryRecord | undefined {
const registered = this.#registered;
return registered.kind === "open" ? registered.record : undefined;
}
/** The composition (or its mapping) at its last good value; throws the failure when there is none. */
get value(): unknown {
const start = this.start();

8
packages/chord/src/next/remote/README.md
View file

@ -91,9 +91,11 @@ within each link's limits, compression at the origin, activation chains and revo
as `failed` (with `retryIn`, the rest of the object's backoff window, for a transient one; absent for broken and
waiting), each new cause of an object parked on a failed dependency too, the current one at once when the object is
failed already; the next attempt (or the parked object's next `loading`) is announced with `loading`; the hold stays,
so a `retry` (from any holder) renews even a broken object for every holder (a parked object passes it on, one retry
operation of this host). Live, it binds through the window (`boundEntry`: the variant and each member's value as a part,
names as indexes into the bound version). Its members' frames arrive as `Subscriber` calls inside the batch; at its
so a `retry` (from any holder) renews even a broken object for every holder (a parked object passes it on, one walk
of this host with one hop less of the message's budget). After a swap in place it gets a fresh `bound` once the new
instance is live, answering calls by the names it was bound with until that is sent (the window may hold it). Live,
it binds through the window (`boundEntry`: the variant and each member's value as a part, names as indexes into the
bound version). Its members' frames arrive as `Subscriber` calls inside the batch; at its
end the hold's entry lists the members that changed with their ops, each encoded with that member's own Delta
encoder (`entry`), or queues them (`queue`, later `pendingEntry`). Each entry counts one `chord.link.encodes`. A
call's method is an index into the bound variant's methods; one outside them is a protocol error, its arguments are

8
packages/chord/src/next/remote/consumer.ts
View file

@ -8,7 +8,6 @@ import type { Context } from "../../context/index.ts";
import { type MethodData, takesNoArgs } from "../contracts/internal.ts";
import { ChordError, isChordError } from "../data/index.ts";
import { asChordError } from "../data/internal.ts";
import type { RetryOperation } from "../host/internal.ts";
import {
type BoundEntry,
type ConsumerMessage,
@ -126,10 +125,9 @@ export class ConsumerSide {
this.forget(hold, error);
}
/** `ref.retry()` on a failed hold, as retry `operation`: the provider is asked for an attempt now. */
retry(hold: number, { id, through }: RetryOperation): void {
const passed = through.length === 0 ? {} : { through };
this.#link.send({ kind: "retry", hold, operation: id, ...passed });
/** `ref.retry()` on a failed hold, with `hops` left of its budget: the provider is asked for an attempt now. */
retry(hold: number, hops: number): void {
this.#link.send({ kind: "retry", hold, hops });
}
/** Drops hold `hold` from the tables; its calls, in flight and waiting, fail with `error` and free their slots. */

179
packages/chord/src/next/remote/hold.ts
View file

@ -3,9 +3,10 @@
// model §1), the object it keeps, the notices of its object's failures and new attempts, the snapshot and subscription
// taken in one synchronous step when a slot of the window is free, and the ops of its members per batch (sent at
// once, or pending while the window is full), each member with its own Delta encoder. Across a reload (slice 7b §6) a
// bound hold is kept: a swap in place re-subscribes it (one `[["r", value]]` per changed member), an object leaving
// `live` unbinds it (`loading` or `failed`, then a fresh `bound`), a version no longer bound or a removed type leaves it
// waiting unbound. A hold of an older version follows its object's adapted members (Q10).
// bound hold is kept: a swap in place sends it a fresh `bound` once the new instance is live (it answers calls
// meanwhile), an object leaving `live` unbinds it (`loading` or `failed`, then a fresh `bound`), a version no longer
// bound or a removed type leaves it waiting unbound. A hold of an older version follows its object's adapted members
// (Q10).
import type { Context } from "../../context/index.ts";
import { type Encoder, encoder, type Op, PendingOps } from "../../delta/index.ts";
@ -18,7 +19,7 @@ import {
type ChordObject,
type Holder,
type Removal,
retryOperation,
retryWalk,
type Serving,
type UnboundRequest,
} from "../host/internal.ts";
@ -55,8 +56,8 @@ export type HoldRequest = Omit<HoldMessage, "kind" | "origin" | "hold" | "princi
*/
type Followed = {
readonly index: number;
/** Its source (subscribed when it changes, a constant's never): a reload's swap may bring a new one. */
source: DeriveSource;
/** Its source (subscribed when it changes, a constant's never). */
readonly source: DeriveSource;
readonly subscriber: Subscriber;
readonly encoder: Encoder;
readonly pending: PendingOps;
@ -65,28 +66,25 @@ type Followed = {
/**
* Where a hold stands: `unplaced` in the unbound index until the host's publication places it; `pending` until its
* object is live (the activation, parking and retries; `toldFailed` once the consumer was told of a failure, so the
* next attempt is announced with `loading`); `binding` while its object is live and its `bound` waits for a slot;
* next attempt is announced with `loading`); `binding` while its object is live and its `bound` waits for a slot
* (after a swap in place with the names it was bound with, `previous`: the consumer still holds it bound, so its calls
* are answered by those names, design §22; only contract data, so the old instance and its sources go meanwhile);
* `bound` with the object's variant and the members it follows.
*/
type HoldState =
| { readonly kind: "unplaced"; readonly request: HoldRequest }
| { readonly kind: "pending"; readonly toldFailed: boolean }
| { readonly kind: "binding" }
| { readonly kind: "binding"; readonly previous?: BoundNames }
| {
readonly kind: "bound";
/** The bound variant's name, and its data at the hold's version. */
readonly variantName: string;
/** The bound variant's data at the hold's version. */
readonly variant: VariantData;
/** The bound version's names of the variant, which the wire's indexes point into. */
readonly names: VariantNames;
/** The members it follows, in the order they were subscribed. */
readonly followed: Followed[];
/** The instance it was bound to: a swap brings another, whose members it re-subscribes to. */
/** The instance it was bound to: a swap brings another, which it binds to afresh. */
readonly instance: unknown;
/** What it was bound to per member: the instance's members, or its mapping at the hold's older version. */
readonly sources: Map<string, DeriveSource>;
/** Bound through its instance's mapping for an older version than the provider's (`AdaptedMembers`). */
readonly adapted: boolean;
}
/**
* Refused where it arrived, with `cause`: `waiting` (not served here, another version, placed elsewhere by the
@ -99,6 +97,9 @@ type HoldState =
/** A bound hold's state. */
type Bound = Extract<HoldState, { readonly kind: "bound" }>;
/** What calls need of a binding: its variant's data at the hold's version and the names the wire's indexes point into. */
type BoundNames = { readonly variant: VariantData; readonly names: VariantNames };
const PENDING: HoldState = { kind: "pending", toldFailed: false };
const BINDING: HoldState = { kind: "binding" };
@ -145,11 +146,17 @@ export class Hold implements Holder, UnboundRequest {
provider.link.host.count("references", 1);
}
/** Whether the hold is live: calls on it reach its object. */
/** Whether the hold is bound: its members' changes are sent. */
get bound(): boolean {
return this.#state.kind === "bound";
}
/** Whether calls on it reach its object: bound, or binding afresh after a swap in place (its `bound` not sent yet). */
get callable(): boolean {
const state = this.#state;
return state.kind === "bound" || (state.kind === "binding" && state.previous !== undefined);
}
/** Whom it acts for (the unbound index's revocation). */
get principal(): Principal | undefined {
return this.caller.principal;
@ -208,11 +215,6 @@ export class Hold implements Holder, UnboundRequest {
this.#refuse(request, served.cause);
return;
}
if (served.kind === "installing") {
this.#state = { kind: "unplaced", request: owned(request) };
link.host.installing(request.type, this);
return;
}
this.#request = owned(request);
const { object } = served;
this.#object = object;
@ -238,7 +240,7 @@ export class Hold implements Holder, UnboundRequest {
* The object's state changed, or the hold took it: a pending hold binds once it is live (when a slot is free),
* sends each failure, and announces the attempt after a failure with `loading`. A bound hold (or one whose `bound`
* waits for a slot) whose object leaves `live` goes back to pending (slice 7b §6), except while a reload's swap runs
* in place; a bound one whose object went live with another attempt (a swap) re-subscribes to its members.
* in place; a bound one whose object went live with another attempt (a swap) gets a fresh `bound` (audit A9).
*/
moved(): void {
const current = this.#state;
@ -247,19 +249,12 @@ export class Hold implements Holder, UnboundRequest {
if (current.kind === "bound" || current.kind === "binding") {
const { kind } = object.state;
if (kind === "live") {
if (current.kind === "bound" && object.instance !== current.instance) {
// In a batch: the changed members' frames leave as one `state` message.
const { link } = this.#provider;
link.host.batch.within(() => this.#rebound(current, object), link.context);
}
if (current.kind === "bound" && object.instance !== current.instance) this.#rebind(current);
return;
}
// Its instance still changes (a swap's old one, a restart's drain, a live failure's admitted commits): it follows
// it on until it stopped (slice 7b, decision "References are the only indirection").
if (current.kind === "bound" && object.attachable === current.instance) {
if (kind === "failed") this.#stopForeign(current, object);
return;
}
if (current.kind === "bound" && object.attachable === current.instance) return;
if (current.kind === "binding" && kind === "loading" && object.swapsInPlace) return;
this.#unbind(object.state.kind === "failed" ? object.state.cause.error : undefined);
}
@ -281,18 +276,6 @@ export class Hold implements Holder, UnboundRequest {
if (this.#failed.length > 0) this.#failed = this.#failed.filter(({ member }) => member !== index);
}
/**
* Its object failed live and still publishes the commits it admitted before: the members following another object's
* sources stop at once, its own follow on until it unbinds (as references do, G1 "own"; Sol).
*/
#stopForeign(state: Bound, object: ChordObject): void {
const { version } = this.contract;
for (const { index, source, subscriber } of state.followed) {
if (!(source instanceof Source)) continue;
if (object.followed(state.names.state[index]!, version).kind !== "live") source.unsubscribe(subscriber);
}
}
/**
* Its object left `live` (a restart, a parked swap, a live failure): no longer subscribed or waiting for a slot, its
* calls in flight rejected (`error`, or `unavailable`), pending again; a `loading` tells the consumer it rebinds
@ -312,68 +295,27 @@ export class Hold implements Holder, UnboundRequest {
}
/**
* A swap of its object went live in place (slice 7b §5): each member it follows is re-subscribed to the new
* attempt's (its mapping at an older version), sending `[["r", value]]` for each whose value changed; another
* variant than the bound one, or a version that became older or current, sends a fresh `bound` instead.
* A swap of its object went live in place (slice 7b §5, audit A9): no longer subscribed to the old instance, it gets
* a fresh `bound` of the new one's members (as a restart, a variant or a version change does), answering calls by
* its old names until that is sent.
*/
#rebound(state: Bound, object: ChordObject): void {
const { version } = this.contract;
const adapted = version !== object.served.contract.version;
if (adapted !== state.adapted || object.variant !== state.variantName) {
unfollow(state);
this.#state = BINDING;
this.#provider.bind(this, this.#provider.link.context);
return;
}
const sources = new Map<string, DeriveSource>();
for (const [index, member] of state.names.state.entries()) {
const source = object.liveMember(member, version);
sources.set(member, source);
// What it was bound to: the replaced instance's member.
const old = state.sources.get(member)!;
if (source === old) continue;
const was = currentOf(old);
// Read before it is followed: a derive computed now publishes no first frame of its own.
let now:
| { readonly kind: "value"; readonly value: JsonValue }
| { readonly kind: "failed"; readonly error: unknown };
try {
now = { kind: "value", value: source.value };
} catch (error) {
now = { kind: "failed", error };
}
// A member it did not follow (a constant) is followed from now on: a later swap may make it change.
let followed = state.followed.find((each) => each.index === index);
if (followed === undefined) followed = this.#follow(state, index, source);
else {
if (followed.source instanceof Source) followed.source.unsubscribe(followed.subscriber);
followed.source = source;
if (source instanceof Source) source.subscribe(followed.subscriber);
}
// Failed on the new instance (its mapping throws): its failure is sent; it heals with `[["r", value]]` (Sol).
if (now.kind === "failed") {
this.#touch();
this.#memberFailed(followed, now.error);
continue;
}
if (now.value === was?.value) continue;
this.#touch();
this.#memberHealed(index);
this.#changes.push([followed, [["r", now.value]]]);
}
this.#state = { ...state, instance: object.instance, sources };
#rebind(state: Bound): void {
unfollow(state);
// Its ops still waiting for a slot are the old instance's: the `bound` replaces them.
this.#provider.unbind(this);
this.#state = { kind: "binding", previous: { variant: state.variant, names: state.names } };
this.#provider.bind(this, this.#provider.link.context);
}
/**
* A `retry` message, of its retry operation (a new one when it names none) through the objects it names: the object is
* asked for an attempt now (it ignores this unless failed, or when the operation reached it before; parked, it
* passes it on); a refused hold is
* started again (a route's `authorize` may admit it now).
* A `retry` message with `hops` left of its budget: the object is asked for an attempt now (it ignores this unless
* failed; parked, it passes it on, to other hosts with one hop less); a refused hold is started again (a route's
* `authorize` may admit it now).
*/
retry({ operation, through = [] }: Extract<Message, { readonly kind: "retry" }>): void {
retry({ hops }: Extract<Message, { readonly kind: "retry" }>): void {
const state = this.#state;
if (state.kind !== "refused") {
this.#object?.retry(operation === undefined ? retryOperation() : { id: operation, through });
this.#object?.retry(retryWalk(hops - 1));
return;
}
if (state.cause.kind === "waiting") this.#provider.link.host.unindex(state.request.type, this);
@ -419,11 +361,11 @@ export class Hold implements Holder, UnboundRequest {
*/
method(index: number): { readonly name: string; readonly resultBytes: boolean } {
const state = this.#state;
const name = state.kind === "bound" ? state.names.methods[index] : undefined;
// Binding afresh after a swap: by the names the consumer still has (the new instance answers by name).
const bound = state.kind === "bound" ? state : state.kind === "binding" ? state.previous : undefined;
const name = bound?.names.methods[index];
if (name === undefined) throw new Error(`hold ${this.id}: no method ${index}`);
// The bound state holds the variant whose names these are.
const { variant } = state as Bound;
return { name, resultBytes: variant.methods.get(name)!.result.bytes };
return { name, resultBytes: bound!.variant.methods.get(name)!.result.bytes };
}
/**
@ -444,7 +386,6 @@ export class Hold implements Holder, UnboundRequest {
for (const [index, member] of state.names.state.entries()) {
// An older version's members are its instance's mapping (its refusals unbound it before: `ChordObject`).
const source = current ? object.stateMembers.get(member)! : object.liveMember(member, version);
state.sources.set(member, source);
let value: JsonValue;
try {
value = source.value;
@ -541,27 +482,19 @@ export class Hold implements Holder, UnboundRequest {
/** The binding: its variant and names at the hold's version, with no member followed yet. */
#bindTo(object: ChordObject): Bound {
const variantName = object.variant!;
const data = variantsAt(object.served.contract, this.contract.version).get(variantName)!;
const instance = object.instance;
const adapted = this.contract.version !== object.served.contract.version;
const names = variantNames(data);
const sources = new Map<string, DeriveSource>();
const variant = variantsAt(object.served.contract, this.contract.version).get(object.variant!)!;
const state: Bound = {
kind: "bound",
variantName,
variant: data,
names,
variant,
names: variantNames(variant),
followed: [],
instance,
sources,
adapted,
instance: object.instance,
};
this.#state = state;
return state;
}
/** A hold over a link follows its object's instances in `moved` (re-subscribed, or a fresh `bound`). */
/** A hold over a link follows its object's instances in `moved` (a fresh `bound` for a new one). */
retarget(): void {}
/** It moves nothing in `retarget`. */
@ -591,7 +524,7 @@ export class Hold implements Holder, UnboundRequest {
* drops what was pending for it and replaces a failure not sent yet (one at most per member, however long the
* consumer stalls); the value after it (a root replacement) replaces the failure. A retirement waits for a value.
*/
#follow(state: Bound, index: number, source: DeriveSource): Followed {
#follow(state: Bound, index: number, source: DeriveSource): void {
const followed: Followed = {
index,
source,
@ -616,10 +549,9 @@ export class Hold implements Holder, UnboundRequest {
encoder: encoder(),
pending: new PendingOps(),
};
// A constant never changes: nothing to subscribe to (a swap may bring a source, `#rebound`).
// A constant never changes: nothing to subscribe to.
if (source instanceof Source) source.subscribe(followed.subscriber);
state.followed.push(followed);
return followed;
}
/** Takes the current batch's changes; the hold is untouched again. */
@ -674,15 +606,6 @@ function unfollow(state: Bound): void {
for (const { source, subscriber } of state.followed) if (source instanceof Source) source.unsubscribe(subscriber);
}
/** The current value of `source`, `undefined` when it throws (a failed member). */
function currentOf(source: DeriveSource): { readonly value: JsonValue } | undefined {
try {
return { value: source.value };
} catch {
return undefined;
}
}
/** `request` with its params owned, not a view of the message they came in (which the link's buffers may hold). */
function owned(request: HoldRequest): HoldRequest {
const { params } = request;

10
packages/chord/src/next/remote/provider.ts
View file

@ -350,10 +350,10 @@ export class ProviderSide {
* A call: one on a hold this side does not know is ignored (Q1), more than this side's N at once and compressed
* arguments from a child (clients never compress) are protocol errors. On a relayed hold it is forwarded, its
* arguments checked against this link's part limit first (the bytes are here). On a served hold that is not bound
* yet it is refused `not_ready`; its method is an index into the bound variant (one outside it is a protocol error)
* and its arguments are opened and parsed here within the hold's limits (past one the call fails `too_large/part`; a
* malformed part is a protocol error), absent ones `null`. Arguments this host must inflate asynchronously are
* inflated first, then the call is handled again.
* yet (nor binding afresh after a swap) it is refused `not_ready`; its method is an index into the bound variant (one
* outside it is a protocol error) and its arguments are opened and parsed here within the hold's limits (past one
* the call fails `too_large/part`; a malformed part is a protocol error), absent ones `null`. Arguments this host
* must inflate asynchronously are inflated first, then the call is handled again.
*/
#call(message: Extract<Message, { readonly kind: "call" }>): void | Promise<void> {
if (this.#calls.has(message.call)) throw new Error(`call ${message.call}: the id is in use`);
@ -374,7 +374,7 @@ export class ProviderSide {
hold.call(message);
return;
}
if (!hold.bound) {
if (!hold.callable) {
this.#refuse(message.call, new ChordError({ code: "not_ready", message: `hold ${hold.id}: not bound yet` }));
return;
}

2
packages/chord/src/next/remote/relay.ts
View file

@ -109,7 +109,7 @@ export class RelayedHold {
upstream?.link.send({ kind: "abort", origin: upstream.origin, call: id });
}
/** A downstream `retry`, forwarded with its operation and the objects it passed through. */
/** A downstream `retry`, forwarded with its hop budget as it is (a relay visits no object). */
retry(message: Extract<Message, { readonly kind: "retry" }>): void {
const upstream = this.#upstream;
upstream?.link.send({ ...message, origin: upstream.origin });

22
packages/chord/src/next/remote/remote-object.ts
View file

@ -26,7 +26,7 @@ import {
LOADING,
type PinnedKey,
type RefState,
type RetryOperation,
type RetryWalk,
type Target,
} from "../host/internal.ts";
import {
@ -112,11 +112,6 @@ export class RemoteObject implements Target {
return this.#state;
}
/** Whether its hold is on a link now (a remote attempt answers for it), not while its path has no link. */
get heldOnLink(): boolean {
return this.#hold !== undefined;
}
/** An object over a link is no blocker of this host's first pass. */
get blocksFirstPass(): false {
return false;
@ -156,15 +151,12 @@ export class RemoteObject implements Target {
attach(link: HostLink): void {
const chain = this.#dependent?.activationChain();
this.#hold = { side: link.consumer, id: link.consumer.attach(this, chain) };
// Held on a link now: a parked dependent checks again (`parked_long` counts a hold on a link only).
this.#dependent?.dependencyMoved();
}
/** The link dropped: loading until a new link's hold is bound (`#startOver`). */
dropped(_error: ChordError): void {
this.#hold = undefined;
this.#startOver();
this.#dependent?.dependencyMoved();
}
/**
@ -250,18 +242,18 @@ export class RemoteObject implements Target {
}
/**
* `ref.retry()`: a failed object's provider is asked for an attempt (a hold it refused is started there again; a
* parked object there passes it on as an operation of its own host). One that failed for a part past the link's
* limits is placed anew and held with a new id, now or on its path's next link (a `hostAddress` that throws fails it
* broken again).
* `ref.retry()`, as a step of `walk`: a failed object's provider is asked for an attempt with the walk's hop budget,
* none left: nothing is sent (a hold it refused is started there again; a parked object there passes it on with one
* hop less). One that failed for a part past the link's limits is placed anew and held with a new id, now or on its
* path's next link (a `hostAddress` that throws fails it broken again).
*/
retry(operation: RetryOperation): void {
retry(walk: RetryWalk): void {
if (this.#state.kind !== "failed") return;
if (this.#path === this.#host.unplaced) {
this.#place();
return;
}
this.#hold?.side.retry(this.#hold.id, operation);
if (walk.hops > 0) this.#hold?.side.retry(this.#hold.id, walk.hops);
}
/** Places a reference whose part was past the limits anew, on its path now (a throwing `hostAddress` fails it). */

3
packages/chord/test/next/documents/address-source.test.ts
View file

@ -6,15 +6,12 @@ import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import type { Op, Prepared } from "../../../src/delta/index.ts";
import { AddressSource } from "../../../src/next/documents/address-source.ts";
import type { Documents } from "../../../src/next/documents/documents.ts";
import type { JsonObject } from "../../../src/next/documents/index.ts";
import { Batch } from "../../../src/next/state/internal.ts";
import { changesOf, delivered, expectCollected } from "../support.ts";
/** An address whose value the test sets. */
class Address extends AddressSource {
// No object's documents: the test reads frames only.
readonly owner = {} as Documents;
current: JsonObject = { n: 0 };
get value(): JsonObject {

8
packages/chord/test/next/host/attempt.test.ts
View file

@ -905,7 +905,7 @@ describe("parked attempts (dependency model §3)", () => {
/** A view with params: no host lifetime, so `load` does not wait for it. */
const Viewer = local("app.viewer", { version: 1, params: { id: int() }, members: { n: int() } });
it("run no timer while parked: no dependency deadline, no backoff, the factory run once", async () => {
it("run no timer while parked but parked_long's: no dependency deadline, no backoff, the factory run once", async () => {
const factories = vi.fn();
const { host, errors } = await loaded((h) =>
h.serve(View, (_, self) => {
@ -921,8 +921,12 @@ describe("parked attempts (dependency model §3)", () => {
cause: { kind: "waiting", error: { message: "waiting for app.missing: app.missing is not served here" } },
retryAt: undefined,
});
// `parked_long`'s one timer fired (once, after readyTimeout); nothing else runs.
expect(vi.getTimerCount()).toBe(0);
expect([factories.mock.calls.length, errors]).toEqual([1, []]);
expect([factories.mock.calls.length, errors]).toEqual([
1,
[{ kind: "parked_long", object: "app.view", waitingFor: "app.missing", state: "failed", since: 0 }],
]);
view.release();
});

243
packages/chord/test/next/host/dependencies.test.ts
View file

@ -1,8 +1,10 @@
import { describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { int, local, remote } from "../../../src/next/contracts/index.ts";
import { int, keyed, local, obj, remote } from "../../../src/next/contracts/index.ts";
import { defineState } from "../../../src/next/documents/index.ts";
import { createHost, defineFacet, type HostError } from "../../../src/next/host/index.ts";
import { constant } from "../../../src/next/state/index.ts";
import { ChordObject } from "../../../src/next/host/object.ts";
import { defineRegistry } from "../../../src/next/registries/index.ts";
import {
bindHold,
failures,
@ -157,70 +159,131 @@ describe("the dependency check", () => {
});
});
describe("parked_long (dependency model §7)", () => {
describe("parked_long (dependency model §7, audit A11)", () => {
fakeTimers();
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
it("starts its clock when the hold gets on a link: a late link is not reported at once (fix round 2, Opus F2)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
const peer = await rawParent(client);
await peer.next("hold");
const linked = Date.now();
await vi.advanceTimersByTimeAsync(10);
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(990);
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", since: linked },
]);
view.release();
});
it("is not reported while the path has no link: no remote attempt answers for it (fix round 1, Opus F2)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
expect(errors).toEqual([]);
expect(view.state.value).toEqual({ kind: "loading" });
// Once a link holds it and the remote attempt is only loading, it is.
const peer = await rawParent(client);
await peer.next("hold");
await vi.advanceTimersByTimeAsync(5_000);
expect(errors.map(({ kind }) => kind)).toEqual(["parked_long"]);
view.release();
});
it("is reported once per parking episode, after readyTimeout on a remote attempt only loading; nothing fails", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await peer.next("hold");
it("is reported once after readyTimeout whatever the blocker: a missing local provider; nothing fails", async () => {
const { host, errors } = await loaded((h) => h.serve(View, (_, self) => ({ n: self.get(Missing, {}).n })), {
readyTimeout: 1_000,
});
const view = host.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(999);
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(1);
expect(errors).toEqual([{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", since: 0 }]);
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.missing", state: "failed", since: 0 },
]);
await vi.advanceTimersByTimeAsync(60_000);
expect(errors).toHaveLength(1);
expect(view.state.value).toEqual({ kind: "loading" });
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
view.release();
await host.close(context);
});
it("is not reported for a remote blocker that failed: the parked attempt shows its cause instead", async () => {
it("names the first blocker in declaration order with its state; its clock runs from parking, with no link yet too", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({
n: self.derive([self.get(Notes, { session: "S1" }).count, self.get(Missing, {}).n], ([a, b]) => a + b),
})),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(1_000);
// The remote one comes first (no link holds it yet), though the attempt shows the failed one.
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", state: "loading", since: 0 },
]);
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
view.release();
await client.close(context);
});
it("keeps declaration order across registries and references: the parked cause and the report name the first (Astra)", async () => {
const Registry = defineRegistry<number, number>("app.missing-registry", { version: 1 });
const { host, errors } = await loaded(
(h) =>
h.serve(View, (_, self) => {
self.registry(Registry);
return { n: self.get(Missing, {}).n };
}),
{ readyTimeout: 1_000 },
);
const view = host.hold(View, { id: 1 }, context);
await flushed();
expect(view.state.value).toMatchObject({
kind: "failed",
cause: {
kind: "waiting",
error: { message: expect.stringMatching(/^waiting for registry app\.missing-registry: /) },
},
});
await vi.advanceTimersByTimeAsync(1_000);
expect(errors).toEqual([
{
kind: "parked_long",
object: "app.view(1)",
waitingFor: "registry app.missing-registry",
state: "failed",
since: 0,
},
]);
view.release();
await host.close(context);
});
it("names a reference whose pinned key still loads: the episode's clock includes the load (Sol)", async () => {
const Rows = defineState("rows", {
version: 1,
keyed: true,
schema: obj({ n: int() }),
initial: () => ({ n: 0 }),
});
const Keys = local("app.keys", { version: 1, params: {}, members: { rows: keyed(obj({ n: int() })) } });
const gate = Promise.withResolvers<void>();
const entered = Promise.withResolvers<void>();
const pinKey = ChordObject.prototype.pinKey;
const held = vi.spyOn(ChordObject.prototype, "pinKey").mockImplementation(async function (
this: ChordObject,
member,
key,
version,
pinContext,
) {
entered.resolve();
await gate.promise;
return pinKey.call(this, member, key, version, pinContext);
});
try {
const { host, errors } = await loaded(
(h) => {
h.serve(Keys, (_, self) => ({ rows: self.state(Rows) }));
h.serve(View, (_, self) => {
const row = self.get(Keys, {}).rows.at("a");
return { n: self.derive([row], ([value]) => value.n) };
});
},
{ readyTimeout: 1_000 },
);
const view = host.hold(View, { id: 1 }, context);
await entered.promise;
await vi.advanceTimersByTimeAsync(1_000);
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.keys", state: "loading", since: 0 },
]);
gate.resolve();
await vi.waitFor(() => expect(view.state.value).toEqual({ kind: "live" }));
view.release();
await host.close(context);
} finally {
gate.resolve();
held.mockRestore();
}
});
it("reports a ring across hosts whose member there waits on a failed dependency (it shows failed here)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
@ -230,34 +293,62 @@ describe("parked_long (dependency model §7)", () => {
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
const { hold } = await peer.next("hold");
const error = { kind: "chord", error: { code: "unavailable", message: "app.notes is not served here" } } as const;
// The provider's object is parked there on a failed dependency (the ring's member back to this host waits too).
const error = {
kind: "chord",
error: { code: "unavailable", message: "waiting for app.c: not served" },
} as const;
peer.send({ kind: "failed", hold, cause: { kind: "waiting", error } });
await vi.advanceTimersByTimeAsync(60_000);
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(1_000);
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", state: "failed", since: 0 },
]);
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
expect(vi.getTimerCount()).toBe(0);
view.release();
await client.close(context);
});
it("is not reported for a local blocker: its own deadline bounds it", async () => {
const Slow = local("app.slow", { version: 1, params: { id: int() }, members: { n: int() } });
const { host, errors } = await loaded(
(h) => {
h.serve(Slow, (_, self) => {
self.open(
() => new Promise(() => {}),
() => {},
);
return { n: constant(1) };
});
h.serve(View, (_, self) => ({ n: self.get(Slow, { id: 1 }).n }));
},
{ readyTimeout: 1_000 },
);
it("is reported once per parking episode: again for an attempt that parks after leaving parking", async () => {
const { host, errors } = await loaded((h) => h.serve(View, (_, self) => ({ n: self.get(Missing, {}).n })), {
readyTimeout: 1_000,
});
const view = host.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
expect(errors.map(({ kind }) => kind)).not.toContain("parked_long");
expect(errors.map(({ kind }) => kind)).toEqual(["parked_long"]);
// A reload of its plugin replaces the parked attempt: a new episode, its clock from its own parking.
const changed = defineFacet((h) => h.serve(View, (_, self) => ({ n: self.get(Missing, {}).n })));
await host.reload([{ name: "@test/app", facet: changed }], context);
await vi.advanceTimersByTimeAsync(999);
expect(errors).toHaveLength(1);
await vi.advanceTimersByTimeAsync(1);
expect(errors[1]).toEqual({
kind: "parked_long",
object: "app.view(1)",
waitingFor: "app.missing",
state: "failed",
since: 5_000,
});
view.release();
await host.close(context);
});
it("is not reported for an attempt that leaves parking within readyTimeout: its timer stops", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
const { hold } = await peer.next("hold");
await vi.advanceTimersByTimeAsync(500);
bindHold(peer, hold, [{ text: "a" }]);
await vi.advanceTimersByTimeAsync(5_000);
expect(view.state.value).toEqual({ kind: "live" });
expect(errors).toEqual([]);
view.release();
await client.close(context);
});
});

35
packages/chord/test/next/host/object.test.ts
View file

@ -378,6 +378,39 @@ describe("parked on dependencies (dependency model §3)", () => {
top.release();
});
it("walks a diamond on one host once: a shared parked dependency passes a retry on once (audit A5)", async () => {
const Top = local("app.top", { version: 1, params: {}, members: { n: int() } });
const Left = local("app.left", { version: 1, params: { id: int() }, members: { n: int() } });
const Mid = local("app.mid", { version: 1, params: {}, members: { n: int() } });
let cores = 0;
const { host } = await loaded((h) => {
h.serve(Core, () => {
if (++cores === 1) throw new Error("bug");
return { n: constant(cores) };
});
h.serve(Mid, (_, self) => ({ n: self.get(Core, {}).n }));
h.serve(Left, (_, self) => ({ n: self.get(Mid, {}).n }));
h.serve(Top, (_, self) => {
const a = self.get(Left, { id: 1 }).n;
const b = self.get(Left, { id: 2 }).n;
return { n: self.derive([a, b], ([x, y]) => x + y) };
});
});
const top = host.hold(Top, {}, context);
await flushed();
const passed = vi.spyOn(Attempt.prototype, "retryDependencies");
try {
top.retry();
// Top, both lefts, and the shared mid once (its second visit through the other left is ignored).
expect(passed).toHaveBeenCalledTimes(4);
} finally {
passed.mockRestore();
}
await flushed();
expect([cores, top.n.value]).toEqual([2, 4]);
top.release();
});
it("fails every object of a cycle broken; none is retried", async () => {
const Ping = local("app.ping", { version: 1, params: {}, members: { n: int() } });
const Pong = local("app.pong", { version: 1, params: {}, members: { n: int() } });
@ -1573,7 +1606,7 @@ describe("reload: the pure swap (slice 7b §5)", () => {
describe("reload: a stuck routine call past the handover", () => {
fakeTimers();
it("fences an old routine call stuck past the handover: its transactions are refused", async () => {
it("an old routine call stuck past the handover: drain_timeout, the old attempt sealed, its commit refused (audit A12)", async () => {
const stuck = Promise.withResolvers<void>();
let late: Promise<unknown> | undefined;
const errors: HostError[] = [];

142
packages/chord/test/next/host/reference-keys.test.ts
View file

@ -15,6 +15,7 @@ import {
expectCollected,
failLive,
flushed,
holdNextCommit,
instancesOf,
loaded,
memoryStore,
@ -102,6 +103,8 @@ describe("re-binding when a new attempt goes live", () => {
?.pins;
expect(pins("a")).toBe(1);
fail();
// Its keys follow the failed instance until its admitted commits published (audit A10), then show the failure.
await delivered();
which = "b";
exposer.retry();
(await host.get(Exposer, {}, context)).release();
@ -560,3 +563,142 @@ describe("a released key leaves its document (final review)", () => {
await host.close(context);
});
});
describe("keys across a restart and a failed re-pin (Astra's coverage note)", () => {
const Keys = local("app.keys", { version: 1, params: {}, members: { notes: keyed(Schema) } });
/** Serves `Keys` over the persisted `Stored` (its handle kept in `handles`), with a native opened by `open`. */
const serveKeys = (handles: KeyedHandle<N>[], open: () => Promise<void>) =>
defineFacet((h) =>
h.serve(Keys, (_, self) => {
self.open(open, () => {});
const notes = self.state(Stored);
handles.push(notes);
return { notes };
}),
);
it("a commit admitted to a watched key, landing after a restart's drain, is in the key's last value", async () => {
const store = await memoryStore();
const handles: KeyedHandle<N>[] = [];
const host = createHost({ store, onError: () => {} });
await host.load([{ name: "@test/app", facet: serveKeys(handles, async () => {}) }], context);
const keys = await host.get(Keys, {}, context);
const change = (n: number) =>
handles.at(-1)!.change(
"k",
(draft) => {
draft.n = n;
},
context,
);
await change(1);
const watch = (await keys.notes.watch("k", context))!;
const calls = changesOf(watch);
const held = holdNextCommit(store);
const admitted = change(5);
await held.entered;
// The new code's native stays opening: the new instance is not live while the key holds its last value.
const opening = Promise.withResolvers<void>();
const reloading = host.reload([{ name: "@test/app", facet: serveKeys(handles, () => opening.promise) }], context);
await delivered();
held.release();
await admitted;
await reloading;
await delivered();
expect([watch.value, valuesOf(calls)]).toEqual([{ n: 5 }, [{ n: 5 }]]);
opening.resolve();
await vi.waitFor(() => expect(keys.state.value).toEqual({ kind: "live" }));
expect(watch.value).toEqual({ n: 5 });
await host.close(context);
await store.close(context);
});
it("a key's re-pin on a new instance that fails to load keeps its last value, one failed call; the next pin heals it", async () => {
const store = await memoryStore();
const seeded: KeyedHandle<N>[] = [];
const seed = createHost({ store, onError: () => {} });
await seed.load([{ name: "@test/app", facet: serveKeys(seeded, async () => {}) }], context);
await seed.get(Keys, {}, context);
await seeded[0]!.change(
"k",
(draft) => {
draft.n = 1;
},
context,
);
await seed.close(context);
// A new host: `Keys`' key is in the store only; the exposer re-exposes provider `b` (in memory) first.
const handles: KeyedHandle<N>[] = [];
let b: KeyedHandle<N> | undefined;
const providers = defineFacet((h) => {
h.serve(Keys, (_, self) => {
const notes = self.state(Stored);
handles.push(notes);
return { notes };
});
h.serve(Provider, (_, self) => {
b = self.state(Notes);
return { notes: b };
});
});
const exposing = (which: "keys" | "b") =>
defineFacet((h) =>
h.serve(Exposer, (_, self) => {
self.state(Own);
const notes = which === "keys" ? self.get(Keys, {}).notes : self.get(Provider, { id: "b" }).notes;
return { notes };
}),
);
const host = createHost({ store, onError: () => {} });
await host.load(
[
{ name: "@test/providers", facet: providers },
{ name: "@test/exposer", facet: exposing("b") },
],
context,
);
await host.get(Provider, { id: "b" }, context);
await b!.change(
"k",
(draft) => {
draft.n = 2;
},
context,
);
const exposer = await host.get(Exposer, {}, context);
const watch = (await exposer.notes.watch("k", context))!;
const calls = changesOf(watch);
// The cold provider's next read of the key fails once.
const persistence = (handles[0] as unknown as { slot: { persistence: { read(...args: unknown[]): unknown } } })
.slot.persistence;
const read = persistence.read.bind(persistence);
persistence.read = async () => {
persistence.read = read;
throw new Error("disk");
};
// A swap in place: the new instance re-exposes the cold provider's notes; the key's pin there fails to load.
await host.reload(
[
{ name: "@test/providers", facet: providers },
{ name: "@test/exposer", facet: exposing("keys") },
],
context,
);
await flushed();
expect(exposer.state.value).toEqual({ kind: "live" });
expect([watch.value, valuesOf(calls)]).toEqual([{ n: 2 }, [{ failed: expect.stringContaining("disk") }]]);
// The next instance pins it again, loaded this time: one frame heals it.
await host.reload(
[
{ name: "@test/providers", facet: providers },
{ name: "@test/exposer", facet: exposing("keys") },
],
context,
);
await vi.waitFor(() => expect(watch.value).toEqual({ n: 1 }));
expect(valuesOf(calls)).toEqual([{ failed: expect.stringContaining("disk") }, { n: 1 }]);
await host.close(context);
await store.close(context);
});
});

12
packages/chord/test/next/host/reference-source.test.ts
View file

@ -180,7 +180,7 @@ describe("a failed object's members", () => {
});
describe("the window of a live failure", () => {
it("members following another object's sources hold at once, while an admitted commit of the object runs", async () => {
it("every member follows on while an admitted commit of the object runs, then all hold (audit A10)", async () => {
const Counter = defineState("counter", { schema: obj({ n: int() }), version: 1, initial: () => ({ n: 0 }) });
const Persisted = defineState("own", {
schema: obj({ n: int() }),
@ -229,12 +229,14 @@ describe("the window of a live failure", () => {
failLive(attempted, exposer!, new ChordError({ code: "unavailable", message: "boom" }));
await provider.set(9, context);
await delivered();
expect([exposed.n.value, frames.map((call) => call.kind)]).toEqual([1, ["failed"]]);
// A re-exposed member shows one more change of its provider in the window.
expect([exposed.n.value, frames.map((call) => call.kind)]).toEqual([9, ["value"]]);
held.release();
await change;
await delivered();
// The object's own admitted commit still publishes once; then its member holds (the failure once).
// The object's own admitted commit still publishes once; then every member holds (the failure once).
expect(valuesOf(owned)).toEqual([{ n: 5 }, { failed: "boom" }]);
expect([exposed.n.value, frames.map((call) => call.kind)]).toEqual([9, ["value", "failed"]]);
await host.close(context);
await store.close(context);
});
@ -443,9 +445,9 @@ describe("a member the new attempt's variant lacks", () => {
two.retry();
(await host.get(Two, {}, context)).release();
await delivered();
// Its object's failure once, then the new variant's lack of it.
// The failed instance is followed until its admitted commits published (audit A10), when the retry's attempt
// loads already (no failure call for it); then the new variant's lack of it.
expect(calls).toEqual([
{ kind: "failed", error: expect.objectContaining({ code: "unavailable" }) },
{ kind: "failed", error: expect.objectContaining({ code: "invalid", reason: "member" }) },
]);
expect(() => n.value).toThrow('app.two (variant view) has no state member "n"');

30
packages/chord/test/next/host/staging.test.ts
View file

@ -1,12 +1,11 @@
// Tests of staging (slice 7b §2): a reload's setups and recorded factories; each kind of throw leaves the host on its
// old code; recorded declarations decide an object's fate; the host is installing meanwhile.
// old code; recorded declarations decide an object's fate.
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { int, local, method, obj, remote, str } from "../../../src/next/contracts/index.ts";
import { defineState } from "../../../src/next/documents/index.ts";
import { createHost, defineFacet, type FacetHost, type StoreSelf } from "../../../src/next/host/index.ts";
import type { ChordHost } from "../../../src/next/host/internal.ts";
import { defineJob } from "../../../src/next/jobs/index.ts";
import { defineRegistry } from "../../../src/next/registries/index.ts";
import { constant } from "../../../src/next/state/index.ts";
@ -184,33 +183,6 @@ describe("staging: recorded declarations decide the fate", () => {
});
});
describe("staging: the host is installing (model §1)", () => {
it("a request for a type the running code does not serve waits loading until the reload ends", async () => {
const Page = local("app.page", { version: 1, params: { id: int() }, members: { id: int() } });
const host = createHost({ onError: () => {} }) as unknown as ChordHost;
await host.load([{ name: "@test/app", facet: serveStore("v1") }], context);
let held: ReturnType<typeof host.hold<typeof Page>> | undefined;
const states: string[] = [];
// A request made while the reload stages (from a recorded factory: staging is synchronous).
const facet = defineFacet((h) => {
h.serve(Store, () => {
if (held === undefined) {
held = host.hold(Page, { id: 1 }, context);
states.push(held.state.value.kind);
}
return { label: constant("v2"), bump: async () => 0 };
});
h.serve(Page, ({ id }) => ({ id: constant(id) }));
});
await host.get(Store, {}, context);
await host.reload([{ name: "@test/app", facet }], context);
await flushed();
expect(states).toEqual(["loading"]);
expect(held!.state.value).toEqual({ kind: "live" });
await host.close(context);
});
});
describe("staging: declaration parity with an attempt (fix round 1)", () => {
it("rejects retryOn of an attempt-owned writable without changing the live host", async () => {
const T = local("astra.staging", { version: 1, params: {}, members: { n: int() } });

4
packages/chord/test/next/host/unbound.test.ts
View file

@ -60,7 +60,9 @@ describe("the index", () => {
// The others stay indexed until their own names are served.
index.wake(["app.b"]);
expect(b.log).toEqual(["wake"]);
expect(index.delete("app.a", a)).toBe(false);
// Woken out of the index: a later publication wakes none of them again.
index.wakeAll();
expect([a.log, b.log, c.log]).toEqual([["wake"], ["wake"], ["wake"]]);
});
it("ends the requests of a revoked principal only, and every request at the host's close", () => {

32
packages/chord/test/next/link/message.test.ts
View file

@ -115,11 +115,9 @@ const EVERY_KIND: Message[] = [
cause: { kind: "transient", error: { kind: "chord", error: { code: "unavailable", message: "m" } } },
retryIn: 60_000,
},
{ kind: "retry", hold: 300 },
// The retry operation it belongs to (fix round 1): an object reached twice handles it once.
{ kind: "retry", hold: 300, operation: incarnation },
// And the instance ids of the objects it passed through (fix round 2): a ring stops after one lap.
{ kind: "retry", hold: 300, operation: incarnation, through: [incarnation, incarnation] },
// With its hop budget (audit A5): the first host's, and the last one passed on.
{ kind: "retry", hold: 300, hops: 16 },
{ kind: "retry", hold: 300, hops: 1 },
{ kind: "loading", hold: 300 },
{ kind: "result", call: 3, value: '{"n":1}' },
{ kind: "result", call: 3 },
@ -141,7 +139,7 @@ const EVERY_KIND: Message[] = [
{ kind: "release", origin: 2, hold: 3 },
{ kind: "call", origin: 2, call: 1, hold: 3, method: 0, args: "[1]" },
{ kind: "abort", origin: 2, call: 1 },
{ kind: "retry", origin: 2, hold: 3 },
{ kind: "retry", origin: 2, hold: 3, hops: 16 },
{ kind: "bound", origin: 2, holds: [{ hold: 3, variant: 0, members: [], failed: [] }] },
{ kind: "state", origin: 2, holds: [] },
{
@ -438,16 +436,8 @@ describe("the binary codec", () => {
expect(() => binaryCodec.decode(new Uint8Array([1, 0, 1, 0xff]))).toThrow();
});
it("writes retry as its kind, hold, flags, operation (16 raw bytes) and objects passed through, loading as its kind and hold, and failed's retryIn as one more than it (0: absent)", () => {
expect([...binaryCodec.encode({ kind: "retry", hold: 300 })]).toEqual([11, 0xac, 0x02, 0]);
expect(binaryCodec.encode({ kind: "retry", hold: 300, operation: incarnation })).toHaveLength(4 + 16);
const through = [incarnation, incarnation];
expect(binaryCodec.encode({ kind: "retry", hold: 300, operation: incarnation, through })).toHaveLength(
4 + 16 + 1 + 32,
);
const flagged = binaryCodec.encode({ kind: "retry", hold: 300 });
flagged[3] = 4;
expect(() => binaryCodec.decode(flagged)).toThrow("an unknown retry flag");
it("writes retry as its kind, hold and hops, loading as its kind and hold, and failed's retryIn as one more than it (0: absent)", () => {
expect([...binaryCodec.encode({ kind: "retry", hold: 300, hops: 16 })]).toEqual([11, 0xac, 0x02, 16]);
expect([...binaryCodec.encode({ kind: "loading", hold: 1 })]).toEqual([12, 1]);
const cause = { kind: "broken", error: { kind: "chord", error: { code: "released", message: "" } } } as const;
const absent = binaryCodec.encode({ kind: "failed", hold: 0, cause });
@ -467,7 +457,7 @@ describe("the text codec", () => {
expect(text({ kind: "bound", holds: [{ hold: 0, variant: 1, members, failed: [] }] })).toBe(
'["b",[[0,1,[[0,[0,5]]]]]]',
);
expect(text({ kind: "retry", hold: 2 })).toBe('["T",2]');
expect(text({ kind: "retry", hold: 2, hops: 16 })).toBe('["T",2,16]');
expect(text({ kind: "loading", hold: 2 })).toBe('["l",2]');
const cause = {
kind: "transient",
@ -564,6 +554,14 @@ describe("the text codec", () => {
const hold = { kind: "hold", hold: 0, type: "t", version: 1, fingerprint: "0123456789abcdef", params: "{}" };
expect(() => checkMessage(hold, context)).toThrow("message schema");
});
it("refuses a retry without its hop budget, or with one outside 1..16 (audit A5)", () => {
const decode = (tuple: string) => textCodec.decode(new TextEncoder().encode(tuple));
expect(() => decode('["T",2]')).toThrow('a "T" message with 1 fields');
for (const hops of [0, 17]) {
expect(() => checkMessage({ kind: "retry", hold: 2, hops }, context)).toThrow("message schema");
}
});
});
describe("describeMessage", () => {

181
packages/chord/test/next/registries/record.test.ts
View file

@ -11,6 +11,7 @@ import {
type RegistryTable,
type TableItem,
} from "../../../src/next/registries/index.ts";
import type { Writable } from "../../../src/next/state/index.ts";
import type { NativeWatchCall } from "../../../src/watch.ts";
import {
delivered,
@ -411,104 +412,124 @@ describe("contributions", () => {
await host.close(context);
});
it("refuses a contribution closing a cycle with the registries' tables; one through a derive is cut and reported", async () => {
const A = defineRegistry<unknown, unknown>("app.a", { version: 1 });
const B = defineRegistry<unknown, unknown>("app.b", { version: 1 });
const Direct = local("app.direct", { version: 1, params: {}, members: {} });
const Late = local("app.late", { version: 1, params: {}, members: {} });
const Derived = local("app.derived", { version: 1, params: {}, members: {} });
const Back = local("app.back", { version: 1, params: {}, members: {} });
let lateAttempts = 0;
it("catches a ring of registries at its first composition: the contributors go live, the rest works (audit A2)", async () => {
const A = defineRegistry<unknown, number>("app.ring-a", { version: 1 });
const B = defineRegistry<unknown, number>("app.ring-b", { version: 1 });
const ToA = local("app.to-a", { version: 1, params: {}, members: {} });
const ToB = local("app.to-b", { version: 1, params: {}, members: {} });
const { host, errors } = await loadedPlugins({
"@app/cycles": (host) => {
host.serve(A, { compose: (table) => table.length });
host.serve(B, { compose: (table) => table.length });
host.serve(Direct, (_, self) => {
"@app/ring": (h, self) => {
h.serve(A, { compose: (table) => table.length });
h.serve(B, { compose: (table) => table.length });
h.serve(ToA, (_, self) => {
self.contribute(A, self.registry(B));
return {};
});
// Its first attempt is discarded (the factory throws), then, once `Direct` is live, it closes a cycle.
host.serve(Late, (_, self) => {
h.serve(ToB, (_, self) => {
self.contribute(B, self.registry(A));
if (++lateAttempts === 1) throw new Error("first attempt");
return {};
});
host.serve(Derived, (_, self) => {
self.contribute(
A,
self.derive([self.registry(B)], ([n]) => n as number),
);
self.contribute(A, "plain");
self.contribute(B, "plain");
},
});
const live = [await host.get(ToA, {}, context), await host.get(ToB, {}, context)];
expect(live.map((reference) => reference.state.value)).toEqual([{ kind: "live" }, { kind: "live" }]);
// A composes first: B composes for it and its contribution of A, read while A composes, is left out. A has its
// plain entry and B's value, B its plain entry.
expect([(await host.registry(A, context)).value, (await host.registry(B, context)).value]).toEqual([2, 1]);
expect(errors).toEqual([
{
kind: "contribution_failed",
registry: "app.ring-b",
plugin: "@app/ring",
type: "app.to-b",
error: expect.objectContaining({ code: "invalid", reason: "cycle" }),
},
]);
await host.close(context);
});
it("keeps a ring's later recompositions bounded: a change composes each registry once, then nothing (audit A2)", async () => {
const A = defineRegistry<unknown, number>("app.later-a", { version: 1 });
const B = defineRegistry<unknown, number>("app.later-b", { version: 1 });
const Ring = local("app.later-ring", { version: 1, params: {}, members: {} });
const Back = local("app.later-back", { version: 1, params: {}, members: {} });
const Extra = local("app.later-extra", { version: 1, params: {}, members: {} });
let composes = 0;
let extra: Writable<number> | undefined;
const sum = (table: RegistryTable<unknown>): number => {
composes++;
return table.reduce((total, { value }) => total + (typeof value === "number" ? value : 0), 0);
};
const { host, errors } = await loadedPlugins({
"@app/later": (h) => {
h.serve(A, { compose: sum });
h.serve(B, { compose: sum });
h.serve(Ring, (_, self) => {
self.contribute(A, self.registry(B));
return {};
});
h.serve(Back, (_, self) => {
self.contribute(B, self.registry(A));
return {};
});
h.serve(Extra, (_, self) => {
extra = self.writable(1);
self.contribute(B, extra);
return {};
});
},
"@app/back": (host) =>
host.serve(Back, (_, self) => {
});
await host.get(Ring, {}, context);
await host.get(Back, {}, context);
await host.get(Extra, {}, context);
const a = await host.registry(A, context);
const b = await host.registry(B, context);
const watches = [await a.watch(context), await b.watch(context)];
await flushed();
const before = composes;
extra!.set(5);
await flushed();
// B changed and composes first now: it reads A, whose contribution of B, read while B composes, is cut instead.
expect(composes - before).toBe(2);
expect([a.value, b.value]).toEqual([0, 5]);
expect(errors.map((error) => (error.kind === "contribution_failed" ? error.registry : error.kind))).toEqual([
"app.later-b",
"app.later-a",
]);
// Nothing composes on by itself.
await flushed();
expect(composes - before).toBe(2);
for (const watch of watches) await watch.stop();
await host.close(context);
});
it("catches a ring through a derive, and one declared within one factory, the same way (audit A2)", async () => {
const A = defineRegistry<unknown, number>("app.both-a", { version: 1 });
const B = defineRegistry<unknown, number>("app.both-b", { version: 1 });
const Both = local("app.both", { version: 1, params: {}, members: {} });
const { host, errors } = await loadedPlugins({
"@app/both": (h) => {
h.serve(A, { compose: (table) => table.length });
h.serve(B, { compose: (table) => table.length });
h.serve(Both, (_, self) => {
self.contribute(A, self.registry(B));
self.contribute(
B,
self.derive([self.registry(A)], ([n]) => n as number),
);
return {};
}),
});
const late = host.hold(Late, {}, context);
late.retry();
await flushed();
expect(late.state.value).toMatchObject({
kind: "failed",
cause: { kind: "broken", error: { code: "invalid", reason: "cycle" } },
});
// A reads B's derive, which reads A while A composes: that contribution is cut, A composes without the cycle.
expect((await host.registry(A, context)).value).toBe(2);
expect(errors.find((error) => error.kind === "contribution_failed")).toMatchObject({
registry: "app.b",
plugin: "@app/back",
error: { code: "invalid", reason: "cycle" },
});
late.release();
await host.close(context);
});
it("refuses a cycle declared within one factory: broken, invalid/cycle (Sol)", async () => {
const A = defineRegistry<unknown, number>("app.fix-a", { version: 1 });
const B = defineRegistry<unknown, number>("app.fix-b", { version: 1 });
const Both = local("app.direct-cycle", { version: 1, params: {}, members: {} });
const { host, errors } = await loadedPlugins({
"@app/cycle": (h) => {
h.serve(A, { compose: (table) => table.length });
h.serve(B, { compose: (table) => table.length });
h.serve(Both, (_, self) => {
self.contribute(A, self.registry(B));
self.contribute(B, self.registry(A));
return {};
});
},
});
expect(
errors.find((error) => error.kind === "activation_failed" && error.object === "app.direct-cycle"),
).toMatchObject({ failure: { kind: "broken", error: { code: "invalid", reason: "cycle" } } });
await host.close(context);
});
it("leaves no cycle edge from a contribution that never entered a table (Sol)", async () => {
const A = defineRegistry<number, number>("app.edge-a", { version: 1 });
const B = defineRegistry<number, number>("app.edge-b", { version: 1 });
const Failed = local("app.failed", { version: 1, params: {}, members: {} });
const Good = local("app.good", { version: 1, params: {}, members: {} });
const { host, errors } = await loadedPlugins({
"@app/a": (h) => {
h.serve(A, { compose: (table) => table.length });
h.serve(B, { compose: (table) => table.length });
h.serve(Failed, (_, self) => {
self.contribute(A, self.registry(B));
throw new Error("factory discarded");
});
h.serve(Good, (_, self) => {
self.contribute(B, self.registry(A));
return {};
});
},
});
expect(failures(errors)).toEqual(["app.failed: broken"]);
expect((await host.get(Both, {}, context)).state.value).toEqual({ kind: "live" });
await host.registry(A, context);
await host.registry(B, context);
const cut = errors.filter((error) => error.kind === "contribution_failed");
expect(cut.length).toBeGreaterThan(0);
for (const error of cut) expect(error).toMatchObject({ error: { code: "invalid", reason: "cycle" } });
expect(failures(errors)).toEqual([]);
await host.close(context);
});

205
packages/chord/test/next/remote/hold.test.ts
View file

@ -18,10 +18,12 @@ import {
expectCollected,
failLive,
fakeTimers,
flushed,
forEachCodec,
instancesOf,
linked,
PlacedNotes,
partValue,
RawPeer,
rawLinked,
rejection,
@ -436,7 +438,7 @@ forEachCodec(() => {
cause: { kind: "broken", error: { kind: "chord", error: offline } },
};
expect([await peer.next("failed"), await other.next("failed")]).toEqual([broken, broken]);
peer.send({ kind: "retry", hold: 0 });
peer.send({ kind: "retry", hold: 0, hops: 16 });
for (const consumer of [peer, other]) {
expect(await consumer.next("loading")).toEqual({ kind: "loading", hold: 0 });
expect((await consumer.next("bound")).holds[0]).toMatchObject({ hold: 0 });
@ -450,7 +452,7 @@ forEachCodec(() => {
await peer.next("failed");
hold(other, 0, Flaky);
await other.next("failed");
other.send({ kind: "retry", hold: 0 });
other.send({ kind: "retry", hold: 0, hops: 16 });
for (const consumer of [peer, other]) {
await consumer.next("loading");
await consumer.next("bound");
@ -461,8 +463,8 @@ forEachCodec(() => {
const { peer } = await rawLinked((h) => serveFlaky(h, { left: 0, broken: false }));
hold(peer, 0, Flaky);
await peer.next("bound");
peer.send({ kind: "retry", hold: 0 });
peer.send({ kind: "retry", hold: 9 });
peer.send({ kind: "retry", hold: 0, hops: 16 });
peer.send({ kind: "retry", hold: 9, hops: 16 });
await transported();
expect([peer.kinds(), peer.closed]).toEqual([["hello"], false]);
});
@ -479,7 +481,7 @@ forEachCodec(() => {
expect(failed.retryIn).toBeUndefined();
expect(server.activity.value.references).toBe(1);
expect(server.references(Halves).value).toEqual([]);
peer.send({ kind: "retry", hold: 0 });
peer.send({ kind: "retry", hold: 0, hops: 16 });
expect(await peer.next("loading")).toEqual({ kind: "loading", hold: 0 });
expect(await peer.next("failed")).toMatchObject({ hold: 0, cause: { kind: "waiting" } });
peer.close();
@ -519,9 +521,9 @@ forEachCodec(() => {
expect(await peer.next("failed")).toEqual(waiting("waiting for app.core: app.core: bug 1"));
// A retry passes on through the parked object to its broken dependency (one host's retry operation).
// (Its new attempt failed within the batch of changes the parked object checks once: no loading between.)
peer.send({ kind: "retry", hold: 0 });
peer.send({ kind: "retry", hold: 0, hops: 16 });
expect(await peer.next("failed")).toEqual(waiting("waiting for app.core: app.core: bug 2"));
peer.send({ kind: "retry", hold: 0 });
peer.send({ kind: "retry", hold: 0, hops: 16 });
await peer.next("loading");
expect((await peer.next("bound")).holds[0]).toMatchObject({ hold: 0 });
});
@ -794,9 +796,9 @@ async function flush(): Promise<void> {
/** A local type nobody serves. */
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
describe("retries across links: the hop budget (audit A5)", () => {
it.each([1, 9, 50])(
"quiesce after %i concurrent Retries round a waiting ring across hosts: each stops after one lap",
"quiesce after %i concurrent Retries round a waiting ring across hosts: each stops at its budget, no attempt started",
async (operations) => {
type P = { kind: "server" } | { kind: "worker" };
const Server = hostKind("app.retry-server");
@ -815,10 +817,12 @@ describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
hostAddress: () => Worker(),
variants: { reader: { state: { n: int() } } },
});
let retries = 0;
const hops: number[] = [];
let factories = 0;
const count = (link: Link): Link => ({
send: (bytes) => {
if (binaryCodec.decode(bytes).kind === "retry") retries++;
const message = binaryCodec.decode(bytes);
if (message.kind === "retry") hops.push(message.hops);
link.send(bytes);
},
onMessage: (listener) => link.onMessage(listener),
@ -843,11 +847,14 @@ describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
{
name: "@test/a",
facet: defineFacet<P>((h) => {
h.serve(LocalA, (_, self) => ({ n: self.get(RemoteB, {}).n }));
h.serve(RemoteA, (_, _principal, self) => ({
variant: "reader",
members: { n: self.get(LocalA, {}).n },
}));
h.serve(LocalA, (_, self) => {
factories++;
return { n: self.get(RemoteB, {}).n };
});
h.serve(RemoteA, (_, _principal, self) => {
factories++;
return { variant: "reader", members: { n: self.get(LocalA, {}).n } };
});
}),
},
],
@ -859,13 +866,14 @@ describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
name: "@test/b",
facet: defineFacet<P>((h) => {
h.serve(LocalB, (_, self) => {
factories++;
self.get(Missing, {});
return { n: self.get(RemoteA, {}).n };
});
h.serve(RemoteB, (_, _principal, self) => ({
variant: "reader",
members: { n: self.get(LocalB, {}).n },
}));
h.serve(RemoteB, (_, _principal, self) => {
factories++;
return { variant: "reader", members: { n: self.get(LocalB, {}).n } };
});
}),
},
],
@ -875,13 +883,16 @@ describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
await flush();
const a = server.hold(LocalA, {}, context);
expect(a.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
// More at once than an object remembers ids: `through` stops each at the object it passed through already.
const made = factories;
for (let i = 0; i < operations; i++) a.retry();
await flush();
// Each went round once: server → worker → server, then stopped at the object it started from.
expect(retries).toBe(2 * operations);
// Each passed round the ring with one hop less per host (16 from the first), never sent at 0.
const lap = Array.from({ length: 16 }, (_, i) => 16 - i);
expect(hops.toSorted((x, y) => y - x)).toEqual(lap.flatMap((hop) => Array(operations).fill(hop)));
await flush();
expect(retries).toBe(2 * operations);
expect(hops).toHaveLength(16 * operations);
// Parked objects only passed it on: no attempt started.
expect(factories).toBe(made);
} finally {
await server.close(context);
await worker.close(context);
@ -956,7 +967,7 @@ function reloadServer(server: Host<TestPrincipal>, setup: (host: FacetHost<TestP
}
describe("holds across a reload (slice 7b §6)", () => {
it("a swapped object keeps its holds: one [[r, value]] per changed member, never loading", async () => {
it("a swapped object keeps its holds: a fresh bound once the new instance is live, never loading (audit A9)", async () => {
const { server, client } = await linked((h) => serveShown(h, "v1"));
const shown = await client.get(Shown, {}, context);
await shown.bump(context);
@ -966,7 +977,8 @@ describe("holds across a reload (slice 7b §6)", () => {
await reloadServer(server, (h) => serveShown(h, "v2"));
await transported();
expect(label).toEqual([{ kind: "value", value: "v2", ops: [["r", "v2"]], retired: false }]);
expect(n).toEqual([]);
// The fresh bound carries every member: an unchanged one may show one frame (nothing lost or stale).
expect(valuesOf(n)).toEqual([1]);
expect(states).toEqual([]);
expect(await shown.bump(context)).toBe(2);
await server.close(context);
@ -1068,6 +1080,138 @@ describe("holds across a reload (slice 7b §6)", () => {
await server.close(context);
});
it("a swap sends each bound hold one fresh bound with the new instance's values, no state (audit A9)", async () => {
const server = createHost<TestPrincipal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
await server.load(
[{ name: "@test/app", facet: defineFacet<TestPrincipal>((h) => serveShown(h, "v1")) }],
context,
);
const [a, b] = memoryLinkPair();
server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b);
const SHOWN = wireNames(Shown, "v");
await peer.next("hello");
hold(peer, 0, Shown);
hold(peer, 1, Shown);
await peer.next("bound");
await transported();
peer.received.length = 0;
await reloadServer(server, (h) => serveShown(h, "v2"));
await transported();
expect(new Set(peer.kinds())).toEqual(new Set(["bound"]));
const entries = peer.received.flatMap((message) => (message.kind === "bound" ? message.holds : []));
expect(entries.map(({ hold }) => hold).sort()).toEqual([0, 1]);
for (const entry of entries) expect(SHOWN.named(entry.members)).toEqual({ label: "v2", n: 0 });
await server.close(context);
});
it("a swap during a full window: calls are answered by the old names until the fresh bound goes (audit A9)", async () => {
const Two = remote("app.two-variants", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: {
a: { state: { label: str() }, methods: { bump: method(undefined, int()), ping: method(undefined, int()) } },
b: { state: { label: str() }, methods: { ping: method(undefined, int()) } },
},
});
const serveTwo = (variant: "a" | "b", label: string) =>
defineFacet<TestPrincipal>((h) =>
h.serve(Two, (_, _principal, self) => {
const doc = self.state(N);
const ping = async () => 7;
const bump = async (ctx: Context) => {
await doc.change((draft) => {
draft.n += 1;
}, ctx);
return doc.value.n;
};
return variant === "a"
? { variant, members: { label: constant(label), bump, ping } }
: { variant, members: { label: constant(label), ping } };
}),
);
const server = createHost<TestPrincipal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
await server.load([{ name: "@test/app", facet: serveTwo("a", "v1") }], context);
const [a, b] = memoryLinkPair();
server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b, { ack: false });
const A = wireNames(Two, "a");
await peer.next("hello");
hold(peer, 0, Two);
await peer.next("bound");
// A second message in flight: the window (two) is full.
const changed = defineFacet<TestPrincipal>((h) =>
h.serve(Two, (_, _principal, self) => ({
variant: "a",
members: {
label: self.derive([self.state(N)], ([d]) => `n ${d.n}`),
bump: async () => 0,
ping: async () => 7,
},
})),
);
await server.reload([{ name: "@test/app", facet: changed }], context);
await peer.next("bound");
// Swaps in place while the window is full: the fresh bound waits for a slot, the hold answers by its old names.
await server.reload([{ name: "@test/app", facet: serveTwo("a", "v2") }], context);
await transported();
peer.send({ kind: "call", call: 0, hold: 0, method: A.methods.bump! });
expect(partValue((await peer.next("result")).value!)).toBe(1);
// The new variant lacks a method of the old names: the attempt refuses it.
await server.reload([{ name: "@test/app", facet: serveTwo("b", "v3") }], context);
await transported();
peer.send({ kind: "call", call: 1, hold: 0, method: A.methods.bump! });
expect(await peer.next("error")).toMatchObject({ error: { error: { code: "invalid", reason: "member" } } });
peer.send({ kind: "call", call: 2, hold: 0, method: A.methods.ping! });
expect(partValue((await peer.next("result")).value!)).toBe(7);
expect(peer.kinds().filter((kind) => kind === "bound")).toEqual([]);
peer.ack();
const fresh = await peer.next("bound");
expect(fresh.holds[0]).toMatchObject({ hold: 0, variant: wireNames(Two, "b").variant });
expect(wireNames(Two, "b").named(fresh.holds[0]!.members)).toEqual({ label: "v3" });
await server.close(context);
});
it("collects the swapped-out instance while the hold's fresh bound waits for a slot (Astra, audit A9 GC)", async () => {
const Type = remote("astra.audit-gc", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { main: { state: { n: int() } } },
});
let declared: Self<TestPrincipal> | undefined;
const facet = (n: number) =>
defineFacet<TestPrincipal>((h) =>
h.serve(Type, (_, _principal, self) => {
declared = self;
return { variant: "main", members: { n: constant(n) } };
}),
);
const server = createHost<TestPrincipal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
await server.load([{ name: "@astra/audit-gc", facet: facet(1) }], context);
const [a, b] = memoryLinkPair();
server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b, { ack: false });
await peer.next("hello");
hold(peer, 0, Type);
await peer.next("bound");
// A second bound in flight: the window is full.
await server.reload([{ name: "@astra/audit-gc", facet: facet(2) }], context);
await peer.next("bound");
const retired = new WeakRef(declared!);
declared = undefined;
await server.reload([{ name: "@astra/audit-gc", facet: facet(3) }], context);
await transported();
await flushed();
// Its fresh bound waits for an ack; the hold keeps the old names for calls, not the old instance.
expect(peer.kinds().filter((kind) => kind === "bound")).toEqual([]);
await expectCollected(retired);
peer.ack();
await peer.next("bound");
await server.close(context);
});
it("through a relay: a worker's reload reaches the tab as it reaches a direct consumer", async () => {
const { tab, workers, server } = await relayed();
const notes = await tab.get(PlacedNotes, { session: "S1" }, context);
@ -1144,7 +1288,7 @@ describe("holds across more than one swap (fix round 1)", () => {
describe("a hold after its object's live failure (final review)", () => {
const attempted = instancesOf(ChordObject, "attempt");
it("stops a member following another object's source at once (Sol, G1 own)", async () => {
it("follows a member re-exposing another object's source until the admitted commits published (audit A10)", async () => {
const Point = obj({ n: int() });
const A = local("app.foreign", { version: 1, params: {}, members: { x: Point } });
const B = remote("app.reexposing", {
@ -1168,17 +1312,18 @@ describe("a hold after its object's live failure (final review)", () => {
const ref = await client.get(B, {}, context);
const calls = changesOf(await ref.x.watch(context));
failLive(attempted, self!, new ChordError({ code: "unavailable", message: "boom" }));
// A's change lands after B failed: B's re-exposed member no longer follows it.
// A's change lands after B failed, in the window before B's admitted commits published: it shows (one more
// change), then the member holds with the failure.
foreign!.set({ n: 1 });
await delivered();
expect([ref.x.value, valuesOf(calls)]).toEqual([{ n: 0 }, [{ failed: "boom" }]]);
expect([ref.x.value, valuesOf(calls)]).toEqual([{ n: 1 }, [{ n: 1 }, { failed: "boom" }]]);
await client.close(context);
await server.close(context);
});
});
describe("a hold across a swap whose mapping fails (final review)", () => {
it("sends the member's failure and heals with the new source (Sol)", async () => {
it("gets the member's failure in its fresh bound and heals with the new source (Sol, audit A9)", async () => {
const W1 = remote("app.swap-failed", {
version: 1,
params: {},

5
packages/chord/test/next/remote/relay.test.ts
View file

@ -143,8 +143,9 @@ forEachCodec(() => {
const result = await raw.next("result");
expect(result).toMatchObject({ kind: "result", call: 5 });
expect(result.origin).toBeUndefined();
raw.send({ kind: "retry", hold: 7 });
expect(await worker.next("retry")).toEqual({ kind: "retry", origin: 1, hold: 7 });
raw.send({ kind: "retry", hold: 7, hops: 5 });
// Forwarded with its budget as it is: a relay visits no object.
expect(await worker.next("retry")).toEqual({ kind: "retry", origin: 1, hold: 7, hops: 5 });
});
it("give each downstream link its own origin, never reused", async () => {

10
packages/chord/test/next/remote/remote-object.test.ts
View file

@ -648,7 +648,7 @@ forEachCodec(() => {
await transported();
expect(notes.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } });
notes.retry();
expect(await peer.next("retry")).toEqual({ kind: "retry", hold, operation: expect.any(String) });
expect(await peer.next("retry")).toEqual({ kind: "retry", hold, hops: 16 });
bindHold(peer, hold);
await transported();
expect(notes.state.value).toEqual({ kind: "live" });
@ -794,7 +794,7 @@ forEachCodec(() => {
expect(errors).toEqual([]);
});
it("dropped and never bound again, stays parked loading: nothing fails, no parked_long without a link", async () => {
it("dropped and never bound again, stays parked loading: nothing fails, one parked_long (audit A11)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
@ -805,8 +805,10 @@ forEachCodec(() => {
peer.close();
await transported();
await vi.advanceTimersByTimeAsync(10_000);
// No remote attempt answers for it while its path has no link (fix round 1, Opus F2).
expect(errors).toEqual([]);
// A diagnostic whatever the blocker (audit A11): its path without a link too; nothing failed.
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view", waitingFor: "app.notes(S1)", state: "loading", since: 0 },
]);
expect(client.hold(View, {}, context).state.value).toEqual({ kind: "loading" });
});