sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat(chord): Chord v2 slice 7a, the dependency model

Browse source 
A reference is a request that outlives its provider: a missing provider or a version that does not fit leaves
it unbound, waiting for the name (failed { waiting }, a new FailureCause kind), local and remote alike. An
attempt parks until its dependencies are live, with no timer and no backoff, woken on every state change of a
dependency; its readyTimeout runs only for its own steps. Requests wait only for the host's publication (the
startup deadlock is gone); load resolves at the first outcomes of its host-lifetime objects. Getters wait through
waiting until their readyTimeout. Registry readers are waiting dependencies, woken at every usable value; a last
good value is dropped when a contribution it was built from leaves. Unbound holds count for keep-alive and are
revoked; retry passes on to failed blockers once per operation (operation id and visited list on the wire); the
dependency deadline is gone and parked_long reports a long wait on a remote attempt.
This commit is contained in:
Mario Zechner 2026-10-29 11:40:57 +00:00
parent babe30a7e6
commit 339847eb14
54 changed files with 3832 additions and 962 deletions
Download patch file Download diff file Expand all files Collapse all files

40
packages/chord/docs/next/benchmarks.md
View file

@ -633,3 +633,43 @@ p95, and swapping host files moves it without a pattern (code layout, not work o
`writable.set` 0.87 µs, the same value 0.011-0.016 µs and 0 B (the seal check now comes first: a set of an ended
attempt's writable throws whatever its value); one of 10 contributions set 0.71-0.78 µs, of 100 1.43-1.61 µs (one run
in six 3.4 µs on either variant tried: JIT).
## Slice 7a: the dependency model (2026-10-29)
Node 24.4.1, one CPU (`taskset -c 6`), runs alternating. HEAD is `babe30a7e`, extracted at a path of the same length
(`/home/badlogic/pi-h7`); the build in the worktree (`/home/badlogic/pi-7a`).
**Paths under every object** (`test/next/bench/calls.ts`), median per operation over nine bursts, heap allocated per
operation, three alternating runs each, HEAD → build:
| Case | HEAD | build |
| --- | --- | --- |
| local method call (its ctx) | 2.81-2.82 µs, 6,894 B | 2.80-2.82 µs, 6,894 B |
| commit through a watched derive | 10.43-10.70 µs, 15,310 B | 10.38-10.53 µs, 15,310 B |
| activation churn: get, live, release (an object with params and one dependency) | 58.5-59.9 µs, 43,716-43,719 B | 53.7-55.4 µs, 41,540-41,559 B |
| retained per live object with one dependency (10,000 kept, an ad-hoc probe) | 7,398-7,406 B | 6,931-6,936 B |
An activation whose dependencies are live at the check does not park: one check of its bindings, then its own steps a
microtask later (as HEAD's wait did). The first build parked every attempt with a dependency (a waiter list, a
`parked_long` record, a microtask per wake even outside parking) and cost 62.0-63.6 µs and 580 B more per activation;
now a wake outside parking is a flag, the first-outcome promise is made only for whoever waits (the start barrier),
and an object keeps no `Set` of waits and no settlement promise per attempt (R18's), hence the lower numbers.
**Links** (`links.ts`, three runs each): wire sizes, encodes, decodes, heap per message and retained heap per hold
(176.0 → 175.4-176.0 KiB) unchanged. Hold to live over two hops (mean of 100), binary: 1,000-item list 0.781-0.817 →
0.807-0.811 ms, 500 KB document 1.250-1.278 → 1.289-1.310 ms (+1-3%: the hold counted at its arrival, the index
check of the publication); text 1.545-1.580 → 1.521-1.573 and 3.299-3.379 → 3.317-3.393 ms. Commits to frames,
streamed appends, call round trips and reconnects within the runs' spread.
**Existing benchmarks**, HEAD → build: the derive diamond (`derives.ts`, three runs) 9.4-10.1 → 9.5-9.6 µs median,
p95 19.3-19.9 → 19.3-19.4 µs. Registries (`registries.ts`, three runs): `writable.set` 0.82-0.97 → 0.77-0.87 µs; one of
10 contributions set 0.72-2.71 → 0.70-1.06 µs; of 100 1.50-3.46 → 1.44-3.40 µs (one slow run per side: JIT);
allocation within 20 B. A registry's last good composition keeps the list of contributions it was built from: the
list is replaced (not spliced) when a contributor comes or goes, so a composition costs no copy.
**Fix round 1** (2026-10-29: revocation over paths, retry operations on the wire, the provider's own host-address
check for same-host remote requests, mapped values' provenance), `calls.ts`, three alternating runs, HEAD → build:
local call 2.81 → 2.79-2.82 µs (6,894 B both), watched derive commit 10.45-10.70 → 10.35-10.47 µs, activation churn
58.6-59.8 → 53.9-54.2 µs (43,716-43,720 → 41,559 B); retained per live object with one dependency 7,399-7,405 →
6,930-6,938 B. The address check runs only for a remote type whose consumer and provider `hostAddress` differ; a
retry's operation id is made per `ref.retry()`, and an object keeps its handled ids only once it was retried.

10
packages/chord/docs/next/dependencies/model.md
View file

@ -1,6 +1,9 @@
# Dependency resolution in Chord v2: one model (decided, 2026-10-29)
Status: proposal for the owner, not decided. Built in slice 7 with reload and rollout. Sources: the owner's direction
Status: decided (owner, 2026-10-29); **built for slice 7a** (§1-4 and §6-11 without what installs and reloads add:
§1's `installing`, §4's install-end and reload wakes, §3.9's reload handover and §5's removal, versions and rollout
are slice 7b's; folded into `design.md` §3, §5, §6, §7, §17, §21's start barrier, §22; `slice-7a.md`). The draft
history below is kept as it was. Sources: the owner's direction
("park until resolved"; "no difference between local and remote resolution: a link going is like a plugin
unloading"), the reviews (`/tmp/dep-mechanism-*.md`, `/tmp/dep-graph-*.md`, `/tmp/barrier-*.md`,
`/tmp/model/review{1,2}-*.md`), the code at pi `358f7d46a`. **Owner** marks an open decision.
@ -138,6 +141,7 @@ through, or recovers by its own backoff.
| any `loading`, none failed | `loading` |
| any `failed` | `failed { waiting, "waiting for X: <X's cause>" }`, `retryAt` undefined; the message keeps the nearest blocker and the root cause within the 500-character rule |
| one failed with `invalid/cycle` (a dependent off the ring) | `broken` with the same error: it would wait on a ring that never heals by itself |
| one failed with `forbidden` (an `authorize` refused it: authority, §3.6) | `broken` with the same error: only `retry()` or a reload helps (slice 7a fix round 1) |
| one `ended` (revoked, or its host closed) | `broken` with that error: the request cannot come back; `retry()` or a reload makes new requests (a guard: a dependency ends only with its holder's principal or host, so the dependent ends too; Fable) |
2. **The dependency check is the activation boundary:** once every dependency is `live`, the attempt runs its own
@ -152,7 +156,9 @@ through, or recovers by its own backoff.
failed, a new cause compared by value, ended; Opus: otherwise a ring of waiting attempts passes its causes round and
round), updating what it shows; it leaves parking when all are live. The factory never re-runs
while parked. Wakes are coalesced per object and run after the batch that caused them; a change during the
attempt's own steps or its teardown is remembered.
attempt's own steps or its teardown is remembered. (As built in 7a, with the parent's note 5: a change while the
parked attempt checks or loads its pinned keys is remembered and checked again; after the dependency check, during
the own steps, a change is ignored, §3.2's R18 rule.)
5. **Kept while parked:** its documents (the object's), its declarations and requests, its seal token; no natives.
Ended with the object (its last holder; a host-lifetime object when its plugin is removed or the host closes).
6. **`broken` only for the object's own code** (a throwing factory, a derive throwing at activation, hydration,

258
packages/chord/docs/next/design.md
View file

@ -124,23 +124,50 @@ export const Catalog = remote("catalog", { hostAddress: () => Server(), … });
- **`params`** (owner, 2026-10-21; was `input`) select the instance among objects of its type and are part of its
identity (section 5).
- **The caller computes the address** and sends it in the `hold`; a host that relays routes by that address alone and
never needs the contract or parses params. **The provider computes it again** from the params and refuses an object
that is not its own: broken `invalid/host_address` (misrouted holds are never forwarded). During a rollout the refusal
is handled like a mismatch (provider on another revision: park; same revision: fail; slice 7).
- **Where a reference made on a host, or a hold arriving on one of its links, goes** (first match; slice 4a):
never needs the contract or parses params. **The provider computes it again** from the params: a hold naming this
host whose params the provider's code places elsewhere (code skew) waits unbound `version` (`failed { waiting }`,
`invalid/host_address`, dependency model §1; slice 7a); a hold naming another host's address is relayed, never served
here (misrouted holds are never captured by a relay that serves the same name).
- **Every reference is a request** (dependency model, slice 7a): it has one holder (an attempt, a call, outside code,
a link's peer) and ends only when its holder ends it (release, a call settling, a getter giving up, the peer's link
or origin closing, revocation, the host closing), never because its provider is missing, went away or does not fit.
- **Where a reference made on a host, or a hold arriving on one of its links, goes**, by address alone (slice 4a;
slice 7a: no same-name check on the consumer's host):
| Case | Goes |
| --- | --- |
| the type is served here and its host address (computed here from the params) is this host's address | here |
| a reference made here (`host.get`/`hold`, `self.get`, `ctx.get`) to a served type placed on another host | on, as if not served here (next rows): worker S1 reaches `Notes { session: "S2" }` through the parent |
| an arriving hold for a served type whose params place it on another host | broken `invalid/host_address`; the address the hold names only routes it |
| the hold names an address of a kind this host has a route for | over that route's link (the address's params checked against the kind, then `authorize`) |
| a `local` type | here |
| a `remote` type whose host address (computed here from the params) is this host's address, served here or not | here |
| an arriving hold naming this host's address | here |
| the address (computed, or named by the hold) is of a kind this host has a route for | over that route's link (the address's params checked against the kind, then `authorize`) |
| otherwise (a remote type) | over the parent link; a hold that arrived *on* the parent link is never sent back to it: `unavailable` |
| no parent link: the host never had one | `unavailable` |
| no parent link: it is down (a drop) | waits on the parent path, `loading`, until the next link (design audit, 2026-10-27) |
A `hostAddress` or `authorize` that throws fails the hold broken with that error; an address whose params fail the
kind's check is broken `invalid/host_address`.
A `hostAddress` or `authorize` that throws fails the request broken with that error; an address whose params fail
the kind's check is broken `invalid/host_address`.
- **The publication gate** (slice 7a): a request made on this host, or a hold arriving on its links, before the
host's first publication (`load`'s batch of declarations) is not placed yet: it is `loading`, and the publication
places it. The gate lasts only for the synchronous setups, never for activation, so it cannot deadlock.
- **Then the selected provider is checked** (here: served, the version and fingerprint, the params, the provider's
own host-address check; over a path: at the provider host). Version checks never look at a same-name registration
elsewhere. What the check finds:
| The check finds | The request |
| --- | --- |
| a fitting provider | bound (to the object here, or to the hold over a path) |
| not published yet | unbound `unpublished`: `loading` |
| nobody serves the name at its address (or serves it as the other kind) | unbound `not_served`: `failed { waiting }` (`unavailable` "X is not served here") |
| no fitting version, a fingerprint that differs at the same version, a host address the provider computes elsewhere | unbound `version`: `failed { waiting }` (`contract_mismatch`, `invalid/host_address`) |
| a registry served, but its first `compose` (or the reader's version mapping) threw | unbound `compose`: `failed { waiting }` (§17) |
| the path has no link (a drop, a route resolving) | on the path: `loading` |
| `authorize` refused, invalid params, a cycle | `failed { broken }`, never indexed |
The unbound reasons that wait for a name are indexed under it on the host that decided them: the consumer's host
for a request made there, the provider's host for a hold that arrived over a link (kept refused there; a `retry`
re-runs its start). A publication serving a name (in 7a: `load`'s) wakes its requests, which run their check again
(a refused hold sends `loading`, then `bound` or `failed`); no new message kind. Index entries go when their request
ends (GC-tested). The consumer never re-holds a request by itself: no backoff, no polling.
### The low-level host API
@ -203,16 +230,16 @@ parent link closes. `resolve` returns the link before the worker has loaded (the
| Call | Meaning |
| --- | --- |
| `createHost<P>({ address?, principal?, ...options })` | with an address: addressable, serves only objects placed there, acts as `principal` on its own host; without: a client. `P` is the app's principal type |
| `load([{ name, facet }], context)` | facets with their plugin names (the plugin's `package.json` name, npm style; duplicates and invalid names are refused); runs every facet's setup, then waits for the start barrier (section 21); a throwing setup rejects and closes the host; runs once. Loaders are adapters; base code passes its core facet the same way |
| `load([{ name, facet }], context)` | facets with their plugin names (the plugin's `package.json` name, npm style; duplicates and invalid names are refused); runs every facet's setup, publishes all of them in one batch (types and registries served, registries composed once, contributions; requests waiting for the publication are placed after it), then waits for the start barrier (section 21: the first outcomes of its host-lifetime objects, slice 7a); a throwing setup rejects and closes the host; runs once. Loaders are adapters; base code passes its core facet the same way |
| `connect(link, { principal, trusted?, codec? }, context)` | the link to the parent, at most one open; once it closed (a drop), `connect` takes the next, and every remote reference over the parent is held again on it. `principal` = who the parent is |
| `accept(link, { principal, trusted?, codec? }, context)` | a link from a child; holds and calls on it act as `principal` |
| `route(Kind, { authorize, resolve }, context)` | how to reach hosts of a kind (below); called before `load`, once per kind; work arriving on its links runs under `context` |
| `get(Type, params, context, { readyTimeout? })` | a reference once its object is live (section 7) |
| `get(Type, params, context, { readyTimeout? })` | a reference once its object is live, waiting through `loading`, `failed { transient }` and `failed { waiting }` up to its `readyTimeout` (section 7) |
| `hold(Type, params, context, { readyTimeout? })` | a reference at once that follows its object until released (section 7) |
| `close(context)` | closes the links (children, routes, then the parent), then ends every object at once and tears them down, the last to go live first (reason `close`); rejects with an `AggregateError` of the native closes that threw |
| `revoke(r, context)` | revocation (section 4; slice 4b) |
| `root(fn, context, { principal? })` | a call scope for code outside objects (a Durable hook, a CLI): `fn` gets a handler's `ctx`, sealed when `fn` settles, `context` aborts or the host closes (slice 8 D3, section 8) |
| `registry(R, context)` | registry `R`'s composed value for code outside objects; waits for `load`, rejects when it has none (slice 8, section 17) |
| `registry(R, context, { readyTimeout? })` | registry `R`'s composed value for code outside objects once it is usable; waits for the publication, then while it has none up to its `readyTimeout`, then rejects with the last error (slice 8, 7a, section 17) |
**Options of `createHost`:** `address`, `principal`, `principalKey` (section 4; the same function on every host),
`telemetry` (section 29), `onError`, `closeTimeout`, `readyTimeout`, `heartbeat: { ping?, close? }`; later `revokedBy`
@ -221,7 +248,7 @@ and closed by the app with `store.close(ctx)` after `host.close`; the host never
background, as a tagged `HostError`: `listener_error` (a watch listener threw), `delivery_failed` (a frame delivery
threw), `activation_failed { object, failure }`, `close_failed { object, native, error }`, `close_timeout { object,
native }`, `drain_timeout { object }`, `protocol_error { error }`, `resolve_failed { address, error }`; later routine
errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), the registries' `registry_conflict { registry, message, plugins }`, `registry_failed { registry, error }`, `contribution_failed { registry, plugin, type?, error }` and `contribution_refused { registry, plugin, type?, version }` (each once per episode, slice 8), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6) and `storage_failed { error }`
errors (slice 5), `derive_failed { object, member?, error }` (a derive's compute threw, once per failure episode, slice 6a), the registries' `registry_conflict { registry, message, plugins }`, `registry_failed { registry, error }`, `contribution_failed { registry, plugin, type?, error }` and `contribution_refused { registry, plugin, type?, version }` (each once per episode, slice 8), `object_failed { object, failure }` (a live object failing, slice 6a, `broken`; `transient` when a `self.follow` source failed, slice 6b R4; `failure` where the value is a `FailureCause`, `error` otherwise, 2026-10-27), `job_interrupted { object, job, id, restarts }` (a persisted run ended at the restart limit, slice 6b D6), `parked_long { object, waitingFor, since }` (an attempt parked longer than its `readyTimeout` on a remote attempt that is only `loading`, once per parking episode: a diagnostic of a ring across hosts started at the same moment, slice 7a, §6) and `storage_failed { error }`
(slice 6a), which every app must handle: Chord never ends the process,
the app exits with its own code for its supervisor or reopens.
@ -230,7 +257,7 @@ the app exits with its own code for its supervisor or reopens.
| Timeout | Bounds | Host default | Override |
| --- | --- | --- | --- |
| `closeTimeout` | a native's `close` (then reported as a possible leak; the rest continues); the drain of an ending object's running handlers (callers are rejected at once) | 30 s | per native: `self.open(open, close, { closeTimeout })` |
| `readyTimeout` | an activation attempt (its wait for the previous instance's teardown included); a getter's whole wait; how long a call made while its reference is loading waits (the park deadline) | 60 s | per `get`/`hold`: `{ readyTimeout }` |
| `readyTimeout` | an activation attempt's own steps (its wait for the previous instance's teardown included; paused while the attempt is parked on its dependencies, slice 7a); a getter's whole wait; how long a call made while its reference is loading waits (the park deadline) | 60 s | per `get`/`hold`/`registry`: `{ readyTimeout }` |
The park deadline's per-type override is not built (only if a real case needs it). Timeouts of later slices
(routine handover 10 s, job `finish` retry backoff 1 s to 60 s, rollout install 60 s, provider-side `linger` per type)
@ -251,7 +278,9 @@ bounded by the app.
- **Worker keep-alive** works as in p2: the worker counts, the app's supervisor decides. **`host.activity`** (slice 4b;
owner, 2026-10-24) is a JSON `Watchable` of `{ references, calls, jobs, lingering }`: holds arriving over links,
method calls running on this host, running jobs, provider-side lingering objects. Counters are kept where the work
happens; a change marks it dirty and it is published at most once per microtask turn with changes. Core code
happens; a change marks it dirty and it is published at most once per microtask turn with changes. A hold placed
here counts from its arrival to its end, bound or not (one waiting for a missing provider keeps the worker alive;
slice 7a); a hold this host relays counts at its destination; `host.references` lists bound references only. Core code
reports it plus Durable tasks (the core `Harness`); the server's supervisor stops the worker when all are zero for
the idle time (p2: 5 minutes, checked every second). Chord provides the counts, no idle logic.
@ -521,9 +550,10 @@ host.serve(Notes, ({ session }, principal, self) =>
- **Compatibility (strict, like documents; R6):** a hold carries only the consumer's contract `version` and
fingerprint, never a description. The provider binds when both equal its current ones, or (slice 7) when the
version is one it accepts and the fingerprint equals the one computed from its own history (section 27). Every
contract change, additive ones included, bumps `version`. Anything else is `contract_mismatch` (or `updating` during
a rollout, section 21). The fingerprint covers params, variants and members, not the host address. A consumer whose contract
does not fit gets a mismatch on its own binding; the provider object and its other bindings are not affected.
contract change, additive ones included, bumps `version`. Anything else leaves the request unbound `version`:
`failed { waiting }` with `contract_mismatch` (slice 7a: not the consumer's fault; a publication that fits wakes it,
§3). The fingerprint covers params, variants and members, not the host address. A consumer whose contract does not
fit waits on its own request; the provider object and its other bindings are not affected.
- **Variants:** a member name has the same kind (state or method) in every variant that has it, and a method the same
signature (arguments and result) in every variant; state schemas may differ per variant (redaction). Enforced at
the type level in `remote(...)` and at declaration at runtime. A remote factory returns one variant, `{ variant,
@ -567,7 +597,7 @@ host.serve(Notes, ({ session }, principal, self) =>
### Lifecycle
```
(get) → activating → live → ending → ended
(get) → activating (parked on its dependencies → own steps) → live → ending → ended
└→ failed (backoff) → activating …
live → failed (a runtime failure of the owner, slice 6a: broken, by reload or retry only)
```
@ -584,11 +614,11 @@ host.serve(Notes, ({ session }, principal, self) =>
- **Activation** (one fresh attempt each time, bounded by `readyTimeout`, its wait for the previous instance's
teardown included): the factory runs; singleton documents take `initial()` and, on the object's first activation,
persisted documents load and its tables' definitions are checked and migrated (slice 6a; the deadline paused during
eager migration work); references are bound (before the
previous attempt's are released, so a dependency held only by it never ends in between) and waited for until live
and hydrated; natives open one after another; the returned members are checked (section 5); derives compute;
routines start (slice 5). The object goes live when all of that succeeded. Calls wait behind its **gate** until
then.
eager migration work); its requests are made (`self.get`, `self.registry`, keyed pins), and the attempt is
**parked** until every one is live and hydrated (below; the deadline paused); then its own steps run under the
deadline: natives open one after another, followed sources attach, the returned members are checked (section 5),
derives compute; routines start (slice 5). The object goes live when all of that succeeded. Calls wait behind its
**gate** until then.
- **The seal fence** (owner, 2026-10-26): every attempt has a seal token (a job run invocation its own, for its state
changes and outcome). After the seal, new work through the attempt's handles is refused at use (`self.tx`, `ctx.get`,
a job or routine start); commits are checked when they enter the store's commit step (sealed: rejected `released`,
@ -609,18 +639,36 @@ host.serve(Notes, ({ session }, principal, self) =>
with the cause, state members keep their last values, natives close, documents stay on the object; recovery by
reload or `ref.retry()`; live dependents stay live. The members freeze only after the commits admitted before the
seal adopted and published once, and a retry's factory waits for that (2026-10-27, slice 6a build reviews).
- **Waiting for dependencies (R18):** an object whose references are not live yet stays `loading` and goes live as
soon as they are (bounded by `readyTimeout`; a link drop while a dependency is still loading is no failure: the
reference re-holds on the next link and the dependent keeps waiting, design audit 6, 2026-10-27). A dependency
going live while the dependent still activates is remembered and retries it at once if that attempt fails. A
dependency that fails (`broken` or `transient`), or does not become
live in time, fails the dependent **`transient`** with the dependency's cause inside, never `broken`: the cause is
the dependency's. The dependent keeps its references while failed, and Chord installs their state as an automatic
retry trigger: the moment the dependency goes live, the dependent is retried at once (the remaining backoff is
cancelled; the backoff stays as a safety net). Factory authors write `self.get` and know nothing of this. When a
dependent's last reference goes, it ends and releases its dependencies. After `live`, a dependency leaving `live`
does not change the dependent: it stays live, and its calls that use the dependency get its error (or park while it
is loading). Only R18 failures keep references; any other failure releases them.
- **Parked on its dependencies** (dependency model §3, slice 7a; supersedes R18's dependency deadline and backoff): a
dependency is what an attempt declares and needs before it goes live: `self.get` (local or remote), `self.registry`,
keyed pins (`at(key)` on another object's keyed member); `self.follow` is the attempt's own step, not a dependency.
While any dependency is not live (and hydrated), the attempt is parked and its object shows:
| Its dependencies | The object shows |
| --- | --- |
| any `loading`, none failed | `loading` |
| any `failed` | `failed { waiting }`, `retryAt` undefined, error `unavailable` "waiting for X: <root cause>" (X the first failed one in declaration order, the root cause found through its message's "waiting for …: "; always these two, so a ring of waiting attempts settles after one round; within 500 characters, X's label clipped first, fix round 1) |
| one failed with `invalid/cycle` (a dependent off the ring) or `forbidden` (an `authorize` that refused it: authority, M§3.6) | `broken` with that error: nothing heals it but `ref.retry()` or a reload |
| one `ended` (revoked, its host closed) | `broken` with that error: the request cannot come back |
**No timer runs while parked:** no dependency deadline, no backoff; the attempt's `readyTimeout` deadline is paused
(as eager migrations pause it) and resumes with the budget left; the identity backoff stays for the object's own
`transient` failures only. **Wakes:** the parked attempt checks again at every change of a dependency (live,
loading, failed, a new cause compared by value, ended, and its first outcome, below), coalesced per object after the
batch that caused them (a change while it checks or loads pinned keys is remembered); it never re-runs its factory.
A parked attempt keeps its documents, declarations, requests and seal token, and holds no natives. **The dependency
check is the boundary:** once all are live (and the pinned keys loaded), the own steps run; a dependency that leaves
`live` afterwards changes nothing (R18 after live: the object goes live; its calls through that reference meet its
state). **Retry passes on:** `ref.retry()` (or a `retry` message) on an object parked `failed { waiting }` passes on
to its failed dependencies, once per retry operation: the operation (a uuidv7) travels in the `retry` message
(`operation`) with the instance ids of the objects it passed through (`through`, added hop by hop, at most 32), and
an object already in `through` drops it, so a ring across hosts stops after one lap however many retries run at
once; an object also remembers the last few ids it handled, so a diamond retries it once. A `broken` or
`transient` one starts an attempt, a parked one passes it on, a request waiting unbound in an index is left as it is
(an event wakes it), a refused hold over a link re-runs its start. A `retryOn` wake while parked does nothing. When
a parked object's last reference goes, it ends at once (it is not live: no `linger`) and releases its requests.
`broken` is only for an object's own code, cycles and authority; no failure keeps references (the R18 retry trigger
is gone: parking replaces it).
- **Dependency cycles are rejected (Q5):** every object instance has a uuidv7 instance id, minted at its first
attempt; an object that restarts gets a new one, so an old chain never matches (slice 4b, the first user; D4,
owner, 2026-10-24). On one host, at
@ -629,23 +677,32 @@ host.serve(Notes, ({ session }, principal, self) =>
(bounded; ignored from untrusted links): the provider rejects the hold when the instance being activated is in the
chain (slice 4b; on one host slice 1; at most 32 entries; locally created objects inherit the activating chain). A
cycle fails the activation `broken invalid/cycle` with the cycle as cause ("A → B → A"), and every other activating
object on it the same way; no references are kept. A dependency failing with `invalid/cycle` fails its dependent
**broken** with the same error, an exception to R18 (owner, 2026-10-24: otherwise every backoff retry hits the same
cycle); dependents that are not on the cycle break too, and a reload or `ref.retry()` recovers them. Checked again
after a reload. Cost: O(chain) per declared reference, only during activation.
**Known limitation** (owner, 2026-10-24): a chain is fixed when its object is created, so a cycle entering an object
that is already activating is not detected. Example: A on the server and B on a worker start at the same moment, A
declares B and B declares A; the holds carry `[A]` and `[B]`, neither finds the other, both fail `transient` at
`readyTimeout` and retry with backoff (later attempts usually come in order and detect it). A probe along waiting
holds was not built.
object on it the same way; no references are kept. A dependency failing with `invalid/cycle` makes its dependent
**broken** with the same error (owner, 2026-10-24: it would wait on a ring that never heals); dependents that are not
on the cycle break too, and a reload or `ref.retry()` recovers them. The walk on one host follows the waiting edges
of activating and parked attempts (exact). Checked again after a reload. Cost: O(chain) per declared reference,
only during activation.
**The gap: rings across hosts started at the same moment** (owner, 2026-10-29: no probe): a chain is fixed when its
object is created, so a cycle entering an object already activating is not detected. Example: A on the server and B
on a worker start at the same moment, A declares B and B declares A; the holds carry `[A]` and `[B]`, neither finds
the other, and without a dependency deadline both stay parked. It is made visible: an attempt parked longer than its
`readyTimeout` on a remote attempt that is itself only `loading` (its hold on a link; the clock and `since` start
when the hold gets on a link and stop while the path has no link)
is reported once per parking episode as
`parked_long { object, waitingFor, since }`; nothing fails or retries. A cycle is a plugin bug; **the guide's four
rules** for plugin authors and agents (host README): (1) a factory declares only what the object needs, never a type
that needs it back; (2) back-references and occasional lookups go into methods and routines (`ctx.get`); (3) data
flows up through state and registries, not mutual `self.get`; (4) host-lifetime objects on the server never declare
worker types in their factory.
- **Failed:** a broken part fails the whole object: a throwing factory, document hydration or `migrate`, a failed
open, a derive that throws on its first computation during activation, or a dependency that did not become live.
After the object is live, a throwing derive fails only its own member (`broken`, W1). The cause is tagged and decided by Chord from where it
failed:
open, a derive that throws on its first computation during activation. After the object is live, a throwing derive
fails only its own member (`broken`, W1). The cause is tagged and decided by Chord from where it failed:
```ts
cause: { kind: "transient"; error: ChordError } // an open threw, a reference was unreachable: automatic backoff
| { kind: "broken"; error: ChordError } // factory, derive, hydration or migrate threw: no automatic retries
cause: { kind: "transient"; error: ChordError } // the object's own effect failed (an open, a follow): backoff
| { kind: "broken"; error: ChordError } // its own code (factory, derive, hydration, migrate), a cycle, authority
| { kind: "waiting"; error: ChordError } // not its fault (slice 7a): no provider, no fitting version, a
// dependency failed; retryAt undefined, woken by an event
```
Only an `open` may choose `broken` itself (`throw broken(error)`, e.g. invalid configuration). **Transient** causes
@ -718,13 +775,17 @@ location are dropped.
| a call | `await ctx.get(Type, params, { readyTimeout? })` | when the handler settles |
| outside | `await host.get(Type, params, context, { readyTimeout? })` / `host.hold(Type, params, context, { readyTimeout? })` | `release()` |
- **`host.get`** (and `ctx.get`) resolves with a reference once its object is live and **hydrated** (every state member
of its variant has a value). A broken failure rejects at once; through transient ones it waits up to its
`readyTimeout`, then rejects with the last failure's error; a getter that gives up releases its reference.
**`host.hold`** returns a reference at once that follows its object until released (loading, live, failed and
retried as its object is, loading again while its link is down) and never rejects; invalid params (checked at once, also before `load`: design audit 7, 2026-10-27) and a local type
not served here throw at once (before `load` resolved, these fail the reference broken once it did); its `variant` is
`undefined` until its first binding (slice 3b). There is no `state: false` option (C8) and no `optional` flag.
- **`host.get`** (and `ctx.get`, `host.registry`) resolves with a reference once its object is live and **hydrated**
(every state member of its variant has a value). It waits while the reference is `loading`, `failed { transient }`
or `failed { waiting }`, up to its `readyTimeout` (default 60 s, the caller's choice), then rejects with the last
cause; only `failed { broken }` rejects at once (owner, 2026-10-29, decision "Getters wait through `waiting`"). A
getter that gives up releases its reference. **`host.hold`** returns a reference at once that follows its object
until released (loading, live, failed and retried as its object is, loading again while its link is down) and never
rejects; only invalid params throw (checked at once, also before `load`: design audit 7, 2026-10-27). A type not
served where it goes is `loading` before the host's publication and `failed { waiting }` after it, before and after
`load` alike (§3; slice 7a). Its `variant` is `undefined` until its first binding (slice 3b). A UI holds and shows
`ref.state`; code that needs the object now gets it, with the patience it passes. There is no `state: false` option
(C8) and no `optional` flag.
- **Narrowing by variant:** `ref.variant` is the discriminant (C15: no `inVariant`/`expectVariant`); `RefIn<C, V>`
narrows to some variants (strict unions, never weakened).
- **Members across attempts** (2026-10-27, slice 6a build reviews): on the same host a reference's state member
@ -741,7 +802,8 @@ location are dropped.
```ts
| { kind: "loading" } | { kind: "live" }
| { kind: "failed"; cause: FailureCause; retryAt: number | undefined } // retryAt undefined: broken, no automatic retry
| { kind: "failed"; cause: FailureCause; retryAt: number | undefined } // retryAt: a transient cause's backoff;
// undefined for broken and waiting
| { kind: "ended"; error: ChordError } // released by its holder, or its object ended
```
@ -749,13 +811,16 @@ location are dropped.
| --- | --- | --- |
| link drop | park until reconnect | readable (`loading`) |
| object loading or restarting | park until live | readable (`loading`) |
| rollout: provider ahead or installing | park until installed, with a deadline (slice 7) | readable |
| rollout: the provider host installing a revision the hold does not fit (slice 7b) | park until installed, bounded | readable (`loading`) |
| unbound (not served, another version) or parked on a failed dependency | reject at once with the cause | `failed { waiting }` (slice 7a) |
| object `failed` | reject at once with the cause | `failed` with cause and `retryAt` |
- A failed reference stays alive: while held, Chord retries per the backoff; `ref.retry()` tries at once (a Retry
button; over a link the `retry { hold }` message, which also re-runs a refused hold's start: authorize, the host address,
binding; design audit 2026-10-27); success goes `loading`, then `live`. A provider tells a pending
hold's consumer each failure (`failed`, with the rest of its backoff window) and the next attempt (`loading`).
- A failed reference stays alive: while held, Chord retries a `transient` failure per the backoff and wakes a
`waiting` one at the event it waits for; `ref.retry()` tries at once (a Retry button; over a link the `retry
{ hold }` message, which also re-runs a refused hold's start: the host address, binding; design audit 2026-10-27;
on a parked object it passes on to the failed dependencies, §6); success goes `loading`, then `live`. A provider
tells a pending hold's consumer each failure (`failed`, with the rest of its backoff window for a transient one),
each new cause of an object parked on a failed dependency, and the next attempt (`loading`).
- **Parked calls** (slice 3b): calls made while the reference is loading are queued in order, each waiting at most the
reference's `readyTimeout` from the moment it was made (never renewed), then fail `unavailable/park_timeout`; they
are sent in order when it goes live (a new call never overtakes them) and rejected with the cause when it fails.
@ -1490,7 +1555,7 @@ export default defineFacet((host, self) => { // `self` (contributions) a
- A plugin's per-host setup, **declarations only** (owner, 2026-10-26): `host.serve`, `self.contribute` (slice 8),
`host.references`; no natives, no documents (C16 accepted), no routines, no jobs. Everything a plugin does at runtime
is done through objects: shared or persisted state and standing behaviour live in a `local` store object with empty
params (host lifetime, its own failure domain, R18 recovers its dependents), as p2 does (its 63 facets take only
params (host lifetime, its own failure domain; its dependents park while it is failed and go on when it recovers), as p2 does (its 63 facets take only
`host`). A facet cannot fail after `load`.
- `host.serve(Type, factory, { linger? })`: `linger` (milliseconds, 0 by default; slice 3b) keeps a live object with
params alive that long after its last reference, so a new reference within that time reuses it with its documents
@ -1524,8 +1589,11 @@ const router = self.registry(Http); // a declared dependency; read rou
live or ends, when a dynamic contribution changes). It always delivers a value with its fallback built in (e.g. the
first by plugin name wins) and reports conflicts through `report` (owner notices and telemetry events); readers get
a plain value and never handle conflicts. If `compose` throws, the failure is reported and its
watches get a `failed` call, while `.value` keeps the last good composition (slice 8); only before any good
composition do readers get the error. Work that started earlier keeps the value it read (a pinned Durable phase keeps
watches get a `failed` call, while `.value` keeps the last good composition (slice 8), **but only while every
contribution it was built from is still in the table** (dependency model decision 5, slice 7a): when one leaves (its
contributor ended, failed or restarted), the last good value is dropped and `.value` throws until `compose`
succeeds, so it never hands out a native whose contributor ended; only before any good composition, or after such a
drop, do readers get the error. Work that started earlier keeps the value it read (a pinned Durable phase keeps
its old table). Helpers: `keyedEntries(table, entriesOf, report)` (a map, the first by plugin name wins, a duplicate
reported), `orderedEntries(table, itemsOf, orderOf?)`; both keep each item's plugin (named so beside the contracts'
`keyed`, slice 8 build).
@ -1538,7 +1606,8 @@ const router = self.registry(Http); // a declared dependency; read rou
(`compose`, version, `accepts`); other plugins' tokens only identify it by name and types (contribute and read,
never bring their own `compose`). A reload of the defining plugin replaces the declaration atomically and the table
recomposes. Without a declaration (the defining plugin is not loaded) the registry has no composed value: readers
get an error like a failed reference; contributions wait in the table.
wait `failed { waiting }` (`unavailable/undeclared`), like a request for a type nobody serves; contributions wait in
the table.
- **Versions (Q8, the owner-adapter pattern):** a contribution carries the registry version its plugin was built
against; older ones are adapted with the defining facet's pure `accepts` when the table is built. A contribution
without a matching version or adapter (older without adapter, or newer) is left out of the table and reported (an
@ -1548,12 +1617,16 @@ const router = self.registry(Http); // a declared dependency; read rou
use site catches synchronous throws and rejections when it calls an entry, reports naming the contributor (the
composed table keeps each entry's plugin name), and continues with the other entries. Durable already does this for
extensions (`extension_error`; a throwing tool becomes an error result); `compose` reports through `report`.
- **Slice 8's rules** (decisions "Slice 8 D1" to "Slice 8 S3", `slice-8.md`): the token is shared, the provider serves it
(`host.serve(R, { compose, accepts })`); `self.registry(R)` in a factory is a declared dependency (activation waits
for a value; with none after `load`, the reader fails `transient` and is retried when one arrives); contributions
never fail a registry (a failed one is left out and reported); a throwing `compose` keeps the last good composition;
older readers get the value mapped back (`accepts[old].value`); `retryOn` retries on value calls only; `host.root(fn,
context, { principal? })` seals its context at settle, abort and `host.close`.
- **Slice 8's rules** (decisions "Slice 8 D1" to "Slice 8 S3", `slice-8.md`; S2's `broken` and S3's backoff superseded
by slice 7a): the token is shared, the provider serves it (`host.serve(R, { compose, accepts })`); `self.registry(R)`
in a factory is a declared dependency: the attempt parks until the registry has a usable value at its version
(composed; mapped by `accepts[old].value` for an older reader), otherwise it shows `failed { waiting }` naming the
registry (`not_served`: `unavailable/undeclared`; `version`: a reader newer than the provider or without a mapping,
`contract_mismatch`; `compose`: a first `compose` or the mapping threw; `loading` before the publication), woken at
every change of whether the value is usable; `host.registry` waits the same way up to its `readyTimeout`;
contributions never fail a registry (a failed one is left out and reported); a throwing `compose` keeps the last
good composition (above); `retryOn` retries on value calls only; `host.root(fn, context, { principal? })` seals its
context at settle, abort and `host.close`.
- **Registries may carry natives.** An entry lives exactly as long as its contributor; consumers look up at each use
or follow the composed value and keep nothing from an entry beyond a use.
- **Contributions that change (owner, 2026-10-19; W4):** registries are host-local, so contributions are usually
@ -1716,7 +1789,9 @@ agent asks everyone present. Its store must live in an object (it predates "nati
keeps the hold: a pure object re-subscribes it through the version's mappings and sends `[["r", value]]` for each
member whose value changed; a restarted object sends a fresh `bound` for the same hold (a `bound` on a bound hold
replaces its replicas). A served hold whose object leaves `live` while its `bound` waits for a slot goes back to
pending (slice 7 builds it: nothing in 4a leaves `live`). A reload that drops the version ends its holds with `gone` (`contract_mismatch`) before
pending (slice 7 builds it: nothing in 4a leaves `live`). *(Pending slice 7b's reconciliation with the dependency
model M§5, which unbinds such holds `version` instead of ending them; until then this item and the rollout's
`updating` deadline below are not current rules.)* A reload that drops the version ends its holds with `gone` (`contract_mismatch`) before
the new code serves anything. A consumer whose own code moves to a new version re-holds (a new hold id) and releases
the old.
- **Old-version calls:** `checkArgs`'s `{ kind: "call" }` result runs the provider's replacement; `{ kind: "method";
@ -1729,14 +1804,22 @@ agent asks everyone present. Its store must live in an object (it predates "nati
- **Consumers elsewhere** see a restarting object like any object that briefly goes away.
- No factory trial, no migrate dry run, no consumer contract check during staging: incompatible consumers fail at bind
with `contract_mismatch` and a notice naming plugin and type.
- **Start barrier** (host start and first load): load facets (the core facet with its start data among them), wait
until every facet and host-lifetime object is live or failed (an object that must
contribute at start, like the model runtime, is a `local` object with empty params). Then the registries hold the contributions of every
plugin that loaded, and dependent work
resumes: base code calls `harness.resume()` after `load` (the harness is an ordinary host-lifetime object that
opened Durable paused during the barrier; owner, 2026-10-21, refines R17). Nothing
waits for specific contributions: what is missing shows where it is used (section 24). Failed plugins are reported
with their error. Plugin reloads need no barrier.
- **Start barrier** (host start and first load; dependency model §6, decision (a), slice 7a): `load` runs the facets'
setups (the core facet with its start data among them) and **publishes** them in one batch (types served,
registries served and composed once, contributions in their tables); requests that waited for the publication are
placed after that batch (holds and `host.hold` wait for the publication only, never for the barrier: two hosts
whose objects need each other's types start without a deadlock). Then it activates the host-lifetime objects and
resolves when each has its **first outcome**: live, failed by its own cause, or parked with no blocker that is a
local object still without its first outcome (`local`: executing on this host, a keyed pin still loading here
included; a wait on another host or on a failed dependency ends the first pass), or ended (`host.close` settles a
waiting `load`). One first-outcome signal per object, separate from the readiness dependents wait for, counted once,
never reopened by retries. So `CodeMode → Harness` waits for the harness's outcome. **What it promises:** every
host-lifetime object here had its first try, and the contributions of those that went live are in their tables;
**not** every tool that will arrive (a parked or slow contributor, tools found in `onLive`, MCP): what a first turn
needs is p3's explicit admission check. Base code calls `harness.resume()` after `load` (the harness is an ordinary
host-lifetime object that opened Durable paused during the barrier; owner, 2026-10-21, refines R17). Nothing waits
for specific contributions: what is missing shows where it is used (section 24). Failed plugins are reported with
their error. Plugin reloads need no barrier.
### Rollout across hosts
@ -1800,14 +1883,15 @@ validation, no clone of snapshots or ops.
| `hello` | both, first | `protocol`, `incarnation`, `calls` (the call window N) |
| `hold` | to provider | `origin?`, `hold`, `type`, `version`, `fingerprint`, `params` (part), `address`, `principal?`, `incarnation?`, `chain?` (activation chain, Q5), `limit?` (decoded size limit, 4b) |
| `revoke` | parent to child (route link) | the revocation value (4b) |
| `release`, `retry` | to provider | `origin?`, `hold` |
| `release` | to provider | `origin?`, `hold` |
| `retry` | to provider | `origin?`, `hold`, `operation?` (the retry it belongs to, a uuidv7), `through?` (with `operation`: the instance ids of the objects it passed through, at most 32; an object in it drops the retry, so a ring stops after one lap); relays forward both (slice 7a fix rounds 1, 2) |
| `call` | to provider | `origin?`, `call`, `hold`, `method` (index), `args?` (part; absent: no arguments), `segments?` |
| `abort` | to provider | `origin?`, `call` (named `cancel` until 2026-10-27) |
| `ack` | to provider | `origin?`; one per applied `bound`/`state` message |
| `closed` | relay to provider | `origin`: the downstream origin ended |
| `bound` | to consumer | `origin?`, `holds: [{ hold, variant, members: [{ member, value }], failed }]`; one batch |
| `state` | to consumer | `origin?`, `holds: [{ hold, members: [{ member, ops }], failed, retired }]`; one batch (`retired`: members retired since the last message, slice 6a) |
| `failed` | to consumer | `origin?`, `hold`, `cause { kind, error }`, `retryIn?` (absent: broken) |
| `failed` | to consumer | `origin?`, `hold`, `cause { kind, error }` (`kind`: `transient`, `broken` or `waiting`, slice 7a), `retryIn?` (a transient cause's backoff; absent: broken, waiting); each new cause of a parked object is a message of its own |
| `loading` | to consumer | `origin?`, `hold`: a new attempt after a failure, or a relay's upstream dropped |
| `result` | to consumer | `origin?`, `call`, `value?` (part; absent: `null`), `segments?` |
| `error` | to consumer | `origin?`, `call`, `error` (a ChordError's JSON or a plugin's code, message, details) |
@ -2132,8 +2216,8 @@ function doing counted work counts it through its context; call sites never coun
site labels.
**Testing the core:** every slice spec contains an explicit test list, one group per mechanism with its edge cases,
tested individually and exhaustively: activation and dependency recovery (R18: broken dependency, trigger, backoff as
safety net, a dependency failing while live, cycles), revocation over running jobs (R11), compatibility (R6), params
tested individually and exhaustively: activation and parked dependencies (the dependency model: unbound requests, waits without timers, wakes on every
change, retry passing on, a dependency failing while live, cycles, the start barrier's first outcomes), revocation over running jobs (R11), compatibility (R6), params
and arguments for every caller (R7), migration results and old calls on migrated documents (R8, R15), flow control
(the window per origin, folding, relays forwarding and returning acks), the incarnation fence (R20), the harness start
after the barrier (R17), native restart patterns (R16), and W1's publication list (a throwing derive, a chain A→B→C

10
packages/chord/docs/next/slices.md
View file

@ -76,6 +76,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 3 | park deadline (= `readyTimeout`, C13) | 3b |
| 3 | `host.activity` (keep-alive counts) | 4b |
| 3 | `host.root(fn, …)` for code outside objects | 8 |
| 3 | requests that outlive providers: the destination by address, the publication gate, the unbound index, the check against the selected provider only; the start barrier's first outcomes (decision (a)) | 7a |
| 4 | principal type `P`, links stamp principals, principal keys in identity | 3a |
| 4 | revocation (`revokedBy`, `host.revoke`, propagation to children) | 4b |
| 5 | `local` types: params, members, checked arguments (Q15), results unchecked | 1 |
@ -85,6 +86,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 6 | lifecycle, one object per identity, lifetime from kind and params (Q-A), factory, activation, ending | 1 |
| 6 | failure causes, backoff per identity, dependency recovery (R18) | 2 |
| 6 | `retryOn` | 8 (first independently living source: registries) |
| 6 | parked attempts (no dependency deadline or backoff, wakes on every change, retry passing on), `FailureCause` `waiting`, `parked_long` and the authoring rules against cycles | 7a |
| 6 | cycle rejection on one host (Q5) | 1 |
| 6 | cycle rejection across hosts (activation chain in hold headers) | 4b |
| 6 | shared store with per-principal views; worker-wide native owners; natives following a relocation | doc (1, 3, 2) |
@ -92,6 +94,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 7 | per-reference watches (released with their reference) | 1 |
| 7 | `ctx.get` (call-scoped references) | 3a |
| 7 | `ref.state` (tagged), `ref.retry()`, `host.hold`, calls parking, failed references | 3b |
| 7 | getters wait through `waiting`; `host.hold` throws only for invalid params | 7a |
| 7 | `ref.variant` as discriminant, `RefIn` (C15: no `inVariant`/`expectVariant`) | 3a |
| 7 | `state: false` (C8: cut) | - |
| 7 | optional dependencies as registries | doc (8) |
@ -119,6 +122,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 15 | `defineJob`, `self.job`, intent/run/finish, state documents, dedupe (Q9), `busy`, `running`, blocked `finish` (C3 decided); persistence, resume, restart limit; `accepts` in use | 5 (persistence: 6b; `accepts` and reload: 7) |
| 16 | `defineFacet<P>`, facet as declarations only (documents and routines removed in 6a) | 1, 6a |
| 17 | registries: `defineRegistry`, contributions, `compose`, versions (Q8), use-site guards | 8 |
| 17 | registry readers as parked dependencies (`waiting`), `host.registry` waiting, the last good value dropped when a contribution leaves | 7a |
| 18 | `constant(value)`, start data as a core object (`SessionInfo`) | 1 (core facet pattern: 4a) |
| 18 | platform bindings as core objects | doc (8) |
| 19 | `host.references` (presence) | 4b |
@ -185,6 +189,12 @@ deleted (the slice's spec and code are the record). Moved here from `carry-over.
(`local`, empty params) stays broken until the host restarts; slice 7 maps the records to the new version (or blocks
the old runs) on reload.
- **Publications wake per name** (from slice 7a): `UnboundIndex.wakeAll()` wakes every indexed request, right for the
one publication 7a has; a reload's or an install's publication wakes only the requests of the names it serves
(`wake(names)`), or every refused hold on the host would re-run its start.
- **The reload handover of a parked attempt** (from slice 7a): the new attempt's requests bind before the old ones are
released (M§3.9); the mechanism (an attempt releasing its predecessor's references after binding its own) was removed
in 7a with R18's kept references, its only feeder.
- **Routines on reload** (from slice 5): the handover per object, bounded by `closeTimeout`; calls do not wait for it.
- **Jobs on reload** (from slice 5): `accepts` in use (args and result mapped forwards once per run; `blocked
{ version }` without an adapter or for a newer version); when an older run's state document may migrate while its

70
packages/chord/src/next/data/failure.ts
View file

@ -1,11 +1,14 @@
// Failure causes of an open: transient (retry later) or broken (bad configuration). The failing stage decides.
// Failure causes: transient (the object's own effect failed: retry later), broken (its own code, a cycle or authority:
// retry by hand) or waiting (not its fault: no provider, no fitting version, a dependency that failed; woken by an
// event). The failing stage decides, never the error code.
import { asChordError, type ChordError } from "./errors.ts";
import { asChordError, ChordError } from "./errors.ts";
/** Why an open failed; the kind comes from the failing stage, never from the error code. */
/** Why an object, or a request, is failed; the kind comes from the failing stage, never from the error code. */
export type FailureCause =
| { readonly kind: "transient"; readonly error: ChordError }
| { readonly kind: "broken"; readonly error: ChordError };
| { readonly kind: "broken"; readonly error: ChordError }
| { readonly kind: "waiting"; readonly error: ChordError };
/** Marks an open's failure as broken (bad configuration): `throw broken(error)`. */
export function broken(cause: unknown): Error {
@ -18,6 +21,65 @@ export function openFailure(thrown: unknown, label: string): FailureCause {
return { kind: "transient", error: asChordError(thrown, label) };
}
/** The longest message a `waiting` cause of a dependent carries (telemetry's rule for error messages). */
const MESSAGE_LIMIT = 500;
/** How a dependent's message names its blocker: the nearest one first. */
const PREFIX = "waiting for ";
/** The separator after a blocker's label. */
const SEPARATOR = ": ";
/** The marker of a clipped label or root. */
const CLIPPED = "…";
/** The most of the 500 characters the root cause takes when both are long: the nearest blocker keeps the rest. */
const ROOT_LIMIT = 320;
/**
* The `waiting` cause of a dependent parked on `blocker`, which failed with `error`: `unavailable`, its message always
* "waiting for <blocker>: <root cause>" (the root found through any "waiting for …: " of `error`'s message), so a ring
* of waiting attempts passing causes round reaches the same message after one round; within 500 characters, the
* label clipped first, the root only past its share. `error` is its cause.
*/
export function waitingFor(blocker: string, error: ChordError): FailureCause {
let root = rootMessage(error.message);
let label = blocker;
const room = MESSAGE_LIMIT - PREFIX.length - SEPARATOR.length;
if (label.length + root.length > room) {
const rootRoom = Math.max(ROOT_LIMIT, room - label.length);
if (root.length > rootRoom) root = clipped(root, rootRoom);
if (label.length + root.length > room) label = clipped(label, room - root.length);
}
const message = `${PREFIX}${label}${SEPARATOR}${root}`;
return { kind: "waiting", error: new ChordError({ code: "unavailable", message }, { cause: error }) };
}
/** `text` within `length` characters, clipped with "…" when longer. */
function clipped(text: string, length: number): string {
return text.length <= length ? text : `${text.slice(0, length - CLIPPED.length)}${CLIPPED}`;
}
/** The root cause in a "waiting for X: root" message: past every leading "waiting for …: ". */
function rootMessage(message: string): string {
let rest = message;
while (rest.startsWith(PREFIX)) {
const end = rest.indexOf(SEPARATOR, PREFIX.length);
if (end < 0) break;
rest = rest.slice(end + SEPARATOR.length);
}
return rest;
}
/** Whether two failure causes say the same (a wake compares causes by value): kind, code, reason and message. */
export function sameCause(a: FailureCause, b: FailureCause): boolean {
const { error: left } = a;
const { error: right } = b;
return (
a.kind === b.kind && left.code === right.code && left.reason === right.reason && left.message === right.message
);
}
class BrokenOpen extends Error {
constructor(cause: unknown) {
super(cause instanceof Error ? cause.message : String(cause), { cause });

2
packages/chord/src/next/data/internal.ts
View file

@ -8,7 +8,7 @@ export {
invalid,
PROTOCOL_VERSION,
} from "./errors.ts";
export { openFailure } from "./failure.ts";
export { openFailure, sameCause, waitingFor } from "./failure.ts";
export { checkAddress, checkParams, identityKey } from "./params.ts";
export { canonicalJson, principalKey } from "./principal.ts";
export { check, validatorFor } from "./validator.ts";

197
packages/chord/src/next/host/README.md
View file

@ -62,12 +62,13 @@ const host = createHost<AppPrincipal>({
closeTimeout: 30_000,
readyTimeout: 60_000,
});
await host.load([{ name: "@app/notes", facet }], ctx); // every host-lifetime object live or failed (reported)
const notes = await host.get(Notes, {}, ctx); // live and hydrated; waits through transient retries
await host.load([{ name: "@app/notes", facet }], ctx); // published, then every host-lifetime object's first outcome
const notes = await host.get(Notes, {}, ctx); // live and hydrated; waits through loading, transient, waiting
await notes.add({ text: "milk" }, ctx);
notes.release();
const draft = host.hold(Draft, { conversation: "c1" }, ctx); // a reference at once: loading, then live
(await draft.state.watch(ctx)).start(async (call) => call.kind === "value" && render(call.value)); // loading, live, failed, ended
// failed { waiting }: "waiting for X: …", not this object's fault; it goes on when X is live
if (draft.state.value.kind === "failed") draft.retry(); // a Retry button
host.accept(link, { principal: { kind: "guest", session: "S1" } }, ctx); // a child's link, authenticated by its entry point
await host.close(ctx); // closes links, then natives with reason "close"
@ -130,7 +131,8 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
`principalKey` maps a principal
to its part of an identity (L1's canonical JSON by default; the same function on every host). `closeTimeout` (30 s)
bounds each native's close and the drain of an ending object's calls; `readyTimeout` (60 s) bounds every activation
attempt (its wait for the previous instance's teardown included) and is the default deadline of a `get`. `heartbeat: { ping?, close? }`: a link silent for `ping` ms (15 s)
attempt (its wait for the previous instance's teardown included; paused while it is parked on its dependencies) and
is the default deadline of a `get` and of `host.registry`. `heartbeat: { ping?, close? }`: a link silent for `ping` ms (15 s)
sends a `ping`, one silent for `close` ms (30 s), its handshake included, is closed (its references go `loading`);
one interval timer per host (`min(ping, close) / 10`), running only while it has links. A handler that blocks the
event loop longer than `close` gets its links closed.
@ -139,8 +141,11 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
principal has `r` among its keys ends at once, whatever references remain (R11): each holder gets `forbidden` with
reason `revoked` (a UI can say "access revoked"; a hold over a link sends `gone` with it), running calls are aborted
with it (callers rejected), an activation in progress is abandoned; teardown follows as for any end (drain, closes
with reason `end`). Holds of the principal that arrived before the revoke but still wait (for `load`, or relayed for
their route's link) end too (`gone`).
with reason `end`). Requests of the principal that hold nothing end too, bound to nothing yet or never: holds that
arrived before the publication, holds refused here (`failed { waiting }`), relayed holds waiting for their route's
link (each gets `gone`), references made here still unbound, and this host's own remote references of the
principal over its paths, a route still resolving or a link down included (`ended`, released at the provider if
held there): no later link holds them again.
Then every route link is sent `revoke { revocation }`, which the child applies with its own `revokedBy` (and passes
down its own route links), and every child link whose principal it revokes closes (a relay sends
`closed { origin }` upstream for it, after the `revoke`). Never this host's own principal, never the parent link; a
@ -157,8 +162,9 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
splice op per removal, one for the additions: a revoke of many holds is one list and one frame), otherwise it is
built on its next read. A departure discovered while a commit's links send (a transport failing on send) lands in
the next batch, so one commit stays one message per link and origin.
- `host.activity`: a JSON watchable `{ references, calls, jobs, lingering }`: holds arriving over links and served
here, method calls running here (local and remote), job runs running or finishing here, objects lingering after their
- `host.activity`: a JSON watchable `{ references, calls, jobs, lingering }`: holds arriving over links and placed
here, from their arrival to their end, bound or not (a hold waiting for a missing provider keeps a worker alive;
`host.references` lists bound references only; a relayed hold counts at its destination), method calls running here (local and remote), job runs running or finishing here, objects lingering after their
last reference. Counted where the work happens; the changes of one microtask turn are one publication
(`[["r", counts]]`), a slow watcher gets the newest counts; nothing is scheduled while nobody watches or derives from
it. Chord counts; the app's supervisor decides.
@ -176,10 +182,12 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
here, plus its own id; the last 32); honoured from trusted links only, relays forward it as it is. A provider refuses
a hold whose chain names the instance it would bind while that activates: broken `invalid/cycle` naming the cycle by
label where known ("app.a(1) → … → app.a(1)"); a local `self.get` of an activation in the chain is refused the same
way. A dependency failing with `invalid/cycle` fails its dependent broken with that error (owner, an exception to
R18's transient dependency failures: a retry would meet the same cycle). Known limitation: a chain is fixed when its
object is created, so a cycle entering an object that is already activating (two activations started at the same
moment) is not detected: both fail `transient` at `readyTimeout` and retry with backoff.
way. A dependency failing with `invalid/cycle` makes its dependent broken with that error (it would wait on a ring
that never heals). The gap (owner, 2026-10-29: no probe): a chain is fixed when its object is created, so a ring
across hosts whose objects start at the same moment is not detected, and without a dependency deadline it stays
parked; an attempt parked longer than its `readyTimeout` on a remote attempt that is only `loading` (its hold on a
link: the clock and `since` run only while it is, so a late link is not reported at once) is reported once per
parking episode (`parked_long`), nothing fails. A cycle is a plugin bug: see the authoring rules below.
- `onError(error: HostError)` gets everything that fails in the background, by `kind` (`failure` where the value is a
`FailureCause`, `error` otherwise; a position is `{ kind: "on" | "onLive" | "native", position }`, from 1):
`activation_failed` (`object`, `failure`: every failed attempt), `object_failed` (`object`, `failure`: a live object
@ -192,7 +200,9 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
route's `resolve` rejected; the address waits its backoff), `routine_failed` (`object`, `routine`: its position,
`error`: a routine call threw, or its watch could not be taken; a source's failure calls never reach a routine), `job_failed` (`object`, `job`,
`id`, `error`: a run threw, not after an abort), `job_finish_failed` (the same: a `finish` threw; it is retried) and
`job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end), and
`job_dropped` (`object`, `job`, `id`: a blocked run dropped without `finish`, by `abort` or its object's end),
`parked_long` (`object`, `waitingFor`: the remote blocker's label, `since`: when it parked; once per parking episode,
slice 7a), and
the registries' (slice 8, each once per episode) `registry_conflict` (`registry`, `message`, `plugins`: a conflict
`compose` resolved), `registry_failed` (`registry`, `error`: `compose` or an older version's `value` adapter threw),
`contribution_failed` (`registry`, `plugin`, `type?`, `error`: a watchable contribution failed or ended, or its
@ -207,11 +217,13 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
batch, anything else is a plain value whatever methods it has (registries carry natives); a registry value closing
a cycle of registries (through the tables and the factory's own earlier contributions) is refused `invalid/cycle`
at `contribute` (an edge lives as long as its contribution is in a table). Every served registry composes once at the end of `load`'s setups, before any object activates.
`self.registry(R)` is a declared dependency: with no value at activation (no provider, a first `compose` that threw,
a version not served here) the attempt fails `dependency_failed` (`transient`, the registry's error as the cause,
the references kept) at once, and the registry's first value retries it (R18); a version the provider does not
serve fails it `broken` (`contract_mismatch`: only a reload helps). `host.registry(R, ctx)` waits for `load` and
rejects without a value.
`self.registry(R)` is a declared dependency: without a usable value at its version (no provider:
`unavailable/undeclared`; a first `compose` or the version mapping that threw; a version the provider does not serve:
`contract_mismatch`) the attempt parks `failed { waiting }` naming the registry, and every change of whether the value
is usable wakes it. `host.registry(R, ctx, { readyTimeout? })` waits for the publication, then the same way up to its
deadline, and rejects with the last error. The last good composition is kept only while every contribution it was
built from is still in the table (a contributor that fails or ends drops it: `.value` throws until `compose`
succeeds).
- **Wake signals** (`self.retryOn(...sources)`, D2): registry values (constants are accepted and never wake; anything
else throws: a reference's member, the attempt's writable, `ref.state`). The object follows them from the factory's
return (before hydration, references and natives, so a failed open still has them) until the next attempt's
@ -227,37 +239,47 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
`host.root` at once (an uncooperative `fn` runs on and fails at its next use of Chord); an aborted `ctx` runs no
`fn`; `host.close` rejects it at once `released/closed`, as a method's caller when its object ends; after `close`
it rejects `released/closed`. Not counted in `host.activity`.
- `host.load([{ name, facet }], ctx)`: once per host; runs every setup, then activates every type with empty params
and resolves when each has had its first outcome, live or failed (the start barrier; later retries never hold it).
A setup that throws rejects it with that error and closes the host as `close` does, its links included. Gets wait
for it.
- `host.load([{ name, facet }], ctx)`: once per host; runs every setup and publishes them in one batch (types and
registries served, registries composed once, contributions); requests made here and holds arriving before that
waited in the unbound index (`loading`) and are placed after the batch (they never wait for the barrier: two hosts
whose objects need each other's types start without a deadlock). Then it activates every type with empty params and
resolves when each has its first outcome (decision (a)): live, failed by its own cause, parked with no blocker that
is an object here still without its first outcome (a keyed pin still loading counts as here; a wait on another host
or on a failed dependency ends the first pass), or ended (`close` settles a waiting `load`); later retries never
hold it. A setup that throws rejects it with that error and closes the host as `close` does, its links included.
- `host.hold(Type, params, ctx, { readyTimeout? })`: a reference at once (`Held<C>`: its `variant` is `undefined` until
its first binding), which follows its object until released and never rejects: `loading` while `load` runs, while
the object activates or retries, while there is no parent link or it is down; `failed` while the object is failed
and no attempt runs; `live`. Throws at once for invalid params (`invalid/params`) and a local type not served here
(`unavailable`); before `load` resolved
these are decided once it did, and the reference fails `broken` with that error. A `hostAddress` or `authorize` that
throws or refuses, or a host address whose params fail its kind's check (`invalid/host_address`), fails the reference
`broken`. A local type never crosses a link. Where a reference made here (`host.get`/`hold`, `self.get`, `ctx.get`)
or a hold arriving on a link goes, first match (`Destination`; the address is the contract's `hostAddress` of the params,
computed by the caller; a hold carries it to be routed, and a host serving the type computes it again):
its first binding), which follows its object until released and never rejects: `loading` before the publication,
while the object activates, retries or is parked on dependencies under way, while there is no parent link or it is
down; `failed` while the object is failed and no attempt runs, or while nothing fits it (`waiting`: not served where
it goes, another version; bound when a publication serves it) or its object is parked on a failed dependency;
`live`. Throws at once only for invalid params (`invalid/params`, also before `load`). A `hostAddress` or
`authorize` that throws or refuses, or a host address whose params fail its kind's check (`invalid/host_address`),
fails the reference `broken`. A local type never crosses a link. Where a reference made here (`host.get`/`hold`,
`self.get`, `ctx.get`) or a hold arriving on a link goes, by address alone (`Destination`; the address is the
contract's `hostAddress` of the params, computed by the caller; a hold carries it to be routed, and a host serving
the type computes it again):
| Case | Goes |
| --- | --- |
| the type is served here and its address is this host's | here |
| a reference made here to a served type whose address is another host's | on, as if not served here (rows below): worker S1 reaches `Notes { session: "S2" }` through the parent |
| an arriving hold for a served type whose params place it on another host | broken `invalid/host_address` (misrouted; never forwarded) |
| a local type; a remote type whose address is this host's, served here or not; a hold naming this host's address | here, checked against what is served here (below) |
| the address is of a kind this host has a route for | over that route's link (the params checked against the kind, `authorize` first; `forbidden` when it refuses) |
| otherwise (a remote type) | over the parent link, open yet or not; a hold that arrived *on* the parent link is never sent back: `unavailable` |
| an arriving hold and no parent link | `unavailable` |
A hold that arrives and goes on is relayed (`remote/`): this host holds it upstream as the arriving hold's caller
and forwards between the two without parsing params, arguments, results or state.
Placed here, a request is checked against the type served here only (never a same-name registration elsewhere):
not served (or served as the other kind) or not at the caller's version and fingerprint, or params the provider's
own `hostAddress` places on another host (code skew; checked for an arriving hold and for a remote request made
here alike), it waits unbound `failed { waiting }` (`unavailable`, `contract_mismatch`, `invalid/host_address`),
indexed under its name on this host (`unbound.ts`) until a publication wakes it (a refused hold also by a `retry`,
which re-runs its start: `loading`, then `bound` or `failed`); invalid params, a cycle and a provider's
`hostAddress` that throws (its own code) are `broken`, never indexed. A hold that arrives and goes on is relayed (`remote/`): this host
holds it upstream as the arriving hold's caller and forwards between the two without parsing params, arguments,
results or state.
- `host.get(Type, params, ctx, { readyTimeout? })`: a hold, awaited until live. Each getter has one deadline for its
whole wait, `load` and reconnects included. A broken failure rejects it at once with its error; through transient
failures and drops it waits, and at its deadline rejects with the last failure's error (`unavailable` "did not
become live within …" when none failed yet). A getter whose deadline passes or whose context is cancelled releases
only its own reference.
whole wait, the publication and reconnects included. A broken failure rejects it at once with its error; through
`loading`, `failed { transient }`, `failed { waiting }` and drops it waits, and at its deadline rejects with the last
failure's error (`unavailable` "did not become live within …" when none failed yet). A getter whose deadline passes
or whose context is cancelled releases only its own reference.
- `host.serve(Type, factory, { linger? })` in a facet's setup: a live object with params whose last reference is
released stays live for `linger` ms (0 by default), with its documents and dependencies; a reference within that
time reuses it, otherwise it ends. An object waiting, activating or failed when its last reference goes ends at once;
@ -298,7 +320,9 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
at `host.close`. When this host holds over a route link it sends whom each hold acts for (the table in `remote/`):
a guest's handler that `ctx.get`s a worker type reaches the worker as the guest.
- `host.close(ctx)`: aborts every route's `resolve`, closes its links first, children, routes, then the parent (no
`gone` goes out), ends every remote reference (`released/closed`, parked calls rejected with it), then ends every
`gone` goes out), ends every remote reference (`released/closed`, parked calls rejected with it), every request
still unbound (waiting for the publication or a provider) and every waiting `host.registry` (`released/closed`, at
once, not after the slow closes below), then ends every
object at once (calls fail `released/ended`, `self.tx` throws; documents stay readable until it is done), aborts pending opens, waits for endings already running, then drains and
closes object by object, the last to go live first (a dependent before its dependencies), with reason `close`.
Every call returns the same closing. It rejects with an `AggregateError` of the closes that threw. It does not wait
@ -366,7 +390,8 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
loading or failed; at runtime it carries every member of every variant, and one outside the bound variant fails
`invalid/member`.
- `ref.state` (a `NativeWatchable<RefState>`, made on first access): `{ kind: "loading" } | { kind: "live" } | { kind:
"failed"; cause; retryAt }` (`retryAt`: when the object's backoff window ends, `undefined` for a broken failure) `|
"failed"; cause; retryAt }` (`retryAt`: when the object's backoff window ends, `undefined` for a `broken` or
`waiting` cause) `|
{ kind: "ended"; error }` (for good: `released` without a reason when its holder released it, `released/ended` when
its object ended, `released/closed` at host close, `forbidden/revoked`). A watch gets the current state, then every
change; it ends `ended` when the reference ends. What a
@ -382,16 +407,18 @@ server.accept(webSocketLink(ws), { principal, limits: { part: 16 * MiB, decoded:
Calls made after the reference went live never overtake queued ones (a call a handler makes while they are sent
queues behind them). A parked call is checked against its deadline when it is about to be sent, and against the
variant it is sent on; it also leaves the queue when its caller cancels (rejected with the reason), the reference
fails (the cause's error) or is released (`released`). A `host.hold` released before `load` resolved makes nothing.
fails (the cause's error) or is released (`released`). A `host.hold` released before the publication makes nothing.
- A local reference follows its object's state members through member sources (`member-source.ts`, one per object and
member): when a new attempt goes live every member re-binds by name to it (watches get one frame from the last
value, other objects' derives over the member recompute, a failed one included), as a remote reference's re-hold
does. The keys a reference holds on a keyed member (`at(key)`, `watch(key)`) follow the object the same way
(`keyed-members.ts`, one source per member and key; the new attempt's member loads them before it goes live). A key
whose first load is still in flight when its object fails or ends is given up: the activation waiting on it fails
`transient` (R18) and retries on the next attempt; the late pin is released once.
whose first load is still in flight when its object fails or ends is given up: the parked attempt waiting on it shows
`failed { waiting }` and loads it again at the next change of that reference (its object's new attempt going live)
or a `ref.retry()`; the late pin is released once.
- `ref.retry()`: a failed object (broken too) gets an attempt now, its backoff window skipped; every reference to it
goes `loading`. Ignored while an attempt runs, once live and once released; over a link it is a `retry` message.
goes `loading`. One parked `failed { waiting }` passes it on (above). Ignored while an attempt runs otherwise, once
live and once released; over a link it is a `retry` message carrying its operation.
Rules:
@ -402,23 +429,48 @@ Rules:
`self.get` in a remote object's factory as that object's principal, in a local object's factory as the host's. Over
a route link they travel with the hold; over the parent link the parent acts as this host (upward is a boundary: a
worker that acts for a user towards the server passes the user as data and the server object decides).
- **Activation**, one attempt: the factory; singleton documents hydrated; references bound and live; natives opened one
after another in declaration order (an opener may read documents, references and earlier natives, not derives);
members checked; derives computed; live. An attempt not live within `readyTimeout` is abandoned: its openers'
contexts are aborted, what it opened is closed (`end`), an open resolving later is closed at once.
- **Activation**, one attempt: the factory; singleton documents hydrated; its requests made (references bound,
registries read); parked until every dependency is live (below); then its own steps: natives opened one after
another in declaration order (an opener may read documents, references and earlier natives, not derives), followed
sources attached, members checked, derives computed; live. An attempt whose own steps do not finish within
`readyTimeout` (the time parked not counted) is abandoned: its openers' contexts are aborted, what it opened is
closed (`end`), an open resolving later is closed at once.
- **Parked attempts** (dependency model §3, slice 7a; `dependencies.ts`): a dependency is a `self.get` reference
(local or remote), a `self.registry` value or a keyed pin (`ref.member.at(key)`, loaded once the reference is live);
`self.follow` is an own step. While one is not live the attempt is parked and its object shows `loading` (all under
way), `failed { waiting }` with `unavailable` "waiting for X: <root cause>" (X the first failed one, the root cause
found through X's message; always these two, so a ring of waiting attempts settles after one round; within 500
characters, X's label clipped first; X's error as `cause`), or fails `broken` with the error of a dependency that
failed for a cycle, for authority (`forbidden`: an `authorize` refused it) or ended. A getter's deadline rejects
with the last cause it showed, also after the dependency fell back to `loading`. No timer runs (no dependency deadline, no backoff; the `readyTimeout`
deadline paused); the factory never runs again; every change of a dependency (its state, a new cause by value, its
first outcome) makes it check again once after the batch that made it; once all are live it goes on to its own
steps, and a dependency leaving `live` after that changes nothing. `ref.retry()` on a parked `failed { waiting }`
object passes on to its failed dependencies once per retry operation (a uuidv7 that travels in the `retry`
message with the instance ids of the objects it passed through: one already in that list drops it, so a ring across
hosts stops after one lap whatever the number of concurrent retries, and the last few ids an object handled
shortcut a diamond: a `broken` or
`transient` one starts an attempt, a parked one passes it on, a request waiting in an index is left alone, a
refused hold over a link re-runs its start); a `retryOn` wake does nothing while parked. A parked object whose last reference goes ends at once.
- **Authoring rules against cycles** (dependency model §7; a cycle is a plugin bug, and a ring across hosts started at
the same moment is only reported, `parked_long`): (1) a factory declares only what the object needs, never a type
that needs it back; (2) back-references and occasional lookups go into methods and routines (`ctx.get`); (3) data
flows up through state and registries, not mutual `self.get`; (4) host-lifetime objects on the server never declare
worker types in their factory.
- **Failures** become a `FailureCause` with a ChordError; the kind comes from where it failed: the factory, a
singleton's `initial()`, the member check (a native watchable as state included), a first derive compute or a
dependency cycle are `broken`; a throwing open is `transient` unless it threw `broken(error)`; the `readyTimeout` is
`transient`; a dependency that failed (R18) is `transient` with `unavailable` "A: dependency B failed" and B's error
as `cause`. A keyed document's `initial()` runs in a later transaction and fails only that transaction. A cycle
(`a(1) → b(1) → a(1)`) fails every object on it.
dependency cycle are `broken`; a throwing open or a failed `self.follow` attach is `transient` unless it threw
`broken(error)`; the `readyTimeout` is `transient`; a dependency that failed is no failure of the dependent (it
parks `failed { waiting }`, above), one that failed for a cycle, for authority (`forbidden`) or ended makes it
`broken`. A keyed document's
`initial()` runs in a later transaction and fails only that transaction. A cycle (`a(1) → b(1) → a(1)`) fails every
object on it.
- **Retries:** one attempt per object at a time. A transient failure is retried after a backoff per identity: 1 s,
doubling to 60 s, reset by a success; a new object for an identity inside its window waits for the window's end.
A broken failure is retried only by `ref.retry()` (and reload, slice 7, `retryOn`, slice 8); a new object after its
end tries at once. Only a dependency failure, or a `readyTimeout` while waiting for dependencies, keeps the object's
references; when that dependency goes live (a remote one: bound again after a drop) the object is retried at once. A dependency failure is transient whatever the dependency's kind: a dependent of a broken dependency is
retried with its backoff for as long as it is held. A retry binds its references before the previous attempt's are
released, so a dependency held only by the retrying object never ends in between.
A broken failure is retried only by `ref.retry()` (and reload, slice 7b, `retryOn`, slice 8); a new object after its
end tries at once. A failed attempt releases its references once its natives closed; the next attempt makes new
ones. A dependent of a failed dependency is not retried at all: it is parked and goes on when the dependency is
live (no backoff of its own).
- **Ending:** transactions refused and the callers of running calls rejected `released` at once, their handlers'
contexts aborted, routines' and job runs' too (their waiters rejected); then the drain (the handlers', routine
calls' and runs' own promises, a run's `finish` included, up to `closeTimeout`), the natives closed in reverse
@ -428,8 +480,8 @@ Rules:
- Calls check the method against the bound variant first (`invalid/member`), then the arguments against the provider's
current version (`invalid/member`, `invalid/args`); on one host arguments and results pass by reference.
- **Release** rejects the calls made through the reference with `released` and aborts their handlers' contexts, on one
host as over a link. A get needs the provider's version and fingerprint (`contract_mismatch`); an unserved
type is `unavailable`, a closed host `released`.
host as over a link. A get needs the provider's version and fingerprint (`contract_mismatch`, `waiting`); an
unserved type is `unavailable` (`waiting`); both wait up to the getter's deadline; a closed host is `released`.
- Watches taken through a reference end when it is released, even while others keep the object alive.
- A failure's error keeps its stack frames' receivers alive until its `stack` is read (V8), its failed attempt among
them (natives closed, documents dropped). An object keeps its last failure's error until it goes live or ends; an
@ -470,30 +522,35 @@ compression bomb), `18-routines.ts` (a reconciling `on` with a retry, a periodic
job), and (slice 6a) `20-tables.ts` (tables, writes, pages, counts), `21-persisted-documents.ts` (documents across two
hosts on one store file), `22-migrations.ts` (`evolveTable` and an eager migration at activation),
`23-keyed-watches.ts` (`watch(key)`, `at(key)`, the `retired` flag), `24-keyed-members.ts` (a keyed member pinned per
key by its consumers) and `25-export-import.ts` (an archive in JSONL and an import).
key by its consumers) and `25-export-import.ts` (an archive in JSONL and an import), and (slice 7a)
`31-parked-dependencies.ts` (a missing provider arrives: causes shown, no timers) and `32-startup.ts` (two hosts that
would have deadlocked at startup).
## For Chord's layers (`internal.ts`)
For `remote/`: `ChordHost` (`place` decides where an arriving hold goes, here or relayed over a path; `serving`
finds the object a hold placed here binds to; `relays` and `relayOrigin`, the origins this host relays holds for by
their number; `pendingLoad`, `closing`, `linkOpened`, `unlink`), `AnyHost` (a host whatever its principal type: `ChordHost<P>` is invariant in `P`),
their number; `published`, `index`/`unindex` (the unbound index), `closing`, `linkOpened`, `unlink`), `AnyHost` (a host whatever its principal type: `ChordHost<P>` is invariant in `P`),
`Target` (what a binding is bound to, implemented by `ChordObject` and by `remote/`'s `RemoteObject`),
`ChordObject` (`state`, `retry`) and `Holder` (what holds an object: a binding, or a hold over a link: `moved()` when
the object's `state` changed, which a new holder also reads itself; `end(error)`), `Binding` (`moved`, `end`),
`CheckedParams`, `Caller` (principal and incarnation of a reference's maker or a hold's origin), `RefState` with `LOADING`, `LIVE`, `RELEASED`, and
`settlement`. Also on `ChordHost` for `remote/`: `revoke` (a parent's `revoke` message), `compression`, `count(kind,
`CheckedParams`, `Caller` (principal and incarnation of a reference's maker or a hold's origin), `RefState` with `LOADING`, `LIVE`,
`RetryOperation` (one `ref.retry()` passing on), `Serving` (what a hold placed here binds to, or why it waits) and
`UnboundRequest` (what the unbound index holds). Also on `ChordHost` for `remote/`: `revoke` (a parent's `revoke` message), `compression`, `count(kind,
delta)` (the keep-alive counts), `cycle(chain, id, closing)` (the `invalid/cycle` error naming a chain by labels),
`unplaced` (the path of remote references failed for a part past the limits) and `pathOf(Type, params, principal)`
(placing one anew on `retry`).
Inside the directory: `ChordHost` (types, objects by identity, facets, the batch, the backoff map, the endings new
instances wait for, its links and the heartbeat timer, its routes and the parent path; `reference` checks a
reference's route, `bind` binds it to an object here or a remote object over a path, `acquire` gets one for a
caller), `Destination` (where a reference or an arriving hold goes: here, a route's path or the parent's; whether a
hold served here is placed here), `Route` (one per routed kind: per address its `Path`, at most one running
instances wait for, its links and the heartbeat timer, its routes and the parent path, the publication and the
unbound index; `reference` makes a request's binding, `bind` binds it to an object here or a remote object over a
path or leaves it waiting, `acquire` gets one for a caller), `UnboundIndex` (requests that cannot bind yet, by name),
`checkDependencies` and `ParkedLong` (what a parked attempt's dependencies say; the `parked_long` report),
`Destination` (where a reference or an arriving hold goes, by address: here, a route's path or the parent's; whether
a hold served here is placed here), `Route` (one per routed kind: per address its `Path`, at most one running
`resolve`, the backoff per address; an address is kept while it has holders, a link or a running `resolve`),
`ChordObject` (one per identity: its state across attempts, holders, retries, cycle checks with its instance id and
the chain it was created for, lingering, ending and revocation, the count of its running and finishing job runs,
`ChordObject` (one per identity: its state across attempts (what a parked attempt shows), its first outcome, holders,
retries passed on while parked, cycle checks with its instance id and the chain it was created for, lingering, ending and revocation, the count of its running and finishing job runs,
which keeps it like a reference: `countRuns`, its contributions while live and its wake signals), `RetryWakes` (an
object's `retryOn` subscriptions), `RootScopes` (the open `host.root` scopes, sealed once each), `Activity` (`host.activity`: counters, one
publication per microtask turn), `References` (one type's presence list), `RevocationKeys` (each principal's keys,

285
packages/chord/src/next/host/attempt.ts
View file

@ -8,7 +8,7 @@ import type { JsonValue } from "../../json.ts";
import type { NativeWatchable, Watchable, WatchEnd } from "../../watch.ts";
import type { Contract, Implementation, ParamsOf } from "../contracts/index.ts";
import { contractData, type ShapeOf } from "../contracts/internal.ts";
import { ChordError, type FailureCause, isChordError, type Principal } from "../data/index.ts";
import { ChordError, type FailureCause, type Principal } from "../data/index.ts";
import { asChordError, checkArgs, checkParams, invalid, openFailure } from "../data/internal.ts";
import type { HandleOf, JsonObject, StateDefinition, Tx } from "../documents/index.ts";
import {
@ -22,7 +22,12 @@ import {
import type { JobDefinition, JobHandle, JobImplementation } from "../jobs/index.ts";
import { type AnyJobImplementation, Job } from "../jobs/internal.ts";
import type { Contribution, Registry, RegistryValue } from "../registries/index.ts";
import { type RegistryContribution, type RegistryEdge, RegistryValueSource } from "../registries/internal.ts";
import {
type RegistryContribution,
type RegistryEdge,
RegistryValueSource,
type RegistryWaiter,
} from "../registries/internal.ts";
import type { Writable } from "../state/index.ts";
import { Derive, type DeriveSource, WritableSource } from "../state/internal.ts";
import type { Columns } from "../store/columns.ts";
@ -33,6 +38,7 @@ import { type Table, TableHandle } from "../store/table-handle.ts";
import { prepareTables } from "../store/table-migration.ts";
import { CallContext, type Caller, type Ctx, withLinkedAbortSignal } from "./ctx.ts";
import { Deadline } from "./deadline.ts";
import { checkDependencies, type Dependency, ParkedLong, waitingOn } from "./dependencies.ts";
import { type FollowAttach, FollowSource } from "./follow.ts";
import type { HostError } from "./host.ts";
import { attachDerives, checkImplementation, contributionOf, type KeyedBinding, type Members } from "./members.ts";
@ -46,7 +52,8 @@ import {
type NativeSlot,
} from "./native.ts";
import type { ChordObject } from "./object.ts";
import type { Binding, CheckedParams, Reference, Target } from "./reference.ts";
import type { Failed } from "./ref-state.ts";
import type { Binding, CheckedParams, Reference, RetryOperation } from "./reference.ts";
import { type RetrySource, retrySource } from "./retry-on.ts";
import {
type OnHandler,
@ -84,8 +91,9 @@ export interface Self<P extends Principal = Principal> extends RoutineDeclaratio
get<C extends Contract>(Type: C, params: ParamsOf<C>): Reference<C>;
/**
* Registry `R`'s composed value as this plugin's version of `R` sees it: a declared dependency, as `get` is
* (activation waits until it has a value; with none it fails `transient` and is retried when one arrives, R18).
* Read `.value` at use; once live, a later failure of the registry keeps its last good value.
* (the attempt parks until it has a usable value: `failed { waiting }` while it has none, woken when it has). Read
* `.value` at use; once live, a later failure of the registry keeps its last good value while every contribution it
* was built from is still in the table.
*/
registry<C, V>(R: Registry<C, V>): RegistryValue<V>;
/**
@ -150,17 +158,14 @@ export type Factory<C extends Contract, P extends Principal = Principal> = Shape
? (params: ParamsOf<C>, self: keyof ParamsOf<C> extends never ? StoreSelf<P> : Self<P>) => Implementation<C, Ctx<P>>
: (params: ParamsOf<C>, principal: P, self: Self<P>) => Implementation<C, Ctx<P>>;
/** How an activation fails on a declared registry without a value (`#readRegistries`). */
type UnreadRegistry = Extract<Activation, { readonly kind: "failed" | "dependency_failed" }>;
/** No declarations of a kind: the shared empty list. */
const NONE: readonly never[] = [];
/** An attempt's registry reads, contributions and wake signals, and its writables (slice 8). */
type RegistryDeclarations = {
/** The registry values it declared (`self.registry`), registered as read during activation (`reads`). */
/** The registry values it declared (`self.registry`): dependencies, as its references are. */
readonly registries: RegistryValueSource[];
/** Releases its registry reads (with its references). */
/** Releases its registry reads (with its references): a read keeps an older version's mapping. */
readonly reads: (() => void)[];
/** Its wake signals (`self.retryOn`), handed to its object right after the factory. */
readonly retrySources: RetrySource[];
@ -171,26 +176,29 @@ type RegistryDeclarations = {
contributions: readonly RegistryContribution[];
};
/** How an attempt's activation ended. `dependency_failed` (R18) keeps the attempt's references. */
/** How an attempt's activation ended. */
type Activation =
| { readonly kind: "live" }
| { readonly kind: "failed"; readonly cause: FailureCause }
| { readonly kind: "dependency_failed"; readonly cause: FailureCause }
| { readonly kind: "ended"; readonly error: ChordError };
/** How an activation that did not go live ended. */
type NotLive = Exclude<Activation, { readonly kind: "live" }>;
/** The error of an object, or a getter, that waited `ms` without the object going live. */
export function notLive(label: string, ms: number): ChordError {
return new ChordError({ code: "unavailable", message: `${label}: did not become live within ${ms} ms` });
}
/**
* One attempt of an object. Activation: factory, documents hydrated (the object's slots), references bound (before the
* previous attempt's are released) and live, natives opened one after another, members checked, derives computed;
* bounded by the host's `readyTimeout`. Ending: sealed at once (calls aborted, its transactions refused and the one
* holding the line discarded, its job invocations cut off), then the drain, the closes in reverse, derives detached;
* its object releases its references. Documents and job runs stay on the object.
* One attempt of an object. Activation: factory, documents hydrated (the object's slots), references bound, then parked
* until every dependency is live (no timer: the deadline paused), then its own steps: natives opened one after
* another, followed sources attached, members checked, derives computed; bounded by the host's `readyTimeout`.
* Ending: sealed at once (calls aborted, its transactions refused and the one holding the line discarded, its job
* invocations cut off), then the drain, the closes in reverse, derives detached; its object releases its references.
* Documents and job runs stay on the object.
*/
export class Attempt implements StoreSelf, KeyDeclarer {
export class Attempt implements StoreSelf, KeyDeclarer, RegistryWaiter {
readonly #object: ChordObject;
/** The object's context: the natives' closes run under it. */
readonly #context: Context;
@ -228,18 +236,21 @@ export class Attempt implements StoreSelf, KeyDeclarer {
* the call path costs no more than before the cleanups.
*/
readonly #running = new Map<Promise<unknown>, (() => void) | undefined>();
/** The previous attempt while it still holds its references (R18): released once this one has bound its own. */
#previous: Attempt | undefined;
/** What activation returns once the attempt was abandoned: a timeout, a cycle through its object, or its end. */
#abandoned: Exclude<Activation, { readonly kind: "live" }> | undefined;
#abandoned: NotLive | undefined;
/** Its dependencies while it is parked on them (the dependency check); `undefined` otherwise. */
#parked: readonly Dependency[] | undefined;
/** A dependency changed since the last check: the parked attempt checks again (one check per batch of changes). */
#moved = false;
/** What a parked attempt waiting for a change waits on; resolved after the change's batch. */
#changed: PromiseWithResolvers<void> | undefined;
/** A followed source's end while this attempt activated: it fails as it goes live (if its activation returned). */
#endedEarly: ChordError | undefined;
/** Its registry reads, contributions, wake signals and writables, made by the first: most attempts have none. */
#registryDeclarations: RegistryDeclarations | undefined;
constructor(object: ChordObject, previous: Attempt | undefined, context: Context) {
constructor(object: ChordObject, context: Context) {
this.#object = object;
this.#previous = previous;
this.#context = context;
this.#opening = withAbortSignal(this.#abort.signal, context);
}
@ -453,7 +464,7 @@ export class Attempt implements StoreSelf, KeyDeclarer {
}
/** Abandons the activation with `outcome` (once): its waits stop and the openers' contexts are aborted. */
abandon(outcome: Exclude<Activation, { readonly kind: "live" }>): void {
abandon(outcome: NotLive): void {
if (this.#abandoned !== undefined) return;
this.#abandoned = outcome;
this.#abort.abort(outcome.kind === "ended" ? outcome.error : outcome.cause.error);
@ -498,6 +509,11 @@ export class Attempt implements StoreSelf, KeyDeclarer {
});
}
/** Whether it is parked on its dependencies (the dependency check runs). */
get isParked(): boolean {
return this.#parked !== undefined;
}
/** Its contributions, resolved by its activation: in the registries' tables while its object is live. */
get contributions(): readonly RegistryContribution[] {
return this.#registryDeclarations?.contributions ?? NONE;
@ -624,9 +640,8 @@ export class Attempt implements StoreSelf, KeyDeclarer {
if (declarations !== undefined) for (const writable of declarations.writables) writable.detach(ended);
}
/** Releases its references and registry reads, and the previous attempt's if this one never bound its own. */
/** Releases its references and registry reads. */
releaseReferences(): void {
this.#releasePrevious();
for (const { binding } of this.#references) binding.release();
const declarations = this.#registryDeclarations;
if (declarations !== undefined) for (const release of declarations.reads.splice(0)) release();
@ -635,8 +650,7 @@ export class Attempt implements StoreSelf, KeyDeclarer {
/** The activation steps; each catches its own failures with their kind (the table in slice 2's spec). */
async #activate(deadline: Deadline): Promise<Activation> {
let implementation: unknown;
let dependencies: Target[];
let unread: UnreadRegistry | undefined;
const dependencies: Dependency[] = [];
try {
implementation = this.#declare();
// Right after the factory, before anything can fail: a failed attempt keeps its wake signals.
@ -650,49 +664,158 @@ export class Attempt implements StoreSelf, KeyDeclarer {
// A persisted job's runs, loaded once per object: rebuilt before anything runs.
for (const job of this.#jobs.values()) await job.load(this.#opening);
const { host } = this.#object;
dependencies = this.#references.map(({ binding, contract, params }) =>
host.bind(binding, contract, params, this.#opening),
);
for (const dependency of dependencies) this.#object.assertNoCycle(dependency, this.#opening);
if (this.#registryDeclarations !== undefined) unread = this.#readRegistries(this.#registryDeclarations);
for (const { binding, contract, params } of this.#references) {
host.bind(binding, contract, params, this.#opening);
dependencies.push({ kind: "reference", binding });
}
// Once all are bound: a dependency made here may have bound back to this object meanwhile.
for (const dependency of dependencies) {
const target = dependency.kind === "reference" ? dependency.binding.target : undefined;
if (target !== undefined) this.#object.assertNoCycle(target, this.#opening);
}
const declarations = this.#registryDeclarations;
for (const value of declarations?.registries ?? NONE) {
// Read from here until its references are released: an older version's mapping is kept meanwhile.
declarations?.reads.push(value.read());
dependencies.push({ kind: "registry", value });
}
} catch (error) {
return this.#broken(error);
}
// Parked only when something is not live yet (most activations find their dependencies live).
if ((dependencies.length > 0 && checkDependencies(dependencies).kind !== "live") || this.#pinsKeys()) {
const parked = await this.#park(dependencies, deadline);
if (parked !== undefined) return parked;
} else {
// Its own steps start in a later turn: an object ended in the turn it was made opens nothing.
await undefined;
}
return this.#ownSteps(implementation);
}
/**
* Parks while a dependency is not live (dependency model §3), with no timer: the deadline paused, `parked_long`'s
* report the only clock. Checks at every change of a dependency (after its batch) and shows what it found on its
* object; once all are live, the keys the factory pinned on them are loaded (a failed load parks it `waiting`
* until the next change or a retry). Resolves `undefined` once all are live and pinned, else with how it ends:
* abandoned, or broken (a cycle, an ended dependency).
*/
async #park(dependencies: readonly Dependency[], deadline: Deadline): Promise<NotLive | undefined> {
const object = this.#object;
deadline.pause();
this.#parked = dependencies;
object.waitsFor(dependencies);
// Waiters on a registry are woken when its value turns usable or its failure changes.
const waits: (() => void)[] = [];
for (const each of dependencies) if (each.kind === "registry") waits.push(each.value.wait(this));
const long = new ParkedLong(object);
try {
for (;;) {
this.#moved = false;
const check = checkDependencies(dependencies);
if (check.kind === "broken")
return this.#abandoned ?? { kind: "failed", cause: { kind: "broken", error: check.error } };
if (check.kind === "live") {
const pinned = await this.#pinKeys();
if (this.#abandoned !== undefined) return this.#abandoned;
// Changed while the keys loaded: checked again.
if (pinned === undefined && !this.#moved) {
// Its own steps follow: `loading` until its outcome (a failure shows as the failure, not this).
object.unparked(this);
return undefined;
}
if (pinned !== undefined) object.parked(this, pinned, true);
long.stop();
} else {
object.parked(this, check.shown, check.firstPass);
long.waitingOn(check.remoteLoading);
}
if (!(await this.#nextChange())) return this.#abandoned!;
}
} finally {
// Bound first: a dependency held only by the previous attempt never ends in between.
this.#releasePrevious();
long.stop();
for (const release of waits) release();
this.#parked = undefined;
object.waitsFor(NONE);
deadline.resume();
}
// No value after `load`'s setups: none comes before a first composition (which retries it, R18) or a reload.
if (unread !== undefined) return this.#abandoned ?? unread;
}
/**
* Waits for the next change of a dependency (at once when one came during the check); `false` when the attempt was
* abandoned meanwhile (its end, the only thing that stops the wait).
*/
async #nextChange(): Promise<boolean> {
if (this.#moved) return true;
this.#changed = Promise.withResolvers();
try {
await Promise.all(dependencies.map((dependency) => this.#object.waitFor(dependency, this.#opening)));
} catch (error) {
const abandoned = this.#abandoned;
// A timeout while waiting keeps the references (R18): a dependency that goes live later retries this object
// at once (a remote one whose link dropped and is held again).
if (abandoned?.kind === "failed" && abandoned.cause.kind === "transient") {
return { kind: "dependency_failed", cause: abandoned.cause };
}
if (abandoned !== undefined) return abandoned;
// `waitFor` rejects with a ChordError naming the dependency, its error as the cause; or with a cycle's
// `invalid/cycle`, which fails this object broken too (proposal, slice 4b: an exception to R18, since every
// retry would meet the same cycle), keeping no references.
const failed = error as ChordError;
if (failed.code === "invalid" && failed.reason === "cycle") {
return { kind: "failed", cause: { kind: "broken", error: failed } };
}
return { kind: "dependency_failed", cause: { kind: "transient", error: failed } };
await awaitWithContext(this.#changed.promise, this.#opening);
return true;
} catch {
return false;
} finally {
this.#changed = undefined;
}
try {
// The keys the factory pinned on its dependencies' keyed members, loaded before this object goes live.
for (const { binding } of this.#references) {
}
/**
* Loads the keys the factory pinned on its dependencies' keyed members (`at(key)`); `undefined` once all are, else
* what the parked attempt shows for the first that failed to load (`waiting` on its reference).
*/
async #pinKeys(): Promise<Failed | undefined> {
for (const { binding } of this.#references) {
try {
const pinning = binding.pinKeys(this.#opening);
if (pinning !== undefined) await pinning;
} catch (error) {
return waitingOn(binding.label, asChordError(error, binding.label));
}
} catch (error) {
// A pin load that failed its provider is a dependency failure (rule 6, R18).
const cause: FailureCause = { kind: "transient", error: asChordError(error, this.#object.label) };
return this.#abandoned ?? { kind: "dependency_failed", cause };
}
return undefined;
}
/**
* A dependency of this parked attempt changed (its state, its cause, its first outcome): it checks again after the
* batch that changed it, once however many changed. A change while it checks or loads pinned keys is remembered
* (`#moved`); one outside parking costs a flag.
*/
dependencyMoved(): void {
if (this.#moved) return;
this.#moved = true;
if (this.#changed !== undefined) queueMicrotask(() => this.#changed?.resolve());
}
/** A registry it is parked on closed with its host: the parked attempt ends with `error`. */
closed(error: ChordError): void {
this.abandon({ kind: "ended", error });
}
/** Whether its factory pinned keys on its dependencies' keyed members (`at(key)`), which load while it is parked. */
#pinsKeys(): boolean {
for (const { binding } of this.#references) if (binding.pinsKeys) return true;
return false;
}
/**
* `ref.retry()` on its parked object (`failed { waiting }`): passed on to every failed dependency (a broken or
* transient one retries, a parked one passes it on, a request waiting in an index is left alone), as `operation`;
* then it checks again (a key that failed to load is loaded again).
*/
retryDependencies(operation: RetryOperation): void {
for (const dependency of this.#parked ?? NONE) {
if (dependency.kind === "reference" && dependency.binding.refState.kind === "failed") {
dependency.binding.retry(operation);
}
}
this.dependencyMoved();
}
/**
* The steps after the dependency check (its own: natives, followed sources, the member check, derives, the keys
* references pin on its keyed members), under the deadline. A dependency that leaves `live` meanwhile changes
* nothing (R18 after live).
*/
async #ownSteps(implementation: unknown): Promise<Activation> {
for (const native of this.#natives) {
// An attempt abandoned before this open (ended while it waited) closed its natives already: none opens late.
if (this.#abandoned !== undefined) return this.#abandoned;
@ -711,14 +834,16 @@ export class Attempt implements StoreSelf, KeyDeclarer {
return this.#abandoned ?? { kind: "failed", cause };
}
}
let members: Members;
try {
const checked = checkImplementation(this.#object.served.contract, implementation);
this.#members = checked.members;
members = checked.members;
this.#members = members;
this.#variant = checked.variant;
if (this.#registryDeclarations !== undefined)
this.#registryDeclarations.contributions = this.#resolveContributions(this.#registryDeclarations);
const { host, label } = this.#object;
attachDerives(this.#derives, this.#members.state, (member) => (error) => {
attachDerives(this.#derives, members.state, (member) => (error) => {
// During activation a throwing derive fails the attempt, which `activation_failed` reports alone.
if (!this.#isLive) return;
host.report({ kind: "derive_failed", object: label, ...(member === undefined ? {} : { member }), error });
@ -729,40 +854,15 @@ export class Attempt implements StoreSelf, KeyDeclarer {
}
try {
// The keys references pin on the object's keyed members, loaded on this attempt's before it goes live.
const preparing = this.#object.prepareKeys(this.#members.keyed, this.#opening);
const preparing = this.#object.prepareKeys(members.keyed, this.#opening);
if (preparing !== undefined) await preparing;
} catch (error) {
const cause: FailureCause = { kind: "transient", error: asChordError(error, this.#object.label) };
return this.#abandoned ?? { kind: "dependency_failed", cause };
return this.#abandoned ?? { kind: "failed", cause };
}
return this.#abandoned ?? { kind: "live" };
}
/**
* Registers this attempt's object as a reader of every registry it declared (woken by a first value, R18, until its
* references are released); returns how the activation fails on the first with no value, `undefined` when every
* one has a value. A version the provider does not serve (S2) is final until a reload: broken, the
* `contract_mismatch` itself the cause. Otherwise (no provider, a first `compose` that threw) a value may still come:
* `transient` (R18, the references kept), naming the registry, its error as the cause.
*/
#readRegistries({ registries, reads }: RegistryDeclarations): UnreadRegistry | undefined {
let unread: UnreadRegistry | undefined;
for (const value of registries) {
reads.push(value.read(this.#object));
const start = value.start();
if (unread !== undefined || start.kind === "value" || start.last !== undefined) continue;
const { error } = start;
if (isChordError(error, "contract_mismatch")) {
unread = { kind: "failed", cause: { kind: "broken", error } };
continue;
}
const message = `${this.#object.label}: registry ${value.name} has no value`;
const cause = new ChordError({ code: "unavailable", message }, { cause: error });
unread = { kind: "dependency_failed", cause: { kind: "transient", error: cause } };
}
return unread;
}
/** Its declared contributions, as Chord follows them (its references are live now), by its plugin and type. */
#resolveContributions({ declared }: RegistryDeclarations): readonly RegistryContribution[] {
const { served, host } = this.#object;
@ -847,11 +947,6 @@ export class Attempt implements StoreSelf, KeyDeclarer {
return this.#abandoned ?? { kind: "failed", cause };
}
#releasePrevious(): void {
this.#previous?.releaseReferences();
this.#previous = undefined;
}
/** The registry edges its declared contributions would add (another registry's value contributed). */
#pendingEdges(): RegistryEdge[] {
const edges: RegistryEdge[] = [];

2
packages/chord/src/next/host/backoff.ts
View file

@ -58,7 +58,7 @@ export class Backoff {
return entry?.timer === undefined ? undefined : entry.until;
}
/** Ends the window at once (its object is retried now, R18); the delay stays for the next failure. */
/** Ends the window at once (its object is retried now: `ref.retry()`, a wake); the delay stays for the next failure. */
skip(identity: string): void {
const entry = this.#entries.get(identity);
clearTimeout(entry?.timer);

142
packages/chord/src/next/host/dependencies.ts Normal file
View file

@ -0,0 +1,142 @@
// Parked attempts (dependency model §3, §7): the dependency check, what an attempt's declared dependencies (`self.get`
// references and `self.registry` values) say at one moment: all live, or what the parked attempt shows while one is not
// (`loading`, `failed { waiting }` naming the nearest blocker and the root cause, or `broken` for a cycle, authority or
// an ended dependency), whether this host's first pass can count it (no blocker here still without its first outcome, decision
// (a)), and the remote blocker that is only loading; and the `parked_long` report a long wait on that blocker makes.
import { type ChordError, isChordError } from "../data/index.ts";
import { asChordError, waitingFor } from "../data/internal.ts";
import type { RegistryValueSource } from "../registries/internal.ts";
import type { ChordObject } from "./object.ts";
import { type Failed, LOADING, type RefState } from "./ref-state.ts";
import type { Binding } from "./reference.ts";
/** One declared dependency: a reference (`self.get`) or a registry's value (`self.registry`). */
export type Dependency =
| { readonly kind: "reference"; readonly binding: Binding }
| { readonly kind: "registry"; readonly value: RegistryValueSource };
/**
* What the dependencies say. `live`: every one is live (hydrated: a reference's members have values). `parked`: what
* the attempt shows (`loading`, or `failed { waiting }` naming its first failed blocker), whether its first pass is
* over (`firstPass`: no blocker, loading or failed, is an object here still without its first outcome), and a remote
* blocker whose hold is on a link and only loading (its label), for `parked_long`. `broken`: one failed for a cycle
* (`invalid/cycle`), for authority (`forbidden`) or ended: it never comes back by itself.
*/
export type Check =
| { readonly kind: "live" }
| {
readonly kind: "parked";
readonly shown: RefState;
readonly firstPass: boolean;
readonly remoteLoading: string | undefined;
}
| { readonly kind: "broken"; readonly error: ChordError };
const LIVE: Check = { kind: "live" };
/** Checks `dependencies` in declaration order (the first failed one is the one the attempt names). */
export function checkDependencies(dependencies: readonly Dependency[]): Check {
let failed: Failed | undefined;
let firstPass = true;
let remoteLoading: string | undefined;
let loading = false;
for (const dependency of dependencies) {
if (dependency.kind === "registry") {
const { value } = dependency;
const start = value.start();
if (start.kind === "value" || start.last !== undefined) continue;
failed ??= waitingOn(`registry ${value.name}`, asChordError(start.error, `registry ${value.name}`));
continue;
}
const { binding } = dependency;
const state = binding.refState;
switch (state.kind) {
case "live":
continue;
case "ended":
return { kind: "broken", error: state.error };
case "failed": {
const { error } = state.cause;
if (healsNever(error)) return { kind: "broken", error };
failed ??= waitingOn(binding.label, error);
// Parked `waiting` on something here still in its first attempt: no first outcome yet (fix round 1).
if (binding.target?.blocksFirstPass === true) firstPass = false;
continue;
}
case "loading": {
loading = true;
const target = binding.target;
if (target?.blocksFirstPass === true) firstPass = false;
if (target?.heldOnLink === true) remoteLoading ??= target.label;
}
}
}
if (failed === undefined && !loading) return LIVE;
return { kind: "parked", shown: failed ?? LOADING, firstPass, remoteLoading };
}
/**
* Whether a dependency failed with `error` never heals by itself, so its dependent breaks too: a cycle
* (`invalid/cycle`: it would wait on a ring) or authority (`forbidden`: an `authorize` that refused, M§3.6; only
* `retry()` or a reload helps).
*/
function healsNever(error: ChordError): boolean {
return (isChordError(error, "invalid") && error.reason === "cycle") || isChordError(error, "forbidden");
}
/** A parked attempt's state for a blocker `label` that failed with `error`. */
export function waitingOn(label: string, error: ChordError): Failed {
return { kind: "failed", cause: waitingFor(label, error), retryAt: undefined };
}
/**
* The `parked_long` report of one parking episode (dependency model §7): once, when the attempt has waited
* `readyTimeout` on a remote blocker that is only loading with its hold on a link (a ring across hosts started at the
* same moment stays parked so; nothing fails or retries). The clock, and the report's `since`, start when such a
* blocker is there and stop when none is (a link that drops, a path still without one): only the time a remote
* attempt answered for the wait counts. Its one timer runs only meanwhile.
*/
export class ParkedLong {
readonly #object: ChordObject;
/** When the attempt began to wait on a remote blocker that is only loading on a link, while it does. */
#since: number | undefined;
#timer: ReturnType<typeof setTimeout> | undefined;
/** The remote blocker it would name now. */
#blocker: string | undefined;
#reported = false;
constructor(object: ChordObject) {
this.#object = object;
}
/** What the check found: `blocker`, a remote attempt that is only loading on a link, or none. */
waitingOn(blocker: string | undefined): void {
this.#blocker = blocker;
if (blocker === undefined) {
this.stop();
return;
}
if (this.#reported || this.#timer !== undefined) return;
this.#since ??= Date.now();
const due = this.#since + this.#object.host.timeouts.ready - Date.now();
this.#timer = setTimeout(() => this.#report(), Math.max(0, due));
}
/** The episode ended, or no remote blocker is loading on a link now: the clock and its timer stop. */
stop(): void {
clearTimeout(this.#timer);
this.#timer = undefined;
this.#since = undefined;
}
#report(): void {
this.#timer = undefined;
const waitingFor = this.#blocker;
const since = this.#since;
if (this.#reported || waitingFor === undefined || since === undefined) return;
this.#reported = true;
const object = this.#object;
object.host.report({ kind: "parked_long", object: object.label, waitingFor, since });
}
}

53
packages/chord/src/next/host/destination.ts
View file

@ -1,6 +1,6 @@
// Destination (§3, section 1's table): where a reference made on a host, or a hold arriving on one of its links, goes:
// here, over the route of its host address, or over the parent link; and the check that an object served here lives
// here.
// Destination (§3, section 1's table; dependency model §1): where a reference made on a host, or a hold arriving on one
// of its links, goes, by address alone: here when the address is this host's (served here or not), else over the
// route of its host address's kind, or over the parent link; and the check that an object served here lives here.
import type { Context } from "../../context/index.ts";
import type { Address, Contract, ParamsOf } from "../contracts/index.ts";
@ -9,7 +9,6 @@ import { ChordError, type Principal } from "../data/index.ts";
import { invalid } from "../data/internal.ts";
import type { WireAddress } from "../link/internal.ts";
import type { HoldRequest, HostLink, Path } from "../remote/internal.ts";
import type { Served } from "./object.ts";
import type { Route } from "./route.ts";
/** Where something is placed: here, or over a path (with the host address it was computed or named with). */
@ -17,47 +16,42 @@ export type Placed =
| { readonly kind: "here" }
| { readonly kind: "path"; readonly path: Path; readonly address: WireAddress };
/** A host's destinations: its address, its served types, its routes and its parent path, as the host keeps them. */
/** A host's destinations: its address, its routes and its parent path, as the host keeps them. */
export class Destination {
readonly #address: Address | undefined;
readonly #types: ReadonlyMap<string, Served>;
readonly #routes: ReadonlyMap<string, Route>;
readonly #parent: Path;
constructor(
address: Address | undefined,
types: ReadonlyMap<string, Served>,
routes: ReadonlyMap<string, Route>,
parent: Path,
) {
constructor(address: Address | undefined, routes: ReadonlyMap<string, Route>, parent: Path) {
this.#address = address;
this.#types = types;
this.#routes = routes;
this.#parent = parent;
}
/**
* Where a reference to `Type` with `params` made as `principal` goes: here for a local type (served here: the host
* refused it otherwise) or a remote type served here whose host address is this host's; otherwise over the route of
* its host address's kind, or over the parent link (the host address decides where an object lives: a worker
* reaches another worker's object of a type it serves too). Throws `invalid/host_address` for a host address whose
* params do not fit its kind, `forbidden` when the route refuses, and what a `hostAddress` or `authorize` throws.
* Where a reference to `Type` with `params` made as `principal` goes: here for a local type, and for a remote type
* whose host address is this host's, served here or not (the check against what is served is the host's); otherwise
* over the route of its host address's kind, or over the parent link (the host address decides where an object
* lives: a worker reaches another worker's object of a type it serves too). Throws `invalid/host_address` for a host
* address whose params do not fit its kind, `forbidden` when the route refuses, and what a `hostAddress` or
* `authorize` throws.
*/
reference(Type: Contract, params: ParamsOf<Contract>, principal: Principal | undefined, context: Context): Placed {
if (Type.kind === "local") return HERE;
const address = hostAddressOf(Type, params);
if (this.#types.has(Type.name) && this.#isHere(address)) return HERE;
if (this.#isHere(address)) return HERE;
return { kind: "path", path: this.#pathTo(address, principal, context) ?? this.#parent, address };
}
/**
* Where a hold arriving on `link` acting as `principal` goes: here when its type is served here; over the route of
* the address it names; else over the parent link (waiting for it while it is down), unless it came on the parent
* link or this host never had one (`unavailable`). Throws as `reference` does.
* Where a hold arriving on `link` acting as `principal` goes, by the address it names: here when it is this host's
* (a relay never captures a hold for serving the same name); over the route of its kind; else over the parent link
* (waiting for it while it is down), unless it came on the parent link or this host never had one (`unavailable`).
* Throws as `reference` does.
*/
hold(request: HoldRequest, link: HostLink, principal: Principal | undefined, context: Context): Placed {
if (this.#types.has(request.type)) return HERE;
const { address } = request;
if (this.#isHere(address)) return HERE;
const routed = this.#pathTo(address, principal, context);
if (routed !== undefined) return { kind: "path", path: routed, address };
if (link.role === "parent" || !this.#parent.linked) {
@ -67,15 +61,16 @@ export class Destination {
}
/**
* Throws `invalid/host_address` for a hold of `Type` with `params` (checked) served here but whose host address is
* another host (misrouted; never forwarded). The host address is computed here from the params: the address a hold
* names only routes it.
* The provider's own check of a request placed here for the remote `Type` it serves, with `params` (checked):
* `invalid/host_address` when its `hostAddress` puts the object on another host (code skew: the consumer computed
* this host), `undefined` when it lives here. A `hostAddress` that throws throws (the provider's own code). The
* address a request names only routes it.
*/
assertHere(Type: Contract, params: ParamsOf<Contract>): void {
misplaced(Type: Contract, params: ParamsOf<Contract>): ChordError | undefined {
const address = hostAddressOf(Type, params);
if (this.#isHere(address)) return;
if (this.#isHere(address)) return undefined;
// A host serving a remote type has an address (`serve` refuses one without).
throw invalid("host_address", `${Type.name}: placed on ${address.kind}, this host is ${this.#address!.key}`);
return invalid("host_address", `${Type.name}: placed on ${address.kind}, this host is ${this.#address!.key}`);
}
/**

351
packages/chord/src/next/host/host.ts
View file

@ -28,6 +28,7 @@ import {
import { binaryCodec, type Codec, type Compression, type Link, type PartLimits } from "../link/index.ts";
import { linkLimits } from "../link/internal.ts";
import type { Registry, RegistryValue } from "../registries/index.ts";
import type { RegistryValueSource } from "../registries/internal.ts";
import {
type RegistryContribution,
type RegistryEdge,
@ -41,6 +42,7 @@ import {
Path,
type ProviderSide,
RemoteObject,
type Upstream,
} from "../remote/internal.ts";
import { Batch, type BatchError } from "../state/internal.ts";
import type { ChordStore, Store } from "../store/store.ts";
@ -58,6 +60,7 @@ import { Binding, type CheckedParams, type Held, type Reference, type Target } f
import { RevocationKeys, type RevokedBy } from "./revocation.ts";
import { type RootOptions, RootScopes } from "./root.ts";
import { Route, type RouteOptions } from "./route.ts";
import { UnboundIndex, type UnboundRequest } from "./unbound.ts";
import { uuidv7 } from "./uuid.ts";
/** What `createHost` takes; `R` is the app's revocation type (JSON). */
@ -151,7 +154,12 @@ export type HostError =
readonly restarts: number;
}
/** A blocked run was dropped without `finish`: aborted, or its object ended. */
| { readonly kind: "job_dropped"; readonly object: string; readonly job: string; readonly id: string };
| { readonly kind: "job_dropped"; readonly object: string; readonly job: string; readonly id: string }
/**
* An attempt parked `readyTimeout` (since `since`, `Date.now()` time) while `waitingFor`, a remote attempt, is only
* loading: once per parking episode, a diagnostic of a ring across hosts started at the same moment (nothing fails).
*/
| { readonly kind: "parked_long"; readonly object: string; readonly waitingFor: string; readonly since: number };
/**
* What `accept` and `connect` take: the principal of the host at the other end, as this host stamps it; `trusted` skips
@ -184,22 +192,26 @@ export type AnyHost = Omit<ChordHost, "load" | "accept" | "connect" | "route" |
/** A host. `P` is the app's principal type, `R` its revocation type. */
export interface Host<P extends Principal = Principal, R extends JsonValue = JsonValue> {
/**
* Runs every facet's setup, then activates every served type with empty params and resolves when each has had its
* first outcome, live or failed (the start barrier; failures go to `onError`). A setup that throws rejects it and
* closes the host. Runs once.
* Runs every facet's setup and publishes what they declared in one batch (the types and registries served, the
* registries composed once, the contributions); requests that waited for the publication are placed after that
* batch. Then activates every served type with empty params and resolves when each has had its first outcome (the
* start barrier, decision (a)): live, failed by its own cause, parked with no blocker here still without its first
* outcome, or ended (failures go to `onError`). A setup that throws rejects it and closes the host. Runs once.
*/
load(facets: readonly { readonly name: string; readonly facet: Facet<P> }[], context: Context): Promise<void>;
/**
* A reference once its object is live; waits for `load` first. It acts as the host's principal and goes where its
* type's host address puts it: here, over a route, or over the parent link (`Destination`). A broken failure
* rejects at once; through transient ones it waits until its deadline, then rejects with the last failure's error.
* A reference once its object is live (and hydrated). It acts as the host's principal and goes where its type's host
* address puts it: here, over a route, or over the parent link (`Destination`). It waits while the reference is
* `loading` (before the host's publication too), `failed { transient }` or `failed { waiting }` until its
* `readyTimeout`, then rejects with the last cause and releases the reference; a broken failure rejects at once.
*/
get<C extends Contract>(Type: C, params: ParamsOf<C>, context: Context, options?: GetOptions): Promise<Reference<C>>;
/**
* A reference at once, following its object until released: loading, live, failed (retried as its object is) and
* loading again while its link is down; it never rejects. Throws at once for invalid params (also before `load`)
* and a local type not served here; before `load` resolved, a type not served here fails the reference broken once
* it did. Its `variant` is `undefined` until its first binding.
* loading again while its link is down; it never rejects. Throws at once only for invalid params (also before
* `load`). A type not served where it goes is `loading` before the host's publication and `failed { waiting }` after
* it (bound when a publication serves it); a `hostAddress` or route that refuses fails it broken. Its `variant` is
* `undefined` until its first binding.
*/
hold<C extends Contract>(Type: C, params: ParamsOf<C>, context: Context, options?: GetOptions): Held<C>;
/**
@ -244,11 +256,12 @@ export interface Host<P extends Principal = Principal, R extends JsonValue = Jso
*/
readonly activity: Watchable<ActivityCounts>;
/**
* Registry `R`'s composed value as `R`'s version sees it, for code outside objects; waits for `load` first. Rejects
* when it has no value then (`unavailable/undeclared` without a provider, a first `compose` that threw, a version
* not served here: `contract_mismatch`); none can come before a reload.
* Registry `R`'s composed value as `R`'s version sees it, for code outside objects, once it is usable: waits for the
* host's publication, then while it has no value (`unavailable/undeclared` without a provider, a first `compose`
* that threw, a version not served here: `contract_mismatch`) until its `readyTimeout`, then rejects with the last
* error.
*/
registry<C, V>(R: Registry<C, V>, context: Context): Promise<RegistryValue<V>>;
registry<C, V>(R: Registry<C, V>, context: Context, options?: GetOptions): Promise<RegistryValue<V>>;
/**
* Runs `fn` with a call context for code outside objects (a Durable tool, a CLI): the host's principal (or
* `options.principal`), this host's incarnation, `context`'s telemetry and signal, `ctx.get`. When `fn` settles or
@ -263,6 +276,14 @@ export interface Host<P extends Principal = Principal, R extends JsonValue = Jso
close(context: Context): Promise<void>;
}
/**
* What a hold placed here binds to (`serving`): its object, or why it waits unbound (`failed { waiting }`: not served,
* another version, or placed elsewhere by the provider's code).
*/
export type Serving =
| { readonly kind: "object"; readonly object: ChordObject }
| { readonly kind: "waiting"; readonly cause: FailureCause };
/** Creates a host. */
export function createHost<P extends Principal = Principal, R extends JsonValue = JsonValue>(
options: HostOptions<P, R>,
@ -305,7 +326,11 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
readonly #objects = new Map<string, ChordObject>();
/** Teardowns still running, by identity: the next instance of an identity waits for its previous ones. */
readonly #endings = new Map<string, Promise<void>>();
readonly #loaded = Promise.withResolvers<void>();
/** The requests that cannot bind yet, by name: before the publication all of them, then those left unbound. */
readonly #unbound = new UnboundIndex();
/** Resolved at the host's first publication (`load`'s batch), or its close. */
readonly #publication = Promise.withResolvers<void>();
#published = false;
/** Every open link: children, routes and the parent. */
readonly #links = new Set<HostLink>();
/** Where holds go that are not served here and not routed: the parent link, across reconnects. */
@ -320,7 +345,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
/** The downstream origins this host relays holds for, by their number on upstream links (never reused). */
readonly relays = new Map<number, ProviderSide>();
#nextOrigin = 1;
/** Aborted when the host closes: the context of every route's `resolve`. */
/** Aborted when the host closes: the context of every route's `resolve`, and the end of waiting `host.registry`s. */
readonly #routing = new AbortController();
readonly #heartbeat: { readonly ping: number; readonly close: number };
/** Its registries by name, each made by its first use (a serve, a contribution or a reader). */
@ -344,7 +369,7 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
this.#revocationKeys = new RevocationKeys(options.revokedBy as RevokedBy<Principal, JsonValue> | undefined);
this.timeouts = { close: options.closeTimeout ?? 30_000, ready: options.readyTimeout ?? 60_000 };
this.#heartbeat = { ping: options.heartbeat?.ping ?? 15_000, close: options.heartbeat?.close ?? 30_000 };
this.#destination = new Destination(this.address, this.#types, this.#routes, this.#parent);
this.#destination = new Destination(this.address, this.#routes, this.#parent);
// Chord's own: `openStore` made it.
this.store = options.store as ChordStore | undefined;
this.store?.attach({ failed: (error) => this.#storageFailed(error), batch: this.batch });
@ -357,8 +382,8 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
this.#state = "loading";
try {
const contributions = this.#setUp(facets);
// The facets' contributions and the first composition of every served registry, in one batch, before any
// object activates.
// The publication: the facets' contributions and the first composition of every served registry, in one
// batch, before any object activates.
this.batch.commit(() => {
for (const contribution of contributions) contribution.record.add(contribution);
for (const record of this.#registries.values()) record.composeServed();
@ -369,12 +394,15 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
await this.#closing.catch(() => {});
throw error;
}
// After the batch: what waited for the publication is placed (the gate lasted only for the setups).
this.#published = true;
this.#publication.resolve();
this.#unbound.wakeAll();
const objects = [...this.#types.values()]
.filter((served) => served.kind === "local" && served.hostLifetime)
.map((served) => this.#object(served, EMPTY_PARAMS, undefined, context));
await Promise.allSettled(objects.map(({ settled }) => settled));
await Promise.all(objects.map(({ firstOutcome }) => firstOutcome));
if (this.#state === "loading") this.#state = "loaded";
this.#loaded.resolve();
}
async get<C extends Contract>(
@ -435,9 +463,18 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
for (const object of [...this.#objects.values()]) {
if (object.principal !== undefined && revoked(object.principal)) object.end(error(object.label));
}
// Holds admitted before this revoke but still waiting for `load` would take revoked objects afterwards.
const waiting = error("a hold waiting for load");
for (const link of this.#links) link.revokeWaiting(revoked, waiting);
// Requests not bound (waiting for the publication or a provider, refused, waiting for their link) end too: nothing
// revives them.
const unbound = error("an unbound request");
this.#unbound.revoke(revoked, unbound);
for (const link of this.#links) link.revokeUnbound(revoked, unbound);
// This host's own requests over its paths, linked or not (a route still resolving, a link down): a later link
// would hold them again (fix round 1).
for (const holder of this.#pathHolders()) {
if (holder.kind !== "remote") continue;
const { principal } = holder.caller;
if (principal !== undefined && revoked(principal)) holder.revoke(error(holder.label));
}
// Down the route links before the child links close: a worker ends the principal's objects with the revocation's
// error before the closed origins release their holds.
for (const link of this.#links) if (link.role === "route") link.send({ kind: "revoke", revocation });
@ -463,16 +500,35 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
return references as unknown as Watchable<readonly Presence<C, P>[]>;
}
async registry<C, V>(R: Registry<C, V>, context: Context): Promise<RegistryValue<V>> {
async registry<C, V>(R: Registry<C, V>, context: Context, options: GetOptions = {}): Promise<RegistryValue<V>> {
this.batch.assertOutside("registry");
const loading = this.pendingLoad;
if (loading !== undefined) await awaitWithContext(loading, context);
this.#assertOpen();
const value = this.registryRecord(R.name).value(R.version);
const start = value.start();
if (start.kind === "failed" && start.last === undefined) throw start.error;
// Its values are the composition as `R`'s version has it, `V`.
return value as RegistryValue<V>;
const timeout = options.readyTimeout ?? this.timeouts.ready;
const expired = new AbortController();
const waiting = withLinkedAbortSignal(expired.signal, context);
let last: unknown;
const deadline = setTimeout(() => expired.abort(last ?? notLive(`registry ${R.name}`, timeout)), timeout);
// The host's close ends the wait at once, `released/closed` (its timer with it).
const closing = (): void => expired.abort(this.#closedError());
this.#routing.signal.addEventListener("abort", closing, { once: true });
try {
await awaitWithContext(this.#publication.promise, waiting.context);
this.#assertOpen();
const value = this.registryRecord(R.name).value(R.version);
// Its values are the composition as `R`'s version has it, `V`.
const usable = (): boolean => {
const start = value.start();
if (start.kind === "value" || start.last !== undefined) return true;
last = start.error;
return false;
};
if (!usable()) await untilUsable(value, usable, waiting.context);
return value as RegistryValue<V>;
} finally {
clearTimeout(deadline);
this.#routing.signal.removeEventListener("abort", closing);
waiting.release();
}
}
async root<T>(fn: (ctx: Ctx<P>) => Promise<T>, context: Context, options: RootOptions<P> = {}): Promise<T> {
@ -499,11 +555,21 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
}
/**
* What a hold arriving before `load` resolved waits for; `undefined` once it resolved (or the host closed), so a
* later hold takes its reference at once, in the turn it arrived (R18's handover: a new hold before the old's release).
* Whether the host published its setups (`load`'s batch): until then requests made here and holds arriving wait in
* the unbound index, placed by the publication.
*/
get pendingLoad(): Promise<void> | undefined {
return this.#state === "new" || this.#state === "loading" ? this.#loaded.promise : undefined;
get published(): boolean {
return this.#published;
}
/** `request` cannot bind yet: indexed under `name` (the type or registry it waits for) until woken or ended. */
index(name: string, request: UnboundRequest): void {
this.#unbound.add(name, request);
}
/** `request` left the unbound index's wait for `name` (it ended, or a retry checks it again). */
unindex(name: string, request: UnboundRequest): void {
this.#unbound.delete(name, request);
}
/** Whether the host closes: its objects end through `close`, not through its links. */
@ -513,8 +579,9 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
/**
* A reference to `Type` made by `caller` (its principal, its origin's incarnation), once it is live (`host.get`,
* `ctx.get`): a hold, waited for up to the getter's one deadline, across reconnects; a broken failure rejects at
* once, the deadline with the last failure's error. A getter that gives up releases its reference.
* `ctx.get`): a hold, waited for up to the getter's one deadline, across reconnects and through `loading`,
* `failed { transient }` and `failed { waiting }`; a broken failure rejects at once, the deadline with the last
* failure's error. A getter that gives up releases its reference.
*/
async acquire(
Type: Contract,
@ -567,74 +634,82 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
/**
* An unbound reference to an object of `Type` made by `caller`, declared by `dependent` (`undefined` for
* `host.get`, `host.hold` and `ctx.get`). Throws where it cannot go (`#assertRoute`).
* `host.get`, `host.hold` and `ctx.get`).
*/
reference(Type: Contract, dependent: ChordObject | undefined, caller: Caller, options: GetOptions): Binding {
this.#assertOpen();
this.#assertRoute(Type);
return this.#binding(Type, dependent, caller, options);
}
/**
* Binds `binding` to the object for (Type, params) as its principal where its host address puts it (`Destination`):
* the object here, created with its first attempt if there is none, or a remote object held over its path. Throws
* as `Destination.reference` does.
* the object here, created with its first attempt if there is none, or a remote object held over its path. Placed
* here, the check is only against the type served here: not served (or of the other kind) or not at the caller's
* version and fingerprint, the reference waits unbound, `failed { waiting }`, indexed under the type's name. Returns
* the object it bound to; throws as `Destination.reference` does (a broken refusal).
*/
bind(binding: Binding, Type: Contract, params: CheckedParams, context: Context): Target {
bind(binding: Binding, Type: Contract, params: CheckedParams, context: Context): Target | undefined {
const { principal } = binding.caller;
const placed = this.#destination.reference(Type, params.params, principal, context);
let target: Target;
if (placed.kind === "path") {
const { caller, dependent } = binding;
const remote = new RemoteObject(this, Type, params, placed.address, caller, placed.path, dependent);
placed.path.attach(remote);
target = remote;
} else {
// Placed here: its type is served here.
const served = this.#types.get(Type.name)!;
target = this.#object(served, params, principal, context, NO_CHAIN, binding.dependent);
binding.bind(remote);
return remote;
}
binding.bind(target);
return target;
const served = this.#types.get(Type.name);
const unbound =
unboundCause(served, Type.kind, Type.name, Type.version, Type.fingerprint) ??
this.#misplaced(Type, served!, params.params);
if (unbound !== undefined) {
this.#wait(binding, Type.name, unbound, () => this.#place(binding, Type, params, context));
return undefined;
}
// Checked above: served here at the caller's version.
const object = this.#object(served!, params, principal, context, NO_CHAIN, binding.dependent);
binding.bind(object);
return object;
}
/** Where a hold arriving on `link` goes (`Destination.hold`); `released` once the host closed. */
/** Where a hold arriving on `link` goes (`Destination.hold`: by the address it names); `released` once the host closed. */
place(request: HoldRequest, link: HostLink, caller: Caller, context: Context): Placed {
this.#assertOpen();
return this.#destination.hold(request, link, caller.principal, context);
}
/**
* The object a hold placed here binds to, acting as `principal`: a remote type served here (otherwise
* `unavailable`) with the hold's version and fingerprint (`contract_mismatch`), valid params, and placed here, its
* host address computed from the params (`invalid/host_address`); not an activation the hold's chain waits for
* (`invalid/cycle`, Q5). A new object keeps the hold's chain.
* What a hold placed here binds to, acting as `principal`: the object, or why it waits unbound (`failed
* { waiting }`): a remote type not served here (`not_served`), not at the hold's version and fingerprint, or whose
* params put it on another host (code skew: `version`). Throws for a broken refusal: invalid params, an activation
* the hold's chain waits for (`invalid/cycle`, Q5). A new object keeps the hold's chain.
*/
serving(request: HoldRequest, params: unknown, principal: Principal, context: Context): ChordObject {
serving(request: HoldRequest, params: unknown, principal: Principal, context: Context): Serving {
const { type, version, fingerprint, chain = NO_CHAIN } = request;
const served = this.#types.get(type);
if (served?.kind !== "remote") {
throw new ChordError({ code: "unavailable", message: `${type} is not served here` });
}
this.#assertMatch(served, version, fingerprint);
const checked = checkParams(served.contract, params, context);
this.#destination.assertHere(served.contract, checked.params);
const object = this.#object(served, checked, principal, context, chain);
const unbound = unboundCause(served, "remote", type, version, fingerprint);
if (unbound !== undefined) return { kind: "waiting", cause: unbound };
// Checked above: a remote type served here.
const remote = served as Extract<Served, { readonly kind: "remote" }>;
const checked = checkParams(remote.contract, params, context);
const misplaced = this.#destination.misplaced(remote.contract, checked.params);
if (misplaced !== undefined) return { kind: "waiting", cause: { kind: "waiting", error: misplaced } };
const object = this.#object(remote, checked, principal, context, chain);
const activation = object.activationId;
if (activation !== undefined && chain.includes(activation)) throw this.cycle(chain, activation, [object.label]);
return object;
return { kind: "object", object };
}
/**
* The path a remote reference of `Type` with `params` acting as `principal` goes over now: `ref.retry()` places a
* reference failed for a part past the limits anew (a route may have forgotten its old path). Throws as
* `Destination.reference` does, and `unavailable` for a type served here now.
* `Destination.reference` does, and `unavailable` for a type placed here now.
*/
pathOf(Type: Contract, params: ParamsOf<Contract>, principal: Principal | undefined): Path {
// A retry has no context of its own: its admission checks count on none.
const placed = this.#destination.reference(Type, params, principal, BACKGROUND_CONTEXT);
if (placed.kind === "path") return placed.path;
throw new ChordError({ code: "unavailable", message: `${Type.name} is served here now` });
throw new ChordError({ code: "unavailable", message: `${Type.name} is placed here now` });
}
/**
@ -713,47 +788,55 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
/**
* A reference to `Type` made by `caller` for `host.hold` and getters: params checked at once (`invalid/params`,
* also before `load`), where it goes too (`#assertRoute`), then bound (a `hostAddress` or route that fails fails
* the reference broken); before `load` resolved, where it goes once it did (a host closed meanwhile releases it).
* also before `load`), then placed (`#place`); before the host's publication it waits in the unbound index,
* `loading`, and the publication places it.
*/
#hold(Type: Contract, params: unknown, caller: Caller, context: Context, options: GetOptions): Binding {
this.#assertOpen();
const checked = checkParams(Type, params, context);
if (this.pendingLoad === undefined) {
const binding = this.reference(Type, undefined, caller, options);
this.#bindOrFail(binding, Type, checked, context);
return binding;
}
const binding = this.#binding(Type, undefined, caller, options);
// Raced with the reference's end: a release before `load` resolved lets go of the binding, its contract and
// `context` at once (only the race's settled promise stays behind in `load`'s reactions).
void Promise.race([this.#loaded.promise, binding.ended()]).then(() => {
// Released while `load` ran: nothing is made for it.
if (binding.refState.kind === "ended") return;
if (this.#state === "closed") return binding.end(this.#closedError());
this.#bindOrFail(binding, Type, checked, context);
});
const place = (): void => this.#place(binding, Type, checked, context);
if (this.#published) place();
else this.#wait(binding, Type.name, undefined, place);
return binding;
}
/**
* Binds a reference (`bind`); where it cannot go (`#assertRoute`, a `hostAddress` or `authorize` that throws or
* refuses) fails it broken, until `ref.retry()` binds it again.
* Places a reference (`bind`); where it cannot go (a `hostAddress` or `authorize` that throws or refuses) fails it
* broken, until `ref.retry()` places it again. One woken after the host closed ends as the close ended the others.
*/
#bindOrFail(binding: Binding, Type: Contract, checked: CheckedParams, context: Context): void {
#place(binding: Binding, Type: Contract, checked: CheckedParams, context: Context): void {
if (this.#state === "closed") {
binding.end(this.#closedError());
return;
}
try {
this.#assertRoute(Type);
this.bind(binding, Type, checked, context);
} catch (error) {
// A retry after the host closed ends the reference as the close did the others.
const again = () =>
this.#state === "closed"
? binding.end(this.#closedError())
: this.#bindOrFail(binding, Type, checked, context);
const again = (): void => this.#place(binding, Type, checked, context);
binding.refuse({ kind: "broken", error: asChordError(error, Type.name) }, again);
}
}
/**
* The provider's own host-address check of a remote `Type` placed here by the consumer's contract, against the one
* served (`invalid/host_address`, `waiting`: code skew); none when both have the same `hostAddress` (the consumer
* computed it with the provider's function) or the type is local. A provider's `hostAddress` that throws throws.
*/
#misplaced(Type: Contract, served: Served, params: ParamsOf<Contract>): FailureCause | undefined {
if (served.kind !== "remote" || contractData(Type).hostAddress === contractData(served.contract).hostAddress) {
return undefined;
}
const error = this.#destination.misplaced(served.contract, params);
return error === undefined ? undefined : { kind: "waiting", error };
}
/** Leaves `binding` unbound in the index under `name`: `cause` (`waiting`), or none before the publication. */
#wait(binding: Binding, name: string, cause: FailureCause | undefined, place: () => void): void {
binding.wait(cause, place, () => this.#unbound.delete(name, binding));
this.#unbound.add(name, binding);
}
/** A new reference's binding made by `caller`; its parked calls wait its `readyTimeout`. */
#binding(Type: Contract, dependent: ChordObject | undefined, caller: Caller, options: GetOptions): Binding {
return new Binding(Type, this.batch, dependent, caller, options.readyTimeout ?? this.timeouts.ready);
@ -791,22 +874,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
return made;
}
/**
* Throws where a reference to `Type` cannot go: a type served here needs the caller's version and fingerprint
* (`contract_mismatch`); a local type never crosses a link (`unavailable`). A remote type not served here goes on (a
* route or the parent link), open yet or not.
*/
#assertRoute(Type: Contract): void {
const served = this.#types.get(Type.name);
if (served === undefined) {
if (Type.kind === "local") {
throw new ChordError({ code: "unavailable", message: `${Type.name} is not served here` });
}
return;
}
this.#assertMatch(served, Type.version, Type.fingerprint);
}
/**
* The object for `params` (checked by L1) and, for a remote type, `principal` (its key is part of the identity),
* created if there is none; its first attempt starts at once, or when its identity's backoff window ends. A new one
@ -854,12 +921,12 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
const closed = this.#closedError();
// Code outside objects stops at its next use of Chord: an `fn` waiting in `sleep(ms, ctx)` ends.
this.#roots.close(closed);
const holders = [
...this.#parent.holders(),
...this.unplaced.holders(),
...[...this.#routes.values()].flatMap((route) => route.close()),
];
for (const holder of holders) if (holder.kind === "remote") holder.lost(closed);
for (const route of this.#routes.values()) route.close();
for (const holder of this.#pathHolders()) if (holder.kind === "remote") holder.lost(closed);
// At the fence too: requests still waiting for a publication or a provider end as the others did, and gets
// waiting for a publication that will not come fail now, not after the slow closes below (fix round 1).
this.#unbound.close(closed);
this.#publication.resolve();
this.backoff.clear();
// Every object is sealed before any teardown releases a reference, so none ends in the background.
const teardowns = [...this.#objects.values()].reverse().map((object) => object.stop());
@ -870,8 +937,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
for (const record of this.#registries.values()) record.close();
this.#registries.clear();
this.store?.detach();
// Gets waiting for a load that will not come fail now.
this.#loaded.resolve();
if (failures.length > 0) {
const message = `host.close: ${failures.length} native close(s) threw`;
throw new AggregateError(
@ -881,6 +946,12 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
}
}
/** Every holder of the parent path, the unplaced one and every route's paths: this host's requests over links. */
#pathHolders(): Upstream[] {
const routed = [...this.#routes.values()].flatMap((route) => route.holders());
return [...this.#parent.holders(), ...this.unplaced.holders(), ...routed];
}
/** The store stopped after a backend error (it tells the host once): reported, and the host closes. */
#storageFailed(error: unknown): void {
this.report({ kind: "storage_failed", error });
@ -956,13 +1027,6 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
this.#types.set(contract.name, served);
}
#assertMatch(served: Served, version: number, fingerprint: string): void {
if (version !== served.contract.version || fingerprint !== served.contract.fingerprint) {
const message = `${served.contract.name} v${version} does not match the served v${served.contract.version}`;
throw new ChordError({ code: "contract_mismatch", message });
}
}
#assertOpen(): void {
if (this.#state === "closed") throw this.#closedError();
}
@ -971,3 +1035,46 @@ export class ChordHost<P extends Principal = Principal, R extends JsonValue = Js
return new ChordError({ code: "released", reason: "closed", message: "the host is closed" });
}
}
/**
* Why a reference to a type of `kind` at `version` and `fingerprint`, placed here, cannot bind to `served` (the type of
* that name served here): not served, or as the other kind (`unavailable`); another version or fingerprint
* (`contract_mismatch`); `undefined` when it binds. Both wait for a publication: `failed { waiting }`.
*/
function unboundCause(
served: Served | undefined,
kind: Contract["kind"],
name: string,
version: number,
fingerprint: string,
): FailureCause | undefined {
if (served === undefined || served.kind !== kind) {
return { kind: "waiting", error: new ChordError({ code: "unavailable", message: `${name} is not served here` }) };
}
const { contract } = served;
if (version === contract.version && fingerprint === contract.fingerprint) return undefined;
const message = `${name} v${version} does not match the served v${contract.version}`;
return { kind: "waiting", error: new ChordError({ code: "contract_mismatch", message }) };
}
/**
* Waits until `usable()` (a registry value's check, which records its last error) holds, checked at every change the
* value tells its waiters of (after the batch that made it); rejects when `context` aborts or the value closes. It
* waits on the value only while it runs.
*/
async function untilUsable(value: RegistryValueSource, usable: () => boolean, context: Context): Promise<void> {
let changed = Promise.withResolvers<void>();
const release = value.wait({
dependencyMoved: () => queueMicrotask(() => changed.resolve()),
// The value closed with its host: the wait ends with the close's error.
closed: (error) => changed.reject(error),
});
try {
while (!usable()) {
await awaitWithContext(changed.promise, context);
changed = Promise.withResolvers<void>();
}
} finally {
release();
}
}

7
packages/chord/src/next/host/internal.ts
View file

@ -3,9 +3,10 @@
export { nextDelay } from "./backoff.ts";
export { CallContext, type Caller } from "./ctx.ts";
export type { AnyHost, ChordHost } from "./host.ts";
export type { AnyHost, ChordHost, Serving } from "./host.ts";
export type { PinnedKey } from "./keyed-members.ts";
export { type ChordObject, type Holder, settlement } from "./object.ts";
export type { ChordObject, Holder } from "./object.ts";
export { LIVE, LOADING, type RefState } from "./ref-state.ts";
export type { Binding, CheckedParams, Target } from "./reference.ts";
export { type Binding, type CheckedParams, type RetryOperation, retryOperation, type Target } from "./reference.ts";
export type { UnboundRequest } from "./unbound.ts";
export { uuidv7 } from "./uuid.ts";

274
packages/chord/src/next/host/object.ts
View file

@ -1,15 +1,16 @@
// Objects (§6): one per identity, from its first attempt to its end. Its state across attempts (its documents, line
// and job runs, its member sources, slice 6a §1.2), its holders, retries after failures (backoff, dependency recovery
// R18, `ref.retry()`), a live object's failure (slice 6a §1.4), the waits and cycle checks of activations (Q5, its
// instance id and the activation chain it was created for), lingering after its last reference (§3, D8), the job runs
// that keep it alive (§15), its contributions to registries while live and its wake signals (`retryOn`, slice 8), and
// its ending, revocation included (§4, R11).
// and job runs, its member sources, slice 6a §1.2), its holders, retries after failures (backoff, `ref.retry()`, passed
// on while parked), what it shows while its attempt is parked on dependencies (dependency model §3), its first outcome
// (the start barrier, decision (a)), a live object's failure (slice 6a §1.4), the waits and cycle checks of activations
// (Q5, its instance id and the activation chain it was created for), lingering after its last reference (§3, D8), the
// job runs that keep it alive (§15), its contributions to registries while live and its wake signals (`retryOn`, slice
// 8), and its ending, revocation included (§4, R11).
import { awaitWithContext, type Context } from "../../context/index.ts";
import type { Context } from "../../context/index.ts";
import type { Contract, ParamsOf } from "../contracts/index.ts";
import type { ContractShape } from "../contracts/internal.ts";
import { ChordError, type FailureCause, isChordError, type Principal } from "../data/index.ts";
import { invalid, type Provider } from "../data/internal.ts";
import { ChordError, type FailureCause, type Principal } from "../data/index.ts";
import { invalid, type Provider, sameCause } from "../data/internal.ts";
import { Documents, type KeyedSlot } from "../documents/internal.ts";
import type { JobDefinition } from "../jobs/index.ts";
import { JobRecords } from "../jobs/internal.ts";
@ -17,13 +18,14 @@ import { CHAIN } from "../link/internal.ts";
import type { DeriveSource } from "../state/internal.ts";
import { Attempt, type Self } from "./attempt.ts";
import type { Caller } from "./ctx.ts";
import type { Dependency } from "./dependencies.ts";
import type { AnyHost } from "./host.ts";
import { KeyedMembers, type PinnedKey } from "./keyed-members.ts";
import { MemberSource } from "./member-source.ts";
import type { KeyedBinding } from "./members.ts";
import type { CloseFailed, NativeCloseReason } from "./native.ts";
import { type Ended, type Failed, LIVE, LOADING, type RefState } from "./ref-state.ts";
import type { Target } from "./reference.ts";
import { type RetryOperation, retryOperation, type Target } from "./reference.ts";
import { type RetrySource, RetryWakes } from "./retry-on.ts";
import { uuidv7 } from "./uuid.ts";
@ -60,7 +62,7 @@ type ServedType<C extends Contract> = {
/** What holds an object: a reference's binding, or a hold arriving over a link. */
export interface Holder {
/** The object whose attempt declared the reference; retried when this object goes live (R18). */
/** The object whose attempt declared the reference: it checks its dependencies again when this object changes. */
readonly dependent: ChordObject | undefined;
/** Whom the reference acts for, with its origin's incarnation (`host.references`). */
readonly caller: Caller;
@ -71,34 +73,40 @@ export interface Holder {
}
/**
* Where an object stands. `waiting`: for its identity's backoff window before its first attempt. `failed` has the
* state its references show, and keeps the failed attempt in `holding` after a dependency failure (R18): its
* references keep the dependencies.
* Where an object stands. `waiting`: for its identity's backoff window before its first attempt. `activating` has what
* its references show: `loading`, or `failed { waiting }` while its attempt is parked on a failed dependency. `failed`
* has the state its references show.
*/
type ObjectState =
| { readonly kind: "waiting" }
| { readonly kind: "activating"; readonly attempt: Attempt }
| { readonly kind: "activating"; readonly attempt: Attempt; readonly shown: RefState }
| { readonly kind: "live"; readonly attempt: Attempt }
| { readonly kind: "failed"; readonly failure: Failed; readonly holding: Attempt | undefined }
| { readonly kind: "failed"; readonly failure: Failed }
| { readonly kind: "ended"; readonly ended: Ended };
/**
* What an ended object still has to do once its previous teardowns finished (`previous`): drain and close its attempt,
* or release a failed attempt's references; then drop its documents and job runs (`end` alone).
* What an ended object still has to do once its previous teardowns finished (`previous`): drain and close its attempt;
* then drop its documents and job runs (`end` alone).
*/
type Teardown =
| { readonly kind: "none" }
| { readonly kind: "end"; readonly previous: Promise<void> | undefined }
| { readonly kind: "close" | "release"; readonly attempt: Attempt; readonly previous: Promise<void> | undefined };
| { readonly kind: "close"; readonly attempt: Attempt; readonly previous: Promise<void> | undefined };
const END: NativeCloseReason = { kind: "end" };
const CLOSE: NativeCloseReason = { kind: "close" };
/** How many retry ids an object remembers: a diamond's shortcut only (a ring is stopped by a retry's `through`). */
const RETRIES_KEPT = 8;
/** No dependencies waited for: the shared empty list. */
const NO_DEPENDENCIES: readonly Dependency[] = [];
/**
* One object. Each attempt is a fresh `Attempt`; the documents, the line and the job runs, the holders (and, after a
* dependency failure, the failed attempt's references) outlive it. A transient failure is retried after its backoff window, or at once when a
* dependency it failed on goes live or a reference asks (`retry`); a broken one stays failed until a reference asks.
* One object. Each attempt is a fresh `Attempt`; the documents, the line, the job runs and the holders outlive it. An
* attempt parked on its dependencies waits for them without a timer; a transient failure (its own) is retried after its
* backoff window, or at once when a reference asks (`retry`); a broken one stays failed until a reference asks.
* The object ends with its last reference, a live one after its type's `linger` (never, for a host-lifetime object),
* or when the host closes.
*/
@ -127,13 +135,20 @@ export class ChordObject implements Target {
#state: ObjectState = { kind: "waiting" };
/** References and holds of this object. */
readonly #holders = new Set<Holder>();
/** Objects this object's activation waits for; followed to find cycles (Q5). */
readonly #waitingFor = new Set<Target>();
/** The attempt a dependency went live during (R18): if it fails on a dependency, it is retried at once. */
#woken: Attempt | undefined;
/** The current attempt's outcome: resolves when it goes live, rejects with its error; renewed by every retry. */
#outcome = settlement();
/** The error of the last failed attempt, until one goes live. */
/** The dependencies its parked attempt waits for; their objects are followed to find cycles (Q5). */
#waitingFor: readonly Dependency[] = NO_DEPENDENCIES;
/**
* Its first outcome (decision (a)): live, failed, parked with no blocker here still without one, or ended; once.
* The promise is made by the first who waits for it (the start barrier, for host-lifetime objects).
*/
#firstOutcome: PromiseWithResolvers<void> | undefined;
#hasFirstOutcome = false;
/**
* The ids of the last retry operations it handled (at most `RETRIES_KEPT`, the newest last; made by the first): a
* retry reaching it twice through a diamond is handled once.
*/
#retried: string[] | undefined;
/** The error of the last failed attempt, or the last cause a parked attempt showed, until one goes live. */
#lastFailure: ChordError | undefined;
/** Ends the object when its type's `linger` passed after its last reference; cleared by a new one. */
#lingering: ReturnType<typeof setTimeout> | undefined;
@ -189,7 +204,8 @@ export class ChordObject implements Target {
const state = this.#state;
if (state.kind === "failed") return state.failure;
if (state.kind === "live") return LIVE;
return state.kind === "ended" ? state.ended : LOADING;
if (state.kind === "ended") return state.ended;
return state.kind === "activating" ? state.shown : LOADING;
}
/** Whether its state members have values: once live (a local object stays live until it ends). */
@ -202,12 +218,27 @@ export class ChordObject implements Target {
return this.#exposed !== undefined && this.#state.kind !== "ended";
}
/** The current attempt's outcome (the start barrier waits for the first). */
get settled(): Promise<void> {
return this.#outcome.promise;
/** Resolves at its first outcome (the start barrier waits for it); never rejects. */
get firstOutcome(): Promise<void> {
if (this.#hasFirstOutcome) return Promise.resolve();
this.#firstOutcome ??= Promise.withResolvers<void>();
return this.#firstOutcome.promise;
}
/** The error of the last failed attempt while none went live since. */
/** Whether it holds up the start barrier: it has no first outcome yet. */
get blocksFirstPass(): boolean {
return !this.#hasFirstOutcome;
}
/** An object here: it is not held on a link. */
get heldOnLink(): false {
return false;
}
/**
* The error of its last failure while none went live since: a failed attempt's, or the cause its parked attempt
* showed last (kept when the dependency falls back to `loading`, so a getter's deadline names it).
*/
get lastFailure(): ChordError | undefined {
return this.#lastFailure;
}
@ -251,21 +282,31 @@ export class ChordObject implements Target {
const state = this.#state;
if (state.kind !== "waiting" && state.kind !== "failed") return;
this.#instance ??= uuidv7();
const attempt = new Attempt(this, state.kind === "failed" ? state.holding : undefined, this.#context);
this.#state = { kind: "activating", attempt };
if (state.kind === "failed") {
this.#outcome = settlement();
this.#notify();
}
const attempt = new Attempt(this, this.#context);
this.#state = { kind: "activating", attempt, shown: LOADING };
if (state.kind === "failed") this.#notify();
void this.#run(attempt);
}
/**
* `ref.retry()` and a `retry` message: an attempt now when failed (a broken one too) or waiting, its backoff window
* skipped; ignored while an attempt runs and once live.
* `ref.retry()` and a `retry` message, as `operation` (handled once): an attempt now when failed (a broken one too)
* or waiting, its backoff window skipped; passed on to the failed dependencies while parked `failed { waiting }`,
* with this object added to the objects it passed through (at most `CHAIN`: a retry stops past 32 hops); ignored
* otherwise (an attempt runs, or it is live), and when it passed through this object already (a ring) or was handled
* here already (a diamond).
*/
retry(): void {
if (this.#state.kind !== "waiting" && this.#state.kind !== "failed") return;
retry(operation: RetryOperation = retryOperation()): void {
const instance = this.#instance;
if (instance !== undefined && operation.through.includes(instance)) return;
if (this.#handled(operation.id)) return;
const state = this.#state;
if (state.kind === "activating") {
const { through } = operation;
if (state.shown.kind !== "failed" || instance === undefined || through.length >= CHAIN) return;
state.attempt.retryDependencies({ id: operation.id, through: [...through, instance] });
return;
}
if (state.kind !== "waiting" && state.kind !== "failed") return;
this.host.backoff.skip(this.identity);
this.attempt();
}
@ -396,10 +437,8 @@ export class ChordObject implements Target {
if (cause.kind === "transient") this.host.backoff.failed(this.identity);
else this.host.backoff.delete(this.identity);
const failure: Failed = { kind: "failed", cause, retryAt: this.host.backoff.until(this.identity) };
this.#state = { kind: "failed", failure, holding: undefined };
this.#state = { kind: "failed", failure };
this.#lastFailure = error;
this.#outcome = settlement();
this.#outcome.reject(error);
this.#withdraw(attempt);
attempt.seal(error);
// Members following another object's sources freeze now: no commit of this object changes them.
@ -476,37 +515,39 @@ export class ChordObject implements Target {
throw error;
}
/**
* Waits for `dependency`'s current attempt as part of this object's activation. Rejects at once when it is failed,
* and when its attempt fails, with `unavailable` naming it (R18), its error as the cause; a dependency that failed
* for a cycle rejects with that `invalid/cycle` itself.
*/
async waitFor(dependency: Target, context: Context): Promise<void> {
this.#waitingFor.add(dependency);
try {
await awaitWithContext(dependency.settled, context);
} catch (error) {
if (isChordError(error, "invalid") && error.reason === "cycle") throw error;
const message = `${this.label}: dependency ${dependency.label} failed`;
throw new ChordError({ code: "unavailable", message }, { cause: error });
} finally {
this.#waitingFor.delete(dependency);
}
/** Its parked attempt waits for `dependencies` (none once it leaves parking): the waits a cycle check follows. */
waitsFor(dependencies: readonly Dependency[]): void {
this.#waitingFor = dependencies;
}
/**
* A dependency went live: retried at once (backoff window skipped) when it failed on a dependency (R18). A wake
* while it activates is remembered: that activation may still fail on the dependency's earlier failure.
* A dependency of its current attempt changed (its state, or its first outcome): a parked attempt checks again; an
* attempt past its dependency check ignores it (R18 after live), and so does any other state.
*/
dependencyLive(): void {
dependencyMoved(): void {
const state = this.#state;
if (state.kind === "activating") {
this.#woken = state.attempt;
return;
if (state.kind === "activating") state.attempt.dependencyMoved();
}
/**
* Its attempt `attempt` is parked showing `shown` (`loading` or `failed { waiting }`): holders are told when that
* changed (a cause compared by value). `firstPass`: no blocker here is still without its first outcome, so this
* object has its first outcome now.
*/
parked(attempt: Attempt, shown: RefState, firstPass: boolean): void {
const state = this.#state;
if (state.kind !== "activating" || state.attempt !== attempt) return;
if (!sameState(state.shown, shown)) {
this.#state = { kind: "activating", attempt, shown };
if (shown.kind === "failed") this.#lastFailure = shown.cause.error;
this.#notify();
}
if (state.kind !== "failed" || state.holding === undefined) return;
this.host.backoff.skip(this.identity);
this.attempt();
if (firstPass) this.#settleFirstOutcome();
}
/** Its attempt `attempt` left parking for its own steps: it shows `loading` until its outcome. */
unparked(attempt: Attempt): void {
this.parked(attempt, LOADING, false);
}
/**
@ -540,14 +581,11 @@ export class ChordObject implements Target {
case "live":
return this.#wentLive(attempt);
case "failed":
return this.#failed(attempt, outcome.cause, undefined);
case "dependency_failed":
return this.#failed(attempt, outcome.cause, attempt);
return this.#failed(attempt, outcome.cause);
}
}
#wentLive(attempt: Attempt): void {
this.#woken = undefined;
if (this.#wakes !== undefined) this.#wakes.during = undefined;
this.#state = { kind: "live", attempt };
this.#exposed = attempt;
@ -562,21 +600,17 @@ export class ChordObject implements Target {
}
this.#lastFailure = undefined;
this.host.wentLive(this);
this.#outcome.resolve();
this.#notify();
// R18: objects that failed because this one had failed are retried now.
for (const holder of [...this.#holders]) holder.dependent?.dependencyLive();
this.#settleFirstOutcome();
attempt.live();
}
/**
* Records a failed attempt: reported, torn down in the background (reason `end`), its references released unless it
* is `holding` them (R18), its holders told. Transient: a backoff window, whose end is the failure's `retryAt`;
* broken: no automatic retry.
* Records a failed attempt: reported, torn down in the background (reason `end`), its references released once its
* natives closed, its holders told. Transient: a backoff window, whose end is the failure's `retryAt`; broken: no
* automatic retry.
*/
#failed(attempt: Attempt, cause: FailureCause, holding: Attempt | undefined): void {
const woken = this.#woken === attempt;
this.#woken = undefined;
#failed(attempt: Attempt, cause: FailureCause): void {
const wokenBy = this.#wakes?.during === attempt;
if (this.#wakes !== undefined) this.#wakes.during = undefined;
// What its activation loaded for the pinned keys goes; they stay frozen.
@ -584,25 +618,23 @@ export class ChordObject implements Target {
if (cause.kind === "transient") this.host.backoff.failed(this.identity);
else this.host.backoff.delete(this.identity);
const failure: Failed = { kind: "failed", cause, retryAt: this.host.backoff.until(this.identity) };
this.#state = { kind: "failed", failure, holding };
this.#state = { kind: "failed", failure };
this.#lastFailure = cause.error;
attempt.seal(this.ended());
const closing = attempt.close(END).then((failures) => {
attempt.detach();
// Released once its natives closed (a close may use them, §6, §13); R18's attempt keeps them for the retry.
if (holding === undefined) attempt.releaseReferences();
// Released once its natives closed (a close may use them, §6, §13).
attempt.releaseReferences();
return failures;
});
this.host.ending(this.identity, this.#reported(closing));
this.host.report({ kind: "activation_failed", object: this.label, failure: cause });
this.#outcome.reject(cause.error);
this.#notify();
this.#settleFirstOutcome();
// Job runs kept it while it activated; failed, they keep it no longer.
this.#idle();
// A dependency went live while this attempt activated: the wake it missed retries it now (R18).
if (holding !== undefined && woken) this.dependencyLive();
// A wake signal arrived while it activated: it retries it now.
else if (wokenBy) this.retry();
if (wokenBy) this.retry();
}
/**
@ -614,7 +646,8 @@ export class ChordObject implements Target {
// Called by its wakes only.
const wakes = this.#wakes!;
if (state.kind === "activating") {
wakes.during = state.attempt;
// While parked its dependencies decide (dependency model §3.8).
if (!state.attempt.isParked) wakes.during = state.attempt;
return;
}
if (state.kind !== "failed" || wakes.leaving) return;
@ -661,7 +694,7 @@ export class ChordObject implements Target {
for (const [member, source] of attempt.stateMembers) {
if (which === "all" || !attempt.owns(source)) this.#memberSource(member, attempt).freeze();
}
// A pin still loading is given up with the failure: a consumer activating on it fails transient (R18).
// A pin still loading is given up with the failure: a consumer parked on it shows `failed { waiting }`.
this.#keys.freeze(which, this.#lastFailure ?? this.ended());
}
@ -705,7 +738,8 @@ export class ChordObject implements Target {
this.#stopLingering();
this.#wakes?.stop();
this.host.forget(this);
this.#outcome.reject(error);
// An object of the start barrier's cohort that ends before its first outcome settles it as ended.
this.#settleFirstOutcome();
// In a batch: the derives over its members are marked, so none keeps a cached value.
this.host.batch.within(() => {
for (const source of this.#members.values()) source.objectEnded(error);
@ -717,12 +751,8 @@ export class ChordObject implements Target {
this.#holders.clear();
this.host.unreferenced(this, holders, this.#context);
for (const holder of holders) holder.end(error);
if (state.kind === "waiting") return { kind: "end", previous };
if (state.kind === "failed") {
// The failed attempt is closed (or closing) since it failed; only the references it holds (R18) are left.
const { holding } = state;
return holding === undefined ? { kind: "end", previous } : { kind: "release", attempt: holding, previous };
}
// A failed attempt is closed (or closing) since it failed.
if (state.kind === "waiting" || state.kind === "failed") return { kind: "end", previous };
if (state.kind === "live") this.#withdraw(state.attempt);
state.attempt.seal(error);
return { kind: "close", attempt: state.attempt, previous };
@ -767,8 +797,10 @@ export class ChordObject implements Target {
if (teardown.kind === "none") return [];
await teardown.previous;
const failures = teardown.kind === "close" ? await teardown.attempt.close(reason) : [];
if (teardown.kind === "close") teardown.attempt.detach();
if (teardown.kind !== "end") teardown.attempt.releaseReferences();
if (teardown.kind === "close") {
teardown.attempt.detach();
teardown.attempt.releaseReferences();
}
await this.documents.settled();
this.documents.end();
this.host.store?.claims.release(this);
@ -794,12 +826,35 @@ export class ChordObject implements Target {
throw new ChordError({ code: "not_ready", message: `${this.label}: not live yet` });
}
/** The chain of waits from this activation to `target` (see `Target`). */
/**
* Whether the retry operation `id` reached it before (among the last few: the diamond's shortcut; a ring is stopped
* by `through`); records it otherwise.
*/
#handled(id: string): boolean {
this.#retried ??= [];
const retried = this.#retried;
if (retried.includes(id)) return true;
if (retried.length === RETRIES_KEPT) retried.shift();
retried.push(id);
return false;
}
/** The settlement of its first outcome (once): dependents parked on it check their first pass again. */
#settleFirstOutcome(): void {
if (this.#hasFirstOutcome) return;
this.#hasFirstOutcome = true;
this.#firstOutcome?.resolve();
this.#firstOutcome = undefined;
for (const holder of this.#holders) holder.dependent?.dependencyMoved();
}
/** The chain of waits from this activation to `target` (see `Target`), through its parked attempt's dependencies. */
pathTo(target: ChordObject, visited: Set<Target>): ChordObject[] | undefined {
if (this === target) return [this];
visited.add(this);
for (const next of this.#waitingFor) {
if (visited.has(next)) continue;
for (const dependency of this.#waitingFor) {
const next = dependency.kind === "reference" ? dependency.binding.target : undefined;
if (next === undefined || visited.has(next)) continue;
const rest = next.pathTo(target, visited);
if (rest !== undefined) return [this, ...rest];
}
@ -807,9 +862,8 @@ export class ChordObject implements Target {
}
}
/** A settlement nobody may wait for: its rejection is not unhandled. */
export function settlement(): PromiseWithResolvers<void> {
const settlement = Promise.withResolvers<void>();
settlement.promise.catch(() => {});
return settlement;
/** Whether two states of a reference say the same: the same object, or failures with the same cause (by value). */
function sameState(a: RefState, b: RefState): boolean {
if (a === b) return true;
return a.kind === "failed" && b.kind === "failed" && a.retryAt === b.retryAt && sameCause(a.cause, b.cause);
}

12
packages/chord/src/next/host/ref-state.ts
View file

@ -7,11 +7,13 @@ import type { ChordError, FailureCause } from "../data/index.ts";
import { type Batch, NativeSource } from "../state/internal.ts";
/**
* Where a reference stands. `loading`: its object activates or retries, it waits for `load` or a parent link, or its
* link is down (the reference's coarser view of its object's waiting and activating); `failed`: its object failed and
* no attempt runs (`retryAt` undefined: broken, no automatic retry); `live`: bound; `ended`: for good, with the error
* that says why (`released` without a reason: released by its holder; `released/ended`: its object ended;
* `released/closed`: its host closed; `forbidden/revoked`).
* Where a reference stands. `loading`: its object activates, retries or is parked on dependencies still under way, it
* waits for its host's publication or a parent link, or its link is down (the reference's coarser view of its object's
* waiting and activating); `failed`: its object failed and no attempt runs (`transient`: `retryAt` is when its backoff
* retries it; `broken`: no automatic retry), or nothing fits it or its object is parked on a failed dependency
* (`waiting`: an event wakes it, `retryAt` undefined); `live`: bound; `ended`: for good, with the error that says why
* (`released` without a reason: released by its holder; `released/ended`: its object ended; `released/closed`: its
* host closed; `forbidden/revoked`).
*/
export type RefState =
| { readonly kind: "loading" }

159
packages/chord/src/next/host/reference.ts
View file

@ -14,7 +14,7 @@ import {
takesNoArgs,
type VariantImplementation,
} from "../contracts/internal.ts";
import { ChordError, type FailureCause } from "../data/index.ts";
import { ChordError, type FailureCause, type Principal } from "../data/index.ts";
import { Keyed, type KeyedSlot, type SealToken, Singleton } from "../documents/internal.ts";
import { type Batch, Constant, type DeriveSource, Source } from "../state/internal.ts";
import { type Caller, withLinkedAbortSignal } from "./ctx.ts";
@ -23,6 +23,8 @@ import type { KeyedBinding } from "./members.ts";
import type { ChordObject, Holder } from "./object.ts";
import { ParkedCalls } from "./parked.ts";
import { type Ended, type Failed, LOADING, ReferenceState, type RefState } from "./ref-state.ts";
import type { UnboundRequest } from "./unbound.ts";
import { uuidv7 } from "./uuid.ts";
/**
* A reference: a local contract's members, or for a remote contract the members of the variant its object chose,
@ -50,7 +52,10 @@ type InVariants<C extends Contract, Unbound> = {
type Lifecycle = {
/** The reference's state: loading, live, failed or released; watches get every change. */
readonly state: NativeWatchable<RefState>;
/** Asks a failed object for an attempt now (the backoff skipped, a broken one renewed); otherwise nothing. */
/**
* Asks a failed object for an attempt now (the backoff skipped, a broken one renewed); an object parked on failed
* dependencies passes it on to them, once; otherwise nothing.
*/
retry(): void;
release(): void;
};
@ -68,12 +73,20 @@ export interface Target {
readonly hydrated: boolean;
/** The variant bound last; `undefined` for a local object and before the first binding. */
readonly variant: string | undefined;
/** The error of its last failed attempt while none went live since. */
/** The error of its last failure (an attempt's, or the dependencies it is parked on) while none went live since. */
readonly lastFailure: ChordError | undefined;
/** Its instance id while it activates on this host (Q5); `undefined` otherwise and over a link. */
readonly activationId: string | undefined;
/** The current attempt's outcome, which a dependent's activation waits for (R18). */
readonly settled: Promise<void>;
/**
* Whether it is held on a link now: a remote attempt answers for it there (`parked_long` counts it while it is
* loading); not while its path has no link, and never for an object here.
*/
readonly heldOnLink: boolean;
/**
* Whether it holds up this host's start barrier: an object here without its first outcome (decision (a)); a remote
* object never does.
*/
readonly blocksFirstPass: boolean;
/** Counts `binding` as a reference; the binding reads `state` itself. */
hold(binding: Binding): void;
/** `binding` was released. */
@ -97,8 +110,11 @@ export interface Target {
* attempts once loaded (`keyed-members.ts`); only a live local object takes it.
*/
pinKey(member: string, key: string, context: Context): Promise<PinnedKey>;
/** `ref.retry()`: an attempt now if failed; ignored otherwise. */
retry(): void;
/**
* `ref.retry()` (`operation` once per retry, deduping diamonds): an attempt now if failed; an object parked on failed
* dependencies passes it on; ignored otherwise.
*/
retry(operation: RetryOperation): void;
/**
* The chain of activation waits from this object to `target`, both included (Q5); `undefined` when there is none.
* `visited` keeps a search over shared dependencies linear in the waits.
@ -106,14 +122,29 @@ export interface Target {
pathTo(target: ChordObject, visited: Set<Target>): ChordObject[] | undefined;
}
/**
* One `ref.retry()` passing on through parked objects, across links too (the `retry` message's `operation` and
* `through`): its id (a uuidv7) and the instance ids of the objects it passed through, added hop by hop (at most
* `CHAIN`). An object in `through` drops it, so a ring of hosts stops after one lap however many retries run at once;
* an object remembers the last few ids it handled too, so a diamond retries it once.
*/
export type RetryOperation = { readonly id: string; readonly through: string[] };
/** A new retry operation, through nothing yet (`ref.retry()`, a `retry` message that names none). */
export function retryOperation(): RetryOperation {
return { id: uuidv7(), through: [] };
}
/** A reference's params as L1 checked them, with their identity key. */
export type CheckedParams = { readonly params: ParamsOf<Contract>; readonly key: string };
/**
* Where a binding stands. `unbound`: waiting for `load` (a `host.hold`) or for its object's activation to bind it (a
* `self.get`); `refused`: a `host.hold` whose type turned out not to be gettable here once `load` resolved; a released
* binding keeps nothing but its name and its end (`ref.state`'s `ended`). `dependent` is the object whose attempt declared the reference with `self.get`,
* `undefined` for `host.get`, `host.hold` and `ctx.get`.
* Where a binding stands. `unbound`: waiting for its object's activation to bind it (a `self.get`); `waiting`: in the
* host's unbound index, `place` checking it again when woken: before the publication (no `failure`: `loading`), or after
* a check that found no fitting provider (`failed { waiting }`, `not_served` or `version`); `refused`: a broken refusal
* (a `hostAddress` or `authorize` that threw or refused) until `ref.retry()` places it again; a released binding keeps
* nothing but its name and its end (`ref.state`'s `ended`). `dependent` is the object whose attempt declared the
* reference with `self.get`, `undefined` for `host.get`, `host.hold` and `ctx.get`.
*/
type BindingState =
| { readonly kind: "unbound"; readonly batch: Batch; readonly dependent: ChordObject | undefined }
@ -123,7 +154,22 @@ type BindingState =
readonly dependent: ChordObject | undefined;
readonly target: Target;
}
| { readonly kind: "refused"; readonly batch: Batch; readonly failure: Failed; readonly bind: () => void }
| {
readonly kind: "waiting";
readonly batch: Batch;
readonly dependent: ChordObject | undefined;
readonly failure: Failed | undefined;
readonly place: () => void;
/** Takes it out of the index (it ended). */
readonly leave: () => void;
}
| {
readonly kind: "refused";
readonly batch: Batch;
readonly dependent: ChordObject | undefined;
readonly failure: Failed;
readonly bind: () => void;
}
| { readonly kind: "released"; readonly state: Ended };
/** What a wait for a state decides at each change. */
@ -144,7 +190,7 @@ type Waiter = { readonly check: () => void; readonly end: (error: ChordError) =>
* change of its state. Calls run while live, park while loading (sent in order once live) and reject while failed.
* Released, or ended with its object, it ends the watches taken through it and rejects the calls made through it.
*/
export class Binding implements Holder {
export class Binding implements Holder, UnboundRequest {
readonly reference: Reference<Contract>;
/** Who calls through this reference: the principal it was got as, and this host's incarnation. */
readonly caller: Caller;
@ -165,8 +211,6 @@ export class Binding implements Holder {
readonly #waiters = new Set<Waiter>();
/** `ref.state`, made on first access. */
#refState: ReferenceState | undefined;
/** Resolved when the reference ends; made by the first `ended()`. */
#ended: PromiseWithResolvers<void> | undefined;
/** The keys its object's factory pinned with `ref.member.at(key)`: pinned once bound, released with it. */
readonly #keys: KeyedMemberAt[] = [];
@ -206,26 +250,38 @@ export class Binding implements Holder {
/** The object whose attempt declared this reference, until it is released. */
get dependent(): ChordObject | undefined {
const state = this.#state;
return state.kind === "unbound" || state.kind === "bound" ? state.dependent : undefined;
return state.kind === "released" ? undefined : state.dependent;
}
/** Whom it acts for (the unbound index's revocation). */
get principal(): Principal | undefined {
return this.caller.principal;
}
/** The reference's state. */
get refState(): RefState {
const state = this.#state;
if (state.kind === "bound") return state.target.state;
if (state.kind === "waiting") return state.failure ?? LOADING;
if (state.kind === "refused") return state.failure;
return state.kind === "unbound" ? LOADING : state.state;
}
/** The object it is bound to; `undefined` while it is not bound. */
get target(): Target | undefined {
return this.#state.kind === "bound" ? this.#state.target : undefined;
}
/** The object in errors: its label once bound, else the contract's name. */
get label(): string {
return this.#state.kind === "bound" ? this.#state.target.label : this.#name;
}
/** The error of the object's last failed attempt while none went live since. */
/** The error of the object's last failure while none went live since, or of the check that left it unbound. */
get lastFailure(): ChordError | undefined {
const state = this.#state;
if (state.kind === "refused") return state.failure.cause.error;
if (state.kind === "waiting") return state.failure?.cause.error;
return state.kind === "bound" ? state.target.lastFailure : undefined;
}
@ -233,26 +289,48 @@ export class Binding implements Holder {
bind(target: Target): void {
const state = this.#state;
if (state.kind !== "unbound") return;
this.#state = { ...state, kind: "bound", target };
this.#state = { kind: "bound", batch: state.batch, dependent: state.dependent, target };
target.hold(this);
this.moved();
}
/**
* A `host.hold` that cannot be bound (where it goes refused it): failed with `cause` until `ref.retry()` runs `bind`
* again (a route's `authorize` may admit it later).
* Leaves the reference unbound in the host's unbound index (`leave` takes it out): `loading` before the publication
* (no `cause`), else `failed` with `cause` (`waiting`); `place` checks it again when the index wakes it.
*/
wait(cause: FailureCause | undefined, place: () => void, leave: () => void): void {
const state = this.#state;
if (state.kind !== "unbound") return;
const failure: Failed | undefined =
cause === undefined ? undefined : { kind: "failed", cause, retryAt: undefined };
this.#state = { kind: "waiting", batch: state.batch, dependent: state.dependent, failure, place, leave };
// Unchanged while loading: nothing to tell.
if (failure !== undefined) this.moved();
}
/** Woken by the unbound index (out of it already): checked again where it goes. */
wake(): void {
const state = this.#state;
if (state.kind !== "waiting") return;
this.#state = { kind: "unbound", batch: state.batch, dependent: state.dependent };
state.place();
}
/**
* A reference that cannot be bound (where it goes refused it): failed broken with `cause` until `ref.retry()` runs
* `bind` again (a route's `authorize` may admit it later).
*/
refuse(cause: FailureCause, bind: () => void): void {
const state = this.#state;
if (state.kind !== "unbound") return;
const failure: Failed = { kind: "failed", cause, retryAt: undefined };
this.#state = { kind: "refused", batch: state.batch, failure, bind };
this.#state = { kind: "refused", batch: state.batch, dependent: state.dependent, failure, bind };
this.moved();
}
/**
* The state may have changed: `ref.state`'s watches are told; live sends the parked calls in order, failed
* rejects them with the cause; waits for a state decide.
* rejects them with the cause; waits for a state decide; the object whose attempt declared it re-evaluates.
*/
moved(): void {
this.#refState?.moved();
@ -266,6 +344,7 @@ export class Binding implements Holder {
this.#parked?.reject(state.cause.error);
}
for (const waiter of [...this.#waiters]) waiter.check();
this.dependent?.dependencyMoved();
}
/** Resolves when the reference is live; rejects with a broken failure's error, or its end. Getters wait for it. */
@ -313,6 +392,11 @@ export class Binding implements Holder {
}
case "unbound":
throw new ChordError({ code: "not_ready", message: `${this.#name}: not live yet` });
case "waiting":
if (state.failure === undefined) {
throw new ChordError({ code: "not_ready", message: `${this.#name}: not live yet` });
}
throw state.failure.cause.error;
case "refused":
throw state.failure.cause.error;
case "released":
@ -369,6 +453,11 @@ export class Binding implements Holder {
return at;
}
/** Whether its object's factory pinned keys on it (`at(key)`). */
get pinsKeys(): boolean {
return this.#keys.length > 0;
}
/**
* Pins the keys of `at(key)` on the live object's keyed members and loads those not in memory, before the
* dependent goes live (slice 6a §12); `undefined` when there are none. A failed load rejects.
@ -381,24 +470,18 @@ export class Binding implements Holder {
}
/**
* Resolves when the reference ends (released, or its object ended); never rejects. A wait that must not keep the
* reference races it (a `host.hold` waiting for `load`).
* `ref.retry()`, as `operation` (one per `ref.retry()`, passed on through parked objects): the object is asked for
* an attempt now (ignored unless failed; a parked one passes it on); a refused reference is placed again; one
* waiting in the index is left as it is (an event wakes it).
*/
ended(): Promise<void> {
if (this.#state.kind === "released") return Promise.resolve();
this.#ended ??= Promise.withResolvers<void>();
return this.#ended.promise;
}
/** `ref.retry()`: the object is asked for an attempt now (ignored unless failed); a refused hold is bound again. */
retry(): void {
retry(operation: RetryOperation = retryOperation()): void {
const state = this.#state;
if (state.kind === "bound") {
state.target.retry();
state.target.retry(operation);
return;
}
if (state.kind !== "refused") return;
this.#state = { kind: "unbound", batch: state.batch, dependent: undefined };
this.#state = { kind: "unbound", batch: state.batch, dependent: state.dependent };
this.moved();
state.bind();
}
@ -415,13 +498,15 @@ export class Binding implements Holder {
/**
* Ends the reference without releasing it from its object (it ended): watches end, calls (running and parked)
* reject with `error`, `ref.state` is `ended` with it.
* reject with `error`, `ref.state` is `ended` with it; the object whose attempt declared it checks again.
*/
end(error: ChordError): void {
// Kept by `ref.state`: its stack is made text now, so the error holds none of the frames' objects.
void error.stack;
const ended: Ended = { kind: "ended", error };
const previous = this.#state;
this.#state = { kind: "released", state: ended };
if (previous.kind === "waiting") previous.leave();
for (const at of this.#keys) at.release();
this.#calls?.abort(error);
this.#calls = undefined;
@ -431,8 +516,8 @@ export class Binding implements Holder {
this.#refState?.released(ended);
// Each removes itself; promise reactions run later.
for (const waiter of this.#waiters) waiter.end(error);
this.#ended?.resolve();
this.#ended = undefined;
// An ended dependency breaks a parked dependent (it cannot come back).
if (previous.kind !== "released") previous.dependent?.dependencyMoved();
}
#call(method: string, args: unknown, context: Context): Promise<unknown> {

5
packages/chord/src/next/host/route.ts
View file

@ -98,6 +98,11 @@ export class Route {
/** The host closes: no address resolves again. Returns every holder of every path, for the host to end. */
close(): Upstream[] {
this.#backoff.clear();
return this.holders();
}
/** Every holder of every path of this route, linked or not (revocation ends a revoked principal's). */
holders(): Upstream[] {
return [...this.#addresses.values()].flatMap(({ path }) => path.holders());
}

62
packages/chord/src/next/host/unbound.ts Normal file
View file

@ -0,0 +1,62 @@
// The unbound index (dependency model §1, §4): the requests a host decided cannot bind yet, by the name they wait for.
// Before the host's first publication every request made here, and every hold arriving on its links, waits here
// (`unpublished`); after it, those its check left unbound (`not_served`, `version`): a reference made here, a hold that
// arrived over a link and was refused for a name. A publication wakes the requests of the names it serves; a request
// leaves when it ends (released, settled, given up, its link or origin closed, revoked) or binds.
import type { ChordError, Principal } from "../data/index.ts";
/** A request in the index: a reference made here or a hold that arrived over a link. */
export interface UnboundRequest {
/** Whom it acts for: revocation ends the requests of a revoked principal. */
readonly principal: Principal | undefined;
/** The name it waits for became usable (or the host published): it runs its check again, leaving the index first. */
wake(): void;
/** It is revoked, or its host closes: it ends with `error` (a hold sends `gone`), leaving the index. */
end(error: ChordError): void;
}
/** A host's unbound requests by name. */
export class UnboundIndex {
readonly #byName = new Map<string, Set<UnboundRequest>>();
/** Indexes `request` under `name` (once; again after a wake). */
add(name: string, request: UnboundRequest): void {
let requests = this.#byName.get(name);
if (requests === undefined) {
requests = new Set();
this.#byName.set(name, requests);
}
requests.add(request);
}
/** `request` ended or binds: it leaves the index. */
delete(name: string, request: UnboundRequest): void {
const requests = this.#byName.get(name);
if (requests === undefined || !requests.delete(request)) return;
if (requests.size === 0) this.#byName.delete(name);
}
/** The host's publication: every request indexed (by any name) runs its check again, out of the index first. */
wakeAll(): void {
const all = [...this.#byName.values()].flatMap((requests) => [...requests]);
this.#byName.clear();
for (const request of all) request.wake();
}
/** Every indexed request acting for a principal `revoked` names ends with `error`. */
revoke(revoked: (principal: Principal) => boolean, error: ChordError): void {
for (const requests of [...this.#byName.values()]) {
for (const request of [...requests]) {
if (request.principal !== undefined && revoked(request.principal)) request.end(error);
}
}
}
/** The host closed: every indexed request ends with `error`. */
close(error: ChordError): void {
const all = [...this.#byName.values()].flatMap((requests) => [...requests]);
this.#byName.clear();
for (const request of all) request.end(error);
}
}

35
packages/chord/src/next/link/binary.ts
View file

@ -6,6 +6,7 @@
import {
type BoundEntry,
CAUSE_KINDS,
type Codec,
type FailedMember,
type Message,
@ -49,6 +50,11 @@ const PRESENT = 1;
const SEGMENTS = 2;
/** A retry's optional fields, one bit each: its operation (16 raw bytes), then the ids it passed through (a list). */
const OPERATION = 1;
const THROUGH = 2;
/** A hold's optional fields, one bit each. */
const PRINCIPAL = 1;
@ -122,10 +128,20 @@ function encodeFields(message: Message, writer: Writer): void {
return;
}
case "release":
case "retry":
case "loading":
writer.varint(message.hold);
return;
case "retry": {
const { operation, through } = message;
writer.varint(message.hold);
writer.byte((operation === undefined ? 0 : OPERATION) | (through === undefined ? 0 : THROUGH));
if (operation !== undefined) writer.raw(hex(operation.replaceAll("-", ""), 16));
if (through !== undefined) {
writer.varint(through.length);
for (const instance of through) writer.raw(hex(instance.replaceAll("-", ""), 16));
}
return;
}
case "call":
writer.varint(message.call);
writer.varint(message.hold);
@ -162,7 +178,7 @@ function encodeFields(message: Message, writer: Writer): void {
return;
case "failed":
writer.varint(message.hold);
writer.byte(message.cause.kind === "transient" ? 0 : 1);
writer.byte(CAUSE_KINDS.indexOf(message.cause.kind));
writer.json(message.cause.error);
// 0 when absent, else one more than the delay.
writer.varint(message.retryIn === undefined ? 0 : message.retryIn + 1);
@ -224,9 +240,16 @@ function decodeFields(kind: (typeof KINDS)[number], reader: Reader): Message {
};
}
case "release":
case "retry":
case "loading":
return { kind, hold: reader.varint() };
case "retry": {
const hold = reader.varint();
const flags = reader.byte();
if (flags > (OPERATION | THROUGH)) throw new Error("binary: an unknown retry flag");
const operation = flags & OPERATION ? { operation: uuid(reader.raw(16)) } : {};
const through = flags & THROUGH ? { through: reader.list(() => uuid(reader.raw(16))) } : {};
return { kind, hold, ...operation, ...through };
}
case "call": {
const [call, hold, method] = [reader.varint(), reader.varint(), reader.varint()];
return { kind, call, hold, method, ...reader.payload("args") };
@ -255,15 +278,15 @@ function decodeFields(kind: (typeof KINDS)[number], reader: Reader): Message {
}
case "failed": {
const hold = reader.varint();
const cause = reader.byte();
if (cause > 1) throw new Error("binary: an unknown failure kind");
const cause = CAUSE_KINDS[reader.byte()];
if (cause === undefined) throw new Error("binary: an unknown failure kind");
// The schema check after decoding establishes the wire error.
const error = reader.json() as Extract<Message, { kind: "error" }>["error"];
const retryIn = reader.varint();
return {
kind,
hold,
cause: { kind: cause === 0 ? "transient" : "broken", error },
cause: { kind: cause, error },
...(retryIn === 0 ? {} : { retryIn: retryIn - 1 }),
};
}

22
packages/chord/src/next/link/message.ts
View file

@ -3,7 +3,7 @@
import Type from "typebox";
import type { Context } from "../../context/index.ts";
import { arr, int, lit, obj, opt, type Static, str, union, unknown } from "../contracts/index.ts";
import { arr, int, lit, obj, oneOf, opt, type Static, str, union, unknown } from "../contracts/index.ts";
import { validatorFor } from "../data/internal.ts";
import type { Part } from "./part.ts";
@ -50,6 +50,9 @@ const incarnation = str({ pattern: "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f
/** The longest activation chain a hold carries (Q5): a longer one keeps its last entries. */
export const CHAIN = 32;
/** The kinds of a `failed` message's cause, in their codes' order (the binary codec's byte, the text codec's letter). */
export const CAUSE_KINDS: ["transient", "broken", "waiting"] = ["transient", "broken", "waiting"];
/** The protocol's messages, one tagged union. Parts are opaque here (the owners parse them). */
const MESSAGE = union([
// `calls`: how many concurrent calls this side accepts (the call window).
@ -75,8 +78,16 @@ const MESSAGE = union([
obj({ kind: lit("release"), origin: opt(origin), hold: id }),
obj({ kind: lit("call"), origin: opt(origin), call: id, hold: id, method: index, args: opt(part), segments }),
obj({ kind: lit("abort"), origin: opt(origin), call: id }),
// Asks the provider for an attempt of a failed hold's object now (`ref.retry()`).
obj({ kind: lit("retry"), origin: opt(origin), hold: id }),
// Asks the provider for an attempt of a failed hold's object now (`ref.retry()`); `operation` names the one retry it
// belongs to (a uuidv7) and `through` the instance ids of the objects it passed through (with `operation` only), so
// an object it reaches twice (a diamond, a ring) handles it once. Relays forward both.
obj({
kind: lit("retry"),
origin: opt(origin),
hold: id,
operation: opt(incarnation),
through: opt(Type.Array(incarnation, { maxItems: CHAIN })),
}),
// Binds several holds at once (first bindings, re-holds), applied in one batch.
obj({
kind: lit("bound"),
@ -103,8 +114,9 @@ const MESSAGE = union([
origin: opt(origin),
hold: id,
// A failure cause's error in the wire error format (a ChordError, checked by L1's envelope check, or a plugin's).
cause: obj({ kind: union([lit("transient"), lit("broken")]), error: wireError }),
// The rest of the object's backoff window in milliseconds: when it retries by itself (absent: broken).
// `waiting`: not the object's fault (no provider, no fitting version, a dependency failed); an event wakes it.
cause: obj({ kind: oneOf(...CAUSE_KINDS), error: wireError }),
// The rest of the object's backoff window in milliseconds: when it retries by itself (absent: broken, waiting).
retryIn: opt(int({ min: 0 })),
}),
// The failed object of a hold started a new attempt.

22
packages/chord/src/next/link/text.ts
View file

@ -8,6 +8,7 @@
import {
type BoundEntry,
CAUSE_KINDS,
type Codec,
type FailedMember,
type Message,
@ -91,7 +92,7 @@ function ownTuple(message: Message): string {
case "state":
return `["s",[${message.holds.map(stateEntry).join(",")}]]`;
case "failed": {
const kind = message.cause.kind === "transient" ? "t" : "b";
const kind = message.cause.kind[0];
const retryIn = message.retryIn === undefined ? [] : [message.retryIn];
return json(["f", message.hold, kind, message.cause.error, ...retryIn]);
}
@ -104,7 +105,10 @@ function ownTuple(message: Message): string {
case "revoke":
return json(["V", message.revocation]);
case "retry":
return json(["T", message.hold]);
// `through` travels only with an `operation` (Chord sends both).
if (message.operation === undefined) return json(["T", message.hold]);
if (message.through === undefined) return json(["T", message.hold, message.operation]);
return json(["T", message.hold, message.operation, message.through]);
case "loading":
return json(["l", message.hold]);
case "ack":
@ -257,10 +261,11 @@ function fromFields(fields: unknown[], rest: Uint8Array): Message {
};
case "f": {
arity(3, 4);
if (f[1] !== "t" && f[1] !== "b") throw new Error("text: an unknown failure kind");
const causeKind = CAUSE_KINDS.find((each) => each[0] === f[1]);
if (causeKind === undefined) throw new Error("text: an unknown failure kind");
// The schema check after decoding establishes the wire error.
const error = f[2] as Extract<Message, { kind: "error" }>["error"];
const cause = { kind: f[1] === "t" ? "transient" : "broken", error } as const;
const cause = { kind: causeKind, error };
return { kind: "failed", hold: field(0), cause, ...(f.length === 4 ? { retryIn: field<number>(3) } : {}) };
}
case "r":
@ -276,8 +281,13 @@ function fromFields(fields: unknown[], rest: Uint8Array): Message {
arity(1);
return { kind: "revoke", revocation: f[0] };
case "T":
arity(1);
return { kind: "retry", hold: field(0) };
arity(1, 2, 3);
return {
kind: "retry",
hold: field(0),
...(f.length >= 2 ? { operation: field<string>(1) } : {}),
...(f.length === 3 ? { through: field<string[]>(2) } : {}),
};
case "l":
arity(1);
return { kind: "loading", hold: field(0) };

32
packages/chord/src/next/registries/README.md
View file

@ -39,8 +39,10 @@ host.serve(Server, (_, self) => {
a change of the table marks it, and it composes when read or, while someone watches, observes or waits for it, in
the batch that changed the table (after the commit's adoptions); an unchanged table (every entry the same) composes
nothing. A `compose` that throws is the owner's bug: `registry_failed` once per episode, watches get one `failed`
call, `.value` keeps the last good composition (before any, it throws the error); the next good composition is a
value call, also when its value is the same.
call, `.value` keeps the last good composition while every contribution it was built from is still in the table
(before any, or once one of them left, it throws the error: a contributor that failed or ended may have closed the
native its entry held; decision 5, slice 7a); the next good composition is a value call, also when its value is the
same.
- The table (`RegistryTable<C>`): one `RegistryEntry { plugin, type?, version, value }` per live contributor, sorted by
plugin name, then type (a facet's entry first): never load order. A contribution of an older version is adapted with
`accepts[version].contribution` once per contribution value; a newer one, or an older one without an adapter, is left
@ -53,16 +55,18 @@ host.serve(Server, (_, self) => {
`RegistryValue<V>`: a native watchable (`.value`, `watch`) with `observe(observer, ctx)`, a synchronous observer of
its compositions (Durable's shape, for `registryReader`). A reader whose token is older than the provider's gets the
composition mapped back with `accepts[version].value`, once per composition, only while read or followed; a mapping
that throws fails that version only (`registry_failed`, its last good mapping kept). A reader without a mapping, or
newer than the provider, gets `contract_mismatch` (its object fails `broken`: only a reload helps). The mapping is
kept only while a reader or follower is there. Without a provider a registry has no value
(`unavailable/undeclared`); its contributions wait in the table.
that throws fails that version only (`registry_failed`, its last good mapping kept while every contribution the
composition it was mapped from was built from is still in the table: decision 5 holds for every version). A reader without a mapping, or
newer than the provider, gets `contract_mismatch`. The mapping is kept only while a reader or follower is there.
Without a provider a registry has no value (`unavailable/undeclared`); its contributions wait in the table. A
reader's attempt without a usable value parks `failed { waiting }` (`host/`), woken when the value turns usable; a
`host.registry` waits the same way up to its `readyTimeout`.
- `keyedEntries(table, entriesOf, report)`: a map of every entry's keyed items, the first by plugin name keeping a key,
a duplicate reported through `compose`'s `report`; `orderedEntries(table, itemsOf, orderOf?)`: every entry's items in
table order, or by `orderOf`. Both keep each item's `plugin` (and `type`): a contribution is another plugin's code, so
a use site catches its throws and rejections, reports the plugin, and goes on.
- Registries carry natives: an entry lives exactly as long as its contributor. A last good composition kept after a
throwing `compose` may still name a contributor that left: use sites look up at each use and keep nothing.
- Registries carry natives: an entry lives exactly as long as its contributor, and a last good composition no longer
than the contributions it was built from; use sites look up at each use and keep nothing.
## For Chord's layers (`internal.ts`)
@ -75,7 +79,11 @@ host.serve(Server, (_, self) => {
(`value(version)`); `close()` at the host's close.
- `RegistryContribution` (`contribution.ts`): one contributor's contribution, a `Dependent` of its source;
`entry(version, provider)` is its table entry (the same object while its value is), or `undefined` when left out.
- `RegistryValueSource`: a `FollowedSource` per reader version; `read(waiter)` registers an activation's object
(`RegistryWaiter`), woken with `dependencyLive()` (on a microtask) when the value gets its first good value; readers
waiting for one count as followers, so a change composes for them. At the host's close it keeps its last start and
lets the record (and the host) go.
- `RegistryValueSource`: a `FollowedSource` per reader version; `read()` registers a reader (an attempt that declared
it, until its references are released: an older version's mapping is kept meanwhile); `wait(waiter)` registers a
`RegistryWaiter` (a parked attempt, a waiting `host.registry`), told with `dependencyMoved()` (inside the batch: it
only records) at every change of whether the value is usable or of its failure while it is not (errors compared by
value: a version mismatch is made per read), and with `closed(error)` when the host closes (`released/closed`: the
wait ends); waiters count as followers, so a change composes for them. At the host's close it keeps its last start
and lets the record (and the host) go. `RegistryRecord.composedFrom` and `inTable` give a mapping the provenance
decision 5 checks.

43
packages/chord/src/next/registries/record.ts
View file

@ -44,9 +44,12 @@ export type RegistryEdge = { readonly from: RegistryRecord; readonly to: Registr
/** The provider as served: the defining plugin's version, `compose` and adapters. */
type Provider = { readonly version: number; readonly provider: RegistryProvider<unknown, unknown> };
/** Where a composition stands: never composed, its last good value, and its failure while `compose` throws. */
/**
* Where a composition stands: never composed, its last good value (with the contributions it was built from), and its
* failure while `compose` throws.
*/
type Composition = {
readonly good: { readonly value: unknown } | undefined;
readonly good: { readonly value: unknown; readonly from: readonly RegistryContribution[] } | undefined;
readonly failure: { readonly error: unknown } | undefined;
};
@ -59,7 +62,7 @@ export class RegistryRecord {
readonly #report: (error: RegistryError) => void;
#provider: Provider | undefined;
/** The contributions of live contributors, sorted by plugin, then type (a facet's first). */
readonly #contributions: RegistryContribution[] = [];
#contributions: readonly RegistryContribution[] = [];
/** The last table composed, kept so an unchanged table composes nothing. */
#table: RegistryTable<unknown> | undefined;
#dirty = true;
@ -128,7 +131,8 @@ export class RegistryRecord {
/** Adds a live contributor's contribution, inside a batch: the table changes. */
add(contribution: RegistryContribution): void {
const at = this.#contributions.findIndex((other) => compareContributors(contribution, other) < 0);
this.#contributions.splice(at < 0 ? this.#contributions.length : at, 0, contribution);
// A new list per change (contributors come and go rarely): a composition keeps the one it was built from.
this.#contributions = this.#contributions.toSpliced(at < 0 ? this.#contributions.length : at, 0, contribution);
const { reads } = contribution;
if (reads !== undefined) this.#reads.set(reads, (this.#reads.get(reads) ?? 0) + 1);
contribution.follow();
@ -139,7 +143,7 @@ export class RegistryRecord {
remove(contribution: RegistryContribution): void {
const at = this.#contributions.indexOf(contribution);
if (at < 0) return;
this.#contributions.splice(at, 1);
this.#contributions = this.#contributions.toSpliced(at, 1);
const { reads } = contribution;
if (reads !== undefined) {
const left = this.#reads.get(reads)! - 1;
@ -176,6 +180,20 @@ export class RegistryRecord {
return { kind: "value", value: good!.value };
}
/** The contributions the current good composition was built from; `undefined` while there is none. */
get composedFrom(): readonly RegistryContribution[] | undefined {
return this.#composition.good?.from;
}
/**
* Whether every contribution of `from` (what a composition, or a mapping of it, was built from) is still in the
* table (decision 5: one that left may have closed the native its entry held).
*/
inTable(from: readonly RegistryContribution[]): boolean {
const current = this.#contributions;
return from === current || from.every((each) => current.includes(each));
}
/** The adapter mapping the composed value back for readers of `version`; `undefined` when there is none. */
valueAdapter(version: number): ((current: never) => unknown) | undefined {
return this.#provider?.provider.accepts?.[version]?.value;
@ -203,7 +221,7 @@ export class RegistryRecord {
*/
close(): void {
for (const contribution of this.#contributions) contribution.unfollow();
this.#contributions.length = 0;
this.#contributions = [];
this.#reads.clear();
this.#table = undefined;
this.#dirty = false;
@ -226,8 +244,9 @@ export class RegistryRecord {
/**
* Builds the table and composes it, unless it is the last one composed, entry for entry (`compose` is pure). A
* throw is a failure episode: reported once, the last good composition kept; conflicts are reported once per
* episode, as their message and plugins.
* throw is a failure episode: reported once, the last good composition kept while every contribution it was built
* from is still in the table (decision 5: one whose contributor ended may hold a closed native); conflicts are
* reported once per episode, as their message and plugins.
*/
#recompose({ version, provider }: Provider): void {
this.#dirty = false;
@ -247,10 +266,14 @@ export class RegistryRecord {
value = this.#batch.compute(() => provider.compose(table, context));
} catch (error) {
if (this.#composition.failure === undefined) this.failed(error);
this.#composition = { good: this.#composition.good, failure: { error } };
const { good } = this.#composition;
this.#composition = {
good: good !== undefined && this.inTable(good.from) ? good : undefined,
failure: { error },
};
return;
}
this.#composition = { good: { value }, failure: undefined };
this.#composition = { good: { value, from: this.#contributions }, failure: undefined };
for (const [key, conflict] of conflicts) {
if (this.#conflicts.has(key)) continue;
this.#report({ kind: "registry_conflict", registry: this.name, ...conflict });

105
packages/chord/src/next/registries/value.ts
View file

@ -1,20 +1,40 @@
// Registry values (slice 8 §1 "Reading", S2, S3): a registry's composed value as the readers of one version see it, a
// native source. The provider's version reads the composition itself; an older one reads it mapped back with
// `accepts[old].value`, once per composition, kept only while someone reads or follows it; a version the provider
// cannot serve fails `contract_mismatch`. Readers declared by attempts are woken (R18) when the value gets its first
// good value.
// cannot serve fails `contract_mismatch`. Its readers (attempts that declared it) keep the mapping; its waiters
// (attempts parked on it, a waiting `host.registry`) are woken at every change of whether it is usable, and of its
// failure while it is not (dependency model §9).
import { ChordError } from "../data/index.ts";
import { ChordError, isChordError } from "../data/index.ts";
import { type Batch, FollowedSource, type NativeStart } from "../state/internal.ts";
import type { RegistryContribution } from "./contribution.ts";
import type { RegistryRecord } from "./record.ts";
/** An object waiting for a registry's value (`self.registry`): retried at once when its first value arrives (R18). */
/** Who waits for a registry's value (a parked attempt, a `host.registry`): told when it may have changed. */
export interface RegistryWaiter {
dependencyLive(): void;
/** The value may have changed (inside its batch: it only records). */
dependencyMoved(): void;
/** The value closed with its host: the wait ends with `error` (`released/closed`). */
closed(error: ChordError): void;
}
/** The last good mapping for an older version, and the composition it was made from. */
type Mapped = { readonly from: unknown; readonly value: unknown };
const USABLE = { kind: "usable" } as const;
/** Whether two failures say the same: the same error, or ChordErrors alike (a version mismatch is made per read). */
function sameError(a: unknown, b: unknown): boolean {
if (a === b) return true;
return isChordError(a) && isChordError(b) && a.code === b.code && a.reason === b.reason && a.message === b.message;
}
/**
* The last good mapping for an older version, the composition it was made from, and the contributions that composition
* was built from (kept only while all are in the table, decision 5).
*/
type Mapped = {
readonly from: unknown;
readonly value: unknown;
readonly contributions: readonly RegistryContribution[] | undefined;
};
/** A failed mapping of composition `from`, until another composition maps. */
type MapFailure = { readonly from: unknown; readonly error: unknown };
@ -32,10 +52,15 @@ export class RegistryValueSource extends FollowedSource<unknown> {
readonly name: string;
readonly version: number;
#registered: Registered;
/** Readers declared by attempts, each a token of its own (one object may read twice), until released. */
readonly #readers = new Set<{ readonly waiter: RegistryWaiter }>();
/** Whether it had a good value: the first one wakes the readers. */
#valued = false;
/** Its readers, each a token of its own (one object may read twice), until released. */
readonly #readers = new Set<object>();
/** Its waiters, each a token of its own, until released. */
readonly #waiters = new Set<{ readonly waiter: RegistryWaiter }>();
/**
* What its waiters were last told: usable (a value, or a failure with a last good one), or the error it has none
* for; a change wakes them.
*/
#told: { readonly kind: "usable" } | { readonly kind: "failed"; readonly error: unknown } | undefined;
#mapped: Mapped | undefined;
#mapFailure: MapFailure | undefined;
@ -60,16 +85,15 @@ export class RegistryValueSource extends FollowedSource<unknown> {
return start.last.value;
}
/** Its composition (mapped for an older version), composed now if its table changed; a first value wakes readers. */
/**
* Its composition (mapped for an older version), composed now if its table changed; a change of whether it is
* usable, or of its failure while it is not, wakes its waiters.
*/
override start(): NativeStart<unknown> {
const registered = this.#registered;
if (registered.kind === "closed") return registered.last;
const start = this.#start(registered.record);
if (!this.#valued && (start.kind === "value" || start.last !== undefined)) {
this.#valued = true;
// Not inside the batch that composed: a retry runs a factory, which may not run in a batch.
for (const { waiter } of this.#readers) queueMicrotask(() => waiter.dependencyLive());
}
if (this.#waiters.size > 0) this.#tell(start);
return start;
}
@ -80,14 +104,14 @@ export class RegistryValueSource extends FollowedSource<unknown> {
return start.value;
}
/** Followed while someone watches or observes it, and while readers wait for its first value. */
/** Followed while someone watches or observes it, and while waiters wait on it. */
override get watched(): boolean {
return super.watched || (this.#readers.size > 0 && !this.#valued);
return super.watched || this.#waiters.size > 0;
}
/** Registers `waiter` (an activation's declared dependency) until the returned release. */
read(waiter: RegistryWaiter): () => void {
const token = { waiter };
/** Registers a reader (an attempt that declared it) until the returned release: its mapping is kept meanwhile. */
read(): () => void {
const token = {};
this.#readers.add(token);
return () => {
this.#readers.delete(token);
@ -95,6 +119,20 @@ export class RegistryValueSource extends FollowedSource<unknown> {
};
}
/**
* Registers `waiter` (a parked attempt's object, a waiting `host.registry`) until the returned release: it is told
* of every change of whether this value is usable (inside the batch that made it: it only records).
*/
wait(waiter: RegistryWaiter): () => void {
const token = { waiter };
if (this.#waiters.size === 0) this.#told = undefined;
this.#waiters.add(token);
return () => {
this.#waiters.delete(token);
if (this.#waiters.size === 0) this.#told = undefined;
};
}
/** The registry's table changed (inside a batch): its dependents are marked, its followers get the next frame. */
recomposed(): void {
this.changed();
@ -119,6 +157,11 @@ export class RegistryValueSource extends FollowedSource<unknown> {
}
this.#registered = { kind: "closed", last };
this.#readers.clear();
const waiters = [...this.#waiters];
this.#waiters.clear();
const error = new ChordError({ code: "released", reason: "closed", message: `registry ${this.name}: closed` });
for (const { waiter } of waiters) waiter.closed(error);
this.#told = undefined;
this.#dropMapping();
this.end();
}
@ -128,6 +171,17 @@ export class RegistryValueSource extends FollowedSource<unknown> {
if (this.#readers.size === 0) this.#dropMapping();
}
/** Wakes the waiters when `start` says something else than they were told (by usability, then by the error). */
#tell(start: NativeStart<unknown>): void {
const told = this.#told;
const usable = start.kind === "value" || start.last !== undefined;
if (usable ? told?.kind === "usable" : told?.kind === "failed" && sameError(told.error, start.error)) return;
this.#told = usable ? USABLE : { kind: "failed", error: (start as { readonly error: unknown }).error };
// The first start of a wait wakes nobody: the waiter reads it itself.
if (told === undefined) return;
for (const { waiter } of this.#waiters) waiter.dependencyMoved();
}
#dropMapping(): void {
this.#mapped = undefined;
this.#mapFailure = undefined;
@ -156,13 +210,18 @@ export class RegistryValueSource extends FollowedSource<unknown> {
if (current.kind === "failed" && current.last === undefined) return current;
const from = current.kind === "value" ? current.value : current.last!.value;
// `undefined` is a composition too: the records, not their fields, say whether there is one.
// A mapping built from a contribution that left is no last good value any more (decision 5).
if (this.#mapped?.contributions !== undefined && !record.inTable(this.#mapped.contributions)) {
this.#mapped = undefined;
}
const mapped = this.#mapped;
const failure = this.#mapFailure;
if (mapped !== undefined && mapped.from === from) {
this.#mapFailure = undefined;
} else if (failure === undefined || failure.from !== from) {
try {
this.#mapped = { from, value: record.adapt(() => map(from as never)) };
const value = record.adapt(() => map(from as never));
this.#mapped = { from, value, contributions: record.composedFrom };
this.#mapFailure = undefined;
} catch (error) {
if (failure === undefined) record.failed(error);

36
packages/chord/src/next/remote/README.md
View file

@ -55,8 +55,8 @@ within each link's limits, compression at the origin, activation chains and revo
one, which a relayed hold carries upstream as `limit`. A call on a relayed hold whose arguments are past the link's
wire limit, or on a served hold whose arguments are past its limits, is answered `too_large/part` and never
admitted. Compressed params (on any link) and compressed arguments from a child are protocol errors: params are never
compressed, and clients never send compressed parts. Holds of a principal being revoked that still wait for `load`
end with `gone` (`revokeWaiting`).
compressed, and clients never send compressed parts. Holds of a principal being revoked that hold nothing (before the
publication, refused, relayed and waiting for their link) end with `gone` (`revokeUnbound`).
- **The window** (K = 2, `ProviderSide`): at most two of this host's own `bound` and `state` messages unacknowledged
per origin; every other message is control and never waits. The consumer sends `ack` with the origin after applying
each. The side keeps the sender of every message in flight in order (`local`, or the upstream link a relayed one came
@ -81,13 +81,18 @@ within each link's limits, compression at the origin, activation chains and revo
Trace (tab D through relay R, worker U, a derive over A and B on the tab): U's window for D's origin is full; a
commit touching A and B joins U's pending queues. D acks one: R returns the `ack` to U, whose freed slot carries one
`state` with A's and B's entries: the derive sees both at once, never A new with B old.
- `Hold` (one per `hold` served here, a `Holder` of its object): takes its object once `load` resolved; the hold's
`params` part is parsed then (malformed: a protocol error). A refused type, a mismatch, invalid params, a wrong
host address or a snapshot that throws is a broken `failed` that ends it (the provider forgets the id). While pending
it follows its object's `state`: each failure, transient or broken, is sent as `failed` with `retryIn`, the rest of
the object's backoff window (absent when broken), the current one at once when the object is failed already; the
next attempt is announced with `loading`; the hold stays, so a `retry` (from any holder) renews even a broken object
for every holder. Live, it binds through the window (`boundEntry`: the variant and each member's value as a part,
- `Hold` (one per `hold` served here, a `Holder` of its object, an `UnboundRequest` of the host's index): before the
host's publication it waits in the unbound index (`unplaced`, nothing sent: the consumer is `loading`) and the
publication starts it; then it takes its object in the turn it arrived; the hold's `params` part is parsed then
(malformed: a protocol error). A type not served here (or as a local type), a mismatch, or params the provider's
code places on another host leave it refused `failed { waiting }`, indexed under its type until a publication re-runs
its start (`loading`, then `bound` or `failed`); invalid params or a cycle leave it refused `failed { broken }`; both
are kept for a `retry`, which re-runs the start. While pending it follows its object's `state`: each failure is sent
as `failed` (with `retryIn`, the rest of the object's backoff window, for a transient one; absent for broken and
waiting), each new cause of an object parked on a failed dependency too, the current one at once when the object is
failed already; the next attempt (or the parked object's next `loading`) is announced with `loading`; the hold stays,
so a `retry` (from any holder) renews even a broken object for every holder (a parked object passes it on, one retry
operation of this host). Live, it binds through the window (`boundEntry`: the variant and each member's value as a part,
names as indexes into the bound version). Its members' frames arrive as `Subscriber` calls inside the batch; at its
end the hold's entry lists the members that changed with their ops, each encoded with that member's own Delta
encoder (`entry`), or queues them (`queue`, later `pendingEntry`). Each entry counts one `chord.link.encodes`. A
@ -97,9 +102,10 @@ within each link's limits, compression at the origin, activation chains and revo
dropping the member's pending ops; a value after it replaces it), and a retirement of a failed member waits for its
next value (never listed beside its failure). Its
object's end sends `gone` with its error (`forbidden/revoked` for a revocation; its running calls aborted with it). A
released pending hold stops its wait for `load` at once. Params past the link's wire limit, or its decoded one, fail
the hold broken `too_large/part`; a cycle its chain closes, broken `invalid/cycle` (`host.serving`). It counts as one
of the host's `references` (`host.activity`) while it keeps its object.
released hold before the publication leaves the index at once. Params past the link's wire limit, or its decoded
one, fail the hold broken `too_large/part`; a cycle its chain closes, broken `invalid/cycle` (`host.serving`). It
counts as one of the host's `references` (`host.activity`) from its arrival to its end, bound or not (relayed, it
counts at its destination instead).
- `RelayedHold` (a hold this host neither serves nor parses, `host.place` decided it): id `h` of its downstream
origin, held with the same id on its path's current link under that origin's number, as the arriving hold's caller,
from its kept `hold` message (params copied once out of the received message, so no view keeps the buffer). `call`,
@ -134,9 +140,9 @@ within each link's limits, compression at the origin, activation chains and revo
tells its owner (a route) when it needs a link or lost its last holder.
- `RemoteObject` (one per remote reference, owned by its host across the links of its path): its `state` (`loading`
until a link's hold is bound and again on every drop or `loading`, `failed` from `failed` with `retryAt = now +
retryIn`, `live` on `bound`, `released`), `settled` (pending while loading, resolved while live, rejected while
failed and by a drop or a relay's `loading` while loading: a dependent's activation waits for it and fails at once
then, R18, and `bound` retries the dependent), `lastFailure`, `variant` (set in `bound`'s batch, before derives recompute) and
retryIn` for a transient cause, `retryAt` undefined for a broken or waiting one, `live` on `bound`, `released`;
every change tells its binding, whose parked dependent checks again: no new hold, no backoff), `remote` and
`blocksFirstPass` (never: a wait on another host ends this host's first pass), `lastFailure`, `variant` (set in `bound`'s batch, before derives recompute) and
`member` (the last binding's, kept while loading or failed; outside the variant `invalid/member`), `call` (only while
live; outside the variant `invalid/member`), `retry` (sends `retry` while failed on an open link; after a part past
the limits, placed anew with `host.pathOf` and held with a new id, now or on that path's next link). Its holds made

8
packages/chord/src/next/remote/consumer.ts
View file

@ -8,6 +8,7 @@ import type { Context } from "../../context/index.ts";
import { type MethodData, takesNoArgs } from "../contracts/internal.ts";
import { ChordError, isChordError } from "../data/index.ts";
import { asChordError } from "../data/internal.ts";
import type { RetryOperation } from "../host/internal.ts";
import {
type BoundEntry,
type ConsumerMessage,
@ -125,9 +126,10 @@ export class ConsumerSide {
this.forget(hold, error);
}
/** `ref.retry()` on a failed hold: the provider is asked for an attempt now. */
retry(hold: number): void {
this.#link.send({ kind: "retry", hold });
/** `ref.retry()` on a failed hold, as retry `operation`: the provider is asked for an attempt now. */
retry(hold: number, { id, through }: RetryOperation): void {
const passed = through.length === 0 ? {} : { through };
this.#link.send({ kind: "retry", hold, operation: id, ...passed });
}
/** Drops hold `hold` from the tables; its calls, in flight and waiting, fail with `error` and free their slots. */

181
packages/chord/src/next/remote/hold.ts
View file

@ -1,7 +1,8 @@
// A hold arriving on a link and served here (§7, §22, W1): where it goes once `load` resolved, the object it keeps,
// the notices of its object's failures and new attempts, the snapshot and subscription taken in one synchronous step
// when a slot of the window is free, and the ops of its members per batch (sent at once, or pending while the window
// is full), each member with its own Delta encoder.
// A hold arriving on a link and served here (§7, §22, W1): where it goes once the host published (waiting in the
// unbound index until then), why it is unbound (refused: kept in the index while it waits for a name, dependency
// model §1), the object it keeps, the notices of its object's failures and new attempts, the snapshot and subscription
// taken in one synchronous step when a slot of the window is free, and the ops of its members per batch (sent at
// once, or pending while the window is full), each member with its own Delta encoder.
import type { Context } from "../../context/index.ts";
import { type Encoder, encoder, type Op, PendingOps } from "../../delta/index.ts";
@ -13,9 +14,16 @@ import {
variantNames,
variantOrder,
} from "../contracts/internal.ts";
import { type ChordError, isChordError } from "../data/index.ts";
import { type ChordError, type FailureCause, isChordError, type Principal } from "../data/index.ts";
import { asChordError } from "../data/internal.ts";
import type { Caller, ChordObject, Holder } from "../host/internal.ts";
import {
type Caller,
type ChordObject,
type Holder,
retryOperation,
type Serving,
type UnboundRequest,
} from "../host/internal.ts";
import {
type BoundEntry,
type FailedMember,
@ -56,12 +64,13 @@ type Followed = {
};
/**
* Where a hold stands: `pending` until its object is live (waiting for `load`, the activation and its retries;
* `toldFailed` once the consumer was told of a failure, so the next attempt is announced with `loading`); `binding`
* while its object is live and its `bound` waits for a slot; `bound` with the object's variant and the members it
* follows.
* Where a hold stands: `unplaced` in the unbound index until the host's publication places it; `pending` until its
* object is live (the activation, parking and retries; `toldFailed` once the consumer was told of a failure, so the
* next attempt is announced with `loading`); `binding` while its object is live and its `bound` waits for a slot;
* `bound` with the object's variant and the members it follows.
*/
type HoldState =
| { readonly kind: "unplaced"; readonly request: HoldRequest }
| { readonly kind: "pending"; readonly toldFailed: boolean }
| { readonly kind: "binding" }
| {
@ -72,8 +81,12 @@ type HoldState =
/** The members it follows, in the order they were subscribed. */
readonly followed: Followed[];
}
/** Refused where it arrived (host address, `authorize`, a mismatch, invalid params, a cycle): `retry` restarts it. */
| { readonly kind: "refused"; readonly request: HoldRequest }
/**
* Refused where it arrived, with `cause`: `waiting` (not served here, another version, placed elsewhere by the
* provider's code) in the unbound index until a publication wakes it; `broken` (invalid params, a cycle, a route that
* refused, a part past the limits) until a `retry` restarts it.
*/
| { readonly kind: "refused"; readonly request: HoldRequest; readonly cause: FailureCause }
| { readonly kind: "released" };
/** A bound hold's state. */
@ -86,11 +99,13 @@ const BINDING: HoldState = { kind: "binding" };
/**
* One hold served here: a reference to its object acting as its caller (the hold's principal). While pending, each
* failure of its object is sent as `failed` (with the rest of its backoff window) and the next attempt as `loading`; a
* hold that cannot be taken (a refused type, a mismatch, invalid params or a wrong host address) is a broken `failed`
* that holds nothing until a `retry` starts it again. Once live it binds when its link's window has a slot, then sends a part per batch that changed its members
* (or keeps their ops pending while the window is full), and `gone` when its object ends.
* hold that cannot be taken holds nothing: `failed { waiting }` (not served, another version) until a publication
* re-runs its start, `failed { broken }` (invalid params, a cycle) until a `retry` does. Once live it binds when its
* link's window has a slot, then sends a part per batch that changed its members (or keeps their ops pending while the
* window is full), and `gone` when its object ends. It counts as one of the host's references from its arrival to its
* end, bound or not.
*/
export class Hold implements Holder {
export class Hold implements Holder, UnboundRequest {
readonly kind = "served";
readonly id: number;
/** A hold is declared by no object on this host. */
@ -103,11 +118,6 @@ export class Hold implements Holder {
#state: HoldState = PENDING;
/** The held object, from its binding until the hold is released. */
#object: ChordObject | undefined;
/**
* Resolved on release: a pending hold stops waiting for `load`. A plain resolution, not an abort: an aborted wait
* rejects with an error whose stack keeps this hold, and the waited promise keeps that rejection.
*/
readonly #released = Promise.withResolvers<void>();
/** The members the current batch changed, with their ops, in order. */
#changes: (readonly [Followed, readonly Op[]])[] = [];
/** The members that failed since the last message that carried this hold. */
@ -115,14 +125,15 @@ export class Hold implements Holder {
/** The members whose document was retired since the last message that carried this hold (ORed while it waits). */
#retired: readonly number[] = NO_RETIRED;
#touched = false;
/** Whether the hold counts as one of the host's references (`host.activity`): while it keeps its object. */
#counted = false;
/** Whether the hold counts as one of the host's references (`host.activity`): from its arrival until it ends. */
#counted = true;
constructor(id: number, provider: ProviderSide, caller: Caller, limits: Limits) {
this.id = id;
this.#provider = provider;
this.caller = caller;
this.limits = limits;
provider.link.host.count("references", 1);
}
/** Whether the hold is live: calls on it reach its object. */
@ -130,33 +141,37 @@ export class Hold implements Holder {
return this.#state.kind === "bound";
}
/** Whether the hold waits for `load`: admitted, nothing taken yet (revocation ends it, §4). */
get waiting(): boolean {
return this.#state.kind === "pending" && this.#object === undefined;
/** Whom it acts for (the unbound index's revocation). */
get principal(): Principal | undefined {
return this.caller.principal;
}
/** Whether the hold holds no object: before the publication, or refused (revocation ends it, §4). */
get unbound(): boolean {
return this.#object === undefined && this.#state.kind !== "released";
}
/**
* Admits the hold once `load` resolved: its params within the link's part limit (`too_large/part`), then relayed
* (it is replaced by a relayed hold), or served here, its params opened and parsed (a malformed part is a protocol
* error), its object taken and followed (`moved`). A hold that cannot go anywhere, a type that does not match,
* invalid params, a wrong host address or a cycle (Q5) is a broken `failed`; the hold is kept, refused, for a
* `retry`.
* Admits the hold: before the host's publication it waits in the unbound index, placed by the publication.
* Then its params within the link's part limit (`too_large/part`), then relayed (it is replaced by a relayed hold),
* or served here, its params opened and parsed (a malformed part is a protocol error), its object taken and followed
* (`moved`) in the turn it arrived. A hold that cannot be served here waits refused (`#refuse`).
*/
async start(request: HoldRequest): Promise<void> {
start(request: HoldRequest): void {
const { link } = this.#provider;
// Waits only for a `load` still running (it never rejects): once loaded, the reference is taken in the turn the
// hold arrived.
const pendingLoad = link.host.pendingLoad;
if (pendingLoad !== undefined) await Promise.race([pendingLoad, this.#released.promise]);
// Released meanwhile (the wait may have settled first): nothing is taken.
if (this.#state.kind !== "pending") return;
let object: ChordObject;
if (!link.host.published) {
this.#state = { kind: "unplaced", request: owned(request) };
link.host.index(request.type, this);
return;
}
let served: Serving;
try {
const oversize = oversized(request.params, link.limits.part);
if (oversize !== undefined) throw oversize;
const placed = link.host.place(request, link, this.caller, link.context);
if (placed.kind === "path") {
this.#state = { kind: "released" };
// Counted where it is served: a relayed hold is not this host's reference.
this.drop();
new RelayedHold(this.id, this.#provider, request, this.caller, placed.path).start();
return;
}
@ -170,18 +185,35 @@ export class Hold implements Holder {
link.close();
return;
}
object = link.host.serving(request, params, this.caller.principal!, link.context);
served = link.host.serving(request, params, this.caller.principal!, link.context);
} catch (error) {
return this.#refuse(request, asChordError(error, request.type));
this.#refuse(request, { kind: "broken", error: asChordError(error, request.type) });
return;
}
// Counted before the object lists it in its presence: a derive over that may close this link synchronously.
if (served.kind === "waiting") {
this.#refuse(request, served.cause);
return;
}
const { object } = served;
this.#object = object;
this.#counted = true;
link.host.count("references", 1);
object.hold(this);
this.moved();
}
/**
* Woken by the unbound index (out of it already): before the publication, placed now; refused `waiting`, its start
* runs again, announced with `loading` (then `bound` or `failed`).
*/
wake(): void {
const state = this.#state;
if (state.kind === "unplaced") {
this.#state = PENDING;
this.start(state.request);
} else if (state.kind === "refused") {
this.#restart(state.request);
}
}
/**
* The object's state changed, or the hold took it: a pending hold binds once it is live (when a slot is free),
* sends each failure, and announces the attempt after a failure with `loading`. A live object stays live until it
@ -195,6 +227,7 @@ export class Hold implements Holder {
this.#state = BINDING;
this.#provider.bind(this, this.#provider.link.context);
} else if (state.kind === "failed") {
// Each failure and each new cause (a parked object's blocker changed) is a message of its own.
const { cause, retryAt } = state;
const retryIn = retryAt === undefined ? {} : { retryIn: Math.max(0, retryAt - Date.now()) };
const failed = { kind: cause.kind, error: wireError(cause.error) };
@ -207,18 +240,26 @@ export class Hold implements Holder {
}
/**
* A `retry` message: the object is asked for an attempt now (it ignores this unless failed); a refused hold is
* started again (a route's `authorize` may admit it now, a type may be served here now).
* A `retry` message, of its retry operation (a new one when it names none) through the objects it names: the object is
* asked for an attempt now (it ignores this unless failed, or when the operation reached it before; parked, it
* passes it on); a refused hold is
* started again (a route's `authorize` may admit it now).
*/
retry(): void {
retry({ operation, through = [] }: Extract<Message, { readonly kind: "retry" }>): void {
const state = this.#state;
if (state.kind !== "refused") {
this.#object?.retry();
this.#object?.retry(operation === undefined ? retryOperation() : { id: operation, through });
return;
}
if (state.cause.kind === "waiting") this.#provider.link.host.unindex(state.request.type, this);
this.#restart(state.request);
}
/** Starts a refused hold again, announced with `loading`. */
#restart(request: HoldRequest): void {
this.#state = PENDING;
this.#provider.send({ kind: "loading", hold: this.id });
void this.start(state.request);
this.start(request);
}
/** Calls a method of the held object as the hold's caller. */
@ -324,18 +365,21 @@ export class Hold implements Holder {
/** Ends the hold without releasing its object (the host closes and ends its objects itself). */
drop(): void {
if (this.#state.kind === "bound") {
for (const { source, subscriber } of this.#state.followed) source.unsubscribe(subscriber);
const state = this.#state;
const { host } = this.#provider.link;
if (state.kind === "bound") {
for (const { source, subscriber } of state.followed) source.unsubscribe(subscriber);
} else if (state.kind === "unplaced" || (state.kind === "refused" && state.cause.kind === "waiting")) {
host.unindex(state.request.type, this);
}
this.#state = { kind: "released" };
this.#released.resolve();
if (this.#counted) this.#provider.link.host.count("references", -1);
if (this.#counted) host.count("references", -1);
this.#counted = false;
}
/**
* The object ended while held (revocation; a reload, slice 7), or the hold was revoked while it waited for `load`:
* the hold sends `gone` with `error`.
* The object ended while held (revocation; a reload, slice 7b), or the hold was revoked (or its host closed) while it
* held no object: the hold sends `gone` with `error`.
*/
end(error: ChordError): void {
if (this.#state.kind === "released") return;
@ -426,16 +470,27 @@ export class Hold implements Holder {
this.#provider.touched(this);
}
/** Refuses the hold: a broken `failed`; it keeps `request` for a `retry`, holding nothing until then. */
#refuse(request: HoldRequest, error: ChordError): void {
const { params } = request;
// Its params owned, not a view of the message they came in (which the link's buffers may hold).
const owned = typeof params === "string" ? params : (params as Uint8Array).slice();
this.#state = { kind: "refused", request: { ...request, params: owned } };
this.#provider.send({ kind: "failed", hold: this.id, cause: { kind: "broken", error: wireError(error) } });
/**
* Refuses the hold with `cause`, holding nothing: it keeps `request`, indexed under its type while `waiting` (a
* publication starts it again), for a `retry` otherwise.
*/
#refuse(request: HoldRequest, cause: FailureCause): void {
this.#state = { kind: "refused", request: owned(request), cause };
if (cause.kind === "waiting") this.#provider.link.host.index(request.type, this);
this.#provider.send({
kind: "failed",
hold: this.id,
cause: { kind: cause.kind, error: wireError(cause.error) },
});
}
}
/** `request` with its params owned, not a view of the message they came in (which the link's buffers may hold). */
function owned(request: HoldRequest): HoldRequest {
const { params } = request;
return typeof params === "string" ? request : { ...request, params: (params as Uint8Array).slice() };
}
/** A `hold` message's fields naming whom it acts for. */
type ActingFor = Pick<HoldMessage, "principal" | "incarnation">;

6
packages/chord/src/next/remote/host-link.ts
View file

@ -193,9 +193,9 @@ export class HostLink implements ChannelHandler {
}
}
/** Ends the holds on this link still waiting for `load` that act for a principal `revoked` names (§4). */
revokeWaiting(revoked: (principal: Principal) => boolean, error: ChordError): void {
for (const provider of this.#providers.values()) provider.revokeWaiting(revoked, error);
/** Ends the holds on this link that hold nothing and act for a principal `revoked` names (§4). */
revokeUnbound(revoked: (principal: Principal) => boolean, error: ChordError): void {
for (const provider of this.#providers.values()) provider.revokeUnbound(revoked, error);
}
closed(): void {

18
packages/chord/src/next/remote/provider.ts
View file

@ -90,7 +90,7 @@ export class ProviderSide {
this.#abort(message.call);
break;
case "retry":
this.#entries.get(message.hold)?.retry();
this.#entries.get(message.hold)?.retry(message);
break;
case "ack":
this.#ack();
@ -128,15 +128,15 @@ export class ProviderSide {
}
/**
* Ends with `error` every hold of this origin that acts for a principal `revoked` names and still waits: for `load`
* (served here), or for its route's link (relayed).
* Ends with `error` every hold of this origin that acts for a principal `revoked` names and holds nothing: served
* here but unbound (before the publication, refused), or relayed and waiting for its route's link.
*/
revokeWaiting(revoked: (principal: Principal) => boolean, error: ChordError): void {
revokeUnbound(revoked: (principal: Principal) => boolean, error: ChordError): void {
for (const hold of [...this.#entries.values()]) {
// A hold arriving on a link acts for a principal: the link's, or one its parent named.
if (!hold.waiting || !revoked(hold.caller.principal!)) continue;
if (hold.kind === "served") hold.end(error);
else hold.revoked(error);
if (!revoked(hold.caller.principal!)) continue;
if (hold.kind === "served" && hold.unbound) hold.end(error);
else if (hold.kind === "relayed" && hold.waiting) hold.revoked(error);
}
}
@ -283,7 +283,7 @@ export class ProviderSide {
/**
* A hold message: a protocol error for a live id or compressed params (params are never compressed), otherwise
* admitted once `load` resolved (`Hold.start`). Its activation chain is kept only from a trusted link (Q5); its
* admitted (`Hold.start`, placed once the host published). Its activation chain is kept only from a trusted link (Q5); its
* decoded limit is the link's, or the smaller one a parent stamped (Q2), and travels with it when it is relayed.
*/
#hold(message: Extract<Message, { readonly kind: "hold" }>): void {
@ -310,7 +310,7 @@ export class ProviderSide {
const limits = limit === link.limits.decoded ? link.limits : { part: link.limits.part, decoded: limit };
const hold = new Hold(id, this, this.#caller(message), limits);
this.#entries.set(id, hold);
void hold.start(request);
hold.start(request);
}
/**

6
packages/chord/src/next/remote/relay.ts
View file

@ -109,10 +109,10 @@ export class RelayedHold {
upstream?.link.send({ kind: "abort", origin: upstream.origin, call: id });
}
/** A downstream `retry`, forwarded. */
retry(): void {
/** A downstream `retry`, forwarded with its operation and the objects it passed through. */
retry(message: Extract<Message, { readonly kind: "retry" }>): void {
const upstream = this.#upstream;
upstream?.link.send({ kind: "retry", origin: upstream.origin, hold: this.id });
upstream?.link.send({ ...message, origin: upstream.origin });
}
/** Released downstream: released upstream (its calls with it) and no later link holds it. */

60
packages/chord/src/next/remote/remote-object.ts
View file

@ -24,7 +24,7 @@ import {
LOADING,
type PinnedKey,
type RefState,
settlement,
type RetryOperation,
type Target,
} from "../host/internal.ts";
import {
@ -78,8 +78,6 @@ export class RemoteObject implements Target {
/** The hold on the path's link while one is open: that link's consumer side and this hold's id on it. */
#hold: { readonly side: ConsumerSide; readonly id: number } | undefined;
#binding: Binding | undefined;
/** Pending while loading, resolved while live, rejected while failed: a dependent's activation waits for it (R18). */
#settled = settlement();
#lastFailure: ChordError | undefined;
constructor(
@ -112,9 +110,14 @@ export class RemoteObject implements Target {
return this.#state;
}
/** Pending while loading; resolves when bound, rejects with a failure or the reference's end. */
get settled(): Promise<void> {
return this.#settled.promise;
/** Whether its hold is on a link now (a remote attempt answers for it), not while its path has no link. */
get heldOnLink(): boolean {
return this.#hold !== undefined;
}
/** An object over a link is no blocker of this host's first pass. */
get blocksFirstPass(): false {
return false;
}
/** The error of the last `failed`, until `bound`. */
@ -151,12 +154,15 @@ export class RemoteObject implements Target {
attach(link: HostLink): void {
const chain = this.#dependent?.activationChain();
this.#hold = { side: link.consumer, id: link.consumer.attach(this, chain) };
// Held on a link now: a parked dependent checks again (`parked_long` counts a hold on a link only).
this.#dependent?.dependencyMoved();
}
/** The link dropped: loading until a new link's hold is bound (`#startOver`). */
dropped(_error: ChordError): void {
this.#hold = undefined;
this.#startOver();
this.#dependent?.dependencyMoved();
}
/**
@ -214,17 +220,18 @@ export class RemoteObject implements Target {
}
/**
* `ref.retry()`: a failed object's provider is asked for an attempt (a hold it refused is started there again). One
* that failed for a part past the link's limits is placed anew and held with a new id, now or on its path's next
* link (a `hostAddress` that throws fails it broken again).
* `ref.retry()`: a failed object's provider is asked for an attempt (a hold it refused is started there again; a
* parked object there passes it on as an operation of its own host). One that failed for a part past the link's
* limits is placed anew and held with a new id, now or on its path's next link (a `hostAddress` that throws fails it
* broken again).
*/
retry(): void {
retry(operation: RetryOperation): void {
if (this.#state.kind !== "failed") return;
if (this.#path === this.#host.unplaced) {
this.#place();
return;
}
this.#hold?.side.retry(this.#hold.id);
this.#hold?.side.retry(this.#hold.id, operation);
}
/** Places a reference whose part was past the limits anew, on its path now (a throwing `hostAddress` fails it). */
@ -313,17 +320,16 @@ export class RemoteObject implements Target {
};
}
/** After a `bound` message's batch: the reference goes live (its parked calls are sent) and R18 retries follow. */
/** After a `bound` message's batch: the reference goes live (its parked calls are sent, a parked dependent goes on). */
wentLive(): void {
this.#lastFailure = undefined;
this.#move(LIVE);
// R18: a dependent that failed while waiting for this object is retried now.
this.#binding?.dependent?.dependencyLive();
}
/**
* `failed`: the provider's object failed and no attempt runs; `retryIn` is the rest of its backoff window. Ignored
* once live (only a pending hold is told).
* `failed`: the provider's object failed and no attempt runs (`retryIn`: the rest of its backoff window), or the hold
* is unbound there or its object parked on a failed dependency (`waiting`), each new cause a new message. Ignored once
* live (only a pending hold is told).
*/
failed(message: Extract<Message, { readonly kind: "failed" }>): void {
if (this.#state.kind === "live") return;
@ -383,6 +389,17 @@ export class RemoteObject implements Target {
};
}
/**
* Its principal was revoked on this host (`host.revoke`): released at its provider if it is held there, then ended
* with `error` as `lost` does; no later link holds it.
*/
revoke(error: ChordError): void {
const hold = this.#hold;
this.#hold = undefined;
hold?.side.release(hold.id, error);
this.lost(error);
}
/** `gone`, or the host closes: the reference ends, its calls reject with `error`. */
lost(error: ChordError): void {
const binding = this.#binding;
@ -399,28 +416,23 @@ export class RemoteObject implements Target {
this.#dependent = undefined;
this.#hold = undefined;
this.#bound = undefined;
this.#settled.reject(error);
this.#path.detach(this);
for (const replica of bound?.replicas.values() ?? []) replica.end(error);
}
/**
* Loading again until the next binding; the members keep their last values. A drop while it is loading already
* changes nothing: a dependent's activation waiting for it goes on waiting for the next link's binding, bounded by
* its `readyTimeout` (§6, R18; a drop is not a dependency failure).
* changes nothing: a dependent parked on it goes on waiting for the next link's binding (a drop is not a dependency
* failure).
*/
#startOver(): void {
const { kind } = this.#state;
if (kind !== "ended" && kind !== "loading") this.#move(LOADING);
}
/** Moves to `state` and tells the reference; `settled` follows (a new wait each time it is loading again). */
/** Moves to `state` and tells the reference. */
#move(state: RefState): void {
const previous = this.#state;
this.#state = state;
if (state.kind === "loading" && previous.kind !== "loading") this.#settled = settlement();
if (state.kind === "live") this.#settled.resolve();
if (state.kind === "failed") this.#settled.reject(state.cause.error);
this.#binding?.moved();
}
}

66
packages/chord/test/next/data/failure.test.ts
View file

@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { broken } from "../../../src/next/data/index.ts";
import { invalid, openFailure } from "../../../src/next/data/internal.ts";
import { broken, ChordError } from "../../../src/next/data/index.ts";
import { invalid, openFailure, sameCause, waitingFor } from "../../../src/next/data/internal.ts";
describe("failure causes", () => {
it("a value thrown by an open is transient unless marked broken", () => {
@ -25,3 +25,65 @@ describe("failure causes", () => {
expect(openFailure(broken(chord), "open")).toEqual({ kind: "broken", error: chord });
});
});
describe("a dependent's waiting cause (dependency model §3)", () => {
const root = new ChordError({ code: "unavailable", message: "app.catalog is not served here" });
it("names the nearest blocker and the root cause, its blocker's error as the cause", () => {
const cause = waitingFor("app.chat(S1)", root);
expect(cause.kind).toBe("waiting");
expect(cause.error.toJSON()).toEqual({
code: "unavailable",
message: "waiting for app.chat(S1): app.catalog is not served here",
});
expect(cause.error.cause).toBe(root);
// A chain names its nearest blocker and its root only (fix round 1, Opus F1).
const chain = waitingFor("app.panel", cause.error);
expect(chain.error.message).toBe("waiting for app.panel: app.catalog is not served here");
});
it("reaches a fixed point when a ring of waiting attempts passes its causes round (Opus F1)", () => {
let b = waitingFor("X", new ChordError({ code: "unavailable", message: "X is not served here" })).error;
let rounds = 0;
for (; rounds < 10; rounds++) {
const next = waitingFor("A", waitingFor("B", b).error).error;
if (next.message === b.message) break;
b = next;
}
expect(rounds).toBeLessThanOrEqual(1);
expect(b.message).toBe("waiting for A: X is not served here");
});
it("keeps the root cause within 500 characters: a long label is clipped first (Sol 8)", () => {
const original = new ChordError({ code: "unavailable", message: "ORIGINAL: missing provider" });
const message = waitingFor(`sol.item(${"x".repeat(600)})`, original).error.message;
expect(message.length).toBeLessThanOrEqual(500);
expect(message).toContain(original.message);
expect(message.startsWith("waiting for sol.item(xxx")).toBe(true);
});
it("keeps a long chain's root cause, never repeated ellipses (Sol 8)", () => {
const original = new ChordError({ code: "unavailable", message: "ORIGINAL: missing provider" });
let error = original;
for (let i = 0; i < 2000; i++) error = waitingFor(`level-${i}`, error).error;
expect(error.message).toBe("waiting for level-1999: ORIGINAL: missing provider");
});
it("clips a long root past its share, and both when both are long", () => {
const long = new ChordError({ code: "unavailable", message: "y".repeat(600) });
const short = waitingFor("app.a", long).error.message;
expect(short).toHaveLength(500);
expect(short.startsWith("waiting for app.a: yyy")).toBe(true);
const both = waitingFor("z".repeat(600), long).error.message;
expect(both).toHaveLength(500);
expect(both.endsWith("y…")).toBe(true);
});
it("is compared by value: kind, code, reason and message", () => {
expect(sameCause(waitingFor("a", root), waitingFor("a", root))).toBe(true);
expect(sameCause(waitingFor("a", root), waitingFor("b", root))).toBe(false);
expect(sameCause({ kind: "broken", error: root }, { kind: "transient", error: root })).toBe(false);
const other = new ChordError({ code: "unavailable", reason: "undeclared", message: root.message });
expect(sameCause({ kind: "waiting", error: root }, { kind: "waiting", error: other })).toBe(false);
});
});

5
packages/chord/test/next/examples/05-errors.ts
View file

@ -78,9 +78,10 @@ await attempt("bad params", () => host.get(Echo, { id: "one" }, context));
// An implementation with a member its contract does not declare fails activation.
await attempt("unknown member", () => host.get(Sloppy, { id: 1 }, context));
// A get needs the provider's version and fingerprint; another declaration of the type does not fit.
// A get needs the provider's version and fingerprint; another declaration of the type does not fit. That is not the
// caller's fault (a reload may bring the version): the get waits up to its readyTimeout, then rejects with the cause.
const EchoV2 = local("example.echo", { version: 2, params: { id: int() }, members: { id: int() } });
await attempt("contract mismatch", () => host.get(EchoV2, { id: 1 }, context));
await attempt("contract mismatch", () => host.get(EchoV2, { id: 1 }, context, { readyTimeout: 100 }));
// A factory that throws fails broken: the get rejects at once with its error, and no object is left behind.
await attempt("throwing factory", () => host.get(Broken, { id: 1 }, context));

3
packages/chord/test/next/examples/07-failures.ts
View file

@ -36,7 +36,8 @@ const facet = defineFacet((host) => {
return { status: constant("connected") };
});
host.serve(Inbox, ({ user }, self) => {
// A dependency that fails fails this object transient too; it is retried the moment the dependency is live.
// A dependency that fails parks this object (`failed { waiting }`, no timer of its own, the factory not run
// again); it goes on the moment the dependency is live (31-parked-dependencies.ts).
const remote = self.get(Remote, {});
console.log(`${at()}: inbox ${user}, attempt`);
return { status: self.derive([remote.status], ([status]) => `${user}: ${status}`) };

148
packages/chord/test/next/examples/31-parked-dependencies.ts Normal file
View file

@ -0,0 +1,148 @@
// Parked dependencies (dependency model, slice 7a): a reference is a request that outlives every provider it meets. An
// object whose dependency is not there yet is parked, not failed: it shows `failed { waiting }` naming what it waits
// for (the nearest blocker and the root cause), no timer runs for it, its factory is not run again, and it goes on the
// moment its dependency is live. Here a tab's panel needs `Notes` from a server that does not serve it yet; the server
// restarts with the notes plugin, the tab reconnects, and the panel goes live without anyone asking.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/31-parked-dependencies.ts
import { BACKGROUND_CONTEXT } from "../../../src/context/index.ts";
import { arr, hostKind, int, local, method, obj, remote, str } from "../../../src/next/contracts/index.ts";
import { isChordError } from "../../../src/next/data/index.ts";
import { defineState } from "../../../src/next/documents/index.ts";
import { createHost, defineFacet, type RefState } from "../../../src/next/host/index.ts";
import { memoryLinkPair } from "../../../src/next/link/index.ts";
import type { NativeWatchable } from "../../../src/watch.ts";
const context = BACKGROUND_CONTEXT;
const started = Date.now();
const at = (): string => `${((Date.now() - started) / 1000).toFixed(1)} s`;
type Principal = { readonly kind: "server" } | { readonly kind: "owner" };
const Server = hostKind("server");
/** The server's notes (the notes plugin), placed on the server. */
const Notes = remote("example.notes", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { editor: { state: { list: arr(str()) }, methods: { add: method(obj({ text: str() })) } } },
});
/** A panel in the tab: a local object over the server's notes. */
const Panel = local("example.panel", { version: 1, params: { id: int() }, members: { count: int() } });
/** A local object the panel's sibling needs, which no plugin of the tab serves. */
const Theme = local("example.theme", { version: 1, params: {}, members: { name: str() } });
const Header = local("example.header", { version: 1, params: { id: int() }, members: { title: str() } });
const List = defineState("example.list", {
schema: obj({ list: arr(str()) }),
version: 1,
initial: () => ({ list: [] }),
});
/** The notes plugin's server facet. */
const notesPlugin = defineFacet<Principal>((host) => {
host.serve(Notes, (_params, _principal, self) => {
const list = self.state(List);
return {
variant: "editor",
members: {
list: self.derive([list], ([l]) => l.list),
add: async ({ text }, ctx) => {
await list.change((draft) => void draft.list.push(text), ctx);
return null;
},
},
};
});
});
/** A server with the given plugins, its link to the tab accepted. */
async function startServer(plugins: { readonly name: string; readonly facet: typeof notesPlugin }[]) {
const server = createHost<Principal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
await server.load(plugins, context);
const [up, down] = memoryLinkPair();
server.accept(up, { principal: { kind: "owner" } }, context);
return { server, tabLink: down, close: () => up.close() };
}
// The tab: its panel declares the notes (a dependency); its header a theme nobody serves.
let panels = 0;
const tab = createHost<Principal>({ onError: (error) => console.log(`${at()}: reported ${error.kind}`) });
await tab.load(
[
{
name: "@example/panel",
facet: defineFacet<Principal>((host) => {
host.serve(Panel, (_, self) => {
panels++;
const notes = self.get(Notes, {});
return { count: self.derive([notes.list], ([list]) => list.length) };
});
host.serve(Header, (_, self) => ({ title: self.get(Theme, {}).name }));
}),
},
],
context,
);
/** Prints every state of `state` as it changes. */
async function follow(label: string, state: NativeWatchable<RefState>): Promise<void> {
const watch = await state.watch(context);
watch.start(async (call) => {
if (call.kind === "value") console.log(`${at()}: ${label} ${describe(call.value)}`);
});
}
function describe(state: RefState): string {
if (state.kind !== "failed") return state.kind;
return `failed { ${state.cause.kind} }: ${state.cause.error.message}`;
}
// 1. The first server has no notes plugin. Its answer is `failed { waiting }`: not the panel's fault.
const first = await startServer([]);
tab.connect(first.tabLink, { principal: { kind: "server" } }, context);
const panel = tab.hold(Panel, { id: 1 }, context);
await follow("panel:", panel.state);
await new Promise((resolve) => setTimeout(resolve, 10));
console.log(`${at()}: panel now ${describe(panel.state.value)}`);
// 2. A local type nobody serves here is the same answer: the request waits unbound, indexed under the type's name.
const header = tab.hold(Header, { id: 1 }, context);
await new Promise((resolve) => setTimeout(resolve, 10));
console.log(`${at()}: header ${describe(header.state.value)}`);
// 3. A getter waits through `waiting` up to its readyTimeout, then rejects with the cause (a hold just shows it).
try {
await tab.get(Panel, { id: 1 }, context, { readyTimeout: 200 });
} catch (error) {
if (isChordError(error)) console.log(`${at()}: get gave up after 200 ms: ${error.message}`);
}
// 4. Time passes: no timer runs for the parked panel, its factory is not run again.
await new Promise((resolve) => setTimeout(resolve, 1500));
console.log(`${at()}: still ${panel.state.value.kind}, factory runs: ${panels}`);
// 5. The server restarts with the notes plugin (the link drop shows as loading); the tab reconnects: its request is
// held again and binds, and the parked panel goes on.
first.close();
await new Promise((resolve) => setTimeout(resolve, 10));
const second = await startServer([{ name: "@example/notes", facet: notesPlugin }]);
tab.connect(second.tabLink, { principal: { kind: "server" } }, context);
const live = await tab.get(Panel, { id: 1 }, context);
console.log(`${at()}: panel live, count ${live.count.value}, factory runs: ${panels}`);
// The panel's notes are the owner's object on the server (a remote object per principal): the tab adds through its own.
const notes = await tab.get(Notes, {}, context);
await notes.add({ text: "first" }, context);
await new Promise((resolve) => setTimeout(resolve, 10));
console.log(`${at()}: panel count ${live.count.value}`);
header.release();
panel.release();
live.release();
notes.release();
await tab.close(context);
await second.server.close(context);
await first.server.close(context);

104
packages/chord/test/next/examples/32-startup.ts Normal file
View file

@ -0,0 +1,104 @@
// Startup (dependency model §6, slice 7a): `load` publishes every facet's declarations in one batch first, then starts
// the host-lifetime objects; requests made here and holds arriving on links wait only for that publication, never for
// the start barrier. Two hosts whose objects need each other's types (an acyclic graph) used to deadlock: the server's
// eager `Bootstrap` needs the worker's `Agent`, the `Agent` needs the server's `Catalog`, and the worker's hold for
// `Catalog` waited for the server's whole `load`, which waited for `Bootstrap`. Now it starts at once, no timeout.
// `load` resolves when each host-lifetime object had its first outcome: live, failed by its own cause, or parked with
// no blocker here still without one (a wait on another host ends the first pass).
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/next/examples/32-startup.ts
import { BACKGROUND_CONTEXT } from "../../../src/context/index.ts";
import { hostKind, int, local, remote, str } from "../../../src/next/contracts/index.ts";
import { createHost, defineFacet } from "../../../src/next/host/index.ts";
import { memoryLinkPair } from "../../../src/next/link/index.ts";
import { constant } from "../../../src/next/state/index.ts";
const context = BACKGROUND_CONTEXT;
const started = Date.now();
const at = (): string => `${Date.now() - started} ms`;
type Principal = { readonly kind: "server" } | { readonly kind: "worker"; readonly session: string };
const Server = hostKind("server");
const Session = hostKind("session", { id: str() });
/** On the server, a host-lifetime object (local, empty params): it starts at `load`. */
const Bootstrap = local("example.bootstrap", { version: 1, params: {}, members: { agents: int() } });
/** On the session's worker. */
const Agent = remote("example.agent", {
version: 1,
params: { session: str() },
hostAddress: ({ session }) => Session({ id: session }),
variants: { v: { state: { models: int() } } },
});
/** On the server. */
const Catalog = remote("example.catalog", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { v: { state: { models: int() } } },
});
/** The worker of a session, started by the server's route; it connects first, then loads (W6). */
function startWorker(id: string) {
const [up, down] = memoryLinkPair();
const worker = createHost<Principal>({
address: Session({ id }),
principal: { kind: "worker", session: id },
onError: (error) => console.log(`${at()}: worker reported ${error.kind}`),
});
worker.connect(down, { principal: { kind: "server" }, trusted: true }, context);
const agents = defineFacet<Principal>((host) =>
host.serve(Agent, (_params, _principal, self) => {
console.log(`${at()}: worker: Agent declares the server's Catalog`);
return { variant: "v", members: { models: self.get(Catalog, {}).models } };
}),
);
void worker
.load([{ name: "@example/agents", facet: agents }], context)
.then(() => console.log(`${at()}: worker loaded`));
return { worker, link: up };
}
const workers: ReturnType<typeof startWorker>[] = [];
const server = createHost<Principal>({
address: Server(),
principal: { kind: "server" },
// No readyTimeout is needed to break anything: a long one shows nothing waits for it.
readyTimeout: 600_000,
onError: (error) => console.log(`${at()}: server reported ${error.kind}`),
});
server.route(
Session,
{
authorize: () => true,
resolve: async (address) => {
const started = startWorker(address.params.id);
workers.push(started);
return { link: started.link, principal: { kind: "worker", session: address.params.id }, trusted: true };
},
},
context,
);
const core = defineFacet<Principal>((host) => {
host.serve(Catalog, () => ({ variant: "v", members: { models: constant(3) } }));
host.serve(Bootstrap, (_, self) => {
console.log(`${at()}: server: Bootstrap declares the worker's Agent`);
return { agents: self.get(Agent, { session: "S1" }).models };
});
});
// 1. The server's load: publication (Catalog served), then Bootstrap starts and parks on the worker's Agent. A wait on
// another host is a first outcome, so `load` resolves without waiting for the worker.
await server.load([{ name: "@example/core", facet: core }], context);
console.log(`${at()}: server loaded; Bootstrap is ${server.hold(Bootstrap, {}, context).state.value.kind}`);
// 2. The worker's hold for Catalog bound on the published server; Agent went live, then Bootstrap.
const bootstrap = await server.get(Bootstrap, {}, context);
console.log(`${at()}: Bootstrap live, it sees ${bootstrap.agents.value} models through the worker`);
bootstrap.release();
await server.close(context);
for (const { worker } of workers) await worker.close(context);

13
packages/chord/test/next/host/activity.test.ts
View file

@ -27,6 +27,19 @@ describe("host.activity", () => {
expect(server.activity.value.references).toBe(0);
});
it("counts a hold placed here from its arrival, bound or not: a missing provider keeps the worker alive (M§10)", async () => {
const { server, client } = await linked(() => {});
const missing = client.hold(Notes, { session: "S1" }, context);
await transported();
expect(missing.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
expect(server.activity.value.references).toBe(1);
// Presence lists bound references only.
expect(server.references(Notes).value).toEqual([]);
missing.release();
await transported();
expect(server.activity.value.references).toBe(0);
});
it("counts method calls running on this host, local and remote, until they settle", async () => {
const { server, client } = await linked((h) => void serveNotes(h));
const remoteRef = await client.get(Notes, { session: "S1" }, context);

97
packages/chord/test/next/host/attempt.test.ts
View file

@ -2,7 +2,7 @@ import { createHook } from "node:async_hooks";
import { afterEach, beforeEach, describe, expect, expectTypeOf, it, vi } from "vitest";
import { type Context, BACKGROUND_CONTEXT as context, withCancel } from "../../../src/context/index.ts";
import { arr, int, local, method, obj, opt, remote, str } from "../../../src/next/contracts/index.ts";
import { broken, isChordError } from "../../../src/next/data/index.ts";
import { broken, ChordError, isChordError } from "../../../src/next/data/index.ts";
import { defineState, type SingletonHandle } from "../../../src/next/documents/index.ts";
import {
createHost,
@ -12,6 +12,7 @@ import {
type Self,
type StoreSelf,
} from "../../../src/next/host/index.ts";
import { ChordObject } from "../../../src/next/host/object.ts";
import { defineRegistry } from "../../../src/next/registries/index.ts";
import { constant } from "../../../src/next/state/index.ts";
import type { StagedWrite } from "../../../src/next/store/step.ts";
@ -22,9 +23,11 @@ import { evolveTable } from "../../../src/next/store/table-evolve.ts";
import { TableHandle } from "../../../src/next/store/table-handle.ts";
import { prepareTables } from "../../../src/next/store/table-migration.ts";
import {
failLive,
failures,
fakeTimers,
flushed,
instancesOf,
loaded,
memoryStore,
metered,
@ -33,6 +36,8 @@ import {
tableOwner,
} from "../support.ts";
const objects = instancesOf(ChordObject, "attempt");
const List = defineState("list", { schema: obj({ items: arr(str()) }), version: 1, initial: () => ({ items: [] }) });
const Page = local("app.page", { version: 1, params: { id: int() }, members: { label: str(), slow: method() } });
@ -892,3 +897,93 @@ describe("eager migrations at activation (slice 6a §9)", () => {
await store.close(context);
});
});
describe("parked attempts (dependency model §3)", () => {
fakeTimers();
const View = local("app.view", { version: 1, params: {}, members: { n: int() } });
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
/** A view with params: no host lifetime, so `load` does not wait for it. */
const Viewer = local("app.viewer", { version: 1, params: { id: int() }, members: { n: int() } });
it("run no timer while parked: no dependency deadline, no backoff, the factory run once", async () => {
const factories = vi.fn();
const { host, errors } = await loaded((h) =>
h.serve(View, (_, self) => {
factories();
return { n: self.get(Missing, {}).n };
}),
);
const view = host.hold(View, {}, context);
await flushed();
await vi.advanceTimersByTimeAsync(10 * 60_000);
expect(view.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.missing: app.missing is not served here" } },
retryAt: undefined,
});
expect(vi.getTimerCount()).toBe(0);
expect([factories.mock.calls.length, errors]).toEqual([1, []]);
view.release();
});
it("pause the readyTimeout deadline while parked: the own steps get the budget left", async () => {
const Slow = local("app.slow", { version: 1, params: { id: int() }, members: { n: int() } });
const { host, errors } = await loaded(
(h) => {
h.serve(Slow, (_, self) => {
self.open(
() => new Promise((resolve) => setTimeout(resolve, 5_000)),
() => {},
);
return { n: constant(1) };
});
h.serve(Viewer, (_, self) => {
const slow = self.get(Slow, { id: 1 });
self.open(
() => new Promise(() => {}),
() => {},
);
return { n: slow.n };
});
},
{ readyTimeout: 10_000 },
);
const view = host.hold(Viewer, { id: 1 }, context);
// Parked 5 s on its dependency; then its own open hangs: it fails 10 s after the boundary, not 10 s after start.
await vi.advanceTimersByTimeAsync(5_000 + 9_999);
expect(failures(errors)).toEqual([]);
await vi.advanceTimersByTimeAsync(1);
expect(failures(errors)).toEqual(["app.viewer(1): transient"]);
view.release();
});
it("go on past the dependency check: a dependency leaving live during the own steps changes nothing (R18)", async () => {
const Dependency = local("app.dependency", { version: 1, params: {}, members: { n: int() } });
const gate = Promise.withResolvers<void>();
let dependencySelf: Self | undefined;
const { host, errors } = await loaded((h) => {
h.serve(Dependency, (_, self) => {
dependencySelf = self;
return { n: constant(1) };
});
h.serve(Viewer, (_, self) => {
const dependency = self.get(Dependency, {});
self.open(
() => gate.promise,
() => {},
);
return { n: dependency.n };
});
});
const view = host.hold(Viewer, { id: 1 }, context);
await flushed();
// The view opens its own native; its dependency fails live meanwhile.
failLive(objects, dependencySelf!, new ChordError({ code: "unavailable", message: "offline" }));
await flushed();
gate.resolve();
await flushed();
expect(view.state.value).toEqual({ kind: "live" });
expect(failures(errors)).toEqual([]);
view.release();
});
});

294
packages/chord/test/next/host/dependencies.test.ts Normal file
View file

@ -0,0 +1,294 @@
import { describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { int, local, remote } from "../../../src/next/contracts/index.ts";
import { createHost, defineFacet, type HostError } from "../../../src/next/host/index.ts";
import { constant } from "../../../src/next/state/index.ts";
import {
bindHold,
failures,
fakeTimers,
flushed,
loaded,
Notes,
rawParent,
rejection,
relayed,
Server,
Session,
statesOf,
type TestPrincipal,
transported,
} from "../support.ts";
const View = local("app.view", { version: 1, params: { id: int() }, members: { n: int() } });
describe("the dependency check", () => {
it("breaks a dependent off a cycle with the cycle's error: it would wait on a ring that never heals", async () => {
const Ping = local("app.ping", { version: 1, params: {}, members: { n: int() } });
const Pong = local("app.pong", { version: 1, params: {}, members: { n: int() } });
const { host, errors } = await loaded((h) => {
h.serve(Ping, (_, self) => ({ n: self.get(Pong, {}).n }));
h.serve(Pong, (_, self) => ({ n: self.get(Ping, {}).n }));
h.serve(View, (_, self) => ({ n: self.get(Ping, {}).n }));
});
const view = host.hold(View, { id: 1 }, context);
await flushed();
expect(view.state.value).toMatchObject({
kind: "failed",
cause: { kind: "broken", error: { code: "invalid", reason: "cycle" } },
});
expect(failures(errors)).toContain("app.view(1): broken");
view.release();
});
it("breaks a dependent whose dependency ended (gone): the request cannot come back", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ onError: (error) => errors.push(error) });
const peer = await rawParent(client);
await client.load(
[
{
name: "@test/view",
facet: defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
),
},
],
context,
);
const view = client.hold(View, { id: 1 }, context);
const { hold } = await peer.next("hold");
const revoked = { code: "forbidden", reason: "revoked", message: "revoked" } as const;
peer.send({ kind: "gone", hold, error: { kind: "chord", error: revoked } });
await transported();
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken", error: revoked } });
expect(failures(errors)).toEqual(["app.view(1): broken"]);
view.release();
});
it("goes live once every dependency is: one parked on two, the second binding last", async () => {
const client = createHost<TestPrincipal>({ onError: () => {} });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => {
const a = self.get(Notes, { session: "S1" }).count;
const b = self.get(Notes, { session: "S2" }).count;
return { n: self.derive([a, b], ([x, y]) => x + y) };
}),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
const first = await peer.next("hold");
const second = await peer.next("hold");
bindHold(peer, first.hold, [{ text: "a" }]);
await transported();
expect(view.state.value).toEqual({ kind: "loading" });
bindHold(peer, second.hold, [{ text: "b" }, { text: "c" }]);
await transported();
expect([view.state.value, view.n.value]).toEqual([{ kind: "live" }, 3]);
view.release();
});
it("breaks a dependent whose dependency a relay's authorize refused: forbidden heals only by retry() (fix round 1, Fable)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
const { hold } = await peer.next("hold");
const forbidden = { code: "forbidden", message: "session|S1: not authorized" } as const;
peer.send({ kind: "failed", hold, cause: { kind: "broken", error: { kind: "chord", error: forbidden } } });
await transported();
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken", error: forbidden } });
expect(failures(errors)).toEqual(["app.view(1): broken"]);
view.release();
});
it("settles a waiting ring across hosts after one round: the root cause kept (fix round 1, Opus F1)", async () => {
const A = remote("app.a", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { v: { state: { n: int() } } },
});
const B = remote("app.b", {
version: 1,
params: {},
hostAddress: () => Session({ id: "S1" }),
variants: { v: { state: { n: int() } } },
});
const X = local("app.x", { version: 1, params: {}, members: { n: int() } });
const Kick = local("app.kick", { version: 1, params: {}, members: { n: int() } });
const Front = local("app.front", { version: 1, params: {}, members: { n: int() } });
const Back = local("app.back", { version: 1, params: {}, members: { n: int() } });
const { workers } = await relayed({
trusted: true,
server: (h) => {
// The server's host object starts the worker; its A and the worker's B then start in the same turns, so
// the activation chains miss the ring (M§7's gap).
h.serve(Kick, (_p, self) => ({ n: self.get(B, {}).n }));
h.serve(A, (_p, _principal, self) => ({ variant: "v", members: { n: self.get(B, {}).n } }));
},
worker: (h) => {
h.serve(Front, (_p, self) => ({ n: self.get(A, {}).n }));
h.serve(Back, (_p, self) => ({ n: self.get(B, {}).n }));
h.serve(B, (_p, _principal, self) => {
const a = self.get(A, {});
self.get(X, {});
return { variant: "v", members: { n: a.n } };
});
},
});
await transported();
const worker = workers.get("session|S1")!;
const b = worker.hold(B, {}, context);
for (let round = 0; round < 50; round++) await transported();
const states = statesOf(await b.state.watch(context));
for (let round = 0; round < 50; round++) await transported();
expect(states).toEqual([]);
expect(b.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: expect.stringMatching(/: app\.x is not served here$/) } },
});
b.release();
});
});
describe("parked_long (dependency model §7)", () => {
fakeTimers();
it("starts its clock when the hold gets on a link: a late link is not reported at once (fix round 2, Opus F2)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
const peer = await rawParent(client);
await peer.next("hold");
const linked = Date.now();
await vi.advanceTimersByTimeAsync(10);
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(990);
expect(errors).toEqual([
{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", since: linked },
]);
view.release();
});
it("is not reported while the path has no link: no remote attempt answers for it (fix round 1, Opus F2)", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
expect(errors).toEqual([]);
expect(view.state.value).toEqual({ kind: "loading" });
// Once a link holds it and the remote attempt is only loading, it is.
const peer = await rawParent(client);
await peer.next("hold");
await vi.advanceTimersByTimeAsync(5_000);
expect(errors.map(({ kind }) => kind)).toEqual(["parked_long"]);
view.release();
});
it("is reported once per parking episode, after readyTimeout on a remote attempt only loading; nothing fails", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
await peer.next("hold");
await vi.advanceTimersByTimeAsync(999);
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(1);
expect(errors).toEqual([{ kind: "parked_long", object: "app.view(1)", waitingFor: "app.notes(S1)", since: 0 }]);
await vi.advanceTimersByTimeAsync(60_000);
expect(errors).toHaveLength(1);
expect(view.state.value).toEqual({ kind: "loading" });
view.release();
});
it("is not reported for a remote blocker that failed: the parked attempt shows its cause instead", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await client.load([{ name: "@test/view", facet }], context);
const view = client.hold(View, { id: 1 }, context);
const { hold } = await peer.next("hold");
const error = { kind: "chord", error: { code: "unavailable", message: "app.notes is not served here" } } as const;
peer.send({ kind: "failed", hold, cause: { kind: "waiting", error } });
await vi.advanceTimersByTimeAsync(60_000);
expect(errors).toEqual([]);
expect(view.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
expect(vi.getTimerCount()).toBe(0);
view.release();
});
it("is not reported for a local blocker: its own deadline bounds it", async () => {
const Slow = local("app.slow", { version: 1, params: { id: int() }, members: { n: int() } });
const { host, errors } = await loaded(
(h) => {
h.serve(Slow, (_, self) => {
self.open(
() => new Promise(() => {}),
() => {},
);
return { n: constant(1) };
});
h.serve(View, (_, self) => ({ n: self.get(Slow, { id: 1 }).n }));
},
{ readyTimeout: 1_000 },
);
const view = host.hold(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(5_000);
expect(errors.map(({ kind }) => kind)).not.toContain("parked_long");
view.release();
});
});
describe("a getter of a parked object (fix round 1, Astra 6)", () => {
it("rejects with the last parked cause after its dependency fell back to loading, the newest of several", async () => {
const host = createHost<TestPrincipal>({ onError: () => {} });
const peer = await rawParent(host);
await host.load(
[
{
name: "@test/view",
facet: defineFacet<TestPrincipal>((h) =>
h.serve(View, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
),
},
],
context,
);
const pending = rejection(host.get(View, { id: 1 }, context, { readyTimeout: 80 }));
const { hold } = await peer.next("hold");
for (const message of ["first absence", "specific provider absence"]) {
const error = { kind: "chord", error: { code: "unavailable", message } } as const;
peer.send({ kind: "failed", hold, cause: { kind: "waiting", error } });
await transported();
}
peer.send({ kind: "loading", hold });
await transported();
expect(await pending).toMatchObject({
code: "unavailable",
message: "waiting for app.notes(S1): specific provider absence",
});
await host.close(context);
});
});

413
packages/chord/test/next/host/host.test.ts
View file

@ -1,6 +1,6 @@
import { describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context, withCancel } from "../../../src/context/index.ts";
import { int, local, method, remote, str } from "../../../src/next/contracts/index.ts";
import { hostKind, int, local, method, remote, str } from "../../../src/next/contracts/index.ts";
import { broken } from "../../../src/next/data/index.ts";
import {
type Ctx,
@ -13,6 +13,7 @@ import {
import type { ChordHost } from "../../../src/next/host/internal.ts";
import { memoryLinkPair } from "../../../src/next/link/index.ts";
import { jsonPart } from "../../../src/next/link/internal.ts";
import { defineRegistry } from "../../../src/next/registries/index.ts";
import { constant } from "../../../src/next/state/index.ts";
import { sqliteBackend } from "../../../src/next/store/backend.ts";
import type { StagedWrite } from "../../../src/next/store/step.ts";
@ -31,6 +32,7 @@ import {
partValue,
RawPeer,
rejection,
relayed,
SERVER_ADDRESS,
Server,
Session,
@ -153,19 +155,296 @@ describe("the start barrier", () => {
await vi.advanceTimersByTimeAsync(1);
expect(done).toBe(true);
});
it("waits for a local blocker's first outcome: a host object parked on another one here (CodeMode → Harness)", async () => {
const Harness = local("app.harness", { version: 1, params: {}, members: { n: int() } });
const CodeMode = local("app.codemode", { version: 1, params: {}, members: { n: int() } });
const opened = Promise.withResolvers<void>();
const host = createHost(noErrors);
const facet = defineFacet((h) => {
h.serve(Harness, (_, self) => {
self.open(
() => opened.promise,
() => {},
);
return { n: constant(1) };
});
h.serve(CodeMode, (_, self) => ({ n: self.get(Harness, {}).n }));
});
let done = false;
const loading = host.load([{ name: "@app/core", facet }], context).then(() => {
done = true;
});
await flushed();
expect(done).toBe(false);
opened.resolve();
await loading;
expect(host.hold(CodeMode, {}, context).state.value).toEqual({ kind: "live" });
});
it("ends its pass at a local blocker's own failure: the dependent parked failed { waiting } has its first outcome", async () => {
const Harness = local("app.harness", { version: 1, params: {}, members: { n: int() } });
const CodeMode = local("app.codemode", { version: 1, params: {}, members: { n: int() } });
const { host, errors } = await loaded((h) => {
h.serve(Harness, (_, self) => {
self.open(
async () => {
throw new Error("offline");
},
() => {},
);
return { n: constant(1) };
});
h.serve(CodeMode, (_, self) => ({ n: self.get(Harness, {}).n }));
});
expect(failures(errors)).toEqual(["app.harness: transient"]);
expect(host.hold(CodeMode, {}, context).state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.harness: app.harness: offline" } },
});
});
it("ends its pass at a remote blocker: a wait on another host is a first outcome", async () => {
const CodeMode = local("app.codemode", { version: 1, params: {}, members: { n: int() } });
const host = createHost<TestPrincipal>(noErrors);
const [a, b] = memoryLinkPair();
host.connect(a, { principal: { kind: "server" }, codec: testCodec() }, context);
const parent = new RawPeer(b);
const facet = defineFacet<TestPrincipal>((h) =>
h.serve(CodeMode, (_, self) => ({ n: self.get(Notes, { session: "S1" }).count })),
);
await host.load([{ name: "@app/codemode", facet }], context);
expect(await parent.next("hold")).toMatchObject({ type: Notes.name });
expect(host.hold(CodeMode, {}, context).state.value).toEqual({ kind: "loading" });
});
it("is settled by host.close: an object of its cohort that ends before its first outcome settles it as ended", async () => {
const host = createHost(noErrors);
const facet = defineFacet((h) =>
h.serve(Info, (_, self) => {
self.open(
() => new Promise(() => {}),
() => {},
);
return { title: constant("t") };
}),
);
let done = false;
const loading = host.load([{ name: "a", facet }], context).then(() => {
done = true;
});
await flushed();
expect(done).toBe(false);
await host.close(context);
await loading;
expect(done).toBe(true);
});
it("waits for a local blocker parked failed { waiting } on one still in its first attempt (fix round 1, Sol 4)", async () => {
const Core = local("app.core", { version: 1, params: {}, members: { n: int() } });
const Middle = local("app.middle", { version: 1, params: { id: int() }, members: { n: int() } });
const Slow = local("app.slow", { version: 1, params: { id: int() }, members: { n: int() } });
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
const gate = Promise.withResolvers<void>();
const host = createHost(noErrors);
const facet = defineFacet((h) => {
h.serve(Slow, (_, self) => {
self.open(
() => gate.promise,
() => {},
);
return { n: constant(1) };
});
h.serve(Middle, ({ id }, self) => {
const slow = self.get(Slow, { id });
self.get(Missing, {});
return { n: slow.n };
});
h.serve(Core, (_, self) => ({ n: self.get(Middle, { id: 1 }).n }));
});
let done = false;
const loading = host.load([{ name: "@app/core", facet }], context).then(() => {
done = true;
});
await flushed();
expect(host.hold(Middle, { id: 1 }, context).state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting" },
});
expect(done).toBe(false);
gate.resolve();
await loading;
await host.close(context);
});
it("waits for it whichever it declares first (fix round 1, Astra 3)", async () => {
const A = local("app.a", { version: 1, params: {}, members: {} });
const B = local("app.b", { version: 1, params: { id: int() }, members: {} });
const Slow = local("app.slow", { version: 1, params: { id: int() }, members: {} });
const Missing = local("app.missing", { version: 1, params: {}, members: {} });
const gate = Promise.withResolvers<void>();
const host = createHost(noErrors);
const facet = defineFacet((h) => {
h.serve(A, (_, self) => {
self.get(B, { id: 1 });
return {};
});
h.serve(B, (_, self) => {
self.get(Missing, {});
self.get(Slow, { id: 1 });
return {};
});
h.serve(Slow, (_, self) => {
self.open(
() => gate.promise,
() => {},
);
return {};
});
});
let done = false;
const loading = host.load([{ name: "@app/a", facet }], context).then(() => {
done = true;
});
await flushed();
const premature = done;
gate.resolve();
await loading;
await host.close(context);
expect(premature).toBe(false);
});
});
describe("startup across hosts (dependency model §6)", () => {
// Timers never advance: nothing here may wait for a deadline.
fakeTimers();
/** On the server, waiting for a worker's `Agent`. */
const Bootstrap = local("app.bootstrap", { version: 1, params: {}, members: { n: int() } });
/** On worker S1, waiting for the server's `Catalog`. */
const Agent = remote("app.agent", {
version: 1,
params: { session: str() },
hostAddress: ({ session }) => Session({ id: session }),
variants: { v: { state: { n: int() } } },
});
const Catalog = remote("app.catalog", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { v: { state: { n: int() } } },
});
it("starts two hosts that would have deadlocked (Sol's probe): an acyclic graph needs no timeout", async () => {
const { server, errors } = await relayed({
server: (h) => {
h.serve(Bootstrap, (_, self) => ({ n: self.get(Agent, { session: "S1" }).n }));
h.serve(Catalog, () => ({ variant: "v", members: { n: constant(7) } }));
},
worker: (h) => {
// The worker's own host object waits for the server's catalog during the worker's load.
h.serve(Agent, (_params, _principal, self) => ({ variant: "v", members: { n: self.get(Catalog, {}).n } }));
},
});
await transported();
await transported();
const bootstrap = server.hold(Bootstrap, {}, context);
expect(bootstrap.state.value).toEqual({ kind: "live" });
expect(bootstrap.n.value).toBe(7);
expect([errors.server, errors.worker]).toEqual([[], []]);
});
it("places a hold arriving before the publication here, not over the parent", async () => {
const worker = createHost<TestPrincipal>({
address: Session({ id: "S1" }),
principal: { kind: "worker", session: "S1" },
...noErrors,
});
const [a, b] = memoryLinkPair();
worker.connect(a, { principal: { kind: "server" }, codec: testCodec() }, context);
const parent = new RawPeer(b);
const [c, d] = memoryLinkPair();
worker.accept(c, { principal: { kind: "owner" }, codec: testCodec() }, context);
const child = new RawPeer(d);
child.send({
kind: "hold",
hold: 0,
type: PlacedNotes.name,
version: PlacedNotes.version,
fingerprint: PlacedNotes.fingerprint,
params: jsonPart({ session: "S1" }),
address: { kind: "session", params: { id: "S1" } },
});
const mine = worker.hold(PlacedNotes, { session: "S1" }, context);
await transported();
expect(mine.state.value).toEqual({ kind: "loading" });
await worker.load(
[{ name: "@test/w", facet: defineFacet<TestPrincipal>((h) => serveNotes(h, PlacedNotes)) }],
context,
);
expect((await child.next("bound")).holds[0]).toMatchObject({ hold: 0 });
await transported();
expect(mine.state.value).toEqual({ kind: "live" });
expect(parent.kinds()).toEqual(["hello"]);
});
it("wakes a parked chain over three hosts in order, its causes passed on (tab → worker → server)", async () => {
const Chat = remote("app.chat", {
version: 1,
params: { session: str() },
hostAddress: ({ session }) => Session({ id: session }),
variants: { v: { state: { n: int() } } },
});
const order: string[] = [];
let catalogs = 0;
const { tab } = await relayed({
server: (h) =>
h.serve(Catalog, (_params, _principal, self) => {
const attempt = ++catalogs;
self.open(
async () => {
if (attempt === 1) throw new Error("offline");
},
() => {},
);
self.onLive(async () => void order.push("catalog"));
return { variant: "v", members: { n: constant(attempt) } };
}),
worker: (h) =>
h.serve(Chat, (_params, _principal, self) => {
const catalog = self.get(Catalog, {});
self.onLive(async () => void order.push("chat"));
return { variant: "v", members: { n: catalog.n } };
}),
});
const view = tab.hold(Chat, { session: "S1" }, context);
for (let round = 0; round < 5; round++) await transported();
expect(view.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.catalog: app.catalog: offline" } },
retryAt: undefined,
});
// The catalog's own backoff (1 s) is the only timer: it retries, and the chain goes on in order.
await vi.advanceTimersByTimeAsync(1_000);
for (let round = 0; round < 5; round++) await transported();
expect(view.state.value).toEqual({ kind: "live" });
expect(order).toEqual(["catalog", "chat"]);
expect(view.n.value).toBe(2);
});
});
describe("get", () => {
it("refuses an unserved type and a contract mismatch", async () => {
it("waits for an unserved type and a contract mismatch until its readyTimeout, then rejects with the cause", async () => {
const { host } = await loaded((h) => h.serve(Info, () => ({ title: constant("t") })));
expect(await rejection(host.get(Page, { id: 1 }, context))).toEqual({
const quick = { readyTimeout: 5 };
expect(await rejection(host.get(Page, { id: 1 }, context, quick))).toEqual({
code: "unavailable",
message: "app.page is not served here",
});
const InfoV2 = local("app.info", { version: 2, params: {}, members: { title: str() } });
const Changed = local("app.info", { version: 1, params: {}, members: { title: int() } });
expect(await rejection(host.get(InfoV2, {}, context))).toMatchObject({ code: "contract_mismatch" });
expect(await rejection(host.get(Changed, {}, context))).toMatchObject({ code: "contract_mismatch" });
expect(await rejection(host.get(InfoV2, {}, context, quick))).toMatchObject({ code: "contract_mismatch" });
expect(await rejection(host.get(Changed, {}, context, quick))).toMatchObject({ code: "contract_mismatch" });
const Same = local("app.info", { version: 1, params: {}, members: { title: str() } });
expect((await host.get(Same, {}, context)).title.value).toBe("t");
});
@ -612,7 +891,11 @@ forEachCodec(() => {
it("a local type never crosses a link; a remote type without a parent link waits for one", async () => {
const { client } = await linked(() => {});
expect(await rejection(client.get(Inner, { id: 1 }, context))).toEqual({
expect(client.hold(Inner, { id: 1 }, context).state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting" },
});
expect(await rejection(client.get(Inner, { id: 1 }, context, { readyTimeout: 5 }))).toEqual({
code: "unavailable",
message: "app.inner is not served here",
});
@ -645,6 +928,81 @@ forEachCodec(() => {
});
});
describe("host.registry and the host's close (fix round 1)", () => {
const Absent = defineRegistry<number, number>("app.absent", { version: 1 });
it("ends a waiting host.registry at the close with released/closed, its deadline timer with it (Sol 5, Astra 5)", async () => {
const host = createHost(noErrors);
await host.load([], context);
let outcome: unknown;
void rejection(host.registry(Absent, context, { readyTimeout: 60_000 })).then((error) => {
outcome = error;
});
await flushed();
await host.close(context);
await flushed();
expect(outcome).toMatchObject({ code: "released", reason: "closed" });
});
it("ends one still waiting for the publication at the close", async () => {
const host = createHost(noErrors);
const waiting = rejection(host.registry(Absent, context, { readyTimeout: 60_000 }));
await host.close(context);
expect(await waiting).toMatchObject({ code: "released", reason: "closed" });
});
});
describe("the selected provider's own checks for a request made here (fix round 1)", () => {
const Here = hostKind("app-here");
const Elsewhere = hostKind("app-elsewhere");
const shape = { version: 1, params: {}, variants: { view: { state: { n: int() } } } } as const;
it("waits a same-host remote request the provider's code places elsewhere: invalid/host_address (Astra 4)", async () => {
const Consumer = remote("app.placed", { ...shape, hostAddress: () => Here() });
const Provider = remote("app.placed", { ...shape, hostAddress: () => Elsewhere() });
const host = createHost({ address: Here(), principal: { kind: "owner" }, onError: () => {} });
const facet = defineFacet((h) => h.serve(Provider, () => ({ variant: "view", members: { n: constant(1) } })));
await host.load([{ name: "@app/provider", facet }], context);
const ref = host.hold(Consumer, {}, context);
await flushed();
expect(ref.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { code: "invalid", reason: "host_address" } },
});
ref.release();
await host.close(context);
});
it("fails broken when the provider's hostAddress throws: its own code (Sol 7)", async () => {
const Consumer = remote("app.remote", { ...shape, hostAddress: () => Server() });
const Provider = remote("app.remote", {
...shape,
hostAddress: () => {
throw new Error("provider address bug");
},
});
const server = createHost({ address: Server(), principal: { kind: "server" }, onError: () => {} });
const client = createHost({ onError: () => {} });
const facet = defineFacet((h) => h.serve(Provider, () => ({ variant: "view", members: { n: constant(1) } })));
await server.load([{ name: "@app/provider", facet }], context);
await client.load([], context);
const [down, up] = memoryLinkPair();
server.accept(down, { principal: { kind: "server" } }, context);
client.connect(up, { principal: { kind: "server" } }, context);
const held = client.hold(Consumer, {}, context);
await transported();
expect(held.state.value).toMatchObject({
kind: "failed",
cause: { kind: "broken", error: { message: "app.remote: provider address bug" } },
});
// The same throw for a request made on the provider's host fails it broken too.
const here = server.hold(Consumer, {}, context);
expect(here.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } });
await client.close(context);
await server.close(context);
});
});
describe("a load whose setup throws", () => {
it("closes the host as close does, links included, and rejects with the setup's error", async () => {
const host = createHost<TestPrincipal>({ principal: { kind: "server" }, onError: () => {} });
@ -687,23 +1045,29 @@ describe("acquiring: hold and get", () => {
const offline = { code: "unavailable", message: "app.svc(1): offline" };
it("invalid params and a local type not served here: at once", async () => {
it("invalid params at once; a local type not served here: hold waits failed, get rejects at its readyTimeout", async () => {
const host = createHost<TestPrincipal>({ address: Server(), ...noErrors, principal: { kind: "server" } });
await host.load([{ name: "@test/app", facet: defineFacet<TestPrincipal>((h) => void serveNotes(h)) }], context);
const params = { code: "invalid", reason: "params" };
expect(() => host.hold(Notes, { session: 1 } as never, context)).toThrow("params");
expect(await rejection(host.get(Notes, { session: 1 } as never, context))).toMatchObject(params);
const unserved = { code: "unavailable", message: "app.info is not served here" };
expect(() => host.hold(Info, {}, context)).toThrow(unserved.message);
expect(await rejection(host.get(Info, {}, context))).toEqual(unserved);
expect(host.hold(Info, {}, context).state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: unserved },
retryAt: undefined,
});
const got = rejection(host.get(Info, {}, context, { readyTimeout: 5 }));
await vi.advanceTimersByTimeAsync(5);
expect(await got).toEqual(unserved);
});
it("before load resolved: hold is loading, get waits; once it resolved both bind, or an unserved type fails", async () => {
it("before load resolved: hold is loading, get waits; once it resolved both bind, or an unserved type waits", async () => {
const host = createHost(noErrors);
const held = host.hold(Info, {}, context);
const unserved = host.hold(Page, { id: 1 }, context);
const got = host.get(Info, {}, context);
const refused = rejection(host.get(Page, { id: 1 }, context));
const refused = rejection(host.get(Page, { id: 1 }, context, { readyTimeout: 50 }));
expect([held.state.value, unserved.state.value]).toEqual([{ kind: "loading" }, { kind: "loading" }]);
await host.load(
[{ name: "@test/app", facet: defineFacet((h) => h.serve(Info, () => ({ title: constant("t") }))) }],
@ -715,9 +1079,10 @@ describe("acquiring: hold and get", () => {
const error = { code: "unavailable", message: "app.page is not served here" };
expect(unserved.state.value).toMatchObject({
kind: "failed",
cause: { kind: "broken", error },
cause: { kind: "waiting", error },
retryAt: undefined,
});
await vi.advanceTimersByTimeAsync(50);
expect(await refused).toEqual(error);
});
@ -866,25 +1231,25 @@ forEachCodec(() => {
expect((await child.next("bound")).holds[0]).toMatchObject({ hold: 0 });
});
it("refuses a hold served here whose params place it elsewhere, whatever address it names: invalid/host_address (race 5)", async () => {
it("keeps a hold naming this host whose params the code here places elsewhere waiting: invalid/host_address (code skew, M§1)", async () => {
const { child } = await worker();
holdOf(child, "S2", "S1");
expect(await child.next("failed")).toMatchObject({
hold: 0,
cause: { kind: "broken", error: { kind: "chord", error: misplaced } },
});
// A refused hold is kept (a retry starts it again): the next hold takes the next id.
holdOf(child, "S2", "S2", PlacedNotes.name, 1);
expect(await child.next("failed")).toMatchObject({
hold: 1,
cause: { kind: "broken", error: { kind: "chord", error: misplaced } },
cause: { kind: "waiting", error: { kind: "chord", error: misplaced } },
});
// A refused hold is kept (a publication or a retry starts it again): the next hold takes the next id.
holdOf(child, "S2", "S1", PlacedNotes.name, 1);
expect(await child.next("failed")).toMatchObject({ hold: 1, cause: { kind: "waiting" } });
});
it("places a hold served here by its params, not by the address it names (that only routes it)", async () => {
const { child } = await worker();
holdOf(child, "S1", "S2");
expect((await child.next("bound")).holds[0]).toMatchObject({ hold: 0 });
it("places a hold by the address it names, never by the name it serves (a relay never captures it)", async () => {
const { child, parent } = await worker({ parent: true });
holdOf(child, "S2", "S2");
const relayed = await parent!.next("hold");
expect(relayed).toMatchObject({ type: PlacedNotes.name, address: { kind: "session", params: { id: "S2" } } });
holdOf(child, "S1", "S2", PlacedNotes.name, 1);
expect(await parent!.next("hold")).toMatchObject({ hold: 1 });
});
it("sends a reference made here whose host address is another host over the parent link, though it serves the type", async () => {

68
packages/chord/test/next/host/keyed-members.test.ts
View file

@ -1,18 +1,19 @@
// Tests of keyed members across attempts (slice 6a §1.2, §1.4, §12): the keys references hold on an object's keyed
// members, frozen while it is failed, re-bound by name (loaded first) when its next attempt goes live, released once.
import { describe, expect, it } from "vitest";
import { describe, expect, it, vi } from "vitest";
import { BACKGROUND_CONTEXT as context, withAbortSignal } from "../../../src/context/index.ts";
import { int, keyed, local, obj, str } from "../../../src/next/contracts/index.ts";
import { ChordError } from "../../../src/next/data/index.ts";
import { defineState, type KeyedHandle } from "../../../src/next/documents/index.ts";
import type { Self } from "../../../src/next/host/index.ts";
import { createHost, defineFacet, type Self } from "../../../src/next/host/index.ts";
import { ChordObject } from "../../../src/next/host/object.ts";
import {
changesOf,
delivered,
expectCollected,
failLive,
flushed,
instancesOf,
loaded,
memoryStore,
@ -291,7 +292,8 @@ describe("a key whose first pin still loads", () => {
await delivered();
expect([viewer.value.value, viewer.double.value]).toEqual([{ n: 2 }, 4]);
expect([pinsOf(providers.a!), pinsOf(providers.b!)]).toEqual([undefined, 1]);
expect(errors.filter((error) => error.kind === "activation_failed").length).toBe(1);
// The pin's failed load parked the viewer (a dependency failure is not its own): nothing failed it.
expect(errors.filter((error) => error.kind === "activation_failed").length).toBe(0);
const values = changesOf(await viewer.value.watch(context));
await providers.a!.change(
"k",
@ -312,10 +314,11 @@ describe("a key whose first pin still loads", () => {
await close();
});
it("the object fails and stays failed: the consumer's activation fails transient, never live on it", async () => {
const { host, errors, exposer, providers, hold, gate, entered, fail, close } = await cold(() => "a");
it("the object fails and stays failed: the consumer parks failed { waiting } on it, never live on it", async () => {
const { host, exposer, providers, hold, gate, entered, fail, close } = await cold(() => "a");
hold();
const abort = new AbortController();
const viewer = host.hold(Viewer, { id: 1 }, context);
const loading = host.get(Viewer, { id: 1 }, withAbortSignal(abort.signal, context));
await entered.promise;
fail();
@ -323,8 +326,8 @@ describe("a key whose first pin still loads", () => {
await delivered();
await new Promise((resolve) => setTimeout(resolve, 10));
expect(exposer.state.value.kind).toBe("failed");
const failure = errors.find((error) => error.kind === "activation_failed");
expect(failure).toMatchObject({ failure: { kind: "transient" } });
expect(viewer.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
viewer.release();
abort.abort();
await expect(loading).rejects.toThrow();
expect(pinsOf(providers.a!)).toBeUndefined();
@ -450,3 +453,54 @@ describe("chains and releases", () => {
await expectCollected(...refs);
});
});
describe("a key whose first pin still loads at startup (scenario 20, Astra)", () => {
it("keeps the first pass pending during an otherwise-ready dependency's delayed keyed pin", async () => {
const Data = defineState("app.pin-data", {
schema: Schema,
version: 1,
keyed: true,
initial: () => ({ n: 4 }),
});
const Store = local("app.pin-store", { version: 1, params: {}, members: { notes: keyed(Schema) } });
const Reader = local("app.pin-reader", { version: 1, params: {}, members: { value: Schema } });
const gate = Promise.withResolvers<void>();
const original = ChordObject.prototype.pinKey;
const spy = vi.spyOn(ChordObject.prototype, "pinKey").mockImplementation(async function (
this: ChordObject,
member,
key,
ctx,
) {
await gate.promise;
return original.call(this, member, key, ctx);
});
const host = createHost({ onError: () => {} });
let done = false;
const loading = host
.load(
[
{
name: "@test/pin",
facet: defineFacet((h) => {
h.serve(Store, (_, self) => ({ notes: self.state(Data) }));
h.serve(Reader, (_, self) => ({ value: self.get(Store, {}).notes.at("k") }));
}),
},
],
context,
)
.then(() => {
done = true;
});
await flushed();
const premature = done;
gate.resolve();
await loading;
spy.mockRestore();
const ref = await host.get(Reader, {}, context);
expect(ref.value.value).toEqual({ n: 4 });
await host.close(context);
expect(premature).toBe(false);
});
});

180
packages/chord/test/next/host/object.test.ts
View file

@ -30,6 +30,7 @@ import {
Server,
Session,
StoreRestartable,
statesOf,
storeRestartable,
testCodec,
transported,
@ -240,7 +241,7 @@ describe("retries", () => {
});
});
describe("dependency recovery (R18)", () => {
describe("parked on dependencies (dependency model §3)", () => {
fakeTimers();
const View = local("app.view", { version: 1, params: { id: int() }, members: { n: int() } });
@ -254,7 +255,7 @@ describe("dependency recovery (R18)", () => {
return { times, factory };
}
it("fails a dependent transient with the dependency's cause and retries it at once when it goes live", async () => {
it("parks a dependent of a failed dependency failed { waiting } naming it, with no timer; it goes on when the dependency is live", async () => {
const core = flaky((attempt) => attempt === 1);
const viewer = view(Core);
const { host, errors } = await loaded((h) => {
@ -262,26 +263,28 @@ describe("dependency recovery (R18)", () => {
h.serve(View, viewer.factory);
});
await vi.advanceTimersByTimeAsync(500);
const held = host.hold(View, { id: 1 }, context);
const got = host.get(View, { id: 1 }, context);
await flushed();
const failed = errors.find((error) => error.kind === "activation_failed" && error.object === "app.view(1)");
expect(failed).toMatchObject({
failure: {
kind: "transient",
error: { code: "unavailable", message: "app.view(1): dependency app.core failed" },
expect(held.state.value).toEqual({
kind: "failed",
cause: {
kind: "waiting",
error: expect.objectContaining({ message: "waiting for app.core: app.core: offline" }),
},
retryAt: undefined,
});
expect(failed?.kind === "activation_failed" && failed.failure.error.cause).toMatchObject({
message: "app.core: offline",
});
// The core's retry at 1 s succeeds; the view is retried then, not at the end of its own window (1.5 s).
// A parked attempt is no failure of its own: nothing is reported for the view.
expect(failures(errors)).toEqual(["app.core: transient"]);
// The core's own backoff retries it at 1 s; the view goes on then, its factory not run again.
await vi.advanceTimersByTimeAsync(500);
expect((await got).n.value).toBe(2);
await vi.advanceTimersByTimeAsync(5000);
expect(viewer.times).toEqual([500, 1000]);
await vi.advanceTimersByTimeAsync(120_000);
expect(viewer.times).toEqual([500]);
held.release();
});
it("keeps the dependency while failed; releasing the dependent releases it", async () => {
it("keeps its references while parked; a getter rejects with the waiting cause at its deadline; releasing ends the wait", async () => {
const svc = flaky(() => true);
const viewer = view(Svc);
const { host } = await loaded((h) => {
@ -290,81 +293,77 @@ describe("dependency recovery (R18)", () => {
});
const got = rejection(host.get(View, { id: 1 }, context, { readyTimeout: 1500 }));
await vi.advanceTimersByTimeAsync(1500);
expect(await got).toEqual({ code: "unavailable", message: "app.view(1): dependency app.svc(1) failed" });
expect(await got).toEqual({
code: "unavailable",
message: "waiting for app.svc(1): app.svc(1): offline",
});
await vi.advanceTimersByTimeAsync(10_000);
// Held by the failed view, the dependency was retried at 1 s; released with it at 1.5 s, it is not any more.
// Held by the parked view, the dependency was retried at 1 s; released with it at 1.5 s, it is not any more.
expect(svc.times).toEqual([0, 1000]);
expect(viewer.times).toEqual([0]);
});
it("binds a retry's references before releasing the previous attempt's", async () => {
const svc = flaky((attempt) => attempt === 1);
const viewer = view(Svc);
it("shows every change of a dependency: loading, failed, a new cause, loading again, live", async () => {
let cores = 0;
const viewer = view(Core);
const { host } = await loaded((h) => {
h.serve(Svc, svc.factory);
h.serve(Core, (_, self) => {
const attempt = ++cores;
self.open(
async () => {
if (attempt < 3) throw new Error(`offline ${attempt}`);
},
() => {},
);
return { n: constant(attempt) };
});
h.serve(View, viewer.factory);
});
const got = host.get(View, { id: 1 }, context);
await vi.advanceTimersByTimeAsync(1000);
// Held only by the view: released first, the live dependency would have ended and started anew.
expect((await got).n.value).toBe(2);
expect([svc.times, viewer.times]).toEqual([
[0, 1000],
[0, 1000],
]);
const held = host.hold(View, { id: 1 }, context);
await flushed();
expect(held.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.core: app.core: offline 1" } },
});
const states = statesOf(await held.state.watch(context));
await vi.advanceTimersByTimeAsync(10_000);
// The core's retries at 1 s and 3 s: each attempt shows as loading, each failure as a new cause.
expect(states).toEqual(["loading", "failed: waiting for app.core: app.core: offline 2", "loading", "live"]);
expect(viewer.times).toEqual([0]);
held.release();
});
it("remembers a dependency going live while the dependent activates: the dependent is retried at once (audit)", async () => {
const viewer = view(Core);
it("passes a retry on to its broken dependency, once through a diamond", async () => {
const Top = local("app.top", { version: 1, params: {}, members: { n: int() } });
const Left = local("app.left", { version: 1, params: { id: int() }, members: { n: int() } });
let cores = 0;
const { host } = await loaded((h) => {
// Its first factory throws (broken); the second goes live at once, without natives.
h.serve(Core, () => {
if (++cores === 1) throw new Error("first factory");
return { n: constant(2) };
if (++cores === 1) throw new Error("bug");
return { n: constant(cores) };
});
h.serve(View, viewer.factory);
});
await flushed();
// The view activates on the core's failure while the core is retried and goes live at once.
const dependency = host.hold(Core, {}, context);
const dependent = host.hold(View, { id: 1 }, context);
dependency.retry();
await flushed();
expect(dependency.state.value).toEqual({ kind: "live" });
expect(dependent.state.value).toEqual({ kind: "live" });
expect(viewer.times).toEqual([0, 0]);
dependent.release();
dependency.release();
});
it("ignores a trigger while an attempt runs: simultaneous triggers give one attempt", async () => {
const gate = Promise.withResolvers<void>();
const svc = flaky((attempt) => attempt === 1, gate.promise);
const viewer = view(Svc);
const { host } = await loaded((h) => {
h.serve(Svc, svc.factory);
h.serve(View, viewer.factory);
});
const got = host.get(View, { id: 1 }, context);
// At 1 s both windows end: the dependency's retry waits at its gate, the view's waits for it.
await vi.advanceTimersByTimeAsync(1000);
gate.resolve();
expect((await got).n.value).toBe(2);
expect(viewer.times).toEqual([0, 1000]);
});
it("retries a dependent of a broken dependency with its backoff, for as long as it is held", async () => {
const viewer = view(Core);
const { host } = await loaded((h) => {
h.serve(Core, () => {
throw new Error("bug");
h.serve(Left, (_, self) => ({ n: self.get(Core, {}).n }));
h.serve(Top, (_, self) => {
const a = self.get(Left, { id: 1 }).n;
const b = self.get(Left, { id: 2 }).n;
return { n: self.derive([a, b], ([x, y]) => x + y) };
});
h.serve(View, viewer.factory);
});
const got = rejection(host.get(View, { id: 1 }, context, { readyTimeout: 16_000 }));
await vi.advanceTimersByTimeAsync(16_000);
expect(await got).toEqual({ code: "unavailable", message: "app.view(1): dependency app.core failed" });
expect(viewer.times).toEqual([0, 1000, 3000, 7000, 15000]);
const top = host.hold(Top, {}, context);
await flushed();
expect(top.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.left(1): app.core: bug" } },
});
await vi.advanceTimersByTimeAsync(60_000);
// A broken dependency is not retried by any timer.
expect(cores).toBe(1);
top.retry();
await flushed();
expect(cores).toBe(2);
expect(top.state.value).toEqual({ kind: "live" });
expect(top.n.value).toBe(4);
top.release();
});
it("fails every object of a cycle broken; none is retried", async () => {
@ -517,6 +516,39 @@ describe("GC of failed objects", () => {
});
});
describe("GC of a parked chain", () => {
it("collects a chain parked on a type not served, released from its top: objects, attempts and index entries", async () => {
const Top = local("app.top", { version: 1, params: { id: int() }, members: { n: int() } });
const Mid = local("app.mid", { version: 1, params: { id: int() }, members: { n: int() } });
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
const refs: WeakRef<object>[] = [];
const { host } = await loaded((h) => {
h.serve(Mid, (_, self) => {
refs.push(new WeakRef(self));
return { n: self.get(Missing, {}).n };
});
h.serve(Top, ({ id }, self) => {
refs.push(new WeakRef(self));
return { n: self.get(Mid, { id }).n };
});
});
const top = host.hold(Top, { id: 1 }, context);
await flushed();
expect(top.state.value).toMatchObject({
kind: "failed",
cause: {
kind: "waiting",
error: { message: "waiting for app.mid(1): app.missing is not served here" },
},
});
top.release();
await flushed();
expect(refs).toHaveLength(2);
await expectCollected(...refs);
expect(host).toBeDefined();
});
});
describe("ref.retry()", () => {
fakeTimers();

4
packages/chord/test/next/host/reference.test.ts
View file

@ -613,7 +613,7 @@ describe("keyed state members (slice 6a §12)", () => {
await store.close(context);
});
it("a pin load that fails its provider fails the consumer as a dependency failure (transient)", async () => {
it("a pin load that fails its provider parks the consumer failed { waiting } on it (a dependency failure)", async () => {
const store = await seeded();
const { host, errors } = await persistedNotes(store, persistedMemo(2));
const provider = await host.get(NoteStore, {}, context);
@ -621,7 +621,7 @@ describe("keyed state members (slice 6a §12)", () => {
await vi.waitFor(() => expect(view.state.value.kind).toBe("failed"));
expect(view.state.value).toMatchObject({
kind: "failed",
cause: { kind: "transient", error: { message: expect.stringContaining("cannot migrate") } },
cause: { kind: "waiting", error: { message: expect.stringContaining("cannot migrate") } },
});
expect(provider.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } });
expect(errors.map(({ kind }) => kind)).toContain("object_failed");

21
packages/chord/test/next/host/retry-on.test.ts
View file

@ -76,6 +76,27 @@ describe("retryOn", () => {
await host.close(context);
});
it("does nothing while the object is parked failed { waiting }: its dependencies decide (dependency model §3.8)", async () => {
const Missing = local("app.missing", { version: 1, params: {}, members: {} });
let factories = 0;
const { host } = await loaded((h) => {
serveModels(h);
h.serve(User, (_, self) => {
factories++;
self.retryOn(self.registry(Models));
self.get(Missing, {});
return {};
});
});
const user = host.hold(User, { id: 1 }, context);
await flushed();
expect(user.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
await (await host.get(Runtime, {}, context)).set(1, context);
await flushed();
expect([user.state.value.kind, factories]).toEqual(["failed", 1]);
await host.close(context);
});
it("wakes on value calls only: a throwing compose wakes nothing, its next good composition does", async () => {
const opens = { n: 0 };
const { host } = await loaded((h) => {

140
packages/chord/test/next/host/revocation.test.ts
View file

@ -1,12 +1,12 @@
// Tests of revocation (§4, R10, R11): revocation keys, \`host.revoke\` on one host (objects, running calls, activations,
// holds still waiting for \`load\`), child links and \`revoke\` messages, GC, and revocation across a relay.
// holds bound or not), child links and \`revoke\` messages, GC, and revocation across a relay.
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { int, local, remote } from "../../../src/next/contracts/index.ts";
import { hostKind, int, local, remote } from "../../../src/next/contracts/index.ts";
import { createHost, defineFacet, type FacetHost, type Host } from "../../../src/next/host/index.ts";
import { ChordObject } from "../../../src/next/host/object.ts";
import { memoryLinkPair } from "../../../src/next/link/index.ts";
import { type Link, memoryLinkPair } from "../../../src/next/link/index.ts";
import { jsonPart } from "../../../src/next/link/internal.ts";
import { constant } from "../../../src/next/state/index.ts";
import {
@ -18,6 +18,7 @@ import {
RawPeer,
rawParent,
rawWorkers,
rejection,
relayed,
SERVER_ADDRESS,
Server,
@ -312,6 +313,22 @@ forEachCodec(() => {
await host.close(context);
});
it("ends an unbound hold of the principal (refused, waiting for a provider): gone, nothing left of it", async () => {
const { host, parent } = await worker();
const Absent = { ...Notes, name: "app.absent" } as unknown as typeof Notes;
holdAs(parent, 0, S1, Absent);
expect(await parent.next("failed")).toMatchObject({ hold: 0, cause: { kind: "waiting" } });
expect(host.activity.value.references).toBe(1);
parent.send({ kind: "revoke", revocation: { session: "S1" } });
expect(await parent.next("gone")).toMatchObject({
hold: 0,
error: { kind: "chord", error: { code: "forbidden", reason: "revoked" } },
});
await transported();
expect(host.activity.value.references).toBe(0);
await host.close(context);
});
it("binds a hold made for a revoked principal, then the parent's revoke after it ends it (race 3, R10's window)", async () => {
const { host, parent } = await worker();
holdAs(parent, 0, S1);
@ -443,3 +460,120 @@ describe("revocation across a relay", () => {
expect(again.state.value.kind).toBe("live");
});
});
describe("revocation of this host's own requests over its paths (fix round 1)", () => {
type Guest = { kind: "owner" } | { kind: "guest"; grant: string };
const There = hostKind("app-there");
const Remote = remote("app.remote", {
version: 1,
params: {},
hostAddress: () => There(),
variants: { view: { state: {}, methods: {} } },
});
const guestRevokedBy = (p: Guest) => (p.kind === "guest" ? [p.grant] : []);
it("ends a guest's ctx.get waiting for its route before any link exists, at once (Astra 1, Sol 1)", async () => {
const host = createHost<Guest, string>({
principal: { kind: "owner" },
revokedBy: guestRevokedBy,
onError: () => {},
});
host.route(There, { authorize: () => true, resolve: () => new Promise(() => {}) }, context);
await host.load([], context);
let outcome: unknown;
void rejection(
host.root((ctx) => ctx.get(Remote, {}, { readyTimeout: 60_000 }), context, {
principal: { kind: "guest", grant: "g" },
}),
).then((error) => {
outcome = error;
});
await transported();
host.revoke("g", context);
await transported();
expect(outcome).toMatchObject({ code: "forbidden", reason: "revoked" });
await host.close(context);
});
it("never lets the revoked request bind on the route's later link (Astra 1)", async () => {
const host = createHost<Guest, string>({
principal: { kind: "owner" },
revokedBy: guestRevokedBy,
onError: () => {},
});
const gate = Promise.withResolvers<{ link: Link; principal: Guest; trusted: boolean }>();
host.route(There, { authorize: () => true, resolve: () => gate.promise }, context);
await host.load([], context);
const pending = rejection(
host.root(
async (ctx) => {
await ctx.get(Remote, {}, { readyTimeout: 500 });
throw new Error("revoked request became live");
},
context,
{ principal: { kind: "guest", grant: "g" } },
),
);
await transported();
host.revoke("g", context);
const worker = createHost<Guest, string>({
address: There(),
principal: { kind: "owner" },
revokedBy: guestRevokedBy,
onError: () => {},
});
const [up, down] = memoryLinkPair();
worker.connect(up, { principal: { kind: "owner" }, trusted: true }, context);
const facet = defineFacet<Guest>((h) => h.serve(Remote, () => ({ variant: "view", members: {} })));
await worker.load([{ name: "@test/late", facet }], context);
gate.resolve({ link: down, principal: { kind: "owner" }, trusted: true });
expect(await pending).toMatchObject({ code: "forbidden", reason: "revoked" });
await transported();
expect(worker.activity.value.references).toBe(0);
await host.close(context);
await worker.close(context);
});
it("ends a guest's reference over a route link that dropped, and nothing holds it on the next one", async () => {
const routed: RawPeer[] = [];
const host = createHost<Guest, string>({
principal: { kind: "owner" },
revokedBy: guestRevokedBy,
onError: () => {},
});
host.route(
There,
{
authorize: () => true,
resolve: async () => {
const [a, b] = memoryLinkPair();
routed.push(new RawPeer(b));
return { link: a, principal: { kind: "owner" }, codec: testCodec() };
},
},
context,
);
await host.load([], context);
const states: string[] = [];
void host.root(
async (ctx) => {
const held = await ctx.get(Remote, {}, { readyTimeout: 60_000 }).catch((error: Error) => error);
states.push(held instanceof Error ? held.message : "live");
},
context,
{ principal: { kind: "guest", grant: "g" } },
);
await transported();
const first = routed[0]!;
await first.next("hold");
first.close();
await transported();
host.revoke("g", context);
await transported();
expect(states).toEqual(["app.remote: revoked"]);
expect(routed.flatMap((peer, i) => (i === 0 ? [] : peer.received.filter(({ kind }) => kind === "hold")))).toEqual(
[],
);
await host.close(context);
});
});

3
packages/chord/test/next/host/routines.test.ts
View file

@ -727,8 +727,9 @@ describe("on", () => {
}, ctx);
});
});
// 0 → 1 → 2 → 3, then the write of 3 changes nothing and nothing calls again (waited for, not timed: Opus F3).
await vi.waitFor(() => expect(calls).toBe(4), { timeout: 2_000 });
await new Promise((resolve) => setTimeout(resolve, 20));
// 0 → 1 → 2 → 3, then the write of 3 changes nothing and nothing calls again.
expect(calls).toBe(4);
});

171
packages/chord/test/next/host/unbound.test.ts Normal file
View file

@ -0,0 +1,171 @@
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import { int, local, str } from "../../../src/next/contracts/index.ts";
import { ChordError, type Principal } from "../../../src/next/data/index.ts";
import { createHost, defineFacet } from "../../../src/next/host/index.ts";
import { Binding } from "../../../src/next/host/reference.ts";
import { UnboundIndex, type UnboundRequest } from "../../../src/next/host/unbound.ts";
import { memoryLinkPair } from "../../../src/next/link/index.ts";
import { jsonPart } from "../../../src/next/link/internal.ts";
import { Hold } from "../../../src/next/remote/hold.ts";
import { constant } from "../../../src/next/state/index.ts";
import {
delivered,
expectCollected,
instancesOf,
RawPeer,
rejection,
SERVER_ADDRESS,
Server,
type TestPrincipal,
testCodec,
transported,
} from "../support.ts";
const bindings = instancesOf(Binding, "release");
const holds = instancesOf(Hold, "release");
const Info = local("app.info", { version: 1, params: {}, members: { title: str() } });
const Missing = local("app.missing", { version: 1, params: { id: int() }, members: { n: int() } });
/** A request recording its wakes and ends. */
function request(principal?: Principal): UnboundRequest & { readonly log: string[] } {
const log: string[] = [];
return { principal, log, wake: () => log.push("wake"), end: (error) => log.push(`end ${error.message}`) };
}
describe("the index", () => {
it("wakes every request at the publication, out of the index first; a deleted one is not woken", () => {
const index = new UnboundIndex();
const [a, b, c] = [request(), request(), request()];
index.add("app.a", a);
index.add("app.a", b);
index.add("app.b", c);
index.delete("app.a", b);
index.wakeAll();
expect([a.log, b.log, c.log]).toEqual([["wake"], [], ["wake"]]);
// Out of it: a second publication wakes nobody.
index.wakeAll();
expect(a.log).toEqual(["wake"]);
});
it("ends the requests of a revoked principal only, and every request at the host's close", () => {
const index = new UnboundIndex();
const guest = request({ kind: "guest" });
const owner = request({ kind: "owner" });
const own = request();
for (const each of [guest, owner, own]) index.add("app.a", each);
const revoked = new ChordError({ code: "forbidden", reason: "revoked", message: "revoked" });
index.revoke((principal) => principal.kind === "guest", revoked);
expect([guest.log, owner.log, own.log]).toEqual([["end revoked"], [], []]);
index.close(new ChordError({ code: "released", reason: "closed", message: "closed" }));
expect([owner.log, own.log]).toEqual([["end closed"], ["end closed"]]);
});
});
describe("references waiting in the index", () => {
it("before the publication: loading, placed by it; a type not served then waits failed { waiting }", async () => {
const host = createHost({ onError: () => {} });
const info = host.hold(Info, {}, context);
const missing = host.hold(Missing, { id: 1 }, context);
expect([info.state.value, missing.state.value]).toEqual([{ kind: "loading" }, { kind: "loading" }]);
await host.load(
[{ name: "@test/a", facet: defineFacet((h) => h.serve(Info, () => ({ title: constant("t") }))) }],
context,
);
await delivered();
expect(info.state.value).toEqual({ kind: "live" });
expect(missing.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { code: "unavailable", message: "app.missing is not served here" } },
retryAt: undefined,
});
// A retry leaves it waiting: an event wakes it, nothing else.
missing.retry();
expect(missing.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
await host.close(context);
expect(missing.state.value).toMatchObject({ kind: "ended", error: { reason: "closed" } });
});
it("leave nothing behind once released, before and after the publication (GC)", async () => {
const host = createHost({ onError: () => {} });
host.hold(Missing, { id: 1 }, context).release();
await host.load([], context);
host.hold(Missing, { id: 2 }, context).release();
const got = host.get(Missing, { id: 3 }, context, { readyTimeout: 1 }).catch(() => {});
await got;
await expectCollected(...bindings);
expect(host).toBeDefined();
});
it("end at the start of the host's close, not after its slow native closes (fix round 1, Sol 6)", async () => {
const Core = local("app.core", { version: 1, params: {}, members: { n: int() } });
const closeGate = Promise.withResolvers<void>();
const host = createHost({ onError: () => {} });
const facet = defineFacet((h) =>
h.serve(Core, (_, self) => {
self.open(
async () => 1,
() => closeGate.promise,
);
return { n: constant(1) };
}),
);
await host.load([{ name: "@test/a", facet }], context);
let outcome: unknown;
void rejection(host.get(Missing, { id: 1 }, context)).then((error) => {
outcome = error;
});
const held = host.hold(Missing, { id: 2 }, context);
const closing = host.close(context);
await delivered();
expect(outcome).toMatchObject({ code: "released", reason: "closed" });
expect(held.state.value).toMatchObject({ kind: "ended", error: { reason: "closed" } });
closeGate.resolve();
await closing;
});
});
describe("holds waiting in the index", () => {
/** A server not loaded yet, a raw consumer peer on a child link, and a hold of `app.missing` it sent. */
async function early() {
const host = createHost<TestPrincipal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
const [a, b] = memoryLinkPair();
host.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b);
peer.send({
kind: "hold",
hold: 0,
type: "app.missing",
version: 1,
fingerprint: "0123456789abcdef",
params: jsonPart({}),
address: SERVER_ADDRESS,
});
await transported();
return { host, peer, link: a };
}
it("wait for the publication without a word, then are refused failed { waiting } and kept", async () => {
const { host, peer } = await early();
expect(peer.kinds()).toEqual(["hello"]);
await host.load([], context);
expect(await peer.next("failed")).toMatchObject({ hold: 0, cause: { kind: "waiting" } });
expect(host.activity.value.references).toBe(1);
await host.close(context);
});
it("leave nothing behind once their link closed, before and after the publication (GC)", async () => {
const before = await early();
before.link.close();
await transported();
await before.host.load([], context);
const after = await early();
await after.host.load([], context);
await after.peer.next("failed");
after.link.close();
await transported();
expect([before.host.activity.value.references, after.host.activity.value.references]).toEqual([0, 0]);
await expectCollected(...holds);
expect([before.host, after.host]).toBeDefined();
});
});

41
packages/chord/test/next/link/message.test.ts
View file

@ -100,6 +100,12 @@ const EVERY_KIND: Message[] = [
cause: { kind: "transient", error: { kind: "chord", error: { code: "unavailable", message: "m" } } },
retryIn: 0,
},
// Not the object's fault (dependency model §2): no backoff window, an event wakes it.
{
kind: "failed",
hold: 0,
cause: { kind: "waiting", error: { kind: "chord", error: { code: "contract_mismatch", message: "m" } } },
},
// The wire error format in `failed` and `gone`: a plugin's error travels too.
{ kind: "failed", hold: 0, cause: { kind: "broken", error: { kind: "plugin", code: "config", message: "m" } } },
{ kind: "gone", hold: 1, error: { kind: "plugin", code: "config", message: "m", details: { at: 1 } } },
@ -110,6 +116,10 @@ const EVERY_KIND: Message[] = [
retryIn: 60_000,
},
{ kind: "retry", hold: 300 },
// The retry operation it belongs to (fix round 1): an object reached twice handles it once.
{ kind: "retry", hold: 300, operation: incarnation },
// And the instance ids of the objects it passed through (fix round 2): a ring stops after one lap.
{ kind: "retry", hold: 300, operation: incarnation, through: [incarnation, incarnation] },
{ kind: "loading", hold: 300 },
{ kind: "result", call: 3, value: '{"n":1}' },
{ kind: "result", call: 3 },
@ -333,6 +343,21 @@ describe("the binary codec", () => {
expect([...binaryCodec.encode({ kind: "closed", origin: 1 })]).toEqual([14 | 0x80, 1]);
});
it("writes a failure's cause as one byte, 0 transient, 1 broken, 2 waiting (dependency model §2); refuses another", () => {
const error = { kind: "chord", error: { code: "unavailable", message: "m" } } as const;
const failed = (kind: "transient" | "broken" | "waiting"): Message => ({
kind: "failed",
hold: 0,
cause: { kind, error },
});
expect(
["transient", "broken", "waiting"].map((kind) => binaryCodec.encode(failed(kind as "waiting"))[2]),
).toEqual([0, 1, 2]);
const unknown = binaryCodec.encode(failed("waiting"));
unknown[2] = 3;
expect(() => binaryCodec.decode(unknown)).toThrow("an unknown failure kind");
});
it("costs a state entry without retirements no byte: the failure count is doubled, plus one when they follow", () => {
const state = (retired: number[]): Message => ({
kind: "state",
@ -413,8 +438,16 @@ describe("the binary codec", () => {
expect(() => binaryCodec.decode(new Uint8Array([1, 0, 1, 0xff]))).toThrow();
});
it("writes retry and loading as their kind and hold, and failed's retryIn as one more than it (0: absent)", () => {
expect([...binaryCodec.encode({ kind: "retry", hold: 300 })]).toEqual([11, 0xac, 0x02]);
it("writes retry as its kind, hold, flags, operation (16 raw bytes) and objects passed through, loading as its kind and hold, and failed's retryIn as one more than it (0: absent)", () => {
expect([...binaryCodec.encode({ kind: "retry", hold: 300 })]).toEqual([11, 0xac, 0x02, 0]);
expect(binaryCodec.encode({ kind: "retry", hold: 300, operation: incarnation })).toHaveLength(4 + 16);
const through = [incarnation, incarnation];
expect(binaryCodec.encode({ kind: "retry", hold: 300, operation: incarnation, through })).toHaveLength(
4 + 16 + 1 + 32,
);
const flagged = binaryCodec.encode({ kind: "retry", hold: 300 });
flagged[3] = 4;
expect(() => binaryCodec.decode(flagged)).toThrow("an unknown retry flag");
expect([...binaryCodec.encode({ kind: "loading", hold: 1 })]).toEqual([12, 1]);
const cause = { kind: "broken", error: { kind: "chord", error: { code: "released", message: "" } } } as const;
const absent = binaryCodec.encode({ kind: "failed", hold: 0, cause });
@ -446,6 +479,10 @@ describe("the text codec", () => {
expect(text({ kind: "failed", hold: 2, cause, retryIn: 750 })).toBe(
'["f",2,"t",{"kind":"chord","error":{"code":"unavailable","message":"m"}},750]',
);
expect(text({ kind: "failed", hold: 2, cause: { ...cause, kind: "waiting" } })).toBe(
'["f",2,"w",{"kind":"chord","error":{"code":"unavailable","message":"m"}}]',
);
expect(() => textCodec.decode(new TextEncoder().encode('["f",2,"x",{}]'))).toThrow("an unknown failure kind");
expect(text({ kind: "ack" })).toBe('["a"]');
// An origin wraps the message's own tuple.
expect(text({ kind: "release", origin: 4, hold: 3 })).toBe('["@",4,["R",3]]');

29
packages/chord/test/next/registries/record.test.ts
View file

@ -320,7 +320,7 @@ describe("compose", () => {
await host.close(context);
});
it("before any good composition, `.value` and `host.registry` throw the error", async () => {
it("before any good composition, `.value` throws the error and `host.registry` rejects with it at its readyTimeout", async () => {
const Broken = defineRegistry<number, number>("app.broken", { version: 1 });
const { host, errors } = await loadedPlugins({
"@app/broken": (host) =>
@ -330,7 +330,7 @@ describe("compose", () => {
},
}),
});
await expect(host.registry(Broken, context)).rejects.toThrow("bug");
await expect(host.registry(Broken, context, { readyTimeout: 5 })).rejects.toThrow("bug");
expect(kinds(errors)).toEqual(["registry_failed"]);
await host.close(context);
});
@ -623,6 +623,31 @@ describe("contributions", () => {
});
});
describe("the last good composition (decision 5)", () => {
it("is dropped when a contribution it was built from leaves: `.value` throws until `compose` succeeds", async () => {
const Count = defineRegistry<number, number>("app.count", { version: 1 });
const compose = (table: RegistryTable<number>): number => {
if (table.length === 0) throw new Error("no contributor");
return table[0]!.value;
};
const { host, fail, retry } = await storeRestartable(
objects,
(attempt, self) => self.contribute(Count, attempt),
{
serve: (host) => host.serve(Count, { compose }),
},
);
const count = await host.registry(Count, context);
expect(count.value).toBe(1);
// Its contributor failed: the last good value was built from its contribution (a native may have closed with it).
fail();
expect(() => count.value).toThrow("no contributor");
await retry();
expect(count.value).toBe(2);
await host.close(context);
});
});
describe("the table's memory", () => {
it("keeps nothing of a closed host: records, tables and followed contributions go with it", async () => {
const Doc = defineState("app.kept", { schema: obj({ on: bool() }), version: 1, initial: () => ({ on: true }) });

173
packages/chord/test/next/registries/value.test.ts
View file

@ -2,7 +2,8 @@ import { getEventListeners } from "node:events";
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context, withAbortSignal, withCancel } from "../../../src/context/index.ts";
import { int, local, method, str } from "../../../src/next/contracts/index.ts";
import type { defineFacet } from "../../../src/next/host/index.ts";
import type { FollowObserver } from "../../../src/next/host/follow.ts";
import { createHost, defineFacet, type Native } from "../../../src/next/host/index.ts";
import { ChordObject } from "../../../src/next/host/object.ts";
import { defineRegistry, type Registry } from "../../../src/next/registries/index.ts";
import type { NativeWatchCall, WatchEnd } from "../../../src/watch.ts";
@ -83,7 +84,7 @@ describe("older readers (S2)", () => {
await host.close(context);
});
it("without a value mapping, and for a reader newer than the provider, fail `contract_mismatch`: broken, reported", async () => {
it("without a value mapping, and for a reader newer than the provider, park readers failed { waiting } (contract_mismatch)", async () => {
const Newer = local("app.newer", { version: 1, params: {}, members: { total: int() } });
const Older = local("app.older", { version: 1, params: {}, members: { total: int() } });
const { host, errors } = await loadedPlugins({
@ -93,11 +94,103 @@ describe("older readers (S2)", () => {
host.serve(Older, (_, self) => ({ total: self.derive([self.registry(SumV1)], ([v]) => v) }));
},
});
expect(await rejection(host.registry(SumV1, context))).toMatchObject({ code: "contract_mismatch" });
expect(await rejection(host.registry(SumV3, context))).toMatchObject({ code: "contract_mismatch" });
// Only a reload helps: no backoff, the mismatch itself the cause.
expect(failures(errors)).toEqual(["app.newer: broken", "app.older: broken"]);
expect(errors[0]).toMatchObject({ failure: { error: { code: "contract_mismatch" } } });
const quick = { readyTimeout: 5 };
expect(await rejection(host.registry(SumV1, context, quick))).toMatchObject({ code: "contract_mismatch" });
expect(await rejection(host.registry(SumV3, context, quick))).toMatchObject({ code: "contract_mismatch" });
// Not their fault: waiting (a reload serving the version wakes them), no backoff, nothing failed.
expect(failures(errors)).toEqual([]);
expect(host.hold(Newer, {}, context).state.value).toMatchObject({
kind: "failed",
cause: {
kind: "waiting",
error: {
message: "waiting for registry app.sum: registry app.sum: version 3 is not served here (version 2 is)",
},
},
retryAt: undefined,
});
await host.close(context);
});
it("drops an older reader's last good mapping after a contributor it was built from failed live (fix round 1, Astra 2)", async () => {
const R = defineRegistry<number, number>("app.mapped", { version: 2 });
const Old = defineRegistry<number, number>("app.mapped", { version: 1 });
const Contributor = local("app.contributor", { version: 1, params: {}, members: { n: int() } });
let observer: FollowObserver<unknown> | undefined;
const { host } = await loadedPlugins({
"@app/mapping": (h) => {
h.serve(R, {
compose: (table) => table.reduce((n, e) => n + e.value, 0),
accepts: {
1: {
value: (n) => {
if (n === 0) throw new Error("old version cannot map the empty table");
return n;
},
},
},
});
h.serve(Contributor, (_, self) => {
self.contribute(R, 42);
return {
n: self.follow(async (followed) => {
observer = followed;
return { value: 42, detach() {} };
}),
};
});
},
});
const old = await host.registry(Old, context);
expect(old.value).toBe(42);
observer!.end({ kind: "failed", error: new Error("runtime ended") });
await flushed();
expect(() => old.value).toThrow("old version cannot map the empty table");
await host.close(context);
});
it("never hands out a mapped native whose contributor failed and closed it (fix round 1, Sol 2)", async () => {
type Resource = { closed: boolean };
const Current = defineRegistry<Native<Resource>, readonly Native<Resource>[]>("app.resources", { version: 2 });
const Old = defineRegistry<Native<Resource>, Native<Resource>>("app.resources", { version: 1 });
const Owner = local("app.owner", { version: 1, params: {}, members: {} });
const resource = { closed: false };
let failRuntime: () => void = () => {};
const { host } = await loadedPlugins({
"@app/resources": (h) => {
h.serve(Current, {
compose: (table) => table.map(({ value }) => value),
accepts: {
1: {
value: (current) => {
if (current.length === 0) throw new Error("no current resource");
return current[0]!;
},
},
},
});
h.serve(Owner, (_, self) => {
const native = self.open(
async () => resource,
() => {
resource.closed = true;
},
);
self.contribute(Current, native);
self.follow(async (observer) => {
failRuntime = () => observer.end({ kind: "failed", error: new Error("runtime closed") });
return { value: 1, detach: () => {} };
});
return {};
});
},
});
const value = await host.registry(Old, context);
expect(value.value.value).toBe(resource);
failRuntime();
await flushed();
expect(resource.closed).toBe(true);
expect(() => value.value).toThrow("no current resource");
await host.close(context);
});
@ -162,29 +255,30 @@ describe("readers (S3)", () => {
R: Registry<number, { readonly total: number }> = Sum,
) => host.serve(Reader, (_, self) => ({ total: self.derive([self.registry(R)], ([v]) => v.total) }));
it("activate once the registry has a value; a reader of a registry nobody serves fails at once, `load` waits for nothing", async () => {
it("activate once the registry has a value; a reader of a registry nobody serves parks failed { waiting }, `load` waits for nothing", async () => {
const started = Date.now();
const { host, errors } = await loadedPlugins({ "@app/reader": (host) => serveReader(host) });
expect(Date.now() - started).toBeLessThan(1_000);
expect(errors).toMatchObject([
{
kind: "activation_failed",
object: "app.reader",
failure: {
kind: "transient",
error: {
code: "unavailable",
message: "app.reader: registry app.sum has no value",
cause: { code: "unavailable", reason: "undeclared" },
},
expect(errors).toEqual([]);
expect(host.hold(Reader, {}, context).state.value).toMatchObject({
kind: "failed",
cause: {
kind: "waiting",
error: {
code: "unavailable",
message: "waiting for registry app.sum: registry app.sum: no plugin serves it here",
cause: { code: "unavailable", reason: "undeclared" },
},
},
]);
expect(await rejection(host.registry(Sum, context))).toMatchObject({ code: "unavailable", reason: "undeclared" });
});
expect(await rejection(host.registry(Sum, context, { readyTimeout: 5 }))).toMatchObject({
code: "unavailable",
reason: "undeclared",
});
await host.close(context);
});
it("a reader failed on a first `compose` that threw is retried at once by the first good composition (R18)", async () => {
it("a reader parked on a first `compose` that threw goes on at the first good composition", async () => {
const { host, errors } = await loadedPlugins({
"@app/sum": (host) => {
host.serve(Sum, {
@ -198,15 +292,46 @@ describe("readers (S3)", () => {
},
"@app/setter": (host) => setter(host, 0),
});
expect(failures(errors)).toEqual(["app.reader: transient"]);
expect(failures(errors)).toEqual([]);
const held = host.hold(Reader, {}, context);
expect(held.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
const setters = await host.get(Setter, {}, context);
await setters.set(3, context);
// The reader's backoff is a second at least: the composition retried it.
// Woken by the composition: no timer of its own.
const reader = await host.get(Reader, {}, context, { readyTimeout: 500 });
held.release();
expect(reader.total.value).toBe(3);
await host.close(context);
});
it("host.registry before the publication waits for it, then resolves with the provider it served", async () => {
const host = createHost({ onError: () => {} });
const waiting = host.registry(Sum, context);
await host.load([{ name: "@app/sum", facet: defineFacet((h) => h.serve(Sum, { compose: total })) }], context);
expect((await waiting).value).toEqual({ total: 0 });
await host.close(context);
});
it("host.registry waits through no value until a first good composition, woken by it, its wait then gone", async () => {
const { host } = await loadedPlugins({
"@app/sum": (host) => {
host.serve(Sum, {
compose: (table) => {
const sum = total(table);
if (sum.total === 0) throw new Error("empty");
return sum;
},
});
setter(host, 0);
},
});
const waiting = host.registry(Sum, context, { readyTimeout: 500 });
await flushed();
await (await host.get(Setter, {}, context)).set(5, context);
expect((await waiting).value).toEqual({ total: 5 });
await host.close(context);
});
it("once live, a later throwing `compose` changes nothing for the reader: `.value` keeps the last good one", async () => {
const { host } = await loadedPlugins({
"@app/sum": (host) => {

171
packages/chord/test/next/remote/hold.test.ts
View file

@ -2,13 +2,13 @@ import { describe, expect, it, vi } from "vitest";
import { type Context, BACKGROUND_CONTEXT as context } from "../../../src/context/index.ts";
import type * as Delta from "../../../src/delta/index.ts";
import { decoder } from "../../../src/delta/index.ts";
import { bool, int, local, method, obj, remote } from "../../../src/next/contracts/index.ts";
import { bool, hostKind, int, local, method, obj, remote } from "../../../src/next/contracts/index.ts";
import { broken } from "../../../src/next/data/index.ts";
import { defineState } from "../../../src/next/documents/index.ts";
import { createHost, defineFacet, type Host, type Self } from "../../../src/next/host/index.ts";
import type { ChordHost } from "../../../src/next/host/internal.ts";
import { type Link, memoryLinkPair } from "../../../src/next/link/index.ts";
import { jsonPart } from "../../../src/next/link/internal.ts";
import { binaryCodec, jsonPart } from "../../../src/next/link/internal.ts";
import { Hold } from "../../../src/next/remote/hold.ts";
import { constant } from "../../../src/next/state/index.ts";
import {
@ -102,7 +102,9 @@ function providerObject(server: unknown) {
const host = server as ChordHost<TestPrincipal>;
const { name: type, version, fingerprint } = Halves;
const request = { type, version, fingerprint, params: "{}", address: SERVER_ADDRESS };
return host.serving(request, {}, { kind: "owner" }, context);
const served = host.serving(request, {}, { kind: "owner" }, context);
if (served.kind !== "object") throw new Error("Halves is served here");
return served.object;
}
/** A remote type whose activation never completes: its open waits for its abort. */
@ -457,6 +459,65 @@ forEachCodec(() => {
});
});
describe("holds that bind to nothing yet (dependency model §1, §3)", () => {
const Absent = { name: "app.absent", version: 1, fingerprint: "0123456789abcdef" };
it("a refused hold waits failed { waiting }, counted; a retry re-runs its start (loading, failed); its link's close drops it", async () => {
const { server, peer } = await rawLinked(() => {});
hold(peer, 0, Absent);
const failed = await peer.next("failed");
expect(failed).toMatchObject({ hold: 0, cause: { kind: "waiting" } });
expect(failed.retryIn).toBeUndefined();
expect(server.activity.value.references).toBe(1);
expect(server.references(Halves).value).toEqual([]);
peer.send({ kind: "retry", hold: 0 });
expect(await peer.next("loading")).toEqual({ kind: "loading", hold: 0 });
expect(await peer.next("failed")).toMatchObject({ hold: 0, cause: { kind: "waiting" } });
peer.close();
await transported();
expect(server.activity.value.references).toBe(0);
});
it("a hold on an object parked on a failed dependency is told each new cause, then loading, then bound", async () => {
const Core = local("app.core", { version: 1, params: {}, members: { n: int() } });
let cores = 0;
const { peer } = await rawLinked((h) => {
h.serve(Core, () => {
if (++cores < 3) throw new Error(`bug ${cores}`);
return { n: constant(cores) };
});
h.serve(Halves, (_, _principal, self) => {
const core = self.get(Core, {});
const n = self.state(N);
return {
variant: "v",
members: {
n,
half: self.derive([core.n], ([c]) => c),
set: async () => null,
wait: async () => null,
},
};
});
});
hold(peer, 0, Halves);
const waiting = (message: string) => ({
kind: "failed",
hold: 0,
cause: { kind: "waiting", error: { kind: "chord", error: { code: "unavailable", message } } },
});
// The core failed at load (its first outcome): the parked object names it at once.
expect(await peer.next("failed")).toEqual(waiting("waiting for app.core: app.core: bug 1"));
// A retry passes on through the parked object to its broken dependency (one host's retry operation).
// (Its new attempt failed within the batch of changes the parked object checks once: no loading between.)
peer.send({ kind: "retry", hold: 0 });
expect(await peer.next("failed")).toEqual(waiting("waiting for app.core: app.core: bug 2"));
peer.send({ kind: "retry", hold: 0 });
await peer.next("loading");
expect((await peer.next("bound")).holds[0]).toMatchObject({ hold: 0 });
});
});
describe("a reconnecting consumer", () => {
const Tally = remote("app.tally", {
version: 1,
@ -715,3 +776,107 @@ forEachCodec(() => {
});
});
});
/** Lets every pending reaction run (a memory link delivers by microtask). */
async function flush(): Promise<void> {
for (let i = 0; i < 100; i++) await Promise.resolve();
}
/** A local type nobody serves. */
const Missing = local("app.missing", { version: 1, params: {}, members: { n: int() } });
describe("retry operations across links (fix rounds 1 and 2, Sol 3)", () => {
it.each([1, 9, 50])(
"quiesce after %i concurrent Retries round a waiting ring across hosts: each stops after one lap",
async (operations) => {
type P = { kind: "server" } | { kind: "worker" };
const Server = hostKind("app.retry-server");
const Worker = hostKind("app.retry-worker");
const LocalA = local("app.a", { version: 1, params: {}, members: { n: int() } });
const LocalB = local("app.b", { version: 1, params: {}, members: { n: int() } });
const RemoteA = remote("app.remote-a", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { reader: { state: { n: int() } } },
});
const RemoteB = remote("app.remote-b", {
version: 1,
params: {},
hostAddress: () => Worker(),
variants: { reader: { state: { n: int() } } },
});
let retries = 0;
const count = (link: Link): Link => ({
send: (bytes) => {
if (binaryCodec.decode(bytes).kind === "retry") retries++;
link.send(bytes);
},
onMessage: (listener) => link.onMessage(listener),
onClose: (listener) => link.onClose(listener),
close: () => link.close(),
});
const [down, up] = memoryLinkPair();
const server = createHost<P>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
const worker = createHost<P>({ address: Worker(), principal: { kind: "worker" }, onError: () => {} });
server.route(
Worker,
{
authorize: () => true,
resolve: async () => ({ link: count(down), principal: { kind: "worker" }, trusted: true }),
},
context,
);
worker.connect(count(up), { principal: { kind: "server" }, trusted: true }, context);
try {
const left = server.load(
[
{
name: "@test/a",
facet: defineFacet<P>((h) => {
h.serve(LocalA, (_, self) => ({ n: self.get(RemoteB, {}).n }));
h.serve(RemoteA, (_, _principal, self) => ({
variant: "reader",
members: { n: self.get(LocalA, {}).n },
}));
}),
},
],
context,
);
const right = worker.load(
[
{
name: "@test/b",
facet: defineFacet<P>((h) => {
h.serve(LocalB, (_, self) => {
self.get(Missing, {});
return { n: self.get(RemoteA, {}).n };
});
h.serve(RemoteB, (_, _principal, self) => ({
variant: "reader",
members: { n: self.get(LocalB, {}).n },
}));
}),
},
],
context,
);
await Promise.all([left, right]);
await flush();
const a = server.hold(LocalA, {}, context);
expect(a.state.value).toMatchObject({ kind: "failed", cause: { kind: "waiting" } });
// More at once than an object remembers ids: `through` stops each at the object it passed through already.
for (let i = 0; i < operations; i++) a.retry();
await flush();
// Each went round once: server → worker → server, then stopped at the object it started from.
expect(retries).toBe(2 * operations);
await flush();
expect(retries).toBe(2 * operations);
} finally {
await server.close(context);
await worker.close(context);
}
},
);
});

16
packages/chord/test/next/remote/provider.test.ts
View file

@ -117,14 +117,14 @@ forEachCodec(() => {
{ fingerprint: "0000000000000000" },
{ code: "contract_mismatch", message: "app.notes v1 does not match the served v1" },
],
])("refuse %s with a broken failed", async (_, override, error) => {
])("keep %s refused, failed { waiting }", async (_, override, error) => {
const { peer } = await notesServer();
const { type, ...rest } = override as { type?: string };
hold(peer, 0, { ...Notes, name: type ?? Notes.name } as Contract, { session: "S1" }, rest);
expect(await peer.next("failed")).toMatchObject({
hold: 0,
cause: { kind: "broken", error: { kind: "chord", error } },
});
const failed = await peer.next("failed");
expect(failed).toMatchObject({ hold: 0, cause: { kind: "waiting", error: { kind: "chord", error } } });
// No backoff window: an event (a publication) wakes it.
expect(failed.retryIn).toBeUndefined();
});
it("refuse invalid params with a broken failed", async () => {
@ -135,7 +135,7 @@ forEachCodec(() => {
});
});
it("arriving before load resolved wait for it", async () => {
it("arriving before the publication wait for it, and bind before the start barrier is over", async () => {
const server = createHost<TestPrincipal>({
address: Server(),
principal: { kind: "server" },
@ -505,12 +505,12 @@ forEachCodec(() => {
});
describe("local types over a link", () => {
it("never cross one", async () => {
it("never cross one: a hold naming one is not served (failed { waiting })", async () => {
const Store = local("app.store", { version: 1, params: {}, members: { n: int() } });
const { peer } = await rawLinked((h) => h.serve(Store, () => ({ n: constant(1) })));
hold(peer, 0, Store, {});
expect(await peer.next("failed")).toMatchObject({
cause: { kind: "broken", error: { kind: "chord", error: { code: "unavailable" } } },
cause: { kind: "waiting", error: { kind: "chord", error: { code: "unavailable" } } },
});
});
});

94
packages/chord/test/next/remote/remote-object.test.ts
View file

@ -302,16 +302,20 @@ forEachCodec(() => {
});
});
describe("self.get of a type over the link (R18)", () => {
it("a failed hold fails the dependent transiently; bound retries it at once with a new hold", async () => {
describe("self.get of a type over the link (dependency model §3)", () => {
it("a failed hold parks the dependent failed { waiting }: no new attempt or hold; bound wakes it", async () => {
const View = local("app.view", { version: 1, params: {}, members: { count: int() } });
const errors: HostError[] = [];
const client = createHost({ onError: (error) => errors.push(error) });
const [a, b] = memoryLinkPair();
client.connect(a, { principal: { kind: "server" }, codec: testCodec() }, context);
const peer = new RawPeer(b);
let factories = 0;
const facet = defineFacet((h) =>
h.serve(View, (_, self) => ({ count: self.get(Notes, { session: "S1" }).count })),
h.serve(View, (_, self) => {
factories++;
return { count: self.get(Notes, { session: "S1" }).count };
}),
);
const loading = client.load([{ name: "@test/view", facet }], context);
const { hold } = await peer.next("hold");
@ -320,25 +324,20 @@ forEachCodec(() => {
hold,
cause: { kind: "transient", error: { kind: "chord", error: { code: "unavailable", message: "warming" } } },
});
// Parked on a remote blocker: its first outcome, so the start barrier is over.
await loading;
expect(errors).toMatchObject([
{
kind: "activation_failed",
object: "app.view",
failure: { kind: "transient", error: { code: "unavailable" } },
},
]);
const members = [
{ member: EDITOR.state.notes!, value: "[]" },
{ member: EDITOR.state.count!, value: "0" },
];
peer.send({ kind: "bound", holds: [{ hold, variant: EDITOR.variant, members, failed: [] }] });
// The retry binds its own hold before the failed attempt's is released.
const retried = await bindNext(peer, [{ text: "a" }]);
expect(retried).toBe(1);
expect(await peer.next("release")).toMatchObject({ hold });
const view = await client.get(View, {}, context);
await transported();
expect(errors).toEqual([]);
const view = client.hold(View, {}, context);
expect(view.state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.notes(S1): warming" } },
});
bindHold(peer, hold, [{ text: "a" }]);
await transported();
expect(view.state.value).toEqual({ kind: "live" });
expect(view.count.value).toBe(1);
expect([factories, peer.kinds().filter((kind) => kind === "hold" || kind === "release")]).toEqual([1, []]);
});
});
@ -370,7 +369,7 @@ forEachCodec(() => {
// Timers stand still: neither the backoff window nor a readyTimeout ends while the test runs.
fakeTimers();
it("fails at once with the real cause when its load holds it (no wait for a timeout)", async () => {
it("parks at once on the real cause when its load holds it (no wait for a timeout)", async () => {
const Flaky = remote("app.flaky", {
version: 1,
params: {},
@ -399,16 +398,12 @@ forEachCodec(() => {
second.connect(b, { principal: { kind: "server" }, codec: testCodec() }, context);
const facet = defineFacet((h) => h.serve(View, (_, self) => ({ n: self.get(Flaky, {}).n })));
await second.load([{ name: "@test/view", facet }], context);
expect(errors).toMatchObject([
{
kind: "activation_failed",
object: "app.view",
failure: {
kind: "transient",
error: { message: "app.view: dependency app.flaky failed", cause: { message: "app.flaky: offline" } },
},
},
]);
await transported();
expect(errors).toEqual([]);
expect(second.hold(View, {}, context).state.value).toMatchObject({
kind: "failed",
cause: { kind: "waiting", error: { message: "waiting for app.flaky: app.flaky: offline" } },
});
await first.close(context);
await failing;
});
@ -633,7 +628,7 @@ forEachCodec(() => {
const toUpgraded = await relink(upgraded, links[0]);
expect(notes.state.value).toMatchObject({
kind: "failed",
cause: { kind: "broken", error: { code: "contract_mismatch" } },
cause: { kind: "waiting", error: { code: "contract_mismatch" } },
retryAt: undefined,
});
notes.retry();
@ -652,7 +647,7 @@ forEachCodec(() => {
await transported();
expect(notes.state.value).toMatchObject({ kind: "failed", cause: { kind: "broken" } });
notes.retry();
expect(await peer.next("retry")).toEqual({ kind: "retry", hold });
expect(await peer.next("retry")).toEqual({ kind: "retry", hold, operation: expect.any(String) });
bindHold(peer, hold);
await transported();
expect(notes.state.value).toEqual({ kind: "live" });
@ -771,15 +766,6 @@ forEachCodec(() => {
h.serve(View, (_, self) => ({ count: self.get(Notes, { session: "S1" }).count })),
);
/** Binds the kept hold (0) on `next`; the retry's own hold (1) is bound and the kept one released (R18). */
async function retried(client: Host<TestPrincipal>, next: RawPeer): Promise<void> {
expect(await bindNext(next, [{ text: "a" }])).toBe(0);
// Retried at once, long before its backoff window (1 s) ends: its new attempt holds anew.
expect(await bindNext(next, [{ text: "a" }])).toBe(1);
expect(await next.next("release")).toEqual({ kind: "release", hold: 0 });
expect((await client.get(View, {}, context, { readyTimeout: 1 })).count.value).toBe(1);
}
it("stays loading through the drop, no failure reported, and goes live when the next link binds it", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ onError: (error) => errors.push(error) });
@ -798,28 +784,32 @@ forEachCodec(() => {
expect(errors).toEqual([]);
});
it("dropped and never bound again, fails transient at its readyTimeout (the bound of the wait)", async () => {
it("dropped and never bound again, stays parked loading: nothing fails, no parked_long without a link", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const peer = await rawParent(client);
const loading = client.load([{ name: "@test/view", facet }], context);
await peer.next("hold");
// A remote blocker ends the start barrier's first pass.
await loading;
peer.close();
await transported();
await vi.advanceTimersByTimeAsync(10_000);
// No remote attempt answers for it while its path has no link (fix round 1, Opus F2).
expect(errors).toEqual([]);
await vi.advanceTimersByTimeAsync(1_000);
await loading;
expect(failures(errors)).toEqual(["app.view: transient"]);
expect(client.hold(View, {}, context).state.value).toEqual({ kind: "loading" });
});
it("without any link fails at its readyTimeout, keeps the reference, and is retried when the first link binds it", async () => {
it("without any link stays loading past its readyTimeout and goes live when the first link binds its one hold", async () => {
const errors: HostError[] = [];
const client = createHost<TestPrincipal>({ readyTimeout: 1_000, onError: (error) => errors.push(error) });
const loading = client.load([{ name: "@test/view", facet }], context);
await vi.advanceTimersByTimeAsync(1_000);
await loading;
expect(failures(errors)).toEqual(["app.view: transient"]);
await retried(client, await rawParent(client));
await client.load([{ name: "@test/view", facet }], context);
await vi.advanceTimersByTimeAsync(500);
expect(failures(errors)).toEqual([]);
const next = await rawParent(client);
expect(await bindNext(next, [{ text: "a" }])).toBe(0);
expect((await client.get(View, {}, context, { readyTimeout: 1 })).count.value).toBe(1);
expect(next.kinds().filter((kind) => kind === "hold" || kind === "release")).toEqual([]);
});
});

27
packages/chord/test/next/store/table-migration.test.ts
View file

@ -218,18 +218,33 @@ describe("eager migrations", () => {
});
});
/**
* Records the `setTimeout` calls the migration's own code makes (its stack passes through `table-migration.ts`); timers
* of the test runner or other work under load pass through unrecorded (fix round 2: the test watched every timer).
*/
function migrationTimers(): { readonly calls: string[]; restore(): void } {
const calls: string[] = [];
const original = globalThis.setTimeout;
const spy = vi.spyOn(globalThis, "setTimeout").mockImplementation(((...args: Parameters<typeof setTimeout>) => {
const stack = new Error().stack ?? "";
if (stack.includes("src/next/store/table-migration.ts")) calls.push(stack);
return original(...args);
}) as typeof setTimeout);
return { calls, restore: () => spy.mockRestore() };
}
describe("the chunk yield", () => {
it("yields between chunks without Node's setImmediate (Workers, browsers), through a message, not a timer", async () => {
await filled(3 * MIGRATION_CHUNK);
const saved = globalThis.setImmediate;
// Runtimes without it: Cloudflare Workers, browsers.
Reflect.deleteProperty(globalThis, "setImmediate");
const timers = vi.spyOn(globalThis, "setTimeout");
const timers = migrationTimers();
try {
await prepared(EditsV2);
expect(timers).not.toHaveBeenCalled();
expect(timers.calls).toEqual([]);
} finally {
timers.mockRestore();
timers.restore();
globalThis.setImmediate = saved;
}
expect(store.table(TABLE)).toMatchObject({ version: 2, state: "ready" });
@ -242,12 +257,12 @@ describe("the chunk yield", () => {
setTimeout(() => {
fired = true;
}, 0);
const timers = vi.spyOn(globalThis, "setTimeout");
const timers = migrationTimers();
try {
await prepared(EditsV2);
expect(timers).not.toHaveBeenCalled();
expect(timers.calls).toEqual([]);
} finally {
timers.mockRestore();
timers.restore();
}
expect(fired).toBe(true);
expect(store.table(TABLE)).toMatchObject({ version: 2, state: "ready" });