pi-mono/packages/chord/docs/slice-2.md

Slice 2: failures and natives (spec, for the owner's approval)

Status: approved (owner, 2026-10-22), after Astra's and Fable's spec reviews. The single source for building slice 2, together with the feature map in slices.md (entries assigned to slice 2), design.md read in full (most relevant: §3 timeouts, §6 lifecycle and failures, §7 references, §13 natives, §21 start barrier), the decision log, the carry-over (L4 section) and the root AGENTS.md style section. Builds on slice 1 (src/state, documents, host, committed d8f5e67e6) and L1.

What an app can do after this slice

host.serve(Inbox, (_, self) => {
  const settings = self.get(Settings, {});
  const watcher = self.open(
    async (context) => watchDir(settings.dir.value.path, context.abortSignal),   // opened during activation
    async (w, reason) => w.close(),                                               // closed in reverse when the object ends
  );
  return { count: …, rescan: async (ctx) => watcher.value.rescan(ctx) };
});

host.serve(Remote, (_, self) => {
  self.open(async (context) => {
    const config = readConfig();
    if (!config) throw broken(new Error("no remote configured"));   // retrying is pointless: broken
    return connect(config, context.abortSignal);                    // a throw here is transient: retried with backoff
  }, (c) => c.close());
  return { … };
});

const host = createHost({ onError, closeTimeout: 30_000, readyTimeout: 60_000 });
await host.load(facets, ctx);          // resolves once every host-lifetime object is live or failed; failures reported
const inbox = await host.get(Inbox, {}, ctx);   // waits through transient retries up to readyTimeout
await host.close(ctx);                 // closes natives (reason "stop"); rejects with every close error

1. Natives (§13)

• self.open(open, close, options?) in a factory returns a handle (Native<T>). Its value is readable from the moment its own open resolved until its close starts (so a later opener and handlers during the drain can use it); before that it throws not_ready, after that released.
• open(context) receives a context whose abort signal fires when the attempt is abandoned (the object ends, the attempt times out). close(value, reason, context) with reason: { kind: "end" } | { kind: "stop" } | { kind: "reload" } (end: last reference gone or a failed attempt; stop: host.close; reload is used from slice 7); the close context's signal fires at the native's closeTimeout, so a slow close can stop its own work. options.closeTimeout overrides the host default for this native.
• Activation order: factory; singleton documents hydrated; references live; natives opened one after another in declaration order; members checked; derives computed; live. An opener may read documents, references and earlier natives, not derives (they are computed after the opens).
• Closing: in reverse order. Each close is awaited up to its closeTimeout; one that takes longer is reported to onError as a possible leak and the others continue. A throwing close does not stop the others: host.close collects every close error and rejects with them (AggregateError); an object ending in the background reports each to onError.
• Abandoned opens: a failed or abandoned attempt closes the natives already open (reason end). An open that resolves after its attempt was abandoned has its value closed at once, exactly once (reason end); an abandoned attempt never comes back to life. A later attempt opens every native anew.
• Facets cannot open natives (FacetSelf has no open). The two restart patterns of R16 (drain in the background, swap through a registry) are documentation in the host README; there is nothing to build.

2. Failures (§6)

• Every activation failure becomes a FailureCause (L1: { kind: "transient" | "broken"; error: ChordError }, foreign errors wrapped by asChordError). The kind comes from where it failed, never from the error:

  Where	Kind
  factory throws; a singleton's initial() fails its schema; member check (incl. a native watchable as state, Q14); first derive compute; a dependency cycle (Q5)	broken
  an open throws	transient, unless it threw broken(error) (L1)
  the attempt did not become live within readyTimeout	transient: unavailable "did not become live within …"
  a dependency failed (R18)	transient: unavailable "A: dependency B failed", with B's error as its cause

  A keyed document's first initial() runs inside a later transaction; a failure there fails that transaction, not the object.

• Every attempt is bounded by the host's readyTimeout: when it passes, the attempt is abandoned (openers' contexts aborted, open natives closed end, a late open closed at once) and fails transient. A hanging open can therefore never block an object or the start barrier.

• What a failed object keeps: only an R18 failure keeps the object's references (so the dependency's recovery can retry it); every other failure releases them at once (a cycle, a factory, an open, a derive). A failed object stays while anything references it (host-lifetime objects always); it ends with its last reference.

• Retries. One attempt per object at a time; a trigger arriving during an attempt is ignored. Transient: retried with a backoff per identity, 1 s doubling to 60 s. The backoff lives in a host map identity → { delay, timer } holding no object (the window is its timer); it is deleted on success, on a broken failure, or when a window has passed with no object left to retry, and cleared by host.close; while its object stays failed the entry keeps the delay for the next doubling. A get inside the window waits until until before the first attempt; a failed object without references is not retried (host-lifetime objects always are). Broken: no automatic retry and no entry; a new object after its end tries at once. In this slice a broken host-lifetime object stays failed until the host is recreated (reload, slice 7, and retryOn, slice 8, add triggers).

• Dependency recovery (R18): when a failed dependency goes live, its R18 dependents are retried at once (their backoff timers cancelled). On every retry the new attempt binds its self.get references before the previous attempt's are released, so a dependency held only by this object never ends between attempts.

• Reporting: every failed attempt is reported to onError (with the object's name and cause); gets waiting for it are rejected in addition (section 3).

• The start barrier (§21): load resolves when every host-lifetime object's first attempt has an outcome (live or failed); later retries never hold load. A facet setup that throws or a facet document's initial() failing its schema still rejects load (code errors).

3. Getting a failing object (§7)

• host.get(Type, params, context, { readyTimeout? }) resolves with a live reference. Each getter has its own deadline (readyTimeout, host default, overridable per get) for its whole wait, including a wait for load; it is separate from the attempt's own bound. A broken cause rejects the getter at once with its error; a transient one keeps it waiting through retries until the object is live or its deadline passes, then rejects with the last cause's error (or unavailable if no attempt has failed yet). A getter whose deadline passes or whose context is cancelled rejects and releases only its own reference; it never cancels the attempt for other getters.

4. Ending with natives (§3, §6)

• When an object ends: its transactions are sealed at once (self.tx throws released); callers of its running calls are rejected released at once and the handlers' contexts aborted (slice 1); Chord keeps each running handler's own promise and waits for those to settle (the drain), bounded by closeTimeout; then it closes its natives in reverse order (a handler still running after closeTimeout sees its natives close; that is reported); its documents stay readable until its closes are done, then are dropped; then the rest as in slice 1.
• A new attempt or a new object for the same identity starts only after the previous instance's ending has finished; gets arriving meanwhile wait.
• host.close(context): marks every object ended at once (every call on the host fails released, every self.tx throws), aborts pending opens (their late values are closed), joins endings already running in the background, then drains and closes object by object in reverse live order (a dependent before its dependencies), with reason stop; resolves when all are done, or rejects with the collected close errors.

5. Timeouts (§3, cut C13)

Host options with per-thing overrides: closeTimeout (default 30 s; per native), readyTimeout (default 60 s; per get; also bounds each attempt). The call drain uses closeTimeout (no separate setting; the design's per-type drain override is not built). Park deadline (slice 3), routine handover and finish retry (slice 5) and rollout install (slice 7) come with their slices.

Cost per hop

Commits and derives are unchanged. A call adds one set entry for its handler's promise (the drain needs it). Activation adds one awaited open per native and one attempt timer; a failure adds one backoff entry; ending adds the drain and the closes.

Carry-over taken by this slice

From carry-over.md (L4): broken()/openFailure() used for opens; a native watchable as state fails broken naming the member; the drain on ending; cuts C5, C6, C7, C13 decided here.

Tests

• Natives: open order and access to earlier handles; not_ready before live; close in reverse with reasons end and stop; a slow close reported after closeTimeout and the others still closed; throwing closes collected by host.close (AggregateError) and reported to onError for a background end; a partly failed attempt closes what it opened; the opener's context aborted when the object ends while opening; facets have no open.
• Failures: each row of the table gives the right kind; the cause's error is a ChordError; transient retries follow the backoff (fake timers) and reset on success; concurrent gets share one attempt; a broken failure is not retried; a failed object ends with its last reference and its backoff entry expires; a get right after a failure waits for the scheduled retry.
• R18: a dependent of a failing dependency fails transient with the dependency's cause, keeps the reference and is retried at once when the dependency goes live; releasing the dependent releases the dependency.
• Start barrier: load resolves with a failed host-lifetime object, reports it to onError, and the object goes live on a later retry; a throwing facet setup still rejects load.
• Gets: broken rejects at once; transient waits and succeeds on retry; transient past readyTimeout rejects with the cause; per-get readyTimeout; a cancelled caller releases only its own reference.
• Ending: the drain waits for a running handler (its own promise, not the rejected caller's) before its native closes, bounded by closeTimeout; documents readable during closes; transactions sealed at once.
• Races: an open resolving after its attempt was abandoned is closed once; a get arriving while the previous instance closes waits and gets a fresh attempt; simultaneous retry triggers give one attempt; two getters with different deadlines; a dependency held only by a retrying dependent is not ended between attempts; host.close during a pending open, during a background ending, and with several objects in dependency order; a cycle releases its references.
• GC: a failed object (documents, dependencies, natives, callbacks) is collected after its last reference while its backoff entry still exists; the entry expires on its own.
• Examples: 06-natives.ts, 07-failures.ts (numbering continues from slice 1).

Decided with the owner (2026-10-22)

• ref.state, ref.retry() and host.hold move to slice 3 (no app-held reference can be failed before then).
• host.get on a failing object and the start barrier as in sections 2 and 3; C13 as in section 5; C5: no natives in facets; openers take a Context; close gets a context aborting at closeTimeout.
• C7: linger is built in slice 3, with links. The plugin error shape is decided in slice 3, where errors first cross a link.

Size

About 700 source lines (both reviewers: the attempt states, backoff map, drain and timers).