feat(chord): a call's answer follows its state

Before a provider sends or forwards a call's result or error, it sends the origin's
waiting state, past the window if it is full, so a resolved call's changes are
applied on the page. TuiScope.optimistic shows a value while calls are pending; the
guide shows useOptimistic over a Chord reference; example 63.
This commit is contained in:
Mario Zechner 2026-11-05 10:45:39 +00:00
parent af79b0c3d7
commit d7a988d18d
13 changed files with 605 additions and 53 deletions

View file

@ -26,6 +26,7 @@
- Chord v2 gains telemetry across hosts (slice 9): `CHORD_TELEMETRY_SCHEMA` (root export) types every span, event and metric Chord records. With a host adapter (`createHost({ telemetry })`), calls (`chord.call`), methods (`chord.method`, local and remote), relays (`chord.relay`), holds (`chord.hold`), activations (`chord.activate`, ending `live`, `failed`, `parked` or `ended`, with `chord.retry` and `chord.park` events), natives' opens and closes, `load`, `reload` (a `chord.reload.fate` event per object), resolves (a worker start included), revocations, routine calls, job phases (`intent`, `run`, `finish`: a persisted run resumed in a new attempt or process stays under its stored carrier) and HTTP requests (`chord.http`: method, route pattern, status, never the path) are spans; failures carry a fixed category and the message cut at 500 characters, a cancel is no error; commits are `chord.commit { documents, ops, rows, duration }` events of the span they ran under. `hold` and `call` carry the caller's trace carrier (`trace`, at most 8 keys, keys and values of at most 256 characters of printable ASCII, else the message fails the message schema and the link closes; the binary codec's kind byte flag `0x40`), relays forward their own span's, and the provider runs the method under it (an absent or unknown carrier: the adapter's root); a reference's carrier rides on its first hold only, a re-hold after a reconnect being a fresh root linked to it. A hold's `type` and its address's `kind` are bounded at 256 characters by the message schema, and span attributes a peer chose at 256 characters. `ctx.trace(name, attributes, fn)` runs `fn` in a child span with a child ctx. Host-wide metrics: `chord.link.bytes`, `chord.link.state_messages`, `chord.commit.duration`, `chord.calls.in_flight`, `chord.holds.active`, `chord.links.open`, `chord.park.duration`, `chord.backpressure.wait`. Without an adapter nothing is started, built or sent: a call over a link costs what it did before. `createHttp`, `startServer` and `startWorker` take a `telemetry` adapter (`createHttp` continues a W3C `traceparent` only from a request a contributed relay verified: an app's proxy must strip it from browsers). A host with an adapter counts the cost counters of its links' and root scopes' work on it also when the app passed a context without a span. The SQLite telemetry store keeps spans of one microsecond in the order they started.
- Chord v2's telemetry closes four gaps found writing the tutorial: `chord.commit` counts the table rows a commit's writes changed (`rows`; a commit writing only rows no longer reads `documents=0 ops=0`); every span Chord starts, `ctx.trace`'s too, carries `chord.host`, the host's address key or `client` without an address (`Host` gains `address`); the SQLite telemetry store keeps span times in whole microseconds from pi-telemetry's `telemetryNow` (store version 2: a recorder migrates a version 1 file in place, its rows kept); and `openTelemetryStore(path)` (new subpath `@earendil-works/chord/telemetry-sqlite/node`) reads a store file read-only, implementing `TelemetryStore` only, refusing a file without a store or with one of another version. `openNodeSqliteDatabase` takes `mode: "read-only"`.
- Chord v2's telemetry is stored in one place, the server (decisions "Telemetry from browser pages", "The server is the one place telemetry is stored", "Who a span was for"). Pages and workers record in memory with Chord's `TelemetryRecorder` (the SQLite store's own recorder too) and send finished spans in batches over their link to the server's `PageTelemetry` and `WorkerTelemetry` (`ingest(batch)`), which the app kit serves when `startServer` has a telemetry store; the store gains `ingest` (never replacing a stored span). `startPage({ telemetry: true })` records the page and sends its spans every 3 s and when hidden; the server checks a page's batch strictly (at most 100 spans, every field bounded, ids printable ASCII, no control characters), keeps only spans under the page's own ids (`browser:<its incarnation>:<n>`), caps them at 1,200 per page and minute, marks them `chord.host: browser` and stamps them with the principal of their link (a page's own `chord.*` attributes are dropped). A worker sends its spans and metrics every second, stamped with its host. `host.root(fn, ctx, { trace: { name, attributes } })` runs `fn` in a new root span (a click). Method and call spans record whom they were for: `chord.principal` (the principal key) by default, or what the host's new `principalAttributes` hook returns (`createHost`, `startServer`, `startWorker`). `ServeOptions.untraced` serves a type whose calls start no method span. `Host` gains `incarnation` and `principalAttributes`.
- Chord v2's `TuiScope` gains `optimistic(read)`: the reader and `show(value, call)`, which shows `value` in place of the reader's until every call shown settled (React's `useOptimistic` for terminal UIs). The guide's "Optimistic UI" shows React's `useOptimistic` and `startTransition` over a Chord call, what it relies on and what it does not cover; example `63-optimistic-answers.ts`.
### Changed
@ -36,6 +37,7 @@
- Chord v2's `host.reload` starts the objects' fates dependents first: removals, then the other fates in the reverse of the order the objects went live (the report keeps its order). A pure dependent reloaded with its provider (a view over a store in one plugin) binds the provider while it is still live and swaps instead of parking until the provider's new instance is live: its contributions are replaced in one batch and its holds over links never show `loading`. A dependent still parks when its new code needs the provider's new version, when its provider went live after it, or when it needs something not live.
- Chord v2 keeps calls over a link across a `loading` unbind (decision "Calls in flight survive a `loading` unbind"): a hold whose object restarts, whose swap parks or whose live failure retries no longer answers its calls in flight `unavailable`; each is answered by the instance it already runs on or by the new instance once it is live (or by its failure, `readyTimeout`, its end), and calls a consumer sent by the old names before it saw the `loading` wait the same way instead of `not_ready`. A `failed` unbind (a removed type, a dropped version, a swap parked on a failed dependency) still rejects them.
- Chord v2's store writes the commits queued behind a running commit step in one backend transaction (group commit, decision S18): each keeps its own seq, outbox commit and adoption in queue order, the metadata is written once; a refusal in a group rolls it back and runs its commits one by one, so only the refusing commit is refused.
- Chord v2 sends a call's answer after its state (decision "A call's answer follows its state"): before a host sends a `result` or `error`, its own or one it forwards as a relay, it sends the origin's state still waiting for a slot of the window (one `state` message, past the window if it is full; acknowledged like any other). A call settling in a page now finds every change its answering host, and a relaying host, committed before answering, so an optimistic value is replaced by the real one instead of flickering back to the old value. Nothing extra is sent when nothing waits.
### Fixed

View file

@ -382,7 +382,22 @@ host.route(Session, {
- a hold that binds waits for a slot; its snapshot and subscription are taken in one synchronous step when its
`bound` is sent; bindings and waiting state take turns when both wait, so neither starves;
- a hold whose object leaves `live` while its `bound` waits goes back to pending (D1, owner, 2026-10-24; built in
slice 7b with the reload, which makes a live object leave `live`); so does a bound one, unbound (§7, §22).
slice 7b with the reload, which makes a live object leave `live`); so does a bound one, unbound (§7, §22);
- **a call's answer follows its state** (decision "A call's answer follows its state", owner, 2026-11-05): before
the provider sends a call's `result` or `error` (its own, or one it forwards as a relay), it sends the origin's
waiting state (every hold with pending ops, one `state` message), past the window if it is full. So when the call
settles at the consumer, every change the answering host committed before answering is applied there, and an
optimistic value (React's `useOptimistic`, `TuiScope.optimistic`) is replaced by the real one, never by the old.
Trace: window full (`bound`, `state` unacknowledged), a call commits (its ops wait), its `result`: the provider
sends `state` (the waiting ops), then `result`; without the rule the result overtook its own commit and the page
showed the old value until the next `ack`. A message past the window is counted and acknowledged like any other
(once, in order; the window opens only when fewer than K are in flight again); there is at most one per answer, and
its consumer acknowledges it before it sends the call that answer frees a slot for, so running calls and messages
past the window stay within N per origin together. Nothing extra is sent when nothing waits (the common path: a
free window sent the commit's `state` already). Bindings waiting for a slot are not sent past it: their `bound`
carries the latest state when it goes (a hold rebinding after a reload's swap shows its old values until then).
Not covered: changes committed after the answer (a method that returns before its effect lands) or on another host
that does not relay the answer.
Not built, by decision: a byte window, credit messages, snapshot admission or debt, an adapter `backlog()` API,
`reset` from anyone.
- **Calls: a call window N** (default 256, the same on every host) announced in `hello`: how many concurrent calls a
@ -433,7 +448,10 @@ drops, folds or resets.
`result` and `error` by its call is checked against the relay's table; an answer for a hold or call not relayed over
the link it came from is dropped (Q1: also one crossing a release). A `bound` or `state` left with some entries is
forwarded with those; left with none, it is acknowledged to that link at once. A message for an origin from a link
the origin relayed nothing to is ignored whole.
the origin relayed nothing to is ignored whole. A `result` or `error` forwarded goes after the relay's own waiting
state for that origin (the rule above: a worker's answer whose effect the worker pushed to the server arrives after
that server state); the upstream host sent its own waiting state before its answer, and the relay forwarded that at
once.
- **Acks:** the relay forwards each upstream `bound`/`state` at once and returns its `ack` to the upstream link that
sent it when the downstream acknowledges it (the senders FIFO). A tab served by W workers may have 2W messages in
flight (plus the relay's own 2), each worker's window bounding its own. A slow tab stalls only its origin at each
@ -2197,7 +2215,9 @@ validation, no clone of snapshots or ops.
provider and consumer), restarted with every snapshot. Two holds of the same principal (two tabs) encode the same
ops separately (O(change) each). A part is encoded once at its origin; relays never re-encode it.
- **Sending:** one `state` message per commit per link and origin while the window has a slot; otherwise the ops wait
in the members' pending queues and leave as one message when an `ack` frees a slot (section 3).
in the members' pending queues and leave as one message when an `ack` frees a slot, or before a call's `result` or
`error` of that origin, past the window (section 3, "a call's answer follows its state"; a relay sends its own before
each answer it forwards). No message kind or field changes.
- **Relays** read and rewrite only envelopes (section 3, "Relays").
- **Size limits per part (R2, Q2; slice 4b):** limits apply to each part (wire and decoded size), configurable per
link, checked from the part's length before it is parsed or inflated; the decoded limit is enforced in the one
@ -2474,7 +2494,8 @@ Decision "Slice 11's points", "Slice 11, P3" to "P9", and "Tutorial gaps" (1) (p
with an empty environment, `--permission` granting nothing and no SQLite; the builtin allowlist and a frozen
substitute `process` inside; a bridge terminal; the trusted launcher outside, confirming copy, open and attach on its
own prompt); a component scope (`hold`, `watch`, `registry`, released on dispose, one `changed` callback for the
app's redraw; `pageState(key, initial)`, the host's page state shared with `usePageState`), no pi-tui dependency.
app's redraw; `pageState(key, initial)`, the host's page state shared with `usePageState`; `optimistic(read)`, a
reader showing an optimistic value while a call is pending, React's `useOptimistic`, §3), no pi-tui dependency.
### Messages and languages (built after slice 11)

View file

@ -405,6 +405,57 @@ const { t } = useMessages(messages); // re-renders when th
draft.notice = messages.message("empty");
```
## Optimistic UI
A user renames a session. The call goes from the tab through the server to the session's worker, which renames it and
reports the new title to the server's index of sessions, then answers. That round trip takes a moment, and the title
should change at once. React's `useOptimistic` shows a value while a transition runs, and Chord's references fit it:
```tsx
// plugins/sessions/web.tsx
function SessionName({ id }: { id: string }) {
const sessions = useHold(Sessions, {}); // the server's index, which every list shows
const session = useHold(SessionTitle, { id }); // on the session's worker
const title = useWatch(sessions?.titles)?.[id] ?? "";
const [shown, showOptimistic] = useOptimistic(title);
const rename = (next: string) =>
startTransition(async () => {
showOptimistic(next); // drawn at once
await session?.rename(next, BACKGROUND_CONTEXT); // settles once the index has `next`
});
return <TitleField value={shown} onRename={rename} />;
}
```
When the transition ends, React drops the optimistic value and draws `title` again. That is only right if `title` is
already the new name by then. **Chord guarantees it:** a call settles in the page only after every change its answering
host committed before answering is applied there, and so does every change a host that relays the answer committed
before it forwarded it. Here the worker's own changes travel before its answer, and the server sends its index change
for this tab before it forwards the worker's answer, even on a slow connection whose window of unacknowledged state is
full (design §3). Without that, the answer could overtake the index change and the old title flickered back.
**What it does not cover:**
- Changes committed after the answer: a method that starts work and returns before it lands (a `void` call, a job, a
routine) answers first; the optimistic value goes and the old one shows until the change arrives.
- Changes on a host that neither answers nor relays the answer. A server method that calls a worker and then answers
the tab orders only the server's own changes before its answer; what the worker changed for the tab travels on its
own and may come later.
So the rule for a method that should feel instant: it awaits its own changes, the pushes to other hosts included (the
worker's `rename` awaits its report to the index), and keeps them on the host that answers or relays the answer.
A terminal UI does the same with its component scope: `scope.optimistic(read)` wraps a reader and returns it with
`show(value, call)`, which shows `value` until the call settles (until every call shown, when they overlap):
```ts
const titles = scope.watch(sessions.titles);
const [title, show] = scope.optimistic(() => titles()?.[id]);
await show(next, session.rename(next, ctx)); // title() is `next` at once, the real one when it settles
```
`test/examples/63-optimistic-answers.ts` runs this with a stalled terminal client and prints what each redraw showed.
# Plugins {#plugins lede="Move the code into a directory that can be added, edited and removed while the app runs."}
So far the facets were part of the app's own code. A **plugin** is the same code in its own directory:

View file

@ -409,6 +409,45 @@ self.<span class="tk-f">contribute</span>(<span class="tk-t">Translations</span>
<span class="tk-k">const</span> { t } = <span class="tk-f">useMessages</span>(messages); <span class="tk-c">// re-renders when the user picks another language</span>
<span class="tk-c">// plugins/notes/worker.ts: a notice as data; every client renders it with t.render(notice)</span>
draft.notice = messages.<span class="tk-f">message</span>(<span class="tk-s">"empty"</span>);</code></pre>
<h3 id="optimistic-ui">Optimistic UI</h3>
<p>A user renames a session. The call goes from the tab through the server to the session&#39;s worker, which renames it and
reports the new title to the server&#39;s index of sessions, then answers. That round trip takes a moment, and the title
should change at once. React&#39;s <code>useOptimistic</code> shows a value while a transition runs, and Chord&#39;s references fit it:</p>
<pre><code><span class="tk-c">// plugins/sessions/web.tsx</span>
<span class="tk-k">function</span> <span class="tk-t">SessionName</span>({ id }: { id: string }) {
<span class="tk-k">const</span> sessions = <span class="tk-f">useHold</span>(<span class="tk-t">Sessions</span>, {}); <span class="tk-c">// the server's index, which every list shows</span>
<span class="tk-k">const</span> session = <span class="tk-f">useHold</span>(<span class="tk-t">SessionTitle</span>, { id }); <span class="tk-c">// on the session's worker</span>
<span class="tk-k">const</span> title = <span class="tk-f">useWatch</span>(sessions?.titles)?.[id] ?? <span class="tk-s">""</span>;
<span class="tk-k">const</span> [shown, showOptimistic] = <span class="tk-f">useOptimistic</span>(title);
<span class="tk-k">const</span> rename = (next: string) =&gt;
<span class="tk-f">startTransition</span>(<span class="tk-k">async</span> () =&gt; {
<span class="tk-f">showOptimistic</span>(next); <span class="tk-c">// drawn at once</span>
<span class="tk-k">await</span> session?.<span class="tk-f">rename</span>(next, BACKGROUND_CONTEXT); <span class="tk-c">// settles once the index has `next`</span>
});
<span class="tk-k">return</span> &lt;<span class="tk-t">TitleField</span> value={shown} onRename={rename} /&gt;;
}</code></pre>
<p>When the transition ends, React drops the optimistic value and draws <code>title</code> again. That is only right if <code>title</code> is
already the new name by then. <strong>Chord guarantees it:</strong> a call settles in the page only after every change its answering
host committed before answering is applied there, and so does every change a host that relays the answer committed
before it forwarded it. Here the worker&#39;s own changes travel before its answer, and the server sends its index change
for this tab before it forwards the worker&#39;s answer, even on a slow connection whose window of unacknowledged state is
full (design §3). Without that, the answer could overtake the index change and the old title flickered back.</p>
<p><strong>What it does not cover:</strong></p>
<ul>
<li>Changes committed after the answer: a method that starts work and returns before it lands (a <code>void</code> call, a job, a
routine) answers first; the optimistic value goes and the old one shows until the change arrives.</li>
<li>Changes on a host that neither answers nor relays the answer. A server method that calls a worker and then answers
the tab orders only the server&#39;s own changes before its answer; what the worker changed for the tab travels on its
own and may come later.</li>
</ul>
<p>So the rule for a method that should feel instant: it awaits its own changes, the pushes to other hosts included (the
worker&#39;s <code>rename</code> awaits its report to the index), and keeps them on the host that answers or relays the answer.</p>
<p>A terminal UI does the same with its component scope: <code>scope.optimistic(read)</code> wraps a reader and returns it with
<code>show(value, call)</code>, which shows <code>value</code> until the call settles (until every call shown, when they overlap):</p>
<pre><code><span class="tk-k">const</span> titles = scope.<span class="tk-f">watch</span>(sessions.titles);
<span class="tk-k">const</span> [title, show] = scope.<span class="tk-f">optimistic</span>(() =&gt; <span class="tk-f">titles</span>()?.[id]);
<span class="tk-k">await</span> <span class="tk-f">show</span>(next, session.<span class="tk-f">rename</span>(next, ctx)); <span class="tk-c">// title() is `next` at once, the real one when it settles</span></code></pre>
<p><code>test/examples/63-optimistic-answers.ts</code> runs this with a stalled terminal client and prints what each redraw showed.</p>
</section>
<section class="chapter" id="plugins"><header class="opener"><div class="numeral">V</div><h2>Plugins</h2><p class="lede">Move the code into a directory that can be added, edited and removed while the app runs.</p></header>

View file

@ -13,6 +13,11 @@ peer), non-React apps never load it. No components, no styling.
re-rendering when the locale or a translation changes; before a language is followed, the first catalog.
- `usePageState(key, initial)`: `useState`'s pair, the value kept by the page (its host's life) under `key`
(`surfaces/page-state.ts`, shared with `TuiScope.pageState`), so a component remounted by a plugin reload keeps it.
- Optimistic UI needs no hook of Chord's: React's `useOptimistic` and `startTransition` over a call. A call settles in
the page after every change its answering host (and a host relaying its answer) committed before answering, so the
optimistic value is replaced by the real one, never by the old (design §3, decision "A call's answer follows its
state"; changes committed after the answer or on a host that does not relay it are not covered). The guide's
"Optimistic UI" (`docs/guide-plugins-hosts/book.md`) shows it.
- A component unmounted by a reload has not left (design §21): its releases are plain releases. A component of a
reloaded plugin is a new function and remounts, its `useState` starting over: what must survive is page state.

View file

@ -1,8 +1,8 @@
// The provider side of one origin on a link (§3, §22, W1; decision "Relay flow control per origin"): the holds the
// peer sent for that origin, served here or relayed, by their ids; the calls on them, at most this side's N at once;
// and the window: at most K unacknowledged `bound` and `state` messages of this host's own, with what waits meanwhile,
// and the sender of every message in flight, so each `ack` reaches the host that sent its message. A served call's
// method runs under the call's carrier (slice 9).
// and the window: at most K unacknowledged `bound` and `state` messages of this host's own, with what waits meanwhile
// (a call's answer sends the waiting state first, past the window), and the sender of every message in flight, so each
// `ack` reaches the host that sent its message. A served call's method runs under the call's carrier (slice 9).
import type { TelemetryContext } from "@earendil-works/pi-telemetry";
import { type Context, withAbortSignal } from "../context/index.ts";
@ -41,15 +41,19 @@ type Sender = "local" | HostLink;
/** A relaying origin: its number on upstream links, and the open upstream links it relayed holds to. */
type Relay = { readonly origin: number; readonly links: Set<HostLink> };
/** At most this many of this host's own `bound` and `state` messages unacknowledged per origin. */
/**
* At most this many of this host's own `bound` and `state` messages unacknowledged per origin; past it only the waiting
* state a call's answer sends first.
*/
const WINDOW = 2;
/**
* The provider side of one origin on a link. Ids are the origin's; a message naming an id this side does not know is
* ignored (Q1), one reusing a live id is a protocol error, and so are more concurrent calls than this side announced.
* This host's own `bound` and `state` messages go through the window: locally served holds wait while it is full
* (their ops in their members' pending queues, their bindings for the next slot). A relay forwards upstream messages
* at once (each upstream host keeps its own window for the origin it sees), so a slow consumer stalls only its origin.
* (their ops in their members' pending queues, their bindings for the next slot); a call's answer, its own or a
* relayed one, sends the waiting state first. A relay forwards upstream messages at once (each upstream host keeps its
* own window for the origin it sees), so a slow consumer stalls only its origin.
*/
export class ProviderSide {
readonly link: HostLink;
@ -173,11 +177,11 @@ export class ProviderSide {
/**
* A message from upstream link `from` for this origin's relayed holds, forwarded with this origin: `bound` and
* `state` join the window's senders (their `ack` goes back to `from`), a reply ends its call, `gone` forgets its
* hold. A link answers only for what was relayed over it: an entry, a notice or a reply naming a hold (or a call)
* this origin did not relay over `from` is dropped (Q1); a `bound` or `state` left without entries is acknowledged
* to `from` at once. Parts travel unchecked and unparsed: the end that parses a part checks it against its own
* link's limits.
* `state` join the window's senders (their `ack` goes back to `from`), a reply ends its call (after this host's own
* waiting state: `reply`), `gone` forgets its hold. A link answers only for what was relayed over it: an entry, a
* notice or a reply naming a hold (or a call) this origin did not relay over `from` is dropped (Q1); a `bound` or
* `state` left without entries is acknowledged to `from` at once. Parts travel unchecked and unparsed: the end that
* parses a part checks it against its own link's limits.
*/
forward(message: ConsumerMessage, from: HostLink): void {
const relay = this.#relay;
@ -247,11 +251,19 @@ export class ProviderSide {
this.#settle();
}
/** A call ended with `reply` (a result or an error): its slot is free; nothing is sent for one already gone. */
/**
* A call ended with `reply` (a result or an error, its own or forwarded from upstream): its slot is free; nothing is
* sent for one already gone. The state waiting for a slot goes first, past the window if it is full (decision "A
* call's answer follows its state"): when the call settles at the consumer, every change this host committed before
* answering is applied there.
*/
reply(id: number, reply: Extract<OriginMessage, { readonly kind: "result" | "error" }>): void {
const call = this.#calls.get(id);
if (call === undefined) return;
this.#calls.delete(id);
// At most one extra message per answer, acknowledged before the call its answer frees a slot for (one direction keeps
// its order): running calls and messages past the window stay within this side's N together.
if (this.#waiting.size > 0) this.#sendWaiting(this.link.context);
this.send(reply);
if (call.kind === "relayed" && call.span !== undefined) relayEnded(call.span, reply);
this.#settle();
@ -559,17 +571,25 @@ export class ProviderSide {
// Every snapshot in one synchronous step: the binding shows all of them at one moment.
this.#sendOwn({ kind: "bound", holds: holds.map((hold) => hold.boundEntry(context)) });
} else if (this.#waiting.size > 0) {
const holds = [...this.#waiting];
this.#waiting.clear();
this.#waited();
this.#bindingTurn = this.#binding.size > 0;
this.#sendOwn({ kind: "state", holds: holds.map((hold) => hold.pendingEntry(context)) });
this.#sendWaiting(context);
} else {
return;
}
}
}
/**
* The waiting state as one `state` message, taking a slot: a freed one, or one past the window before an answer.
* Waiting bindings take the next slot.
*/
#sendWaiting(context: Context): void {
const holds = [...this.#waiting];
this.#waiting.clear();
this.#waited();
this.#bindingTurn = this.#binding.size > 0;
this.#sendOwn({ kind: "state", holds: holds.map((hold) => hold.pendingEntry(context)) });
}
/**
* A call's result message, its part deflated above the threshold when this host compresses; with byte segments when
* its method's result schema has a `bytes()` leaf.

View file

@ -29,7 +29,8 @@ startTerminalRuntime<P>({
- `scope.ts`: `TuiScope(host, changed)`: `hold`, `watch` (a reader, `undefined` while no value), `registry`,
`pageState(key, initial)` (the host's value at `key`, shared with React's `usePageState`: a scope opened again after a
plugin reload finds it), `messages(t)` (a plugin's messages in the host's language at each call; a change of the
locale or a translation calls `changed`), `dispose` (no `changed` after it). References wait and rebind by themselves; no terminal
library dependency.
locale or a translation calls `changed`), `optimistic(read)` (the reader and `show(value, call)`: `value` shown until
every call shown settled, React's `useOptimistic`), `dispose` (no `changed` after it). References wait and rebind by
themselves; no terminal library dependency.
Tests: `test/tui/` and, with the launcher and sandbox, `test/node/tui/`.

View file

@ -2,7 +2,8 @@
// counterpart of the React bindings for terminal UI libraries. It holds references, follows watchables and registries,
// calls one `changed` callback (the app's redraw) on every change, and releases everything at `dispose`. Its page state
// is the host's, shared by every scope (React's `usePageState`): a scope opened again after a plugin reload finds it.
// Its messages render in the host's language, and a change of the locale or a translation redraws.
// Its messages render in the host's language, and a change of the locale or a translation redraws. A reader can show
// an optimistic value while a call is pending (React's `useOptimistic`).
// References wait and rebind by themselves (no retry loop, unlike p2's); no terminal library dependency.
import { BACKGROUND_CONTEXT, withAbortSignal } from "../context/index.ts";
@ -72,6 +73,35 @@ export class TuiScope<P extends Principal = Principal> {
return () => read();
}
/**
* `read` with an optimistic value (React's `useOptimistic`): `show(value, call)` returns `call` and shows `value` in
* place of `read`'s until every call shown so settled, then `read`'s again; each change calls `changed`. A call over
* a link settles after the changes its answering host committed before answering (decision "A call's answer follows
* its state"), so the value shown next is the real one.
*/
optimistic<T>(
read: () => T | undefined,
): readonly [read: () => T | undefined, show: <R>(value: T, call: Promise<R>) => Promise<R>] {
let shown: T | undefined;
let pending = 0;
const redraw = (): void => {
if (!this.#abort.signal.aborted) this.#changed();
};
const settled = (): void => {
if (--pending === 0) shown = undefined;
redraw();
};
const show = <R>(value: T, call: Promise<R>): Promise<R> => {
shown = value;
pending++;
// Its caller handles a rejection of `call`; this only follows it.
call.then(settled, settled);
redraw();
return call;
};
return [() => (pending > 0 ? shown : read()), show];
}
/**
* The host's page state at `key`, kept for the terminal's life, not the scope's (a new key starts at `initial`); its
* changes call `changed` until `dispose`.

View file

@ -0,0 +1,213 @@
// A call's answer follows its state (decision of 2026-11-05), and the optimistic value a terminal UI shows while a
// call is pending (`TuiScope.optimistic`, React's `useOptimistic`). A session's worker renames its session and pushes
// the new title to the server's index inside the method, as an app's worker should. A slow terminal client holds the
// index (served by the server) and calls `rename` (relayed by the server to the worker). The server's window for the
// client is full, so its index change waits; before the server forwards the worker's answer it sends that waiting
// state, so when the call settles in the client the index already has the new title: the optimistic title is replaced
// by the real one and the old one never shows again.
// Run from packages/chord:
// node --conditions=source --experimental-strip-types test/examples/63-optimistic-answers.ts
import { BACKGROUND_CONTEXT } from "../../src/context/index.ts";
import { hostKind, local, map, method, obj, remote, str } from "../../src/contracts/index.ts";
import { defineState } from "../../src/documents/index.ts";
import { createHost, defineFacet, type Host } from "../../src/host/index.ts";
import { binaryCodec, describeMessage, type Link, memoryLinkPair } from "../../src/link/index.ts";
import { TuiScope } from "../../src/tui/scope.ts";
const context = BACKGROUND_CONTEXT;
type P = { kind: "owner" } | { kind: "server" } | { kind: "worker"; session: string };
const Server = hostKind("server");
const Session = hostKind("session", { id: str() });
const Report = obj({ id: str(), title: str() });
/** The server's index of session titles: one store, whoever views it. */
const SessionIndex = local("example.session-index", {
version: 1,
params: {},
members: { titles: map(str()), report: method(Report) },
});
/** The index as clients and workers hold it. */
const Sessions = remote("example.sessions", {
version: 1,
params: {},
hostAddress: () => Server(),
variants: { all: { state: { titles: map(str()) }, methods: { report: method(Report) } } },
});
/** A session, on its worker: its title, and the method that changes it. */
const SessionTitle = remote("example.session-title", {
version: 1,
params: { id: str() },
hostAddress: ({ id }) => Session({ id }),
variants: { owner: { state: { title: str() }, methods: { rename: method(str()) } } },
});
const IndexDoc = defineState("index", {
schema: obj({ titles: map(str()) }),
version: 1,
initial: () => ({ titles: {} }),
});
const TitleDoc = defineState("title", { schema: obj({ title: str() }), version: 1, initial: () => ({ title: "" }) });
const serverFacet = defineFacet<P>((host) => {
host.serve(SessionIndex, (_, self) => {
const doc = self.state(IndexDoc);
return {
titles: self.derive([doc], ([index]) => index.titles),
report: async ({ id, title }, ctx) => {
await doc.change((draft) => {
draft.titles[id] = title;
}, ctx);
return null;
},
};
});
host.serve(Sessions, (_params, _principal, self) => {
const index = self.get(SessionIndex, {});
return { variant: "all", members: { titles: index.titles, report: (report, ctx) => index.report(report, ctx) } };
});
});
const workerFacet = defineFacet<P>((host) => {
host.serve(SessionTitle, ({ id }, _principal, self) => {
const doc = self.state(TitleDoc);
const sessions = self.get(Sessions, {});
return {
variant: "owner",
members: {
title: self.derive([doc], ([current]) => current.title),
// The rule for a method that should feel instant: it awaits its own changes, the push to the server included.
rename: async (title, ctx) => {
await doc.change((draft) => {
draft.title = title;
}, ctx);
await sessions.report({ id, title }, ctx);
return null;
},
},
};
});
});
/** The client's end of its link: while paused, what arrives waits (so it acknowledges nothing); logs what it gets. */
class SlowLink implements Link {
paused = false;
readonly #link: Link;
readonly #waiting: Uint8Array[] = [];
#listener: (message: Uint8Array) => void = () => {};
constructor(link: Link) {
this.#link = link;
link.onMessage((message) => {
if (this.paused) this.#waiting.push(message);
else this.#deliver(message);
});
}
/** Delivers what waited one message per task, as a socket does: what a message settles runs before the next. */
async resume(): Promise<void> {
for (let message = this.#waiting.shift(); message !== undefined; message = this.#waiting.shift()) {
this.#deliver(message);
await new Promise((resolve) => setTimeout(resolve, 0));
}
this.paused = false;
}
send(message: Uint8Array): void {
this.#link.send(message);
}
onMessage(listener: (message: Uint8Array) => void): void {
this.#listener = listener;
}
close(): void {
this.#link.close();
}
onClose(listener: () => void): void {
this.#link.onClose(listener);
}
#deliver(message: Uint8Array): void {
const decoded = binaryCodec.decode(message);
if (decoded.kind === "state" || decoded.kind === "result") {
console.log(" client ←", describeMessage(decoded).slice(0, 100));
}
this.#listener(message);
}
}
const settle = () => new Promise((resolve) => setTimeout(resolve, 10));
// 1. A server routing sessions to workers (in memory here), and a terminal client on a link that can stall.
const server = createHost<P>({ address: Server(), principal: { kind: "server" }, onError: console.error });
const workers: Host<P>[] = [];
server.route(
Session,
{
authorize: () => true,
resolve: async (address) => {
const [a, b] = memoryLinkPair();
const principal = { kind: "worker", session: address.params.id } as const;
const worker = createHost<P>({ address, principal, onError: console.error });
worker.connect(b, { principal: { kind: "server" }, trusted: true }, context);
void worker.load([{ name: "@example/worker", facet: workerFacet }], context);
workers.push(worker);
return { link: a, principal, trusted: true };
},
},
context,
);
await server.load([{ name: "@example/server", facet: serverFacet }], context);
const [up, down] = memoryLinkPair();
server.accept(up, { principal: { kind: "owner" } }, context);
const slow = new SlowLink(down);
const client = createHost<P>({ onError: console.error });
client.connect(slow, { principal: { kind: "server" } }, context);
await client.load([], context);
// 2. A component's scope: it holds the index and the session, and shows the session's title from the index, with an
// optimistic value while a rename is pending. Each redraw records what it shows.
const drawn: (string | undefined)[] = [];
const scope = new TuiScope(client, () => drawn.push(title()));
const sessions = scope.hold(Sessions, {});
const session = scope.hold(SessionTitle, { id: "S1" });
const titles = scope.watch(sessions.titles);
const [title, show] = scope.optimistic(() => titles()?.S1);
await session.rename("Draft", context);
await settle();
console.log("the index shows:", title());
// 3. The client stalls (it acknowledges nothing). Other sessions report to the index: two `state` messages fill the
// server's window for this client, the third change waits there.
slow.paused = true;
const index = await server.get(SessionIndex, {}, context);
for (const id of ["S2", "S3", "S4"]) await index.report({ id, title: `session ${id}` }, context);
// 4. The user renames S1. The title shows the new name at once; the worker renames, pushes the title to the index
// (whose change waits in the server's window too) and answers. The server sends its waiting state before it forwards
// the answer, past its window.
drawn.length = 0;
const renamed = show("Renamed", session.rename("Renamed", context));
await settle();
console.log("while the call is pending, the title shows:", title());
// 5. The client catches up, one message per task: the states, then the answer. When the call settles the index has the
// new title, so the title never shows "Draft" again (an answer that overtook the waiting state showed it once more).
console.log("the client resumes:");
await slow.resume();
await renamed;
await settle();
console.log("the redraws showed:", drawn);
console.log("the index shows:", title());
scope.dispose();
await client.close(context);
await server.close(context);
for (const worker of workers) await worker.close(context);

View file

@ -5,7 +5,7 @@ import { broken, ChordError } from "../../src/data/index.ts";
import type * as Delta from "../../src/delta/index.ts";
import { decoder } from "../../src/delta/index.ts";
import { defineState } from "../../src/documents/index.ts";
import { createHost, defineFacet, type FacetHost, type Host, type Self } from "../../src/host/index.ts";
import { type Ctx, createHost, defineFacet, type FacetHost, type Host, type Self } from "../../src/host/index.ts";
import type { ChordHost } from "../../src/host/internal.ts";
import { ChordObject } from "../../src/host/object.ts";
import { type Link, memoryLinkPair } from "../../src/link/index.ts";
@ -632,7 +632,10 @@ const RETIRING = wireNames(Retiring, "v");
/** Every member of `Retiring` retired with its document, sorted. */
const retiredAll = [RETIRING.state.doc!, RETIRING.state.big!, RETIRING.state.t!].sort();
/** A server serving `Retiring` to a raw peer that acks only when told; `call` calls a method of hold 0. */
/**
* A server serving `Retiring` to a raw peer that acks only when told; `call` calls a method of the object hold 0 holds,
* from the server itself as the hold's principal (an answer over the link would send the waiting state before it).
*/
async function retiring() {
const server = createHost<TestPrincipal>({ address: Server(), principal: { kind: "server" }, onError: () => {} });
await server.load(
@ -673,13 +676,15 @@ async function retiring() {
server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b, { ack: false });
await transported();
let calls = 0;
const call = async (method: "set" | "retire", args?: number) => {
const id = calls++;
const payload = args === undefined ? {} : { args: jsonPart(args) };
peer.send({ kind: "call", call: id, hold: 0, method: RETIRING.methods[method]!, ...payload });
expect((await peer.next("result")).call).toBe(id);
};
const call = (method: "set" | "retire", args?: number) =>
server.root(
async (ctx) => {
const object = await ctx.get(Retiring, {});
await (method === "set" ? object.set(args!, ctx) : object.retire(ctx));
},
context,
{ principal: { kind: "owner" } },
);
const { name: type, version, fingerprint } = Retiring;
peer.send({ kind: "hold", hold: 0, type, version, fingerprint, params: "{}", address: SERVER_ADDRESS });
await peer.next("bound");
@ -1063,14 +1068,12 @@ describe("holds across a reload (slice 7b §6)", () => {
const [a, b] = memoryLinkPair();
server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const peer = new RawPeer(b, { ack: false });
const SHOWN = wireNames(Shown, "v");
await peer.next("hello");
hold(peer, 0, Shown);
await peer.next("bound");
for (let call = 0; call < 3; call++) {
peer.send({ kind: "call", call, hold: 0, method: SHOWN.methods.bump! });
await peer.next("result");
}
// Commits from the server itself as the hold's principal: an answer over the link would send the waiting state.
const bump = async (ctx: Ctx<TestPrincipal>) => (await ctx.get(Shown, {})).bump(ctx);
for (let commit = 0; commit < 3; commit++) await server.root(bump, context, { principal: { kind: "owner" } });
// One state in flight with the bound: the third commit's ops wait in their pending queue.
await peer.next("state");
await server.reload([], context);

View file

@ -4,7 +4,7 @@ import { BACKGROUND_CONTEXT as context } from "../../src/context/index.ts";
import { type Contract, int, local, map, method, obj, remote } from "../../src/contracts/index.ts";
import { PendingOps } from "../../src/delta/index.ts";
import { defineState } from "../../src/documents/index.ts";
import { createHost, defineFacet, type FacetHost } from "../../src/host/index.ts";
import { type Ctx, createHost, defineFacet, type FacetHost } from "../../src/host/index.ts";
import { type Compression, memoryLinkPair, type PartLimits } from "../../src/link/index.ts";
import { DeflatedPart, jsonPart, type Message } from "../../src/link/internal.ts";
import { nodeCompression } from "../../src/node/compression.ts";
@ -645,21 +645,15 @@ forEachCodec(() => {
});
it("overflows past 10,000 pending ops to a replacement with the current value (race 23)", async () => {
const { peer } = await slowConsumer();
const { server, peer } = await slowConsumer();
hold(peer, 0, Grid, {});
await peer.next("bound");
const fill = (id: number, value: number) =>
peer.send({
kind: "call",
call: id,
hold: 0,
method: GRID.methods.fill!,
args: jsonPart({ n: 6_000, value }),
});
fill(0, 1);
fill(1, 2);
fill(2, 3);
for (const _ of [0, 1, 2]) await peer.next("result");
// Calls on the owner's object from the server itself: an answer over the link would send the waiting state first.
const fills = async (ctx: Ctx<TestPrincipal>) => {
const grid = await ctx.get(Grid, {});
for (const value of [1, 2, 3]) await grid.fill({ n: 6_000, value }, ctx);
};
await server.root(fills, context, { principal: { kind: "owner" } });
// The first fill took the second slot; the next two overflowed the queue.
peer.ack();
await transported();
@ -670,6 +664,79 @@ forEachCodec(() => {
});
});
describe('an answer after its state (decision "A call\'s answer follows its state")', () => {
it("sends the waiting state before a result when the window is full: one message, every hold, past the window", async () => {
const { server, peer } = await slowConsumer();
hold(peer, 0, Notes, { session: "S1" });
hold(peer, 1, Notes, { session: "S2" });
await peer.next("bound");
await peer.next("bound");
await addNote(server, "S2", "a");
await addNote(server, "S2", "b");
call(peer, 0, 0, add, { text: "c" });
await transported();
// The two bindings fill the window: both commits and the call's own wait, then go before its result.
expect(peer.kinds()).toEqual(["hello", "state", "result"]);
const { holds } = statesOf(peer)[0]!;
expect(holds.map(({ hold, members }) => [hold, EDITOR.named(members).count])).toEqual([
[1, [["r", 2]]],
[0, [["r", 1]]],
]);
expect(partValue((await peer.next("result")).value!)).toBe(1);
});
it("acknowledges every message once, in order, those past the window too: the window opens after their acks", async () => {
const { server, peer } = await slowConsumer();
hold(peer, 0, Notes, { session: "S1" });
await peer.next("bound");
await addNote(server, "S1", "a");
await addNote(server, "S1", "b");
call(peer, 0, 0, add, { text: "c" });
await peer.next("result");
await addNote(server, "S1", "d");
await transported();
// Three in flight (the bound, a state, the state past the window): the first ack leaves the window full.
expect(windowed(peer)).toEqual(["state", "state"]);
peer.ack();
await transported();
expect(windowed(peer)).toEqual(["state", "state"]);
peer.ack();
await transported();
expect(windowed(peer)).toEqual(["state", "state", "state"]);
for (const _ of [0, 1]) peer.ack();
// Acks crossing nothing are ignored: two slots, not more.
for (const _ of [0, 1]) peer.ack();
for (const text of ["e", "f", "g"]) await addNote(server, "S1", text);
await transported();
expect(windowed(peer)).toEqual(["state", "state", "state", "state", "state"]);
});
it("sends nothing more when nothing waits: a full window, an answer of a call that changed nothing", async () => {
const { server, peer } = await slowConsumer();
hold(peer, 0, Notes, { session: "S1" });
await peer.next("bound");
await addNote(server, "S1", "a");
call(peer, 0, 0, fail, { how: "chord" });
await transported();
expect(peer.kinds()).toEqual(["hello", "state", "error"]);
});
it("costs nothing on the common path: a call's commit goes out once, at its commit, never again with its answer", async () => {
const telemetry = new InMemoryTelemetryContext();
const { peer } = await rawLinked((h) => void serveNotes(h), { telemetry });
hold(peer, 0, Notes, { session: "S1" });
await peer.next("bound");
for (const [id, text] of ["a", "b", "c"].entries()) {
call(peer, id, 0, add, { text });
await peer.next("result");
}
await transported();
const states = telemetry.getMetrics().filter(({ name }) => name === "chord.link.state_messages");
expect(states.reduce((sum, { value }) => sum + value, 0)).toBe(3);
expect(peer.kinds()).toEqual(["hello", "state", "state", "state"]);
});
});
describe("origins at the provider (a relay's: on its parent link)", () => {
it("key holds and calls by (origin, id): the same ids on two origins are two holds; answers carry the origin", async () => {
const { peer } = await slowConsumer("parent");

View file

@ -21,6 +21,7 @@ import {
rawWorkers,
rejection,
relayed,
SERVER_ADDRESS,
Server,
Session,
serveNotes,
@ -337,6 +338,48 @@ forEachCodec(() => {
});
});
describe("a relayed answer after the relay's own state (decision \"A call's answer follows its state\")", () => {
it("forwards a worker's result after the state the relay's own objects keep waiting for the tab; every ack reaches its sender", async () => {
const workers: RawPeer[] = [];
const relay = await relayed({
server: (h) => void serveNotes(h),
resolve: rawWorkers(workers, { ack: false }),
});
const [a, b] = memoryLinkPair();
relay.server.accept(a, { principal: { kind: "owner" }, codec: testCodec() }, context);
const tab = new RawPeer(b, { ack: false });
const { name: type, version, fingerprint } = Notes;
const params = jsonPart({ session: "S1" });
tab.send({ kind: "hold", hold: 0, type, version, fingerprint, params, address: SERVER_ADDRESS });
await tab.next("bound");
holdPlaced(tab, 1);
const worker = await workerAt(workers);
const held = await worker.next("hold");
bindPlaced(worker, held);
await tab.next("bound");
// The relay's own window for the tab: its bound and this state; the next commit waits.
await addNote(relay.server, "S1", "a");
await addNote(relay.server, "S1", "b");
tab.send({ kind: "call", call: 0, hold: 1, method: add, args: jsonPart({ text: "x" }) });
const forwarded = await worker.next("call");
worker.send({ kind: "result", ...originOf(held), call: forwarded.call, value: jsonPart(1) });
await transported();
expect(tab.kinds()).toEqual(["hello", "state", "state", "result"]);
const waited = tab.received.filter((message) => message.kind === "state").at(-1)!;
expect(waited.holds.map(({ hold, members }) => [hold, EDITOR.named(members).count])).toEqual([
[0, [["r", 2]]],
]);
// Four messages in flight to the tab, one of them the worker's: its ack goes to the worker, the others free the
// relay's own window (the state past it included).
for (const _ of [0, 1, 2, 3]) tab.ack();
await transported();
expect(worker.kinds().filter((kind) => kind === "ack")).toEqual(["ack"]);
for (const text of ["c", "d", "e"]) await addNote(relay.server, "S1", text);
await transported();
expect(tab.kinds().filter((kind) => kind === "state")).toHaveLength(4);
});
});
describe("an upstream drop", () => {
fakeTimers();

View file

@ -1,7 +1,8 @@
// Tests of `TuiScope` (slice 11 §4, decision "Slice 11, P9"): references held for a component, watchables and
// registries followed, one `changed` callback for every change, everything released at `dispose`, and the host's page
// state shared by every scope (decision "Tutorial gaps" (1)), and messages in the host's language, a change redrawing
// (decision "Messages and languages in the app kit").
// (decision "Messages and languages in the app kit"), and an optimistic value shown while a call is pending (decision
// "A call's answer follows its state").
import { describe, expect, it } from "vitest";
import { BACKGROUND_CONTEXT as context } from "../../src/context/index.ts";
@ -92,6 +93,62 @@ describe("a component's scope", () => {
});
});
describe("a scope's optimistic values", () => {
it("show the value while its call is pending, then the reader's, calling changed on each change", async () => {
const h = await host();
let changes = 0;
const scope = new TuiScope(h, () => changes++);
const notes = scope.hold(Notes, {});
const [text, show] = scope.optimistic(scope.watch(notes.text));
await until(() => text() === "first");
const before = changes;
const written = show("second", notes.write("second", context));
expect(text()).toBe("second");
expect(changes).toBe(before + 1);
await written;
await until(() => changes > before + 1);
expect(text()).toBe("second");
scope.dispose();
await h.close(context);
});
it("show the last value until every call settled; a rejected call drops it as a resolved one does", async () => {
const h = await host();
const calls: string[] = [];
const scope = new TuiScope(h, () => calls.push(text()!));
const [text, show] = scope.optimistic(() => "real");
const first = Promise.withResolvers<null>();
const second = Promise.withResolvers<null>();
show("one", first.promise);
const failing = show("two", second.promise);
first.resolve(null);
await first.promise;
await Promise.resolve();
expect(text()).toBe("two");
second.reject(new Error("refused"));
await expect(failing).rejects.toThrow("refused");
await Promise.resolve();
expect(calls).toEqual(["one", "two", "two", "real"]);
scope.dispose();
await h.close(context);
});
it("never call changed after dispose", async () => {
const h = await host();
let changes = 0;
const scope = new TuiScope(h, () => changes++);
const [, show] = scope.optimistic(() => "real");
const call = Promise.withResolvers<null>();
show("shown", call.promise);
scope.dispose();
call.resolve(null);
await call.promise;
await Promise.resolve();
expect(changes).toBe(1);
await h.close(context);
});
});
describe("a scope's page state", () => {
it("is the host's: a scope opened after another was disposed (a reload) finds its value, a new key starts fresh", async () => {
const h = await host();