docs(chord): slice 6b D9, self.follow; spec decided

This commit is contained in:
Mario Zechner 2026-10-28 11:50:25 +00:00
parent f6a9db1d83
commit e4c58271b8
3 changed files with 42 additions and 23 deletions

View file

@ -1683,6 +1683,14 @@ validation and copies the v2 kernel added around Delta.
ctx, { persistence: { kind: "deferred", withinMs? } })`, Durable's `CommitPersistence` shape (default 1,000 ms):
adopted and published at once, stored with the next immediate change, before the outcome is recorded, at close, or
after `withinMs`; a crash loses what is not stored. Only run state documents for now.
- **Slice 6b D9: `self.follow(attach)` follows Durable's views (owner, 2026-10-27: "sounds good"):** a generic Chord
primitive, no `durableView` module in Durable: `attach(observer, ctx)` returns `Promise<{ value, detach }>`
(Durable's `Observation`; its observers have Chord's watch shapes since pi `5f312dac4`). Chord makes the observer,
attaches during activation (early calls held, `value` the start value), detaches at the attempt's end; calls inside a
batch mark the source in the step, calls outside one are grouped into one batch on the next microtask; a rejected or
`undefined` attach fails the activation as a failed dependency; Durable's `failed` end ends the watches `failed`.
Returns a `Watchable` (state member, derive source, `on` source). Supersedes the carry-over's `durableView` and D1's
`durableView` part.
- **The performance and memory bar, and converging reviews (owner, 2026-10-27: "yes"):** supersedes "as good as
HEAD in everything" for Chord v2 slices and pi changes. CPU: each hot path (calls, changes, watches, links) within
about 5% of HEAD, measured in bursts with alternating processes pinned to one CPU; worse is fixed or goes to the

View file

@ -1,6 +1,6 @@
# Slice 6b: Durable and jobs on the store (spec)
Status: **draft, open points marked "Decision needed"** (2026-10-27), written by the redesign session after reading
Status: **decided, ready for the spec reviews** (2026-10-27; D1-D9 decided by the owner), written by the redesign session after reading
Durable's `Storage` interface (`packages/durable/src/types.ts`), its SQLite storage, schema and migrations, the storage
conformance suite and benchmark, `session/session.ts`, `session/transaction.ts`, `session/forks.ts`, and Chord v2's
store engine, applier, schema, commit step and jobs, in full. Durable's memory, JSONL and IndexedDB storages were not
@ -12,7 +12,7 @@ W1, W12, W13, "Slice 6 first decisions", "Durable observers", "Durable's synchro
"Failures are watch calls", the "One vocabulary" entries, "The performance and memory bar"; `slice-6a.md` (what 6a
built); the feature map and the carry-over of `slices.md` (slice 6b); the root `AGENTS.md` "Chord v2 Code Style".
6b puts Durable on the store (`StoreStorage`), follows Durable's views as Chord sources (`durableView`), persists jobs,
6b puts Durable on the store (`StoreStorage`), follows Durable's views as Chord sources (`self.follow`), persists jobs,
and extends the applier to every write kind. After 6b a worker has one store: Chord's objects and Durable's Session
commit through one queue, and a commit of either is one seq in one outbox.
@ -28,7 +28,7 @@ host.serve(Harness, (_, self) => {
const harness = self.open(async (context) =>
Harness.open(new StoreStorage(store), { models, registry, … }, context), // one StoreStorage per store
(h, _reason, context) => h.close(context));
const view = self.follow(...) // Decision needed (D9): name and shape
const view = self.follow((observer, ctx) => harness.value.observeConversation(id, observer, ctx)); // section 2
return { … };
});
@ -47,8 +47,8 @@ Durable's `Storage` (types.ts, "Atomic persistence boundary for Session records"
Durable's Session runs unchanged on it and passes Durable's storage conformance suite
(`createStorageConformance`), across reopen too.
**Where it lives (D1, owner, 2026-10-27: "they are only used if durable is used"):** `StoreStorage` and
`durableView` belong to Durable, in its own `src` (Chord v2 is in pi upstream since `93029a0f1`), importing
**Where it lives (D1, owner, 2026-10-27: "they are only used if durable is used"):** `StoreStorage` belongs to
Durable (D9 makes `durableView` a generic Chord primitive, `self.follow`), in its own `src` (Chord v2 is in pi upstream since `93029a0f1`), importing
`@earendil-works/chord/next` (Chord's `./next` package export comes with them). Durable already depends on Chord and
Chord imports nothing of Durable (S13); Durable's conformance suite runs in Durable's tests and CI.
@ -130,22 +130,33 @@ store usable. 6b changes 6a's step and object failure accordingly.
A target to measure, as 6a's: an entry append is the `record_ids` put, the entry put, `lastSeq` (plus the outbox); a
document delta is 6a's three statements. `SqliteStorage` does the same work today (`record_ids`, the row, metadata).
## 2. `durableView`: Durable's views as Chord sources
## 2. `self.follow`: Durable's views as Chord sources (D9, owner, 2026-10-27)
A Chord source over Durable's synchronous observers (`observeConversation`, `observeTaskGraph`, `observeDoc`;
decision "Durable's synchronous observers"): `advance(call, ctx)` records the call; on the store, `adopt` runs inside
the host's batch, so the source is marked in the step and its frame goes out in the flush with the rest of the commit
(one message per link per commit). On a Durable storage that is not the store (tests, other apps), calls arrive outside
any batch: the source opens one batch per Durable commit on the next microtask (carry-over). Durable's failures are
final: `end({ kind: "failed" })` ends the source's watches `failed`; `ended` ends them `ended`. A derive over two
Durable views of one commit sees both changes in one computation.
Durable's synchronous observers (`observeConversation`, `observeTaskGraph`, `observeDoc`; decision "Durable's
synchronous observers") have Chord's watch shapes (`advance(call, ctx)` with a `WatchCall`, `end(end)` with a
`WatchEnd`, pi `5f312dac4`), so Chord follows them with one generic primitive and Durable needs no adapter:
> **Decision needed (D9): the API.** The carry-over names it `self.follow` (working name): `self.follow(observe)` in
> a factory, `observe` attaching an observer, returning a `Watchable` usable as a state member, a derive source and an
> `on` source, detached when the attempt ends. Options: (a) `self.follow((observer, ctx) =>
> harness.observeConversation(id, observer, ctx, window))`; (b) a plain function `durableView(attach)` returning a
> native the object opens (`self.open`). Proposal: (a), named `self.follow`: the attempt owns the attachment as it
> owns a derive.
```ts
host.serve(ChatView, ({ conversation }, self) => {
const harness = self.get(Harness, {});
const view = self.follow((observer, ctx) =>
harness.value.observeConversation(conversation, observer, ctx, { min: 50 }));
return { view, title: self.derive([view], ([v]) => v.title) };
});
```
- `self.follow(attach)` returns a `Watchable`: a state member, a derive source, an `on` source. `attach(observer,
ctx)` returns `Promise<{ value, detach }>` (Durable's `Observation`).
- **Attaching** is that one call during activation: Chord makes the observer, calls `attach`, takes `value` as the
source's value (calls arriving before the promise resolves are held and applied after it), and the object goes live.
The attempt's end calls `detach()`. An attach that rejects, or resolves `undefined` (`observeDoc` of an absent
address), fails the activation like a failed dependency.
- **Calls:** `advance` records the call. Inside a batch (on the store, Durable's `adopt` runs in the host's batch) the
source is marked in the step and its frame goes out in that flush; outside a batch (a Durable storage that is not
the store) the source collects the calls and opens one batch on the next microtask. A derive over two views of one
Durable commit sees both changes in one computation.
- **End:** Durable's failures are final: `end({ kind: "failed" })` ends the watches `failed`, `ended` ends them
`ended`.
## 3. Persisted jobs
@ -215,8 +226,8 @@ applying (S17).
memory (allocation per op, retained after GC) no worse beyond noise; persisted job starts and state changes against
in-memory ones.
- **Tests:** Durable's storage conformance suite on `StoreStorage` (memory `:memory:` and across reopen); Durable's
Session and harness test files that take a storage, run on `StoreStorage`; durableView (one frame per commit, a
derive over two views, outside the store, failure and end); persisted jobs (rerun, abandon, a recorded outcome
Session and harness test files that take a storage, run on `StoreStorage`; `self.follow` (one frame per commit, a
derive over two views, outside the store, early calls held, failure and end, an absent `observeDoc`); persisted jobs (rerun, abandon, a recorded outcome
resumes `finish`, restart counting and reset, the limit, abort stored, deferred state, fingerprints); the applier
(every kind, the outbox replay, refused counts); export and import with Durable records and `rewindable` history;
GC (observers detached with their attempt).
@ -225,7 +236,7 @@ applying (S17).
## Carry-over taken
From `slices.md` "Slice 6b": persisted jobs (all of it), `StoreStorage` (ids, fork copies, `adopt` in the batch),
`durableView` (the batch on and off the store, `self.follow`), the applier for batch apply; `job.abort` async and
the Chord source over Durable's views (`self.follow`, the batch on and off the store), the applier for batch apply; `job.abort` async and
stored (decision of 2026-10-27).
## Carry-over left

View file

@ -109,7 +109,7 @@ pattern, no code. Decided cuts are noted where they apply.
| 10 | the store: backends, tables, query runner, writes, commit queue, outbox, migrations, export/import | 6a |
| 11 | `Watchable`, watches (Chord's `Watch`), the batch (W1), reentrancy | 1 |
| 11 | `NativeWatchable`, `self.writable` | 8 |
| 11 | Durable's views as sources (`durableView` over Durable's `observeXXX`) | 6b |
| 11 | Durable's views as sources (`self.follow` over Durable's `observeXXX`) | 6b |
| 12 | `self.derive`, failures per member, lazy when unwatched, `mapShared` | 1 |
| 12 | container-width benchmark (R24) | 9 |
| 13 | `self.open`, close reasons, order, restart patterns (R16) | 2 (restarts on reload: 7) |