pi-mono/packages/chord/docs/next/slice-1.md
Mario Zechner 93029a0f1e feat(chord): Chord v2 in src/next, experimental and not exported yet
The next object runtime moves from the private repository into pi: src/next (contracts, hosts and
objects, documents, derives, links and relays, routines and jobs, the store) with its tests and
examples in test/next, src/next-node (Node compression), and its design in docs/next (design.md, the
decision log decisions.md, the slice plan slices.md, slice specs, books and benchmarks). AGENTS.md
gets its code style. Not exported: the API changes with every slice until it has an entry point.
2026-10-28 10:30:19 +00:00

16 KiB

Slice 1: one host, local objects (spec, for the owner's approval)

Status: approved (owner, 2026-10-22), after Astra's and Fable's spec reviews. The single source for building slice 1, together with the feature map in slices.md (entries assigned to slice 1) and design.md, read in full (the most relevant sections: §3 host, §5-§7, §8, §9, §11, §12, §16, §18); rules come from the decision log (packages/chord/docs/next/decisions.md); code style: the section "Chord v2 Code Style" of the root AGENTS.md. Builds on L1 (src/next/contracts, src/next/data) and Chord's shared watch handle (src/watch.ts).

What an app can do after this slice

const Notes = local("notes.store", {
  version: 1,
  params: {},
  members: {
    notes: obj({ list: arr(Note) }),
    titles: arr(str()),
    count: int(),
    add: method(obj({ text: str() })),
    saveDraft: method(obj({ conversation: int(), body: str() })),
  },
});
const Draft = local("notes.draft", {
  version: 1,
  params: { conversation: int() },
  members: { summary: str() },
});

const NotesDoc = defineState("notes", { schema: obj({ list: arr(Note) }), version: 1, initial: () => ({ list: [] }) });
const Drafts = defineState("drafts", { schema: obj({ body: str() }), version: 1, initial: () => ({ body: "" }), keyed: true });

const facet = defineFacet<AppPrincipal>((host) => {
  host.serve(Notes, (_, self) => {                                // empty params: lives as long as the host
    const notes = self.state(NotesDoc);
    const drafts = self.state(Drafts);                             // keyed: one document per conversation (P3 pattern)
    const title = mapShared((note: Note) => note.text.slice(0, 20));
    return {
      notes,
      titles: self.derive([notes], ([n]) => title(n.list)),
      count: self.derive([notes], ([n]) => n.list.length),
      add: async ({ text }, ctx) => {
        await self.tx(async (tx) => { (await tx.draft(notes)).list.push({ text }); }, ctx);
        return null;
      },
      saveDraft: async ({ conversation, body }, ctx) => {
        await self.tx(async (tx) => { (await tx.draft(drafts, String(conversation))).body = body; }, ctx);
        return null;
      },
    };
  });
  host.serve(Draft, ({ conversation }, self) => {                  // params: lives while referenced
    const store = self.get(Notes, {});                              // declared here, live before this object is
    return { summary: self.derive([store.count], ([c]) => `conversation ${conversation}, ${c} notes`) };
  });
});

const host = createHost({ onError: (error) => log(error) });       // no address: local types only
await host.load([{ name: "@app/notes", facet }], ctx);
const notes = await host.get(Notes, {}, ctx);                      // live and hydrated
await notes.add({ text: "milk" }, ctx);
notes.count.value;                                                 // 1
const w = await notes.count.watch(ctx);
w.start(async (value) => console.log(value));
notes.release();                                                   // ends w; the object lives on (empty params)
await host.close(ctx);

1. Host and facets

  • createHost({ telemetry?, onError? }): a host without an address; it serves only local types in this slice. telemetry is the adapter (NOOP by default); onError receives background errors (a throwing watch listener).
  • defineFacet<P>(setup) returns a facet typed for the app's principal type P (base code binds it once; nothing is checked at runtime). setup(host, self) runs synchronously inside host.load and only declares: host.serve(Type, factory) and facet documents through self.state(Def) (C16, owner: built now). host.serve outside a setup throws.
  • Facet documents (design §9, §16): the facet is a pure object with the host's lifetime; its documents live as long as the host. Objects the facet serves may read the facet's handles and draft them in their own self.tx, together with their own documents, atomically (lock order: the object's own line, then the facet's line). The facet never drafts a served object's documents; objects of other facets cannot use them.
  • host.load(facets, context): facets is [{ name, facet }] with npm-style plugin names; duplicate names or a type served twice are refused. After every setup ran, every served local type with empty params is activated (Q-A, the start barrier); load resolves when all are live. If one fails, load ends everything it activated and rejects; the host is then empty. A host.get made before load resolves waits for it.
  • host.close(context): rejects new calls, ends every object (in reverse activation order) and resolves when all have ended.

2. Objects

  • Identity: one object per (type, params key) (L1's checkParams and identityKey; local has no principal).
  • Lifetime (Q-A): a local type with empty params is created at load and lives until close; a type with params is created by the first get and ends when its last reference is released.
  • Factory: (params, self) => implementation, synchronous and declarative, run once per object. The implementation has exactly the contract's members: a state member is a JSON Watchable of its value (a document handle, a derive, a member of a reference), a method is (args, ctx) => Promise<result>. L1's Implementation<C> types state members as Watchable<Static<schema>> (owner, 2026-10-22).
  • Activation (design §6): run the factory; acquire the references it declared with self.get (activating their objects if needed) and wait until they are live; check the members (exactly the contract's, state members JSON watchables, Q14); compute every derive's first value; then the object is live and calls proceed. A throw anywhere in activation (factory, a dependency's activation, a member check, a first derive compute, initial() failing its schema) rejects the get (or load) with that error, releases everything acquired and leaves no object. Failure states, backoff and retries are slice 2.
  • Ending: new calls on the object fail released; the context of each of its running calls is aborted; from then on its self.tx throws released (a still-running method can no longer change anything); its derives are detached from their sources (no source keeps a reference to an ended object's derive); its references are released; its documents are dropped; every watch on its members ends ended. Ending does not wait for running calls (the drain arrives with natives in slice 2, design §3).

3. References

  • host.get(Type, params, context) returns a reference once its object is live; self.get(Type, params) declares one in a factory: it is live by the time the object goes live, and released when the object ends.
  • A reference is a plain object built from the contract's member list (no proxy): ref.<state> is a Watchable (value, watch(ctx)), ref.<method>(args, ctx) calls the method, ref.release() releases it (idempotent). After release, every use throws released.
  • Calls: arguments and results pass by reference; arguments are checked at the provider with L1's checkArgs against the current version (bindProvider(contract, {}) at serve): unknown method invalid/member, bad arguments invalid/args; results are not checked; no-argument methods take null.
  • Per-reference watches (owner): watches obtained through a reference end (ended) when it is released, even while other references keep the object alive.
  • Contract match (owner): at get, the caller's contract must have the provider's version and fingerprint (the same declaration, or an identical one), otherwise contract_mismatch. Older versions through accepts come with slice 7.
  • Reserved member names (owner): then, release, state and retry are refused as member names at declaration (an L1 check in remote, local and evolve), because a reference has them (release now, state and retry in slice 2) and then would make an awaited reference thenable.
  • Cycles (Q5, one host; owner): at self.get during an activation, Chord follows the chain of objects waiting for activation; reaching the object being activated fails that activation with invalid/cycle, naming the cycle (a → b → a); nothing is kept.
  • ctx in this slice is a plain Context (cancellation and telemetry); ctx.get and ctx.principal come with slice 3.

4. Documents in memory (§9, P2)

  • defineState(name, { schema, version, initial, keyed? }): the root is a JSON object. checkpointWhen is required only for persisted documents (slice 6; owner, 2026-10-22). migrate comes with slices 6 and 7.
  • self.state(Def) returns a handle bound to this object; every document is in memory, instance state of the object (P2), gone when it ends.
  • Singleton handle: handle.value is the current value, initial() until written or after a retire (Q11); the handle is a Watchable.
  • Keyed handle: await handle.get(key) gives the value or undefined; handle.keys() is an AsyncIterable of the present keys (the shape persistence keeps); keys are strings. A keyed handle is not a Watchable (a live collection is a singleton holding a list or map).
  • initial() is checked against the schema once, the first time a definition's initial() is used (R8).

5. Transactions (§9, R12, R13)

  • self.tx(fn, ctx): an object's transactions run one at a time (its line). fn gets tx: await tx.draft(handle, key?) returns the document's draft (Delta's track, beginChange; the same document gives the same draft within one transaction); tx.retire(handle, key?) removes a document (a no-op when absent). self.tx returns what fn returns.
  • Commit: when fn resolves, if ctx is aborted the transaction is discarded; otherwise every draft is prepared (Delta's prepare: ops recorded, only placed values checked), unchanged drafts commit nothing, and all changed documents are adopted (pointer swap) in one batch (section 6). A throw discards everything.
  • Drafts are revoked when the transaction settles (touching one throws, R12); a transaction returns plain values and data is read from the handle after the commit.
  • A transaction drafts only its own object's documents and its facet's (any other handle throws). Inside fn: no second self.tx, no calls to other objects (Durable's rule; documented, not detected).
  • Shorthands (C14, owner: built now): handle.change(fn, ctx) (keyed: handle.change(key, fn, ctx)) runs one transaction that loads the document and calls the synchronous fn(draft); handle.retire(ctx) (keyed: handle.retire(key, ctx)) retires it. Both are thin wrappers over self.tx.

6. Derives and the batch (W1, §11, §12)

  • self.derive(sources, compute): sources are JSON Watchables (this object's handles and derives, members of its self.get references); only the declared sources are tracked. compute is a synchronous pure function of their values; its result is JSON.
  • Pull model. A commit only marks: every derive that depends on a changed document, directly or through other derives, including derives of other objects on the host, is flagged dirty (O(dependents), flags only). A derive's value recomputes when it is dirty, after reading its sources' values (which recompute first if they are dirty); so each derive computes at most once per batch and always after its sources, diamonds included.
  • The batch (one per host): a commit's adoptions happen inside it; after the last adoption, still synchronously, every watched value that changed is read (watched derives recompute through the pull), its ops computed (diffRevisions(old, new) for a derive, the commit's ops for a document), and its watches get one frame each. A derive is recomputed in the batch only if something watched depends on it; otherwise it stays dirty until read. Reading .value once does not make it watched.
  • A derive that throws fails only itself: its value throws the error and its watches end failed; a derive over it throws the same error when read. A derive that fails during activation fails the activation (section 2). A failed derive stays failed until the object is rebuilt.
  • Reentrancy: inside the batch (adoptions, derive computes, including lazy computes outside a commit), self.tx, watch, release, get, load and close throw invalid/reentrant. One flag per host, cleared in a finally.
  • mapShared(f): maps an array calling f only for object elements not seen before (by identity), returning the previous result for unchanged ones; primitive elements are mapped every time; the cache is a WeakMap.
  • constant(value): a JSON Watchable that never changes (start data, §18); usable as a state member and a source.
  • State is JSON only (L2 decision): no bytes in documents or derives.

7. Watching

watchable.watch(ctx) returns Chord's shared Watch (src/watch.ts): the value at acquisition, then one frame per batch that changed it, (value, ops, ctx), in order, asynchronously, with its overflow, stop and end semantics. Each frame is consistent with its commit; two different watches are not ordered relative to each other. A retired singleton gives the frame [["r", initial]] (it changes back to initial(); a singleton always has a value). ctx's abort cancels the watch. A listener that throws ends its watch listener_error and is reported to onError (taken from the carry-over).

Cost per hop

Hop Work
a call L1's three lookups and one compiled check; arguments by reference
a commit Delta prepare per changed document, pointer swaps, dirty flags per dependent, then per watched changed value one read (derives recompute once) and one frame per watch, by reference
an unwatched derive nothing until read

Counters (telemetry, where the work happens): L1's, plus chord.derive.computes and chord.derive.diffs.

Carry-over taken by this slice

From carry-over.md (L4 section): watch listener errors reported to onError; native watchables refused as state (by the member check: only JSON watchables exist in this slice); defineFacet typed by P. Left for later: plugin error shape, old-version calls (7), broken() (2), cuts C5, C6, C7, C13 (2).

Tests

  • Host: load runs setups, activates empty-params types, rejects duplicates and double serves; a failing activation makes load end what it activated; get during load waits; serve outside setup throws; close ends everything.
  • Objects: one object per params key; params type created on first get and ended on last release (documents gone, watches end ended, its derives detached from other objects' members); empty-params type lives across releases; activation failures (factory, dependency, member check, first derive, initial()) reject the get and leave nothing; a method running while its object ends cannot commit (self.tx throws released) and sees its ctx aborted.
  • References: hydrated on get; members watchable; calls checked; released reference throws; self.get dependency kept alive and released on end; per-reference watches; contract mismatch; reserved member names refused at declaration; a two-object and a three-object cycle fail with invalid/cycle and leave nothing.
  • Documents: singleton initial, retire back to initial; keyed get, keys, retire; initial() checked once.
  • Transactions: serialized per object; same draft twice; unchanged commits nothing; throw and abort discard; drafts revoked after settle; another object's handle refused; a served object's transaction over its own and its facet's documents commits both in one batch; shorthands.
  • Batch and derives: two documents in one transaction reach a derive over both in one compute and a watch in one frame; a derive over another object's member updates in that object's commit's batch; diamond computes once; unwatched derive stays lazy and a one-off read does not make it eager; a throwing derive fails only itself and its dependents; reentrancy throws in commits and lazy computes and the flag clears after a throw; mapShared identity.
  • Watching: frames in order, retire frame, cancellation, listener error reported.
  • Cost guards per the table. GC: an ended object with its documents, derives and watches is collected; a released reference and a stopped watch hold nothing; a detached derive is not held by its former sources.

Size

About 1,100 source lines.