diff --git a/packages/agent/docs/pico-v5-chord-usage.md b/packages/agent/docs/pico-v5-chord-usage.md new file mode 100644 index 000000000..451bc0b2e --- /dev/null +++ b/packages/agent/docs/pico-v5-chord-usage.md @@ -0,0 +1,364 @@ +# Pico5 documents through Chord + +This guide uses the contracts in the [Pico5 specification](pico-v5.md). + +- **Session:** a durable container for conversations, entries, tasks, and documents; + it serializes mutations on one commit line. +- **Facet:** a feature's setup unit in a Chord host. Setup declares services and + dependencies synchronously; `onActivate` runs after dependencies are ready. +- **Context:** explicitly passed cancellation and invocation values. Forward the caller's context. +- **Service:** a typed token and implementation acquired with `env.use`. Remote + services contain replicated state and async methods with JSON arguments plus a trailing `Context`. +- **Durable document:** JSON persisted as a complete base, Chord operations, and + checkpoints. Its definition token declares schema, version, scope, and initialization; + conversation scope also declares history and fork behavior. +- **Transaction draft:** the mutable tracked object from `await tx.doc`, valid + only inside that commit callback, including all nested objects. +- **DocumentSource:** an opaque handle to one document incarnation's committed changes, + not a mutable value or a public subscription API. +- **ReplicatedState:** Chord's immutable complete values through `value` and + `subscribe(listener)`. `value` is `undefined` before hydration or while disconnected. + The listener receives `(value, context, delivery)`; delivery contains `kind` + (`hydrate` or `update`) and a stream `sequence`, not a Session commit number. + +Documents are scoped directly to a Session, conversation, or task. A task is +durable work attached to a conversation; its documents retire when it becomes +terminal and never participate in conversation forks. + +## Imports and the adapter boundary + +Examples build on one another. Pico5 names (`defineDoc`, `defineDocFamily`, +`Session`, `DocumentSource`, `Id`, `TaskRuntime`, `DocumentObserver`) +refer to normative contracts, without a specified import path or runnable Pico5 +package. These Chord imports exist today: + +```ts +import { + createFacetHost, createRemoteServiceBinding, defineFacet, defineService, + type Context, type Facet, type JsonValue, type RemoteServiceTransport, + type ReplicatedState, +} from "@earendil-works/chord"; +import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context"; + +// The spec calls this JsonObject; current Chord exports JsonValue, not JsonObject. +type JsonObject = { [key: string]: JsonValue }; + +// Required by Pico5, but not a current Chord export yet. +declare function documentReplicatedState( + source: DocumentSource, context: Context, +): Promise<{ readonly state: ReplicatedState; dispose(): void }>; +``` + +The adapter must register Chord-recognized state, atomically hydrate matching +committed value/revision and subscribe, and forward committed ops without another +tracker or re-diff. Disposal releases observation, not the document. Even Chord's +hydration requests must never flush an uncommitted draft. + +Current `env.replicatedState(initial)` creates a **mutable, in-memory** tracker +with `.state` and `.publish(context)`, not a durable source adapter. There is no +`replicatedState.own(source)`. Do not mirror documents into another tracker. + +## 1. A Session-wide canvas + +All conversations in this Session share one canvas. Session documents are +current-only and have no history or fork settings. + +```ts +type Stroke = { color: string; points: { x: number; y: number }[] }; +type CanvasState = { strokes: Stroke[] }; + +const CanvasDoc = defineDoc({ + id: "app.canvas", version: 1, scope: "session", + initial: () => ({ strokes: [] }), + // This append-only example adds one stroke per commit. + checkpointWhen: value => value.strokes.length % 100 === 0, +}); + +interface CanvasService { + readonly state: ReplicatedState; + addStroke(stroke: Stroke, context: Context): Promise; +} +const Canvas = defineService("app.canvas"); + +async function createCanvasFacet(session: Session, context: Context): Promise { + const target = { scope: "session" } as const; + // Get-or-create: an absent canvas is initialized in a serialized commit. + const source = await session.documentSource(CanvasDoc, target, context); + const replica = await documentReplicatedState(source, context); + return defineFacet({ + id: "app.canvas/session", + setup(env) { + env.own(() => replica.dispose()); + env.provide(Canvas, { + state: replica.state, + async addStroke(stroke, context) { + await session.commit(async tx => { + const draft = await tx.doc(CanvasDoc, target); + draft.strokes.push(stroke); // The draft now owns stroke; caller must not mutate it. + }, context); + }, + }); + }, + }); +} + +const CanvasConsumer = defineFacet({ + id: "app.canvas/consumer", + setup(env) { + const canvas = env.use(Canvas); // Declare now; access only after activation. + env.onActivate(() => { + env.own(canvas.state.subscribe((value, _context, delivery) => { + console.log(delivery.kind, delivery.sequence, value?.strokes.length ?? "retired"); + })); // subscribe delivers the current hydrated value, then updates. + }); + }, +}); + +async function runCanvasExample(session: Session): Promise { + const context = BACKGROUND_CONTEXT; + const provider = await createCanvasFacet(session, context); + const host = await createFacetHost({ facets: [provider, CanvasConsumer] }); + try { + await host.services.use(Canvas).addStroke({ + color: "black", points: [{ x: 10, y: 20 }, { x: 30, y: 40 }], + }, context); + } finally { + await host.dispose(); // Unsubscribes; does not delete the canvas or close Session. + } +} +``` + +The application owns the open Session. Acquire state **before** synchronous +`setup`; `env.provide` cannot run in `onActivate`. Install one provider per +Session host. Chord may reload presentation facets independently, but v1 does +not use facet reload to replace Session-side task or hook implementations. A +host plugin code change closes and reopens the Harness with the new definition +set. + +Remote clients supply a `RemoteServiceTransport` connected to the host's `services` +provider; Chord prescribes no socket protocol. `CanvasConsumer` also works unchanged +in a UI host configured with a remote service source. + +```ts +async function connectCanvas(transport: RemoteServiceTransport, context: Context) { + const services = createRemoteServiceBinding({ services: [Canvas], transport }); + const canvas = services.use(Canvas); + const stop = canvas.state.subscribe((value, _context, delivery) => { + console.log(delivery.kind, delivery.sequence, value?.strokes ?? "retired"); + }); + try { + await services.ready(context); // Initial snapshot installed; not all future updates. + } catch (error) { + stop(); + await services.dispose(BACKGROUND_CONTEXT); + throw error; + } + return async () => { + stop(); + await services.dispose(BACKGROUND_CONTEXT); + }; +} +``` + +```text +first acquisition -> commit base { strokes: [] } +addStroke(A) -> commit A -> local and remote subscribers see A +worker restarts -> open same durable storage; install canvas facet again +acquire source -> load base + committed deltas, without rerunning initial() +late client -> hydrate complete canvas including A, then ordered updates +conversation fork -> still uses this same Session canvas +``` + +Use persistent storage for restart survival. The memory backend is not persistent. +A method's successful commit does not promise every remote callback has run yet. + +## 2. Conversation-scoped diff reviews + +A **document family** uses one definition for many instances. The logical key is +`(definition, conversationId, instanceId)`; the persisted numeric document ID +identifies an incarnation and is never reused. Initializer input is not identity. + +```ts +type ReviewInput = { path: string; patch: string }; +type ReviewComment = { id: string; line: number; text: string }; +type ReviewState = ReviewInput & { comments: ReviewComment[] }; +const ReviewDoc = defineDocFamily({ + id: "app.diff-review", version: 1, family: true, scope: "conversation", + history: "latest", fork: "current", + initial: input => ({ path: input.path, patch: input.patch, comments: [] }), + checkpointWhen: value => value.comments.length % 50 === 0, +}); +interface DiffReviewService { + readonly state: ReplicatedState; + identity(context: Context): Promise<{ conversationId: Id; instanceId: string }>; + addComment(comment: ReviewComment, context: Context): Promise; +} +const DiffReviews = defineService("app.diff-reviews"); + +function reviewFacet( + session: Session, conversationId: Id, + reviews: readonly { instanceId: string; initial: ReviewInput }[], context: Context, +): Facet { + return defineFacet({ + id: "app.diff-reviews/session", + setup(env) { + const instances = env.provideMany(DiffReviews); + env.onActivate(async () => { + for (const review of reviews) { + const target = { scope: "conversation" as const, conversationId, ...review }; + const source = await session.documentSource(ReviewDoc, target, context); + const replica = await documentReplicatedState(source, context); + env.own(() => replica.dispose()); + // Chord instance keys route services; they are not persisted document IDs. + instances.spawn(JSON.stringify([conversationId, review.instanceId]), { + state: replica.state, + async identity() { return { conversationId, instanceId: review.instanceId }; }, + async addComment(comment, context) { + await session.commit(async tx => { + const draft = await tx.doc(ReviewDoc, target); + draft.comments.push(comment); // Do not mutate comment after this assignment. + }, context); + }, + }); // The facet owns spawned service lifetimes automatically. + } + }); + }, + }); +} +``` + +Example input: `[{ instanceId: "review-7", initial: { path: "a.ts", patch: "-old\n+new" } }]`. +Pass distinct instance IDs and a real conversation ID; install the facet as above. +Keyed consumers use `env.observe(DiffReviews, handler)`, not `env.use`. The handler +receives `(review, context)`; call `review.identity(context)` to identify it and +`review.state.subscribe` to observe comments. + +These comments are conversation-scoped: ending the generating task or unloading +the facet does not retire them. `current` copies every logically +present review at fork-commit time into independent child instances with new +numeric IDs, retaining family instance IDs. Parent and child then diverge. + +```text +parent review-7: comment A -> transcript entry E -> comment B +fork at E with current: child review-7 contains A and B +child adds C: parent still contains only A and B +``` + +With `fork: "initial"`, no instance copies; first child access uses supplied input. +`latest` cannot read history or fork `asOf`; use `history: "rewindable"` with `asOf` +to reflect E's commit. Reaccess ignores `initial`; updating the patch requires +an explicit mutation or a distinct review instance ID. + +## 3. Task-scoped output and a tool/task watch + +This is a bounded progress window, not the permanent result. Scope itself gives +it task lifetime; no history, fork, owner, or conversation setting is needed. + +```ts +type JobInput = { command: string }; +type JobOutput = { stdout: string; chunks: number }; +const JobOutputDoc = defineDoc({ + id: "app.job-output", version: 1, scope: "task", + initial: () => ({ stdout: "", chunks: 0 }), + checkpointWhen: value => value.chunks % 100 === 0, +}); +function jobOutputTarget(taskId: Id) { + return { scope: "task" as const, taskId }; +} +async function appendJobOutput( + runtime: TaskRuntime, + chunk: string, context: Context, +): Promise { + // Read process output outside this callback. The runtime gates the live task. + await runtime.commit(async tx => { + const draft = await tx.doc(JobOutputDoc, jobOutputTarget(runtime.taskId)); + draft.stdout = (draft.stdout + chunk).slice(-50_000); + draft.chunks += 1; + }, context); +} +async function observeJob( + api: DocumentObserver, producerTaskId: Id, + finished: Promise, context: Context, +): Promise { + const target = jobOutputTarget(producerTaskId); + const watch = await api.watchDoc(JobOutputDoc, target, context); + try { + console.log(watch.revision, watch.value.stdout); // Consume captured initial value. + watch.start((value, ops, revision, _context) => { + console.log(revision, value === null ? "retired" : value.stdout, ops); + }); // Drain buffered changes in order, then continue live delivery. + await finished; // Caller-supplied observation lifetime; outside any commit. + } finally { + watch.stop(); // Idempotent; discards any remaining buffer. + } +} +``` + +The running phase calls `appendJobOutput` and must also commit its next checkpoint +or terminal outcome. `api` is the invocation's `DocumentObserver`; `TaskRuntime` +includes that interface. Acquisition validates the target task is live and +derives its conversation from the task record. + +```text +watchDoc: capture value V at revision 8 + register listener atomically on Session line +producer commits revision 9 before start: buffer 9 +consumer reads watch.value (V), calls start: deliver 9, then subsequent revisions +producer becomes terminal: same commit retires output; watch receives null + ["r", null] +``` + +Watches automatically stop when their invocation ends. A Session-acquired watch +is caller-owned and stops on Session close. Retirement ends the incarnation's +stream; recreation requires a new watch/source and a new numeric +document ID. A terminal task cannot create more output. Task-scoped documents +never copy into forks. Preserve required output in result entries or Session- or +conversation-scoped documents in the terminal commit before retirement. Use a +task-scoped family only when one task needs several independently keyed documents. + +## 4. Built-in conversation views and commit delivery + +`ConversationView` contains `conversation`, raw transcript `entries`, and +`docs: Readonly>`. Selected built-in singletons mount +under `docs` by stable definition ID, not numeric incarnation ID. The spec's +illustrative path mapping is: + +```text +document ["s", ["message"], value] + -> view ["s", ["docs", "pi.live", "message"], value] +``` + +Built-in IDs/fields await approval; `pi.live` is illustrative, not an available API. +The mount publishes one batch per complete Session commit: entry/head changes and +changed mounted documents together, without a tracker or semantic projection. +Third-party documents are **not automatically mounted**; use their own Chord +services or trusted invocation watches. A retired source publishes `null`; a +dynamic service should then withdraw its instance rather than expose stale data. + +```text +addStroke -> hold Session mutation line -> await tx.doc -> mutate tracked draft +callback succeeds -> tracker flush: incremental ops + candidate value +atomic storage commit: all document and record writes; checkpoint predicate selects ops or base +storage succeeds -> adopt committed baseline + enqueue source publication, still on line +release line -> deliver committed source ops -> adapter -> local/remote Chord consumers +late subscriber -> atomically capture committed value/revision + subscription +``` + +No visible-undurable path exists. Callback failure restores the unchanged +baseline. Failure after flush, including checkpoint/storage failure, publishes +nothing and poisons the open Session; close and reopen it instead of continuing. + +## Footguns + +- Never retain drafts, nested proxies, or bound array methods across commits. + Never mutate inserted aliases or insert one mutable object at multiple paths. +- Async commit holds the line through storage settlement and baseline adoption. + Await document access there, not models, processes, network calls, or humans. +- Normal `snapshot`, `documentSource`, and `watchDoc` reads are get-or-create. + Family `initial` is first-creation input, not an update. +- An unstarted watch buffers without a fixed bound. Start or stop promptly. +- Definitions own checkpoints, not storage heuristics. Revise the counting predicates + above if mutations change. Keep schema IDs, versions, fork policies, and public + paths stable; schema changes require migration, not a source-only rename. + +Chord sources: [types](../../chord/src/types.ts), [facet examples](../../chord/test/facets.test.ts), +[state](../../chord/src/services/state.ts), [provider](../../chord/src/services/provider.ts), +[Context](../../chord/src/context/index.ts), [Delta](../../chord/src/delta/README.md). diff --git a/packages/agent/docs/pico-v5-handoff.md b/packages/agent/docs/pico-v5-handoff.md index 93372b466..9c3eddf20 100644 --- a/packages/agent/docs/pico-v5-handoff.md +++ b/packages/agent/docs/pico-v5-handoff.md @@ -1,555 +1,266 @@ -# Pico5 handoff and decision register - -> Working coordination document. `pico-v5.md` is the compact design. This file -> records accepted choices, unresolved decisions, examples, and layer sign-offs so -> work survives conversation compaction. No unresolved item may be decided silently -> during implementation. - -## Current status - -- `packages/agent/src/harness/pico/` and its tests were removed. -- The untracked `pico4` source/tests were removed. -- `pico3` remains as a reference, not an implementation base. -- Storage benchmark artifacts are in `/tmp/pi-pico5-storage-bench/`. -- Astra approved starting the memory + SQLite storage/conformance slice after the - storage decisions below are signed off. - -## Accepted direction - -1. Entries are immutable and paged; the Session is not one materialized DOM. -2. Authors mutate ordinary JSON documents; Chord produces nested/list/string ops. -3. History policy is per document: `latest` or `rewindable`. -4. Built-ins use three conversation documents: - - `rewindable`: historical configuration; - - `sticky`: present state such as queues and retry policy; - - `live`: high-churn turn, generation, tool, and compaction presentation state. -5. Typed document definitions and dynamic families are required. Family instances - may be task-owned and retire automatically. -6. Task recovery state is a complete replacement, separate from presentation and - application documents. -7. There is no visible-but-undurable publication. Every Chord-visible update follows - a successful durable commit. -8. Interrupted assistant/tool partials are converted into durable aborted/interrupted - entries by their recovering task kind, then removed from `live`. -9. A built-in needs cross-task document observation, so `watchDoc` is initial scope. -10. UI status is state, not an event protocol. There is no public event channel - initially; clients observe document paths. A bounded `live` notice field may be - designed with the built-in view if concrete transient facts require it. -11. Third-party documents initially use their own Chord services. Automatic mounting - into the agent view is deferred. -12. SQLite is the long-running/cloud backend. JSONL remains local and readable. -13. Implementation proceeds layer by layer and stops for user sign-off after each. - -## Decisions required before storage implementation - -### S1. Document identity and families — accepted - -A definition carries type, logical ID, scope, history, initializer, and optional -family ownership rules: - -```ts -const live = defineDoc({ - id: "pi.live", - scope: "conversation", - history: "latest", - initial: () => ({ tools: [] }), -}); - -const jobOutput = defineDocFamily({ - id: "pi.job", - scope: "conversation", - history: "latest", - owner: "task", - initial: input => ({ command: input.command, stdout: "" }), -}); -``` - -A persisted instance has a unique, never-reused storage identity plus definition ID, -scope owner, family instance ID, creation sequence, history policy, and retirement -sequence. Reusing one logical family key creates a new incarnation. - -### S2. Historical document membership — accepted - -A rewindable document retired today still exists in an as-of query from yesterday. -A latest document can be deleted after retirement. - -Example: - -```text -10 create rewindable review A -20 append entry E -30 retire A -40 fork at E -``` - -The fork enumerates A at commit 20 and copies its as-of value into a new child-owned -instance. Current enumeration at commit 40 omits A. - -### S3. Storage ownership and copying — proposed recommendation - -Simplest safe contract: - -- Commit inputs are borrowed until `Storage.commit()` settles. -- A backend retaining anything after settlement detaches it first. -- Memory uses a small recursive `copyJson`; JSONL/SQLite detach through serialization. -- Every storage read returns a fresh owned JSON copy. Mutating it cannot corrupt the - backend or another caller. Public types remain readonly, but correctness does not - rely on TypeScript at runtime. -- Large strings are immutable and may share their string value during an in-memory - copy; object/array containers never share mutably. - -Example failure this prevents: +# Pico5 implementation handoff -```ts -const task = { phase: "running" }; -await storage.commit([{ type: "task", task }]); -task.phase = "done"; // must not mutate MemoryStorage behind its back -``` - -Decision: approve this contract, or replace it with explicit ownership transfer and -state which caller may still use document ops for Chord publication. - -### S4. Table reads and writes inside `Tx` — proposed recommendation - -Adopt the other design's simple rule: - -```text -table reads first -> first table write -> later table reads throw ReadAfterWrite -document reads/mutations always read their transaction draft -``` +`packages/agent/docs/pico-v5.md` is normative. Implement this list in order. +After every package: run its tests, run `npm run check`, and stop for user review. +Do not redesign later packages while implementing the current one. -Example: +Pico3 is reference material only. Preserve useful behavior, not its capability +facades, membranes, document routing, view projection, events, or clone chains. -```ts -const inputs = await tx.inputs(ids); // read -const head = await tx.newestEntry(c); // read -const entry = tx.appendEntry(c, assistant); // first table write -tx.doc(live, c).message = undefined; // document draft is allowed -// await tx.entry(other) now throws ReadAfterWrite -``` +## Status -Benefit: no shadow indexes for newly created entries/tasks/inputs. Cost: helpers must -load everything before their first table write. Validate all built-in settlement -traces before accepting. +- Obsolete `pico` and `pico4` prototypes were removed. +- `pico3` remains. +- No Pico5 implementation exists. -Decision: adopt this rule, or support read-your-writes for tables and pay for overlays. +## 1. Records, cursors, and memory tables -### S5. Escaped document draft — open +Implement IDs, sequences, conversations, entries, inputs, live/terminal tasks, +document metadata, storage writes, cursors, and detached `MemoryStorage` tables. -A callback can retain a document proxy: +Test mixed atomic commits, rollback, detached reads/writes, cursor boundaries, +fork-aware entry scans through deep ancestor caps, head lookup, +entry-to-commit lookup, and full task replacement. -```ts -let escaped: LiveState; -await session.commit(tx => { escaped = tx.doc(live, c); }); -escaped.message = partial; -``` +## 2. Memory document records -Options: +Add document creation bases, deltas, required/checkpoint bases, retirement, +reincarnation, current/as-of membership, and base-plus-tail reconstruction. -- **Detached stale draft:** mutation succeeds locally but has no effect. Simplest, - but silently hides a bug. -- **Revoked draft:** every later read/write throws. Clearest, but requires tracker - proxies to share an active guard or another membrane. -- **Debug assertion:** stale mutation is detected at the next transaction. Cheaper, - but delayed. +Test Session-, conversation-, and task-scoped documents, retired historical +membership, family queries, and no scans of unrelated document records. -Recommendation: first determine whether Chord tracker proxies can cheaply share one -revocation guard. Do not let an escaped proxy mutate committed state or contaminate a -later commit. +## 3. SQLite backend -### S6. Base policy — partly accepted +Implement the complete storage contract with ordinary rows and indexed document +records. Do not translate Chord operations into SQL JSON patches. -Every document may receive bases. Storage gets both incremental ops and a borrowed -candidate value, serializing only one: +Run the memory conformance suite after reopen. Test SQL transaction rollback, +recent/ancient as-of reads, query plans, latest reclamation, WAL checkpointing, +deleted-page reuse, and representative storage sizes. -```ts -persist(shouldBase(id, ops) ? [["r", value]] : ops); -``` +## 4. JSONL publication -- `latest`: a committed base permits reclaiming older records. -- `rewindable`: bases bound as-of replay; historical segments remain. -- Trigger on both encoded bytes and record count. +Implement table writes in `main.jsonl`, one document sidecar per incarnation, +one sidecar per live task, and one main marker per commit. Do not add a +standalone-sidecar protocol. -Open values: +Fault-test torn/short sidecar writes, failures between sidecars, every marker +boundary, unconfirmed tails, missing confirmed data, and poisoned writes. -- byte threshold; -- record-count threshold; -- whether document definitions may request `rebaseWhenInitial`. +## 5. JSONL reclamation -`pi.live` benefits from `rebaseWhenInitial`: when a turn ends at `{ tools: [] }`, a -tiny base allows immediate reclamation of streamed output. The definition/document -manager can recognize its initial state; storage still owns physical reclamation. +Implement task-document retirement and current-only base reclamation using +committed markers, temporary replacement, rename, and descriptor invalidation. -### S7. Task rows — accepted +Crash-test every rewrite/rename boundary. Verify that rewindable history is +never reclaimed and default no-fsync behavior matches the specification. -Task records are full replacements. SQLite uses one live row and `DELETE` on terminal. -JSONL uses one task sidecar and unlinks it only after terminal publication. +## 6. Tracker transaction core -The runtime envelope owns kind/input/dependencies/memos/effect IDs. Kind-defined -recovery state cannot overwrite memos. A terminal transaction transfers any needed -outcome and retires every task-owned document family instance atomically. +Keep one tracker per loaded document. Add transaction begin, callback rollback +from the unchanged baseline, flush, fatal post-flush failure, committed baseline +adoption, and eviction. -This atomic retirement removes the need for an application-level orphan sweep: +Do not add membranes, transaction proxy graphs, capability facades, or defensive +full-document clones. Test callback failure, no-op transactions, storage failure +poisoning, and unload/reload. -- crash before terminal: task and owned docs remain live and recover; -- crash after terminal marker: logical retirement is known, and storage may clean - leftover files during open. +## 7. Document definitions and access -### S8. SQLite representation — accepted direction +Implement `defineDoc`, `defineDocFamily`, identity validation, the three direct +scopes, and get-or-create `tx.doc`, `snapshot`, and `documentSource` acquisition. -Use one SQL transaction per Session commit: +Test concurrent initialization once, initial bases, detached snapshots, family +initializer use only on first creation, scope/target mismatch, terminal-task +rejection, task-derived conversation identity, retirement, and +reincarnation-bound sources. Include create-task-then-document, +document-after-terminal rejection, and create-document-then-terminal settlement +in one transaction; internal candidate validation must not trigger +`ReadAfterWrite`. -```text -conversations / entries / inputs -document metadata -document base+delta records indexed by document and commit -a current live-task row -``` +## 8. Checkpoints and migration -Do not translate Chord ops into SQLite JSON functions initially. The benchmark found -full JSONB replacement slower than TEXT and generic SQL patching slower with more WAL -writes. Current/as-of reads select a base and indexed tail; they do not scan unrelated -documents. +Inside `Storage.commit()`, evaluate `checkpointWhen(value, ops)` exactly once +for ordinary mutations. Implement required creation/version bases and +all-older-version migration. -Still choose schema details, WAL checkpoint cadence, and `synchronous` default during -the SQLite layer. +Test latest migration persistence, rewindable migration on current/historical +read, first post-migration mutation base, newer-version rejection, absent plugin +preservation, and checkpoint starvation without backend heuristics. -### S9. JSONL physical layout — current lean: uniform sidecars +## 9. Conversation document forks -Simplest layout from the other design: +Using fixture conversations and entry-to-commit mappings, implement the `asOf`, +`current`, and `initial` settings for singleton and family documents. -```text -main.jsonl tables + one publication marker per commit -doc-.jsonl one file per document instance -task-.jsonl one file per live task -``` +Test unknown definitions, retired membership, new child incarnations, lazy +`initial` creation, and exclusion of task- and Session-scoped documents. -Every sidecar record is prepared. Sidecars append first; one main marker is always -last. This deliberately permits records such as: +## 10. Chord structural array operations -```json -{"seq":14,"refs":["task-9"]} -{"seq":15,"refs":["task-9"]} -{"seq":16,"refs":["pi.live-1"]} -``` +Make ordinary positional mutations encode scattered removals without carrying +retained payloads. Callers must not write operations manually. -The markers are cosmetic overhead but make every commit use one protocol. They can be -compact; fields such as `writes:[]` and repeated `maxId` are unnecessary if high-water -is represented elsewhere. Without `fsync`, the second append is unlikely to be the -performance bottleneck; the benchmark's no-fsync sidecar backend still handled about -9.5k–30k commits/s depending on workload while opening/closing each sidecar. +Test front/tail/middle/scattered/all/no removal, retained 256 KiB and 1 MiB +payloads, append plus removal, later nested/index writes, exact replay, and +unchanged previous immutable snapshots. Internal flushes must remain one commit. -Alternative: standalone one-file commits avoid markers but introduce two record modes, -distributed commit discovery, and more recovery cases. +## 11. Chord document source -Decision: approve uniform marker publication or select the standalone optimization. +Add the thin opaque-source adapter to `ReplicatedState`. It must use committed +value/ops directly, with no tracker or re-diff. -### S10. JSONL process-crash contract — accepted direction +Test contiguous revisions, atomic hydrate/subscribe, attachment races, +retirement ending one incarnation, recreation requiring reacquisition, and +listener isolation. -No `fsync` by default. Awaited/full sidecar writes precede the awaited/full main -marker. For an ordinary process crash with the OS/filesystem alive: +## 12. Document watches -- torn final lines are removed; -- sidecar records without a marker are uncommitted and removed; -- a complete marker publishes the transaction. +Implement get-or-create `watchDoc`: capture value/revision and register its +listener atomically, buffer without a cap until `start()` or `stop()`, and always +deliver value plus ops. -Power loss, host/kernel failure, or weak filesystem ordering may lose acknowledged -commits. Optional durable mode may fsync sidecars before the marker. +Test ordered draining, live delivery, retirement as absence, idempotent stop, +listener errors, and explicit cleanup. Invocation-owned cleanup is integrated in +package 15. -Any uncertain write error poisons the open backend until reopen. Reclamation occurs -only after its base/retirement commits and uses temp + rename; cached descriptors are -then reopened. +## 13. Conversations and entries -### S11. Missing confirmed sidecar — open +Implement conversation history/ownership records, entry creation, cursor-based +fork-aware scans, head lookup, and entry edits. -Example: +Test conversation creation and actual forks, deep ancestor caps, same-commit +entry prefixes, newest-edit wins, self-head resolution, raw head-to-tail +transcript, and ownership traversal. -```text -sidecar record 20 was reported written -main marker 20 survives -sidecar record 20 is absent after machine failure -``` +## 14. Context derivation and system messages -This should not arise from the stated ordinary process-crash contract. Options: +Implement model-context reduction, PR #9548 positional `SystemMessage` replay, +tool-result ordering, and missing post-fork tool results. Replay `content`, +ordered named `sections` with `null` removal, then tool removals/additions. -- **Fail open:** report corruption and preserve all bytes for diagnosis/repair. - Simplest and never invents state. -- **Roll back globally:** truncate every file to the latest complete global prefix. - Recovers availability but may discard acknowledged later commits and must interact - correctly with reclaimed prefixes and standalone records if those exist. -- **Best-effort skip:** invalid because it exposes a partial atomic commit. +Test model-less and excluded-stop-reason entries, replacements/omissions, +multiple heads, section replacement/removal/re-addition order, rejection of +integer-like section keys, tool addition/removal/replacement order, and raw-view +versus model context. -Recommendation: fail open initially. Add a separate repair tool only after physical -fault tests establish a safe prefix algorithm. +## 15. Task definitions and invocations -### S12. Main JSONL growth — accepted limitation +Implement `defineTask`, exhaustive phase maps, full checkpoint replacement, +kind migration, runtime commits, memos, and invocation close gates. -Document/task sidecars reclaim high-churn data; `main.jsonl` still grows by entries, -lifecycle records, and commit markers. JSONL may scan it on open. It is the local -backend. Years-long products use SQLite. Do not add global JSONL compaction initially. +Use a fake two-phase effect. Test intent/effect/outcome recovery, +unchanged-checkpoint faulting, same-phase checkpoint progress, cancellation +precedence, thrown-handler faulting, close/reopen without abort marks, outcomes, +or task-document retirement, no fresh phase/abort dispatch while closing, +first-writer-wins memos, and automatic watch cleanup. -### S13. Storage query surface — needs exact signatures +## 16. Scheduler and terminal tasks -The first storage contract must support: +Implement reservation, running-task reopen reconciliation, dependencies, +terminal outcomes, waits, holds, joins, and orphaning. -- create/current/retired conversation records; -- paged immutable entries and entry -> commit mapping; -- current task lookup/scan; -- current and as-of document enumeration; -- current and as-of document value; -- family-instance enumeration by definition/scope/owner; -- one atomic mixed commit; -- close/reopen. +Test result values and entry IDs, terminal records after reopen, dependency +eligibility, unknown kinds, and terminal removal of checkpoints/memos. -As-of results distinguish not-yet-created, retired after target, and unavailable -history. Concrete TypeScript signatures are the first layer's first sign-off. +## 17. Abort and owned conversations -## Decisions required before document/task implementation +Implement durable abort marks, signal/join/fresh-abort invocation, owned +conversation creation, subtree traversal, background behavior, and idle waits. -### D1. Three built-in documents — accepted +Test commit rejection after a run task is marked, crashes at every abort stage, +close precedence over a previously marked task, deep ownership trees, and atomic +retirement of task-scoped documents. -```ts -rewindable // model/tool/config choices inherited by historical fork -sticky // inbox, queue modes, retry policy -live // current message/tools/generation/collapse and maybe notices -``` +## 18. Inputs and positional inbox -`live` is durable because recovery converts its partial output into transcript entries. -Its high churn is independently reclaimable when it returns to initial state. +Define the initial inbox and turn-control documents, then implement request-ID +deduplication, awaitable input handles, busy admission, withdrawal, queue modes, +and `postTools`/`final` boundaries. Use a fake successor task. -### D2. Document adapter to Chord — accepted direction +Table-test every input transition, interleaved steer/follow-up/write selection, +self-head cuts, stale targets, successor triggers, reopen waits, compact +large-payload removals, and orphan/fault cleanup of active turn control. -A thin adapter such as `replicatedState.own(documentSource)` exposes committed value -and ops. It owns no second tracker. Each document and mounted view has a contiguous -publication revision separate from sparse global Session commit sequence. +## 19. Remaining built-in documents and view -Hydration captures value/revision and subscribes atomically. Listener errors after -persistence are isolated. Current Chord needs a small public source adapter; it does -not need harness knowledge. +Define the concrete configuration, preference, and live presentation documents; +reuse the approved inbox/turn definitions. Record all IDs, fields, history, +fork settings, migration, and checkpoint predicates in the normative +specification. -### D3. Cross-task `watchDoc` — accepted, exact scope open +Implement `{ conversation, entries, docs }`. Test direct task writes, one +publication per Session commit, atomic +entry/preview settlement, head changes, contiguous revisions, stable public +paths, retry/collapse late-join status, bounded-output truncation metadata, and +absence of semantic projection. Specify which diagnostics become entries, +terminal details, or bounded document state. -Required semantics: +## 20. Registries, hooks, and sections -```ts -const watch = await api.watchDoc(jobOutput, job.id, context); -consume(watch.value); // snapshot first -watch.start((ops, value) => consume(value)); -``` +Implement task/tool/entry/section registries, Session and owned-subtree hooks, +positional PR #9548 section/tool updates, complete baselines after a head cut, +and preparation revision checks. Do not add a public event stream or +plugin-state router. -- capture + subscription are one Session-line operation; -- callbacks run off the line and cannot re-enter a commit synchronously; -- watch is volatile and invocation-owned; -- recovery re-enters the handler, captures a fresh snapshot, and subscribes again; -- retirement appears as document absence; -- `stop()` is idempotent. +Test registration lifetimes, hook replay with memos, exact persisted rendered +section strings, minimal section patches and `null` removals, complete baseline +tool declarations, and a head cut that retains earlier system messages. Verify +the new baseline entry omits those messages through `ContextEdit` before replay, +including retained `content`, section order, and tool changes. Include a retained +delta whose ID precedes the head-carrying entry: select omissions by retained +context membership, not an ID comparison with the head entry. Test tool loadout +additions/removals and preparation retry after registry movement. Do +not implement in-process replacement of Session-side plugin code; a host plugin +change uses the Harness close/reopen boundary. -Open scope: same conversation only, or any conversation owned by the observing task's -subtree. Initial built-in use must provide the concrete required edge. +## 21. Tool and post-tools tasks -### D4. Reopen conversion — accepted +Implement offered-set checks, argument validation, tool hooks, durable bounded +progress, owned APIs, interrupted/replay-safe recovery, result entries, +post-tools joining, controls, and boundaries using fake tools. -Generation recovery reads `live.message`, appends an assistant entry with -`stopReason: "aborted"`, clears the preview, and retries/fails atomically. Tool -recovery turns persisted output into its interrupted result before clearing the slot. -The entry uses the normal model/render shape while model projection excludes aborted -content. +Test recovery from every phase, both stored/current replay-policy directions, +and atomic assistant/tool/post-tools settlement. Use a fake generation +successor; package 22 replaces it and reruns integration. -Conversion runs before retiring any task-owned family document it needs. +## 22. Generation task -### D5. Document versioning/migration — open +Implement preparation, request intent, durable throttled partials, attempts, +retry policy, response classification, continuation, and deferred polling/ +cancellation with the faux provider. -Document paths and persisted values are public/durable schema. A plugin upgrade may -change `{ comments: [...] }` to `{ threads: [...] }`. +Test every phase before and after reopen, aborted partial conversion, overflow +through a fake collapse kind, input settlement, and no visible-undurable update. +Replace the fake tool successor and rerun the package 21 integration tests. -Options: +## 23. Collapse task -- definition has an integer version and migration from complete current value; -- task/document kind owns migration during open; -- reject mismatched versions initially. +Implement manual, threshold, and overflow collapse; exchange-boundary range +selection; summarization; retries; staleness; and headed summary entries. -Recommendation: persist definition version now even if initial implementation only -rejects mismatches. Retrofitting identity/version after stored products ship is harder. +Test context before/after collapse, provider failure, declined/stale work, and +reopen from every phase. Replace generation's fake overflow target and rerun its +overflow integration test. -## Decisions required before view implementation +## 24. Job and plugin tasks -### V1. Public view shape — open +Implement process jobs with bounded durable output, interruption recovery, +rescheduling, and registered plugin handlers. -Fixed top-level fields: +Test process exit/error/abort/reopen, background idle behavior, notifications as +state or entries, and plugin handler recovery. -```ts -{ conversation, entries, rewindable, sticky, live } -``` +## 25. Harness integration -Typed and direct, but every added built-in document changes the protocol. +Expose conversation/input/task handles, watches, dynamic registries, close, +suspend, service withdrawal/client detach, and product wiring. Implement the v1 +host-plugin reload path as stop admission, close/join, dispose, rebuild with all +new definitions, reopen/migrate, and resume. Test that closing seals commit and +get-or-create admission, lets storage settlement for already-flushed admitted +commits finish despite caller cancellation, writes no abort or terminal outcome, starts no fresh abort +invocation, and does not run old and new generations concurrently. -Document map: - -```ts -{ - conversation, - entries, - docs: { - "pi.rewindable": RewindableState, - "pi.sticky": StickyState, - "pi.live": LiveState, - }, -} -``` - -Makes document identity/lifetime explicit and can add selected documents later, but -paths and IDs become public protocol. It can remain typed for the three built-ins. - -Recommendation: decide before writing any UI adapter. In either shape, mount only the -three built-ins initially and treat every visible path as versioned public API. - -### V2. Public events — accepted: none initially - -A compaction-start event cannot describe current state to a late joiner. Instead: - -```ts -live.collapse = { through, stage: "summarizing" }; // running -live.collapse = undefined; // finished -``` - -Clients react to state ops by path and always hydrate the current truth. Hooks remain -pre-decision interception. Internal commit subscribers may exist mechanically, but -there is no separate public semantic event protocol initially. - -Whether failed attempts need a bounded `live.notices` list is deferred until the UI -is implemented. It remains state with explicit retention, not fire-and-forget events. - -### V3. Atomic publication — accepted - -The built-in mount consumes the complete Session commit, prefixes every changed -built-in document path, combines entry changes, and publishes one Chord batch. It has -no tracker or semantic projection. Independent third-party Chord services may publish -separately. - -### V4. Active transcript bound — open - -A Slack conversation cannot hydrate every entry ever written. Options include: - -- durable `head` entry: active transcript is the range beginning at its target; -- fixed recent entry/count/byte window; -- product-selected window with a stable cursor. - -Example requirement: - -```text -10 years / 500,000 entries -late UI attach -> bounded snapshot -older history -> explicit paged query -model context -> independently derived bounded projection -``` - -Choose before view implementation, not before storage. Entry paging and indexes are -required now regardless. - -### V5. Multi-entry fork boundary — open - -One commit may append entries E and F and leave one final rewindable state. Fork at E: - -- **entry prefix:** child transcript includes E but not F, while document state is the - final state of the containing commit; -- **commit boundary:** selecting E means inherit all entries from its indivisible - commit, including F; -- **restriction:** only designated forkable entries/commits may be targets. - -Choose before fork/view API. Storage records entry -> commit either way. - -### V6. Sticky fork initialization — current lean: defaults - -A historical fork inherits rewindable documents as-of its target and no live tasks. -Copying current sticky/live state can copy queued inputs or running previews from the -future. Simplest initial rule: - -```text -rewindable -> historical copy -sticky -> fresh defaults -live -> fresh defaults -``` - -Explicit sticky-copy options remain deferred. - -### V7. Third-party mounting — accepted defer - -Third-party documents initially expose application-specific Chord services. Do not add -`inView`, privacy flags, dynamic mount registries, or client-selected arbitrary docs. -A document is not a security boundary; services deliberately choose what crosses a -process boundary. - -## Layer plan and sign-off gates - -### Layer 0 — cleanup - -- Remove `pico` source/tests: done. -- Remove `pico4` source/tests: done. -- Keep `pico3`: done. -- Run repository check after deletions and document additions: pending. - -### Layer 1 — logical storage + memory reference - -Before code: sign off S3 and S6, retain accepted S1/S2/S7, and approve the exact S13 interfaces. - -Acceptance: - -- atomic mixed commits; -- document creation, bases, current/as-of membership and values; -- family instances and task ownership; -- full task replacement/retirement; -- entry paging and entry -> commit; -- ownership isolation and failure rollback. - -Stop for user review. - -### Layer 2 — SQLite - -Before code: approve concrete schema and durability defaults. - -Acceptance: Layer 1 conformance plus reopen, indexed query plans, latest reclamation, -retired rewindable history, WAL checkpoint behavior, and realistic storage benchmark. - -Stop for user review. - -### Layer 3 — JSONL - -Before code: decide S9 and S11. - -Acceptance: semantic conformance plus physical fault injection at every sidecar, -marker, rewrite, rename, retirement, and reopen boundary. - -Stop for user review. - -### Layer 4 — document manager + Chord source - -Before code: decide S4, S5, D3 scope, and D5. - -Acceptance: ordinary mutations, no duplicate tracker, failed-commit rollback, -contiguous publication, hydration fencing, listener isolation, eviction, and -`watchDoc` snapshot/update/retirement behavior. - -Stop for user review. - -### Layer 5 — tasks - -Acceptance: phase-narrowed full replacements, effect sandwich, runtime-owned memos, -abortable ownership tree, atomic family retirement/outcome handoff, generation/tool -reopen conversion, and no visible-undurable state. - -Stop for user review. - -### Layer 6 — built-in documents and conversation view - -Before code: decide V1, V4, V5, and V6. - -Acceptance: direct built-in writes, fixed structural mount, one publication per commit, -bounded late hydration, crash-visible progress, and no public event dependency. - -Stop for user review. - -### Layer 7 — remaining harness - -Inputs, hooks, tools, system sections, dynamic tools, deferred providers, and product -wiring. Preserve accepted APIs from earlier design work rather than redesigning them -inside the state implementation. +Run all package-specific tests and the repository check. Verify a local +coding-agent turn and a reopened interrupted turn, then stop for final review. diff --git a/packages/agent/docs/pico-v5-slides/index.html b/packages/agent/docs/pico-v5-slides/index.html index c845fa434..e9272f36e 100644 --- a/packages/agent/docs/pico-v5-slides/index.html +++ b/packages/agent/docs/pico-v5-slides/index.html @@ -3,676 +3,685 @@ - Pico5 — durable documents + Pico5 — one harness, long-running products
← → · space · f
-
-
-

Pico5

-

Durable documents without a whole-session DOM

-

Usage, state → view, and why Lane stops short

+
+

Pico5

+

One durable harness for long-running agent products

+

Conversations · tasks · documents · entries

+
+ +
+

What is the harness?

+
+
product sends
input
→
+
harness runs
model + tools
→
+
harness persists
conversation + work
→
+
clients observe
current truth
-
+

The process may disappear between any two arrows.

+
-
-
-

The target is not one coding session

-
-

Products

  • coding agent
  • personal assistant
  • cloud agents in Durable Objects
  • Slack bots across years of channels and threads
-

Required invariants

  • survive process death mid-effect
  • fork historical conversation state
  • attach many presentations
  • let facets persist application state without inventing storage
-
+
+

One core, different products

+
+

Slack bot

Channels and threads run for years. Most are cold.

+

Personal assistant

Many conversations share preferences, approvals, and application state.

+

Cloudflare DO agent

One durable owner sleeps and restarts between requests.

+

Coding agent

Long tool calls, streaming output, forks, compaction, subagents.

-
+
-
-
-

Lane solved a real problem

-
-

Execution

Complete operation-state replacements make recovery a switch, not inference.

-

Effects

Durable intent → uncertain effect → durable outcome.

-

Transactions

Entries, values, and lists commit atomically in memory, JSONL, and SQLite.

-
-

Keep those properties. Replace the application-state substrate.

+
+

The scale is not “one chat in RAM”

+ 
Session
+├─ #incidents                     active conversation
+├─ thread 1719432.0021            active tool call
+├─ #support / customer-482        idle for 8 months
+├─ personal calendar workflow     shared application state
+└─ 100,000 more conversations     on disk, not materialized
+
    +
  • Entries are paged.
    • +
  • Documents load independently.
    • +
  • A client subscribes to what it needs—not one whole-Session DOM.
    • +
+
+ +
+

Crash behavior is product behavior

+ 
12:00:00  assistant starts streaming
+12:00:02  durable preview: "I found three…"
+12:00:03  bash output checkpoint: 34 KiB
+12:00:04  worker dies
+12:00:10  new worker opens the Session
+
+UI shows the committed assistant and bash prefixes.
+Recovery records them as interrupted, then retries only when safe.
+

A client must never see state that recovery cannot load.

+
+ +
+

The harness must also serve applications

+
+

Canvas

Several clients append strokes to shared durable state.

+

Diff review

Comments are edited collaboratively, then sent to the model.

+

Background job

A tool watches durable output after starting work elsewhere.

-
+

An extension should not have to invent persistence, replay, and replication.

+
-
-
-

Lane exposes values and lists

- 
const review = value<ReviewState>("app.review", id);
-const findings = list<Finding>("app.findings", id);
+  

+    
What Lane got right

+    
TX 10  operation = { phase: "requesting", requestId: "r1" }
+       // durable intent
 
-await session.setValue(review, completeNextReview, context);
-await session.appendList(findings, finding, context);
-await session.deleteList(findings, context);

-      
    
-        
  • A scalar is replaced whole.
    • 
-        
  • A list appends or is deleted whole.
    • 
-        
  • Nested changes, splices, checkpoints, and reconstruction belong to the caller.
    • 
-      

-    

-  

+       provider request runs
+       // process may die here
 
-  

-    

-      
Canvas on Lane: choose your own machinery

-      

-        

-          
Store the whole canvas

-          
const CanvasValue = value<CanvasState>("app.canvas");
-const stored = await session.getValue(CanvasValue, context);
+TX 11  assistant entry + operation = { phase: "tools", calls: [...] }
+       // durable outcome

+    
Atomic transactions and explicit restart points were correct.

+  

+
+  

+    
Lane’s state machine was one fixed machine

+    
type OperationState =
+  | Starting | Checkpoint | AssistantReady | AssistantEffectPending
+  | AssistantRetryWait | Tools | DeferredSuspended | DeferredEffectPending
+  | SummaryDeciding | SummaryReady | SummaryEffectPending
+  | SummaryRetryWait | NavigationReadyToCommit;
+
+switch (state.at) {
+  // one core driver understands every operation and recovery edge
+}

+    
Adding a first-class durable operation kind required changing the core graph and recovery.

+  

+
+  

+    
Lane’s application storage was values + lists

+    
value<T>       set or replace the complete value
+list<T>        append one element or delete the whole list
+entry          append immutable transcript content
+transaction    commit all writes atomically

+    
Good storage primitives. Poor extension programming model for nested shared state.

+  

+
+  

+    
Canvas with a Lane value: naive split read/write

+    
const Canvas = value<CanvasState>("app.canvas");
+
+const stored = await session.getValue(Canvas, context);
 const next = stored?.value ?? { strokes: [] };
 next.strokes.push(stroke);
-await session.setValue(CanvasValue, next, context);

-          
Simple API; complete serialization and replacement for every stroke.

-        

-        

-          
Store stroke deltas

-          
const StrokeLog = list<StrokeOp>("app.canvas.strokes");
-await session.appendList(StrokeLog, toStrokeOp(stroke), context);
-// define reducer and operation vocabulary
-// define checkpoints
-// coordinate append + compact
-// reconstruct after restart

-          
Efficient writes; every facet authors a private state engine.

-        

-
- -
+await session.setValue(Canvas, next, context); +
    +
  • Read, copy, and serialize the complete canvas per stroke.
    • +
  • Use Lane’s existing session.mutate() to serialize read-modify-write.
    • +
  • Maintain a separate replicated UI copy.
    • +
+ -
-
-

Diff review on Lane: durability leaks upward

- 
const ReviewRecord = (id: string) => value<Review>("app.review", id);
-const activity = env.replicatedState({ review: initialReview });
+  

+    
Canvas with a Lane list

+    
const CanvasOps = list<CanvasOp>("app.canvas.ops");
+await session.appendList(CanvasOps, {
+  type: "stroke.added", stroke,
+}, context);
 
-return regionFor(reviewId).run(signal, async () => {
-  const stored = await session.getValue(ReviewRecord(reviewId), context);
-  const next = validateAndReplace(stored!.value, action);
-  await session.setValue(ReviewRecord(reviewId), next, context);
-  activity.state.review = project(next);
-  activity.publish(context);
-});

-      
    
-        
  • The facet owns locking and read-modify-write correctness.
    • 
-        
  • It reconstructs live state after worker restart.
    • 
-        
  • Persistence and replication are two separately coordinated writes.
    • 
-      

-    

-  

+// The extension now owns:
+foldOperations(); checkpoint(); compact(); restore(); publishToClients();
+

Small writes—but every extension starts building its own document store.

+
-
-
-

Lane forks transcript, not application state

- 
branch A:  e1 ─ e2 ─ e3 ─ e4
-                         └──── fork at e3 → branch B
+  

+    
Durability and UI replication split

+    
await session.setValue(ReviewValue, next, context); // durable copy
 
-value("plan-mode")       current replacement only
-list("canvas-strokes")   surviving elements only
+uiState.state.review = next;                            // second mutable copy
+uiState.publish(context);                               // second publication
 
-question: what was either value at e3?    not representable

-      
The JSONL bytes may contain old writes. The storage contract does not expose them as history.

-    

-  

+// On restart: read storage, reconstruct uiState, avoid races.
+

Every facet owns the seam between storage and Chord.

+
-
-
-

The gap is narrow but foundational

-
-

Lane gives facets

  • typed scalar replacement
  • append/delete list
  • atomic storage calls
  • separate volatile ReplicatedState
-

Products need

  • ordinary nested mutation
  • automatic delta + base policy
  • historical document reads
  • commit and replication as one path
-
-

Lane remains a good execution design. It is not enough as the state substrate for products and facets.

-
-
+
+

Lane lacked entry-anchored application-state history

+ 
conversation A:  e1 ─ e2 ─ e3 ─ e4
+                              └── fork B at e3
 
-  

-    

-      
A Session is a store of entries, documents, and live task bookmarks.

-      
Not one giant DOM. Not a bag of extension-owned repositories.

-    

-  

+transcript at e3:            available
+current plan-mode value:     available
+plan-mode value at e3:       not available
+review comments at e3:       not available
+

Tree forks copy current application state; branch forks exclude it. Neither reconstructs state at e3.

+
-
-
-

Four durable primitives

- - - - - - - - -
PrimitiveUseStorage
EntryTranscript and unbounded historyImmutable, paged
DocumentCurrent application and presentation stateChord ops + bases
Task statePrivate recovery phase/bookmarkComplete replacement, retireable
MemoHook/tool idempotency receiptSmall, first writer wins
-
-
+
+

The old extension API mixed owners

+ 
export default function (pi: ExtensionAPI) {
+  let turnCount = 0;                    // process-local
+
+  pi.on("turn_start", (_event, ctx) => {
+    turnCount++;
+    ctx.ui.setStatus("counter", `turn ${turnCount}`); // presentation
+  });
+
+  pi.on("tool_call", async (event, ctx) => {
+    const answer = await ctx.ui.select("Allow?", ["yes", "no"]);
+    if (answer !== "yes")
+      return { block: true, reason: "Not approved" };
+  });
+}
+

One callback contains worker state, agent policy, and mode-bound UI.

+
+ +
+

Legacy binding: one Session, two clients

+ 
Session worker
+  └─ ExtensionRunner
+       └─ one bound ExtensionUIContext
+
+web client A attaches
+terminal client B attaches
+
+tool asks “Allow bash?”
+  → send to A, B, both, or a controller?
+  → what if that client disconnects?
+  → is turnCount Session state or one client’s state?
+

The old contract has no recipient, ownership, or hydration model.

+
+ +
+

RPC transports calls; it does not define ownership

+
    +
  • RPC mode proxies supported dialogs and notifications over one request stream.
    • +
  • Terminal components, focus, raw input, editor objects, and themes stay local or degrade.
    • +
  • Running one extension per client duplicates lifecycle hooks and process-local state.
    • +
  • Running it once still requires routing, reconnect, broadcast, and conflict policy.
    • +
+

Routed services now isolate attachments; durable approval ownership and arbitration remain product policy.

+
+ +
+

Chord separates Session and presentation facets

+ 
Session facet                       client facets
+─────────────                       ─────────────
+agent policy         typed service  terminal UI
+DurableDocument  ─────────────────→ web UI
+ReplicatedState      snapshot+ops   mobile UI
+approval requests ←────────────────  explicit keyed response
+
+one Session state source
+many independently attached presentations
+
    +
  • One Session owner hosts policy and durable state; clients attach independently.
    • +
  • Each client owns its components, focus, editor, and subscriptions.
    • +
  • Authorization and “who may answer” remain explicit product policy; Chord does not make methods exactly-once.
    • +
+
+ +
+

Lane made one agent loop durable.

+

We need a durable substrate for agent and application state.

+
+ +
+

Pico5 in one sentence

+

A Session atomically commits immutable entries, full task records, and Chord-tracked documents.

+

The committed document operations are also the replicated UI operations.

+
+ +
+

Four durable concepts

+ + + + + +
ConceptWhat it storesLifetime
EntryTranscript and permanent factsSession history
DocumentMutable agent/application stateSession, conversation, or task
TaskPhase, input, recovery bookmark, terminal resultTerminal record remains queryable
MemoFirst-writer-wins effect/hook receiptOne live task
+
+ +
+

Conversations are cheap branches

+ 
Conversation A
+  entries: e1 ─ e2 ─ e3 ─ e4
+                    └─ Conversation B { parent: { conversationId: A, at: e3 } }
+
+history parent    → transcript and historical document inheritance
+owner task        → subtree abort and idle waits
+head entry        → model context begins here; old entries remain stored
+

No transcript copy. No whole-Session tree in memory.

+
+ +
+

A document declares its semantics

+ 
type Stroke = { color: string; points: { x: number; y: number }[] };
+type CanvasState = { strokes: Stroke[] };
 
-  

-    

-      
Create a durable document

-      
// Module-level typed declaration token.
 const CanvasDoc = defineDoc<CanvasState>({
-  id: "@app/canvas",
+  id: "app.canvas",
+  version: 1,
   scope: "session",
-  history: "latest",
   initial: () => ({ strokes: [] }),
-});
+  checkpointWhen: (value, _ops) => value.strokes.length % 100 === 0,
+});

+    
Scope defines lifetime. Only conversation documents choose history and fork.

+  

 
-// Tx resolves CanvasDoc to this Session's mutable draft.
-await session.commit(tx => {
-  tx.doc(CanvasDoc).strokes.push(stroke);
+  

+    
Conversation history and fork are explicit

+    
+      
+      
+      
+      
+      
+    
SettingMeaning
history: latestOnly current value matters; old records can be reclaimed after a base.
history: rewindableState can be reconstructed at an entry’s commit.
fork: asOfChild receives historical value at the selected entry.
fork: currentChild receives the source’s value when the fork runs.
fork: initialNo copy; first child access initializes normally.

+  

+
+  

+    
One mutation, one authority

+    
const target = { scope: "session" } as const;
+
+await session.commit(async tx => {
+  const canvas = await tx.doc(CanvasDoc, target);
+  canvas.strokes.push(stroke);
 }, context);

-      
CanvasDoc is the token. tx.doc(CanvasDoc) is this transaction's draft.

+    

+      
tracked draft
→

+      
Chord ops
→

+      
atomic storage
→

+      
replicas

     

-  

+
-
-
-

Expose the same document through Chord

- 
interface CanvasService {
-  readonly state: ReplicatedState<CanvasState>;
+  

+    
Chord: the live delivery half

+    
    
+      
  • Facet — one feature’s setup unit.
    • 
+      
  • Service — typed methods/state exposed to local or remote consumers.
    • 
+      
  • ReplicatedState — complete immutable client value, hydrated once and updated with ops.
    • 
+      
  • DocumentSource — Pico5’s opaque committed source; never mutable.
    • 
+    

+    
Chord transports state. Pico5 commits it.

+  

+
+  

+    
Canvas service: acquire once

+    
interface CanvasService {
+  readonly state: ReplicatedState<CanvasState | null>;
   addStroke(stroke: Stroke, context: Context): Promise<void>;
 }
-const Canvas = defineService<CanvasService>("@app/canvas");
+const Canvas = defineService<CanvasService>("app.canvas");
 
-// Committed source for the CanvasDoc token declared previously.
-const canvasDocument = session.document(CanvasDoc);
-const canvasState = replicatedState.own(canvasDocument);
+const source = await session.documentSource(CanvasDoc, target, context);
+const replica = await documentReplicatedState(source, context);
 
-env.provide(Canvas, {
-  state: canvasState,
-  addStroke: (stroke, context) => session.commit(tx => {
-    tx.doc(CanvasDoc).strokes.push(stroke);
-  }, context),
-});

-      
  • No second tracker.
  • No facet-authored delta protocol.
  • No restart reconstruction code.

-    

-  

+// documentReplicatedState is the small Pico5→Chord adapter.
+
-
-
-

One mutation path

-
-
mutate
transaction draft
→
-
Chord derives
ops
→
-
storage commits
all or nothing
→
-
publish committed ops
to replicas
-
-

If a client can see it, recovery can reconstruct it.

-
-
- -
-
-

No volatile task publication

- 
for await (const event of model.stream(request)) {
-  applyEvent(partial, event);
-
-  if (flushDue()) {
-    await rt.commit(tx => {
-      // LiveDoc is the built-in defineDoc<LiveState> token.
-      tx.doc(LiveDoc, task.conversationId).message = partial;
-    }, context);
-  }
-}
-
  • The UI sees only committed throttle points.
  • A crash loses at most the uncommitted window.
  • There is one tracker baseline: the last durable commit.
-
-
- -
-
-

Three built-in conversation documents

- 
interface ConversationView {
-  conversation: ConversationInfo;
-  entries: Entry[];             // bounded active transcript
-  rewindable: RewindableState; // historical fork state
-  sticky: StickyState;         // inbox and present policy
-  live: LiveState;             // turn, streaming, tools, collapse
-}
- 
rewindable ["s", ["model"], model]
-       view ["s", ["rewindable", "model"], model]
-
-live       ["a", ["message", ...], delta]
-view       ["a", ["live", "message", ...], delta]
-

A fixed structural mount. No config merge, task description pass, or plugin projection.

-
-
- -
-
-

Task state stays private and small

- 
const attempt = task.state.attempt + 1;
-
-// Provisional spelling: transition replaces the complete kind state.
-await rt.transition(
-  { phase: "requesting", attempt, requestId },
-  tx => {
-    tx.doc(LiveDoc, task.conversationId).generation = {
-      stage: "requesting", attempt,
-    };
+  

+    
Canvas service: provide durable methods

+    
const CanvasFacet = defineFacet({
+  id: "app.canvas/session",
+  setup(env) {
+    env.own(() => replica.dispose());
+    env.provide(Canvas, {
+      state: replica.state,
+      addStroke: (stroke, context) =>
+        session.commit(async tx => {
+          (await tx.doc(CanvasDoc, target)).strokes.push(stroke);
+          // draft now owns stroke; caller must not mutate it
+        }, context),
+    });
   },
-  context,
-);

-      
Recovery state and presentation state are updated atomically, but are not the same object.

-    

-  

+});
+

The method resolves after durability—not after every UI callback.

+
-
-
-

Crash-visible tool progress

- 
TX 40  task state = { phase: "running" }
-       live.tools[2] = { status: "running", output: "" }
-
-TX 41  live.tools[2].output = "... first 12 KiB ..."
-TX 42  live.tools[2].output = "... latest 50 KiB window ..."
-       PROCESS DIES
-
-restart  task says running
-         view says exactly how far output reached
-         recovery writes interrupted result or safely resumes
-

The partial is useful UI state even when it never enters model context.

-
-
- -
-
-

Rewind belongs to documents

- 
commit 100  entry E + rewindable plan.enabled = true
-commit 101  live streaming progress
-commit 102  rewindable plan.enabled = false
-
-fork at E:
-  rewindable → state at commit 100
-  sticky     → fresh defaults
-  live       → fresh defaults
-  tasks      → not copied
-  child      → new document IDs + initial bases
-
  • Bases bound replay distance.
  • Rewindable history remains while ancient forks are supported.
  • The parent and its live work remain untouched.
-
-
- -
-
-

One atomic Session commit

- 
// Provisional spelling; all writes below are one transaction.
-await rt.transition({ phase: "tools", calls }, tx => {
-  tx.appendEntry(task.conversationId, finalAssistant);
-
-  const live = tx.doc(LiveDoc, task.conversationId);
-  delete live.message;
-  delete live.generation;
-  live.tools = pendingToolSlots(calls);
-
-  for (const spec of toolTaskSpecs(calls)) tx.createTask(spec);
-}, context);
-

Storage publishes all or none. ConversationView emits one combined operation batch.

-
-
- -
-
-

Storage chooses delta or base

- 
interface DocumentChange {
-  id: DocumentId;
-  ops: readonly Op[];
-  value: JsonValue; // borrowed; not serialized normally
-}
-
-if (bytesSinceBase + encodedSize(ops) < threshold)
-  persist(ops);
-else
-  persist([["r", value]]);
-
  • Latest: committed base allows reclaiming the old tail.
  • Rewindable: bases accelerate reads; old addressable segments remain.
  • Chord clients still receive incremental ops.
-
-
- -
-
-

SQLite: the long-running backend

-
-

Documents

  • metadata row
  • indexed base/delta records
  • range query for current or as-of
  • transactional latest reclamation
-

Tasks + entries

  • one complete row per live task
  • DELETE on retirement
  • immutable paged entry rows
  • all writes in one SQL transaction
-
-

No sidecar publication protocol. No scan of unrelated conversations or documents.

-
-
- -
-
-

JSONL: one uniform publication rule

- 
main.jsonl               tables + one publication marker per commit
-task-<id>.jsonl          retireable task state
-doc-<id>.jsonl           one file per document instance
-
-commit                   append every sidecar first
-                         append one compact main marker last
-retire task              publish terminal marker → unlink task sidecar
-rebase latest document   publish base marker → atomic sidecar rewrite
-

Sidecars are a JSONL backend decision. Harness and facet code never see them.

-
-
- -
-
-

The benchmark changed one decision

-
-

Measured

  • 72 storage runs + 48 SQLite materialization runs validated
  • task sidecars: 1.13 MB → 59 KB after retirement
  • SQLite task deletion + checkpoint: 4.45 MB → 0.30 MB
-

Rejected optimization

  • JSONB replacement 10–18% slower than TEXT
  • SQL JSON patching used 2–4× CPU
  • large canvas DB+WAL: ~80 MB → ~155 MB
-
-

Persist Chord ops and periodic bases. Do not build a second op interpreter in SQL.

-
-
- -
-
-

Years-long sessions stay bounded in memory

- 
Session
-├─ entries                         paged rows; unbounded on disk
-├─ conversation A
-│  ├─ active entry window          bounded
-│  ├─ pi.rewindable                loaded while active
-│  ├─ pi.sticky                    loaded while active
-│  └─ pi.live                      loaded only while needed
-├─ conversation B                 evictable while idle
-├─ @app/canvas                    loaded while service is active
-└─ task 91                        one live row; deleted at terminal
-

There is no whole-session materialization, snapshot, or subscriber payload.

-
-
- -
-
-

What Pico5 does not add yet

-
-
  • generic plugin mounting into ConversationView
  • per-subtree history policies
  • visible-but-undurable updates
  • CRDT or offline writes
-
  • SQL JSON patch translation
  • one whole-Session DOM
  • history pruning semantics
  • public semantic event machinery
  • pico3 config/plugin projections
-
-

The first release mounts only the three built-in conversation documents.

-
-
- -
-
-

Build the owl one piece at a time

-
-
1
memory + SQLite
storage semantics
→
-
2
JSONL
crash matrix
→
-
3
documents + Chord
narrow proof
→
-
4
tasks
effect sandwich
→
-
5
view + rest
policy last
-
-

Do not implement the scheduler while storage semantics are still moving.

-
-
- -
-
-

Families: one typed document per task

- 
const JobOutputDoc = defineDocFamily<JobOutput, JobInput>({
-  id: "pi.job-output",
-  scope: "conversation",
-  history: "latest",
-  owner: "task",
-  initial: input => ({ command: input.command, stdout: "" }),
+  

+    
Consumers see snapshots, then updates

+    
const CanvasConsumer = defineFacet({
+  id: "app.canvas/consumer",
+  setup(env) {
+    const canvas = env.use(Canvas);
+    env.onActivate(() => env.own(
+      canvas.state.subscribe((value, _context, delivery) => {
+        render(value?.strokes ?? []);
+        console.log(delivery.kind, delivery.sequence);
+      }),
+    ));
+  },
 });
 
-// task.id selects the family instance owned by this task
-await rt.commit(tx => {
-  tx.doc(JobOutputDoc, task.id).stdout += chunk;
+const host = await createFacetHost({
+  facets: [CanvasFacet, CanvasConsumer],
+});

+    
Late join: current value. Connected client: ordered committed updates.

+  

+
+  

+    
A generation task commits what clients see

+    
// Schematic: LiveDoc is the conversation-scoped presentation document.
+// Read provider frames off the Session line.
+const batch = await nextFrames(context);
+
+// One commit makes the partial durable and observable.
+await rt.commit(async (tx, current) => {
+  const live = await tx.doc(LiveDoc, {
+    scope: "conversation", conversationId: current.conversationId,
+  });
+  applyFrames(live, batch); // initializes or updates live.message
+  live.generation = { stage: "streaming", attempt };
 }, context);

-      
Terminal settlement retires the task and every task-owned family instance atomically.

+    
Storage and every Chord client receive the same committed operations.

+  

+
+  

+    
Task kinds replace the fixed graph

+    
type GenerationState =
+  | { phase: "prepare" }
+  | { phase: "requesting"; attempt: number; cutoff: Id }
+  | { phase: "retrying"; attempt: number; retryAt: number }
+  | { phase: "deferred"; handle: DeferredHandle };
+
+const Generation =
+  defineTask<null, GenerationState, { entryId: Id }>({
+    name: "pi.generation", version: 1, initial: () => ({ phase: "prepare" }),
+    phases: { prepare, requesting, retrying, deferred }, // supplied handlers
+    abort,                                           // supplied handler
+  });

+    
Core scheduler knows tasks. Each kind knows its own phases.

+  

+
+  

+    
The generation task graph grows by commits

+    
Selected business commits; reservation/preparation/progress omitted
+
+A  create G7 pending
+B  G7 running → requesting intent
+
+   provider returns two tool calls
+
+C  append assistant e20 + clear preview + terminalize G7
+   create T8, T9 and P10 after [T8, T9]
+
+D/E  each tool commits its result entry + terminal receipt
+     either may finish first; P10 becomes eligible after both
+
+F  ordinary post-tools settlement:
+   terminalize P10 + create G11 pending

+    
The scheduler knows dependencies. Generation, tools, and post-tools create the graph.

+  

+
+  

+    
The effect sandwich stays

+    
successful response with no tool calls
+
+commit  replace checkpoint with requesting intent
+
+ effect  await provider outside the Session line
+
+commit  append assistant entry E
+        clear the durable preview
+        replace task with completed result { entryId: E }

+  

+
+  

+    
Private task state ≠ presentation state

+    

+      
Task record
{
+  phase: "requesting",
+  attempt: 2,
+  cutoff: 914
+}
Minimal bookmark for recovery.

+      
Presentation document
{
+  message: partial,
+  generation: {
+    stage: "requesting", attempt: 2
+  }
+}
Durable state clients render directly.

     

-  

+
-
-
-

Watch another task's durable output

- 
const job = await api.task(JobKind, input, { background: true }, context);
-const watch = await api.watchDoc(JobOutputDoc, job.id, context);
+  

+    
Task scope carries bounded output

+    
type JobOutputState = { stdout: string; chunks: number };
 
-consume(watch.value);                    // atomic initial snapshot
-watch.start((_ops, value) => consume(value));
+const JobOutput = defineDoc<JobOutputState>({
+  id: "pi.job-output", version: 1, scope: "task",
+  initial: () => ({ stdout: "", chunks: 0 }),
+  checkpointWhen: value => value.chunks % 100 === 0,
+});
 
-try {
-  return await api.waitForTask(job, context);
-} finally {
-  watch.stop();                           // idempotent, invocation-owned
-}

-      
  • Callbacks run off the Session mutation line.
  • Recovery re-enters and captures a fresh snapshot; no durable subscription cursor.
  • Retirement appears as document absence.

+// taskId identifies the supplied live producer task.
+const target = { scope: "task", taskId } as const;
+const watch = await api.watchDoc(JobOutput, target, context);
+consume(watch.value);
+watch.start((value, ops) => consume(value, ops));
+// Producer commit: output.stdout = (output.stdout + chunk).slice(-50_000)
+// Terminal task commit retires the document.
+

The producer bounds stdout on append; checkpoints only bound replay cost.

+
+ +
+

Abort is durable and semantic

+ 
conversation C
+├─ generation G7 ──owns──→ sub-conversation D ── tool T9
+└─ background job J8                         survives ordinary abort
+
+1  commit abortRequested on selected non-background tasks/subtree
+2  signal and join each active run invocation
+3  scheduler starts fresh abort invocations
+4  abort handlers commit terminal outcomes + retire task documents
+
+crash anywhere → durable marks remain → reopen continues abort
+

Clients observe committed consequences; process-local abort signals are not replicated state.

+
+ +
+

Close is not abort

+ + + + +
OperationDurable writesDistribution and reopen
Cancel a ContextNone by itselfRequests cancellation of one call/wait; remote propagation depends on transport. Committed work is not undone.
Abort task/conversationAbort marks, then terminal outcomesTraverses ownership; reopen finishes marked aborts.
Close SessionNo abort/terminal writes caused by closeAdmitted commits may settle; then join, stop watches, close storage, and withdraw services. Reopen resumes checkpoints.
+

Disconnecting a client does not silently abort shared work.

+
+ +
+

Hooks and tools remain extensible

+
+

Hooks

  • declared by task kind
  • replacement chains or first decision
  • may rerun before their consuming commit
  • memos persist idempotency receipts
+

Tools

  • dynamic registry + schema validation
  • documents for durable progress
  • tasks and owned conversations
  • replay only when stored and current policy are safe
-
+

V1: client facets may reload; Session-side plugin code closes and reopens the Harness.

+ -
-
-

Lane made execution durable.

-

Pico5 makes application state durable by default.

-

Ordinary objects → one commit → storage + view

+
+

Target contract: open pi-ai PR #9548

+ 
// e10 — complete baseline
+{ role: "system", content: basePrompt,
+  sections: { persona: renderedPersona, cwd: renderedCwd },
+  toolsAdded: [read, bash], timestamp }
+
+// e24 — positional delta
+{ role: "system", content: "",
+  sections: { cwd: nextRenderedCwd, legacy: null },
+  toolsRemoved: [{ name: "read" }],
+  toolsAdded: [nextRead], timestamp }
+
+// e40 moves the head beyond e10
+// e41 omits retained system deltas, then emits a complete baseline
+

Replay in transcript order; stored rendered sections remain authoritative.

+
+ +
+

The conversation view is a mount

+ 
type ConversationView = {
+  conversation: Conversation;
+  entries: readonly Entry[];  // raw active head → tail
+  docs: Readonly<Record<string, JsonObject>>;
+};
+
+// Document op: ["s", ["message"], partial]
+// Mounted view op: ["s", ["docs", "pi.live", "message"], partial]
+

Path prefixing, not a second tracker or semantic projection.

+
+ +
+

Status is state, not an event

+ 
live.collapse = { stage: "summarizing", through: 920 }
+live.collapse = { stage: "retrying", retryAt: 1730000000 }
+delete live.collapse
+
+late client during retry:
+  hydrates the current value
+  knows compaction is running and when it retries
+

A “started” event cannot help a client that attaches later.

+
+ +
+

Storage is one contract, two products

+
+

SQLite

  • long-running/cloud Sessions
  • one SQL transaction per commit
  • indexed base + delta ranges
  • current task rows
+

JSONL

  • local readable Sessions
  • document/task sidecars
  • sidecars first, main marker last
  • reclaim after committed base/retirement
-
+

Backends receive the same entries, task records, document ops, and candidate values.

+
+ +
+

Build it in reviewable slices

+
+
1
storage
memory · SQLite · JSONL
→
+
2
documents
tracking · migration · Chord
→
+
3
tasks
phases · abort · recovery
→
+
4
agent loop
inputs · tools · generation
+
+

Each slice has intrinsic conformance tests and stops for sign-off.

+
+ +
+

The deliberate sharp edges

+
+

Ownership

  • draft/proxy escape
  • inserted and duplicated aliases
  • table read after table write
  • large values in terminal task records
+

Liveness

  • external or nested waits on the line
  • get-or-create “reads”
  • unstarted watch buffering
  • checkpoint starvation
  • continuing after storage failure
+
+

Stable definition IDs, mounted paths, and fork settings are persisted protocol.

+
+ +
+

Footguns: drafts have one owner

+ 
let escaped: CanvasState | undefined;
+await session.commit(async tx => {
+  const canvas = await tx.doc(CanvasDoc, { scope: "session" });
+  const stroke: Stroke = { color: "black", points: [] };
+  canvas.strokes.push(stroke);
+  stroke.points.push(point);       // wrong: draft owns stroke now
+  escaped = canvas;
+}, context);
+escaped!.strokes.push(otherStroke); // wrong: transaction ended
+
+const sharedStroke: Stroke = { color: "red", points: [] };
+await session.commit(async tx => {
+  const canvas = await tx.doc(CanvasDoc, { scope: "session" });
+  canvas.strokes.push(sharedStroke, sharedStroke); // wrong: duplicated alias
+}, context);
+

Do not retain drafts, nested proxies, bound methods, or assigned mutable aliases.

+
+ +
+

Footguns: do not block or outrun the line

+ 
// Independent mistake 1: external effect on the Session line.
+await session.commit(async _tx => requestModel(context), context);
+
+// Independent mistake 2: nested line operation.
+await session.commit(async _tx =>
+  session.commit(async _inner => {}, context), context);
+
+// Independent mistake 3: table read after table write.
+await session.commit(async tx => {
+  tx.appendEntry(conversationId, entry);
+  await tx.entry(otherId);
+}, context);
+
+await session.snapshot(Doc, target, context); // may create + commit
+const watch = await session.watchDoc(Doc, target, context);
+// never start()/stop(): buffered commits are unbounded
+checkpointWhen: () => false // replay/storage can grow while live
+
+known post-flush/checkpoint/storage failure → close + reopen
+callback failure before flush              → baseline restored
+

Also remember: normal document reads may create, and IDs/paths are protocol.

+
+ +
+

Entries preserve history.

+

Tasks preserve work.

+

Documents preserve current truth.

+

Persist and replicate the same committed document changes.

+