Chord and Durable change a document one transaction at a time, so a tracker registered a WeakRef per
change for competitors that never existed on their hot path. The WeakRef targets survived scavenges
and V8 could pretenure every overlay context into old space (47.5 MB promoted against 2 MB in a
tracker loop). The tracker now keeps its open change in a plain field and moves it into the weak
registry only when a second change opens; stale semantics are unchanged. A draft dropped without
prepare or abort stays referenced until the tracker's next change or adopt (at most one per tracker).
`stored` and `rawNodes` are created on first use.
Chord's shared watch: listeners take a tagged call, { kind: "value", value, ops, retired } or
{ kind: "failed", error }; a recoverable failure keeps the watch open and the next value arrives as
[["r", value]]; WatchEnd is tagged by kind, and an aborted context ends a watch "aborted".
Durable: observers take the watch's shapes (advance(call, ctx), end(end)); onReport(report) with
report.kind and report.error, session_failed is storage_failed; subscribeCommits is observeCommits;
SessionEnd and ReportFrom are tagged by kind; FileChange is tagged and file watchers stop with
stop(ctx); CompactionFailure and CompactionStatus name their trigger (pi.live version 2 with a
migrate); the task_outdated report uses the scheduler's reasons; TaskInspection's ready is pending;
DocumentCommitChange is tagged. Stored formats are unchanged.
The Node driver takes the count from node:sqlite; the Durable Object driver reads
SELECT changes() after the statement, since the cursor's rowsWritten also counts
index writes (verified in workerd). run is unchanged and costs nothing more.
Durable publishes a stored commit inside Storage.commit's adopt: the
telemetry events, its own observers (document watches, conversation
views, task graph, scheduler, waits), and a dispatch of host
subscribeCommits listeners one microtask later, before every wait and
abort the commit triggers. A commit the storage rejects after adoption is
published before the Session fails; a throwing telemetry adapter is
reported as host_error "telemetry" instead of rejecting a durable commit.
Chord's watch listeners take one change object, { value, ops, retired }
({ value } for native watches); retirement is a flag, not an end.
Durable's document watches follow their address, and a task document's
watch ends after its retirement call. New synchronous observers for
adapters: observeDoc, observeConversation, observeTaskGraph; the watches
are built on them. The env file watcher's change type is renamed
FileChange.
Tracker revisions, applyImmutable() and applyImmutableBatches() copy each container on a changed path with
spread (Object.assign() onto a null-prototype object) instead of one Object.defineProperty() per key: same own
data properties, prototype and own __proto__ member, about 25 times faster per container. The copy-count test
counts key listings through a proxy instead of spying on Object.keys.
Adds a tracker GC regression test: 10,000 adopted changes retain only the document, not a context registration
per change. Follow-up to #9265.
- chord/delta: PendingOps (push, size, take -> PendingTake): raw pending ops, r drops everything
before it, a leaf index merges appends and primitive sets per path (cleared by any other op),
past 10,000 ops only "replace with the current value"; one batch is kept by reference and
received ops are never mutated
- chord: Watch queues frames in PendingOps while its listener is busy or not started and delivers
one call with the newest value, the folded ops (or [["r", newest]] after an overflow) and the
newest frame's context; replaces the 100-frame collapse. NativeWatch is unchanged and takes an
optional replace()
- durable: watchEvents keeps every event batch in order (NativeWatch with a snapshot replace);
document, view and task-graph watches fold; spec, README, example 04 and tests updated
- tests: fold fuzzers (tracker, appendText, splices/reorders, root arrays, diffRevisions) with
codec round trips, adversarial cases, cap, ownership, cost and GC guards
Follow-up to #9265.
- chord: Delta appendText(draft, key, text) / StringKeys<T> record an append op without the
O(length) prefix compare that a plain `draft[key] += text` costs in prepare()
- durable: partial commits append text and thinking deltas with appendText (assignJson plan,
checked by length), and apply tool-call argument changes from their own StreamingJsonParser
of the raw toolcall_delta text instead of reassigning the arguments snapshot
- durable "." entry-graph budget 73 -> 76 for the parser it now imports
- ai: StreamingJsonParser / ToolCallArgumentsStream read each argument delta once (tolerant,
immutable snapshots, linear cost); every provider, pi-messages and the faux provider use it
and keep stream state (partialJson, partialArgs, index, customInput, streamIndex, Bedrock
reasoning bytes) off the content blocks; grammar tool input appends per delta
- agent: streamProxy uses the same parser
- chord: Delta append ops own their text (ownedSlice), so ops and replicas no longer keep the
whole growing string alive
- durable, ai, mcp: bounded error excerpts and summaries own their characters
- coding-agent: tool calls render streamed arguments once per frame, not once per delta
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
ended (was session_closed) and failed (was session_failed); retirement is an
explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
and DocumentState; use watchDoc(), watch(), watchTaskGraph()
Reverts 48c8f3228 and 4759f09fb. Session sync replicates pi-durable commits instead of SQLite pages, so NodeSqliteDatabase.exclusive(), the replicated connection wrapper, the WAL reader, page checksums, the segment container, start-marker keys and the zstd page compressor have no user.
Portable modules under sqlite/replication: a WAL scanner that returns the newest page versions of committed transactions and stops at the last valid commit frame, incremental whole-database checksums from per-page hashes, the shared segment and base container with header validation, and object keys and start markers. A Node-only zstd page compressor sits beside them. None of it is exported from the package yet.
NodeSqliteDatabase.exclusive() runs an operation alone on the connection, outside a transaction, and rolls back a transaction the operation left open.
The internal replicated connection wrapper opens a database file with automatic checkpoints off, reports every operation, and runs a hook before SQLite checkpoints and deletes the WAL on close. It is not exported from the package yet.
- pi-telemetry: StoredSpan/StoredTrace/TelemetryStore (findTraces with name, attribute, error, time-window filters and
cursor paging; getTrace; findMetrics), trace ids carried in carriers, renderTrace in the /render subpath; the in-memory
adapter implements the store; the adapter contract trusts typed inputs (malformed-input conformance cases removed)
- chord: @earendil-works/chord/telemetry-sqlite over chord/sqlite: settled spans with inline events and links, metrics
aggregated per series per 10 s, 1 s batched flush with retention, bounded queue (dropped and counted), store queries
- pi-ai: AI_TELEMETRY_SCHEMA; pi.ai.request and pi.ai.poll spans around Models requests via a lazyStream observer
(no provider changes), first_chunk and block events (never per delta), usage/cost end attributes, provider_error
status with a bounded message, aborted stays ok; duration, first-chunk, token and cost metrics; option renamed
telemetryContext -> telemetry (a parent span)
- pi-telemetry: TelemetryContext carries readonly metrics and propagation capabilities on every span
- chord/context: Context.telemetry is always a TelemetrySpan (NOOP default); withTelemetry takes spans
- durable: DURABLE_TELEMETRY_SCHEMA; pi.durable.submit (inputs start their own trace linked to the caller) and
pi.durable.phase spans for every task phase; stored carriers on submissions and tasks (TaskOptions.trace, links);
entry, submission, checkpoint, replay and report events after commit; resumed tasks stay in their trace; phase
duration, active task and report metrics; failed phases carry fixed categories with bounded messages
Due sleeps are picked up at every scheduling pass, so a pass that drops the timer cannot strand them.
Code mode drops a global's result after the script finished instead of reading an ended invocation's clock.
A foreground timer the host declines falls back to setTimeout; the default timer never fires early; Progress
rechecks its clock; an undelivered onWake is offered again. Spec, README, and tests follow.
Tasks end a phase asleep with runtime.sleepUntil() or { status: "sleeping", until, checkpoint }:
no invocation is held, the deadline survives restarts, and the task resumes at its checkpoint once due.
Generation retry/poll and compaction retry sleep this way. Harness.wake() and HarnessOptions.onWake let
a host such as a Durable Object wake the process itself. HarnessOptions.setTimer arms every Harness
timer, and chord Progress paces on an injectable clock. SQLite migrates to schema version 2 to admit
the sleeping status.
OutputBuffer, Progress, boundOutput, sanitizeOutput, characterEnd and the
output types move from Durable's internal harness/output.ts to the new
Chord subpath, with OutputSkip (the shape of Durable's ShellOutputSkip) and
utf8ByteLength. Durable imports them from Chord; its truncate.ts re-exports
Chord's utf8ByteLength instead of keeping a second copy.
A call through a remote reference checked the ctx against its own host,
so a ctx from another host, or an object not made by Chord, passed and
its root was carried over the link. It now runs the same checks as a
local reference and rejects with invalid/foreign_ctx.
closes#10959
- an awaited get or a ready hold has every state member of its variant, local or remote
- holds subscribe to their variant's state; per-member opt-out with state: false
- remote binding keys use canonical identity; client holds carry an opaque origin
- shape frames no longer carry codeVersion
- self.state(name, schema, { init }) loads before the object goes live
- the principal type is a parameter; createHost self, connect and route principals are required
- remote contracts declare their host (hostNamed, hostFamily); get no longer takes at
serve(Type, factory, { lifetime: "host" }) compiles only for a local type whose input has no required fields, and
not together with linger; the runtime check stays as a backstop. ServeOptions is a union (ReferencedServeOptions,
HostServeOptions), picked per contract by ServeOptionsFor<C>.
A host that wrote to a peer process which had just exited (an ack timer firing after the worker's exit, before the
channel's disconnect) got EPIPE as an unhandled 'error' event. IpcLink now passes a send callback and ends the link on
failure; host-manager control messages to a gone peer are dropped (its exit reports it).
- A child that says hello during a cluster reload is fenced at once and reconciled after the reload ends, to the
committed code version (it was marked in sync without voting); a running reconciliation also waits for the reload.
- A child retires a code version only after its parent did (spec section 12): the coordinator retires before it
prepares, and children learn already retired versions at hello and before a reconciliation's prepare.
- Forced retirement aborts the leftover roots of the old code version, relayed calls included, before it closes the
resources only that code used.
- A task's replacement starts after the old run has ended, at most taskStopDeadline later.
- A kept resource adopts the newest declaration's close at commit.
- Disposed objects and old code generations are released: Ref bindings are weak, finished disposals leave the dying
table, stopped tasks are dropped, and contexts link abort signals without AbortSignal.any (which kept linked signals
and their listeners reachable).
- Facets get FacetApi.signal, aborted when their run is replaced, removed, aborted or the host closes.
- Regression tests for each, including a garbage-collection test of old code generations.
- self.state(name, schema, initial, { version?, migrate? }) returns the cell; the state version and migration live on
the cell (ServeOptions.state is gone). self.derive(sources, fn) has no name and is recomputed at commit.
self.resource(name, { version, deps?, exclusive?, open, close? }) (rev is version). self.task(fn). self.on(name,
source, [select,] handler) replaces listen/listener. cell(), derivation(), resource(), listener() and the
declaration maps are gone; a name declared twice in one factory run is an error.
- A listener on state receives the current value at start and after every reload: handlers must reconcile.
- method(args?, result?): an omitted result is void, omitted arguments make a no-argument method called as
ref.name(ctx) and implemented as (ctx) => ...; the wire omits both.
- References are Ref<C> (was Handle<C>), objects ObjectImpl (record.ts is object.ts), ImplementationOf<C> (was
BehaviorOf<C>), ResourceDeclaration (was ResourceSpec); epochs are code versions (codeVersion in Ctx, RootOptions,
Shape, Host, frames and reload reports). Docs say object and reference.
- A remote object's variant is named variant (was kind): factories return { variant }, references expose ref.variant.
- Links announce a protocol version (PROTOCOL_VERSION = 2) in hello; peers of different versions close the link with
reason protocol-version <n> and report it through LinkOptions.onProtocolMismatch.
- ServeOptions.lifetime ("referenced" | "host") replaces eager: a host-lifetime object is local, has an empty input,
exists once per host and lives until the host closes; awaitEager is awaitHostObjects. A reload that makes a type
referenced releases the host's own reference.
Peers acknowledge processed frames (ack, announced with acks in hello). While a window of frames is unacknowledged, state frames wait at the sender, coalesced per subscription, instead of queuing in front of call results; slots that outgrow their bound become resets, or a downstream resubscription on relays. AcceptOptions gains maxCalls, answering calls beyond it with unavailable/too_many_calls. Link.send returns the encoded frame size.
forkDriver spread process.env into the child, so a child host that forks its own children (or a test run inside a p2 worker) passed its lock on, and children without a lock of their own failed with "locked".
Call args, results and state snapshots whose JSON exceeds 64K characters are deflated
(level 1): over IPC as z: [field, base64], over WebSockets as a binary message
(envelope plus deflated payload). A relaying host forwards the bytes as received; the
host that uses the payload inflates it. Browsers inflate with DecompressionStream in
order; the p2 TUI launcher turns packed messages back into text frames. Servers send
packed frames to clients but refuse them from clients.
On the 8b6ab5cc session: first view 523 KiB -> 78 KiB, all history 23.5 MB -> 4.9 MB.
Frame, RootRef, WireError and the reload types are derived from FrameSchema instead of
being declared twice. Error codes, contract tokens (Description is now a schema in
contract.ts) and ops are checked as typed; ops stay unknown until the delta decoder
validates them.
A facet listed before another facet whose directory it imports from loaded a second copy of that module; register
all roots first so every facet of one load shares one copy, whatever the order.
A deep TMPDIR, such as ./test.sh's below macOS's per-user one, made the
lock socket path exceed 104 bytes and listen() failed with EINVAL. Fall
back to a private per-user directory under /tmp when the path does not fit.
7f022043f added packages/chord/node_modules, a symlink from a local worktree. The ignore pattern matched only
directories, so a node_modules symlink could be committed; it now matches files and links too.
A route that returned a dead link (a child that exited before its link was wrapped) left a closed connection in
the host, so every later call to that host name failed with unavailable until restart. Such a connection is not
added, closed children are skipped when routing by name, and no ping timer is started for it.
