Schema helpers are a thin layer over pi-schema's builders with the same JSON,
so fingerprints and stored data are unchanged (proven by schema-switch tests and
a store written by the TypeBox code). Checks through pi-schema; error messages
are pi-schema's; format is an annotation. v1 contracts take any JSON Schema.
map(bytes()) works.
chord/next/http: a Web fetch handler built from the registry Http (routes, wraps,
sockets, hosts, relays, CSP sources) and Authenticators, responses as descriptions,
a security baseline on by default, Chord's own Node listener as a host-lifetime
object, nodeRequest for apps mounting it, links and plugin sockets.
chord/next/workers: a forked worker per address, started on demand, stopped when
idle, never restarted by itself, plugins built by the server and shipped.
Plugins: kinds with platform and host packages, createPluginRuntime, scripts(kind),
rollout results per host, imports of other plugins by name. Kernel: RouteHandle
(retry, suspend), no route backoff, anonymous host.root, origins only from relays.
ws is an optional peer dependency.
Plugin discovery, esbuild builds (optional peer dependency), vm evaluation with
pluginFile and mapStack, dependency installs, a watcher with dirty marks and
rollout across hosts; host.load and host.reload per plugin with the closure rule;
examples 35-39 as folders with real plugin directories.
The owner's audit decisions (decisions.md, "Simplification audit"): cut the reload-only
installing state (staging is synchronous) and the static registry cycle edges (the
runtime guard stays); retry propagation by a visited set and a hop budget (retry.hops)
instead of operation ids and through lists; a pure swap sends holds a fresh bound; one
rule after a live failure (every member followed until the admitted commits published);
parked_long for any long park, naming the first blocker in declaration order with its
state; one seal token per attempt. Keys follow the old instance during a restart's drain
like members. src/next: about 255 code lines fewer.
`sanitizeOutput()` in `@earendil-works/chord/output` deleted the ESC
byte (0x1b) along with the other control characters, but it left the
rest of each escape sequence in place. Colored tool output such as
`\x1b[31mERROR\x1b[0m` came back from `OutputBuffer.snapshot()` as
`[31mERROR[0m`. Because the ESC byte was already gone, later ANSI
stripping could no longer recognize the leftover fragments.
`sanitizeOutput()` now removes whole 7-bit escape sequences before it
removes single control characters. It handles:
- CSI sequences (colors, cursor and erase commands)
- string sequences (OSC, DCS, APC, PM, SOS) terminated by BEL or `ESC \`
- short `ESC x` forms such as `ESC ( B` or `ESC 7`
`snapshot()` sanitizes the concatenated retained text, so a sequence
split across pushed chunks is also removed.
**Testing**
- Added a regression test in `packages/chord/test/output.test.ts`. It
covers SGR colors, cursor and erase CSI, charset and short escapes, OSC
8 hyperlinks and an OSC title, plus an `OutputBuffer` fed a color
sequence split across three chunks. The test fails without the change
and passes with it.
- `test/output.test.ts` and `test/output-skip.test.ts` pass.
- Biome is clean. The only `tsc` errors in `npm run check` are in
`packages/ai` tests and are unrelated to this change.
- The durable package test results are the same with and without the
change: 71 test files fail on both, and every individual test that runs
passes.
- Reproduction from the issue: before the change both lines printed
`"[31mERROR[0m\n"`; after it, both print `"ERROR\n"`.
Fixes#11604
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
A Session no longer holds its line while a commit is stored. Commits that
arrive while one storage call is in flight go out together in the next call,
each with its own seq and outcome; a job runs until it touches a key an
unpublished commit wrote, then waits for that commit's publication. Outcomes
are delivered in line order; publication is in line order, one commit per
macrotask. The scheduler's decisions run as regions that record the keys they
read and rerun after a wait, with side effects only after a clean check.
Storage.commit(batches, receive, context) replaces commit(writes, adopt,
context): prefix semantics (Durable's own checks refuse a batch and keep the
ones before it; any other failure rejects the call), receipts per stored
batch, optional publishing() scope. Memory, SQLite, JSONL, IndexedDB and
StoreStorage implement it natively; StoreStorage through the new
ChordStore.commitSeries (one store transaction per call), replacing
commitInBatch.
A normal run makes no storage reads: bounded record cache, conversation
tails, task-document index, kept context ranges. A tool slot's running status
and a generation's attempt are stored with the next commit (12 instead of 14
commits per run).
One Harness with 128 conversations over storage with a 4 ms round trip: about
5 -> 440 agent runs per second. Design: packages/durable/docs/group-commit.md.
host.reload(facets, ctx): staging into a staged table with a recording self (all or
nothing), one commit (types, registries, contributions; per-name wakes; installing
holds), and each object's fate: pure swap in place, native restart after its drain,
retry on the new code, or its end by its type's removal, with a report. Old calls
finish on old handles (R15 for documents and tables), routines hand over per object,
jobs keep their versions (accepts, blocked { version }, persisted runs per R3),
registries swap providers and contributions in one batch, holds are kept,
re-subscribed or unbound with failed { waiting } (model §5), accepts maps older
consumers' state, arguments, results and keyed members.
References are the only indirection (owner, 2026-10-29): a consumer's watches and
derives attach through the reference, which passes them straight to the object
instance's own document or derive, moves them to a new instance (one frame) and keeps
its last values while unbound; MemberSource, keyed-members and the per-version
hand-over are removed. Older consumers read through the instance's shared mapping
per version. Examples 33-34, reload benchmark.
A reference is a request that outlives its provider: a missing provider or a version that does not fit leaves
it unbound, waiting for the name (failed { waiting }, a new FailureCause kind), local and remote alike. An
attempt parks until its dependencies are live, with no timer and no backoff, woken on every state change of a
dependency; its readyTimeout runs only for its own steps. Requests wait only for the host's publication (the
startup deadlock is gone); load resolves at the first outcomes of its host-lifetime objects. Getters wait through
waiting until their readyTimeout. Registry readers are waiting dependencies, woken at every usable value; a last
good value is dropped when a contribution it was built from leaves. Unbound holds count for keep-alive and are
revoked; retry passes on to failed blockers once per operation (operation id and visited list on the wire); the
dependency deadline is gone and parked_long reports a long wait on a remote attempt.
The contract field placement is hostAddress, required on every remote() (type and declaration error); the
fallback for remote types without one is gone. invalid/placement is invalid/host_address. host/placement.ts
(Placement) is host/destination.ts (Destination); here is decided by address alone, and a host without an
address refuses to serve a remote type. A hold always carries its address on the wire (the binary codec's
ADDRESS flag is gone, the text codec requires it). Tests and examples declare their host addresses.
Registries: shared tokens (defineRegistry), one provider per host (host.serve(R, { compose, accepts })),
contributions from facets and host-lifetime objects, plain values or followed Chord watchables, a table
sorted by plugin, lazy compose in the batch that changed the table, failures left out and reported with
the last good value kept; self.registry as a declared dependency, host.registry outside objects, older
readers mapped by accepts[old].value; keyedEntries and orderedEntries. Native sources: self.writable,
registry values and ref.state. self.retryOn wakes failed objects on registry values. host.root gives code
outside objects a call scope sealed at settle, abort and host.close.
Durable: registryReader(value, context); a Session close ends synchronous observers failed
("session closed"), watches still end ended.
A failed activation releases its references after its natives closed; a dependency going live while
the dependent activates retries it at once; a call on a local object runs with the host's principal
and incarnation; retiring an unloaded document checks its stored version and fingerprint, and a cut
retirement no longer fails its successor; ref.retry() re-runs a refused hold's start on the same id
(the provider keeps refused holds), and ends released/closed on a closed host; a relayed hold waits
while the parent link is down; a link drop while a dependency is still loading is no failure.
StoreStorage (Durable's storage on Chord v2's store, @earendil-works/pi-durable/storage/store) passes
Durable's storage conformance suite and runs Durable's harness tests (npm run test:store). The store
gets owner names and claims, one id allocator, Durable's Storage semantics for refused steps, history
none/latest/rewindable with reads at a point, archive revisions and applyBatch. self.follow follows
Durable's synchronous observers as Chord sources. Persisted jobs: records and state in the owner's
store, restarts as Durable's tasks, abort stored, deferred state. Chord's ./next export returns for
the adapter surface.
The next object runtime moves from the private repository into pi: src/next (contracts, hosts and
objects, documents, derives, links and relays, routines and jobs, the store) with its tests and
examples in test/next, src/next-node (Node compression), and its design in docs/next (design.md, the
decision log decisions.md, the slice plan slices.md, slice specs, books and benchmarks). AGENTS.md
gets its code style. Not exported: the API changes with every slice until it has an entry point.
Chord and Durable change a document one transaction at a time, so a tracker registered a WeakRef per
change for competitors that never existed on their hot path. The WeakRef targets survived scavenges
and V8 could pretenure every overlay context into old space (47.5 MB promoted against 2 MB in a
tracker loop). The tracker now keeps its open change in a plain field and moves it into the weak
registry only when a second change opens; stale semantics are unchanged. A draft dropped without
prepare or abort stays referenced until the tracker's next change or adopt (at most one per tracker).
`stored` and `rawNodes` are created on first use.
Chord's shared watch: listeners take a tagged call, { kind: "value", value, ops, retired } or
{ kind: "failed", error }; a recoverable failure keeps the watch open and the next value arrives as
[["r", value]]; WatchEnd is tagged by kind, and an aborted context ends a watch "aborted".
Durable: observers take the watch's shapes (advance(call, ctx), end(end)); onReport(report) with
report.kind and report.error, session_failed is storage_failed; subscribeCommits is observeCommits;
SessionEnd and ReportFrom are tagged by kind; FileChange is tagged and file watchers stop with
stop(ctx); CompactionFailure and CompactionStatus name their trigger (pi.live version 2 with a
migrate); the task_outdated report uses the scheduler's reasons; TaskInspection's ready is pending;
DocumentCommitChange is tagged. Stored formats are unchanged.
The Node driver takes the count from node:sqlite; the Durable Object driver reads
SELECT changes() after the statement, since the cursor's rowsWritten also counts
index writes (verified in workerd). run is unchanged and costs nothing more.
Durable publishes a stored commit inside Storage.commit's adopt: the
telemetry events, its own observers (document watches, conversation
views, task graph, scheduler, waits), and a dispatch of host
subscribeCommits listeners one microtask later, before every wait and
abort the commit triggers. A commit the storage rejects after adoption is
published before the Session fails; a throwing telemetry adapter is
reported as host_error "telemetry" instead of rejecting a durable commit.
Chord's watch listeners take one change object, { value, ops, retired }
({ value } for native watches); retirement is a flag, not an end.
Durable's document watches follow their address, and a task document's
watch ends after its retirement call. New synchronous observers for
adapters: observeDoc, observeConversation, observeTaskGraph; the watches
are built on them. The env file watcher's change type is renamed
FileChange.
Tracker revisions, applyImmutable() and applyImmutableBatches() copy each container on a changed path with
spread (Object.assign() onto a null-prototype object) instead of one Object.defineProperty() per key: same own
data properties, prototype and own __proto__ member, about 25 times faster per container. The copy-count test
counts key listings through a proxy instead of spying on Object.keys.
Adds a tracker GC regression test: 10,000 adopted changes retain only the document, not a context registration
per change. Follow-up to #9265.
- chord/delta: PendingOps (push, size, take -> PendingTake): raw pending ops, r drops everything
before it, a leaf index merges appends and primitive sets per path (cleared by any other op),
past 10,000 ops only "replace with the current value"; one batch is kept by reference and
received ops are never mutated
- chord: Watch queues frames in PendingOps while its listener is busy or not started and delivers
one call with the newest value, the folded ops (or [["r", newest]] after an overflow) and the
newest frame's context; replaces the 100-frame collapse. NativeWatch is unchanged and takes an
optional replace()
- durable: watchEvents keeps every event batch in order (NativeWatch with a snapshot replace);
document, view and task-graph watches fold; spec, README, example 04 and tests updated
- tests: fold fuzzers (tracker, appendText, splices/reorders, root arrays, diffRevisions) with
codec round trips, adversarial cases, cap, ownership, cost and GC guards
Follow-up to #9265.
- chord: Delta appendText(draft, key, text) / StringKeys<T> record an append op without the
O(length) prefix compare that a plain `draft[key] += text` costs in prepare()
- durable: partial commits append text and thinking deltas with appendText (assignJson plan,
checked by length), and apply tool-call argument changes from their own StreamingJsonParser
of the raw toolcall_delta text instead of reassigning the arguments snapshot
- durable "." entry-graph budget 73 -> 76 for the parser it now imports
- ai: StreamingJsonParser / ToolCallArgumentsStream read each argument delta once (tolerant,
immutable snapshots, linear cost); every provider, pi-messages and the faux provider use it
and keep stream state (partialJson, partialArgs, index, customInput, streamIndex, Bedrock
reasoning bytes) off the content blocks; grammar tool input appends per delta
- agent: streamProxy uses the same parser
- chord: Delta append ops own their text (ownedSlice), so ops and replicas no longer keep the
whole growing string alive
- durable, ai, mcp: bounded error excerpts and summaries own their characters
- coding-agent: tool calls render streamed arguments once per frame, not once per delta
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
ended (was session_closed) and failed (was session_failed); retirement is an
explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
and DocumentState; use watchDoc(), watch(), watchTaskGraph()
Reverts 48c8f3228 and 4759f09fb. Session sync replicates pi-durable commits instead of SQLite pages, so NodeSqliteDatabase.exclusive(), the replicated connection wrapper, the WAL reader, page checksums, the segment container, start-marker keys and the zstd page compressor have no user.
Portable modules under sqlite/replication: a WAL scanner that returns the newest page versions of committed transactions and stops at the last valid commit frame, incremental whole-database checksums from per-page hashes, the shared segment and base container with header validation, and object keys and start markers. A Node-only zstd page compressor sits beside them. None of it is exported from the package yet.
NodeSqliteDatabase.exclusive() runs an operation alone on the connection, outside a transaction, and rolls back a transaction the operation left open.
The internal replicated connection wrapper opens a database file with automatic checkpoints off, reports every operation, and runs a hook before SQLite checkpoints and deletes the WAL on close. It is not exported from the package yet.
- pi-telemetry: StoredSpan/StoredTrace/TelemetryStore (findTraces with name, attribute, error, time-window filters and
cursor paging; getTrace; findMetrics), trace ids carried in carriers, renderTrace in the /render subpath; the in-memory
adapter implements the store; the adapter contract trusts typed inputs (malformed-input conformance cases removed)
- chord: @earendil-works/chord/telemetry-sqlite over chord/sqlite: settled spans with inline events and links, metrics
aggregated per series per 10 s, 1 s batched flush with retention, bounded queue (dropped and counted), store queries
- pi-ai: AI_TELEMETRY_SCHEMA; pi.ai.request and pi.ai.poll spans around Models requests via a lazyStream observer
(no provider changes), first_chunk and block events (never per delta), usage/cost end attributes, provider_error
status with a bounded message, aborted stays ok; duration, first-chunk, token and cost metrics; option renamed
telemetryContext -> telemetry (a parent span)
- pi-telemetry: TelemetryContext carries readonly metrics and propagation capabilities on every span
- chord/context: Context.telemetry is always a TelemetrySpan (NOOP default); withTelemetry takes spans
- durable: DURABLE_TELEMETRY_SCHEMA; pi.durable.submit (inputs start their own trace linked to the caller) and
pi.durable.phase spans for every task phase; stored carriers on submissions and tasks (TaskOptions.trace, links);
entry, submission, checkpoint, replay and report events after commit; resumed tasks stay in their trace; phase
duration, active task and report metrics; failed phases carry fixed categories with bounded messages
Due sleeps are picked up at every scheduling pass, so a pass that drops the timer cannot strand them.
Code mode drops a global's result after the script finished instead of reading an ended invocation's clock.
A foreground timer the host declines falls back to setTimeout; the default timer never fires early; Progress
rechecks its clock; an undelivered onWake is offered again. Spec, README, and tests follow.
Tasks end a phase asleep with runtime.sleepUntil() or { status: "sleeping", until, checkpoint }:
no invocation is held, the deadline survives restarts, and the task resumes at its checkpoint once due.
Generation retry/poll and compaction retry sleep this way. Harness.wake() and HarnessOptions.onWake let
a host such as a Durable Object wake the process itself. HarnessOptions.setTimer arms every Harness
timer, and chord Progress paces on an injectable clock. SQLite migrates to schema version 2 to admit
the sleeping status.
OutputBuffer, Progress, boundOutput, sanitizeOutput, characterEnd and the
output types move from Durable's internal harness/output.ts to the new
Chord subpath, with OutputSkip (the shape of Durable's ShellOutputSkip) and
utf8ByteLength. Durable imports them from Chord; its truncate.ts re-exports
Chord's utf8ByteLength instead of keeping a second copy.
A call through a remote reference checked the ctx against its own host,
so a ctx from another host, or an object not made by Chord, passed and
its root was carried over the link. It now runs the same checks as a
local reference and rejects with invalid/foreign_ctx.
closes#10959
- an awaited get or a ready hold has every state member of its variant, local or remote
- holds subscribe to their variant's state; per-member opt-out with state: false
- remote binding keys use canonical identity; client holds carry an opaque origin
- shape frames no longer carry codeVersion
- self.state(name, schema, { init }) loads before the object goes live
- the principal type is a parameter; createHost self, connect and route principals are required
- remote contracts declare their host (hostNamed, hostFamily); get no longer takes at
serve(Type, factory, { lifetime: "host" }) compiles only for a local type whose input has no required fields, and
not together with linger; the runtime check stays as a backstop. ServeOptions is a union (ReferencedServeOptions,
HostServeOptions), picked per contract by ServeOptionsFor<C>.
A host that wrote to a peer process which had just exited (an ack timer firing after the worker's exit, before the
channel's disconnect) got EPIPE as an unhandled 'error' event. IpcLink now passes a send callback and ends the link on
failure; host-manager control messages to a gone peer are dropped (its exit reports it).
- A child that says hello during a cluster reload is fenced at once and reconciled after the reload ends, to the
committed code version (it was marked in sync without voting); a running reconciliation also waits for the reload.
- A child retires a code version only after its parent did (spec section 12): the coordinator retires before it
prepares, and children learn already retired versions at hello and before a reconciliation's prepare.
- Forced retirement aborts the leftover roots of the old code version, relayed calls included, before it closes the
resources only that code used.
- A task's replacement starts after the old run has ended, at most taskStopDeadline later.
- A kept resource adopts the newest declaration's close at commit.
- Disposed objects and old code generations are released: Ref bindings are weak, finished disposals leave the dying
table, stopped tasks are dropped, and contexts link abort signals without AbortSignal.any (which kept linked signals
and their listeners reachable).
- Facets get FacetApi.signal, aborted when their run is replaced, removed, aborted or the host closes.
- Regression tests for each, including a garbage-collection test of old code generations.
- self.state(name, schema, initial, { version?, migrate? }) returns the cell; the state version and migration live on
the cell (ServeOptions.state is gone). self.derive(sources, fn) has no name and is recomputed at commit.
self.resource(name, { version, deps?, exclusive?, open, close? }) (rev is version). self.task(fn). self.on(name,
source, [select,] handler) replaces listen/listener. cell(), derivation(), resource(), listener() and the
declaration maps are gone; a name declared twice in one factory run is an error.
- A listener on state receives the current value at start and after every reload: handlers must reconcile.
- method(args?, result?): an omitted result is void, omitted arguments make a no-argument method called as
ref.name(ctx) and implemented as (ctx) => ...; the wire omits both.
- References are Ref<C> (was Handle<C>), objects ObjectImpl (record.ts is object.ts), ImplementationOf<C> (was
BehaviorOf<C>), ResourceDeclaration (was ResourceSpec); epochs are code versions (codeVersion in Ctx, RootOptions,
Shape, Host, frames and reload reports). Docs say object and reference.
- A remote object's variant is named variant (was kind): factories return { variant }, references expose ref.variant.
- Links announce a protocol version (PROTOCOL_VERSION = 2) in hello; peers of different versions close the link with
reason protocol-version <n> and report it through LinkOptions.onProtocolMismatch.
- ServeOptions.lifetime ("referenced" | "host") replaces eager: a host-lifetime object is local, has an empty input,
exists once per host and lives until the host closes; awaitEager is awaitHostObjects. A reload that makes a type
referenced releases the host's own reference.
Peers acknowledge processed frames (ack, announced with acks in hello). While a window of frames is unacknowledged, state frames wait at the sender, coalesced per subscription, instead of queuing in front of call results; slots that outgrow their bound become resets, or a downstream resubscription on relays. AcceptOptions gains maxCalls, answering calls beyond it with unavailable/too_many_calls. Link.send returns the encoded frame size.
forkDriver spread process.env into the child, so a child host that forks its own children (or a test run inside a p2 worker) passed its lock on, and children without a lock of their own failed with "locked".
Call args, results and state snapshots whose JSON exceeds 64K characters are deflated
(level 1): over IPC as z: [field, base64], over WebSockets as a binary message
(envelope plus deflated payload). A relaying host forwards the bytes as received; the
host that uses the payload inflates it. Browsers inflate with DecompressionStream in
order; the p2 TUI launcher turns packed messages back into text frames. Servers send
packed frames to clients but refuse them from clients.
On the 8b6ab5cc session: first view 523 KiB -> 78 KiB, all history 23.5 MB -> 4.9 MB.
A facet listed before another facet whose directory it imports from loaded a second copy of that module; register
all roots first so every facet of one load shares one copy, whatever the order.
A deep TMPDIR, such as ./test.sh's below macOS's per-user one, made the
lock socket path exceed 104 bytes and listen() failed with EINVAL. Fall
back to a private per-user directory under /tmp when the path does not fit.
