Commit graph

43 commits

Author SHA1 Message Date
Mario Zechner
8f9446a060 feat(chord): invalid and forbidden exported, page state across plugin reloads, the tutorial app's API a workspace package 2026-11-03 19:02:56 +00:00
Mario Zechner
ed4f488401 feat(chord): remove Chord v1 (/old, /old/node), its tests and docs
p2 runs on Chord v2; apps still on v1 pin @earendil-works/chord@1.5.0.
2026-11-03 12:11:22 +00:00
Mario Zechner
e9c0ff4d84 feat(chord): Chord v2 is the package's root
src/next is src (the main export), src/next-node is src/node with short subpaths (/http, /workers, /plugins,
/plugins/runtime, /app, /surfaces, /web, /react, /tui, /tui/launcher); the foundations get /json and /watch;
Chord v1 moves to src/old (/old, /old/node) until p2 runs on v2; tests and docs move likewise (test/, docs/,
test/old, docs/old). Durable and env import the foundations from their subpaths.
2026-11-02 22:36:55 +00:00
Mario Zechner
be039fb316 feat(chord): Chord v1 and v2 on pi-schema, drop TypeBox
Schema helpers are a thin layer over pi-schema's builders with the same JSON,
so fingerprints and stored data are unchanged (proven by schema-switch tests and
a store written by the TypeBox code). Checks through pi-schema; error messages
are pi-schema's; format is an annotation. v1 contracts take any JSON Schema.
map(bytes()) works.
2026-11-01 10:15:02 +00:00
Mario Zechner
5f312dac4c feat(chord,durable): failures are watch calls; one vocabulary for watches, observers and reports
Chord's shared watch: listeners take a tagged call, { kind: "value", value, ops, retired } or
{ kind: "failed", error }; a recoverable failure keeps the watch open and the next value arrives as
[["r", value]]; WatchEnd is tagged by kind, and an aborted context ends a watch "aborted".

Durable: observers take the watch's shapes (advance(call, ctx), end(end)); onReport(report) with
report.kind and report.error, session_failed is storage_failed; subscribeCommits is observeCommits;
SessionEnd and ReportFrom are tagged by kind; FileChange is tagged and file watchers stop with
stop(ctx); CompactionFailure and CompactionStatus name their trigger (pi.live version 2 with a
migrate); the task_outdated report uses the scheduler's reasons; TaskInspection's ready is pending;
DocumentCommitChange is tagged. Stored formats are unchanged.
2026-10-27 20:38:08 +00:00
Mario Zechner
8b0caefc22 feat(durable,chord): publish commits at adoption, address-following watches, synchronous observers
Durable publishes a stored commit inside Storage.commit's adopt: the
telemetry events, its own observers (document watches, conversation
views, task graph, scheduler, waits), and a dispatch of host
subscribeCommits listeners one microtask later, before every wait and
abort the commit triggers. A commit the storage rejects after adoption is
published before the Session fails; a throwing telemetry adapter is
reported as host_error "telemetry" instead of rejecting a durable commit.

Chord's watch listeners take one change object, { value, ops, retired }
({ value } for native watches); retirement is a flag, not an end.
Durable's document watches follow their address, and a task document's
watch ends after its retirement call. New synchronous observers for
adapters: observeDoc, observeConversation, observeTaskGraph; the watches
are built on them. The env file watcher's change type is renamed
FileChange.
2026-10-26 20:44:34 +00:00
Mario Zechner
d8d99950ac feat(chord,durable): PendingOps folds pending ops for slow consumers; Watch delivers one call per busy period
- chord/delta: PendingOps (push, size, take -> PendingTake): raw pending ops, r drops everything
  before it, a leaf index merges appends and primitive sets per path (cleared by any other op),
  past 10,000 ops only "replace with the current value"; one batch is kept by reference and
  received ops are never mutated
- chord: Watch queues frames in PendingOps while its listener is busy or not started and delivers
  one call with the newest value, the folded ops (or [["r", newest]] after an overflow) and the
  newest frame's context; replaces the 100-frame collapse. NativeWatch is unchanged and takes an
  optional replace()
- durable: watchEvents keeps every event batch in order (NativeWatch with a snapshot replace);
  document, view and task-graph watches fold; spec, README, example 04 and tests updated
- tests: fold fuzzers (tracker, appendText, splices/reorders, root arrays, diffRevisions) with
  codec round trips, adversarial cases, cap, ownership, cost and GC guards
2026-10-24 09:37:12 +00:00
Mario Zechner
42cb822f1c feat(chord): shared watch-handle implementation (Watch, NativeWatch); Durable uses it 2026-10-23 07:05:23 +00:00
Mario Zechner
92ab536024 feat: Durable storage adopt callback inside the commit, watch types in Chord, explicit watch retirement
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
  synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
  ended (was session_closed) and failed (was session_failed); retirement is an
  explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
  and DocumentState; use watchDoc(), watch(), watchTaskGraph()
2026-10-22 10:52:59 +00:00
Armin Ronacher
89e309581b fix(chord): remove WAL page replication
Reverts 48c8f3228 and 4759f09fb. Session sync replicates pi-durable commits instead of SQLite pages, so NodeSqliteDatabase.exclusive(), the replicated connection wrapper, the WAL reader, page checksums, the segment container, start-marker keys and the zstd page compressor have no user.
2026-10-21 14:26:02 +02:00
Armin Ronacher
48c8f32284 feat(chord): exclusive SQLite operations and a replicated connection wrapper
NodeSqliteDatabase.exclusive() runs an operation alone on the connection, outside a transaction, and rolls back a transaction the operation left open.

The internal replicated connection wrapper opens a database file with automatic checkpoints off, reports every operation, and runs a hook before SQLite checkpoints and deletes the WAL on close. It is not exported from the package yet.
2026-10-20 15:09:25 +02:00
Mario Zechner
d9025a5f5e feat: SQLite telemetry store in Chord, shared TelemetryStore and trace renderer
- pi-telemetry: StoredSpan/StoredTrace/TelemetryStore (findTraces with name, attribute, error, time-window filters and
  cursor paging; getTrace; findMetrics), trace ids carried in carriers, renderTrace in the /render subpath; the in-memory
  adapter implements the store; the adapter contract trusts typed inputs (malformed-input conformance cases removed)
- chord: @earendil-works/chord/telemetry-sqlite over chord/sqlite: settled spans with inline events and links, metrics
  aggregated per series per 10 s, 1 s batched flush with retention, bounded queue (dropped and counted), store queries
2026-10-19 16:50:11 +00:00
Mario Zechner
2ec55d808d feat: Durable and pi-ai telemetry; Context.telemetry is always a span
- pi-ai: AI_TELEMETRY_SCHEMA; pi.ai.request and pi.ai.poll spans around Models requests via a lazyStream observer
  (no provider changes), first_chunk and block events (never per delta), usage/cost end attributes, provider_error
  status with a bounded message, aborted stays ok; duration, first-chunk, token and cost metrics; option renamed
  telemetryContext -> telemetry (a parent span)
- pi-telemetry: TelemetryContext carries readonly metrics and propagation capabilities on every span
- chord/context: Context.telemetry is always a TelemetrySpan (NOOP default); withTelemetry takes spans
- durable: DURABLE_TELEMETRY_SCHEMA; pi.durable.submit (inputs start their own trace linked to the caller) and
  pi.durable.phase spans for every task phase; stored carriers on submissions and tasks (TaskOptions.trace, links);
  entry, submission, checkpoint, replay and report events after commit; resumed tasks stay in their trace; phase
  duration, active task and report metrics; failed phases carry fixed categories with bounded messages
2026-10-19 06:26:02 +00:00
Mario Zechner
48909d987a feat: telemetry propagation, span links and metrics; telemetry member in chord/context
- pi-telemetry: TelemetryPropagation (inject/extract, opaque carriers), span links (SpanOptions.links, addLink),
  TelemetryMetrics (counter, up-down counter, histogram), typed metric schemas (createTypedMetrics), in-memory
  measurement log, conformance tests for propagation and metrics
- chord/context: fixed Context.telemetry member (NOOP default), withTelemetry, traceIn
- durable: HarnessOptions.telemetry option type (no instrumentation yet)
2026-10-18 20:36:08 +00:00
Mario Zechner
c268154ca1 feat(chord): move Durable's SQLite drivers into chord/sqlite 2026-10-17 16:48:29 +00:00
Mario Zechner
4a5d372915 feat(chord,durable): move OutputBuffer and Progress to @earendil-works/chord/output
OutputBuffer, Progress, boundOutput, sanitizeOutput, characterEnd and the
output types move from Durable's internal harness/output.ts to the new
Chord subpath, with OutputSkip (the shape of Durable's ShellOutputSkip) and
utf8ByteLength. Durable imports them from Chord; its truncate.ts re-exports
Chord's utf8ByteLength instead of keeping a second copy.
2026-10-15 09:59:46 +00:00
Mario Zechner
b45e0ebcc5 feat(chord): references hydrate their state before use (protocol 4)
- an awaited get or a ready hold has every state member of its variant, local or remote
- holds subscribe to their variant's state; per-member opt-out with state: false
- remote binding keys use canonical identity; client holds carry an opaque origin
- shape frames no longer carry codeVersion
2026-10-14 14:29:58 +00:00
Mario Zechner
aa9cf15be4 feat(chord): init for state, typed principals, contract placement (protocol 3)
- self.state(name, schema, { init }) loads before the object goes live
- the principal type is a parameter; createHost self, connect and route principals are required
- remote contracts declare their host (hostNamed, hostFamily); get no longer takes at
2026-10-14 11:55:26 +00:00
Mario Zechner
d20c6e9662 refactor(chord): declarations by name, void and no-argument methods, per-cell state versions
- self.state(name, schema, initial, { version?, migrate? }) returns the cell; the state version and migration live on
  the cell (ServeOptions.state is gone). self.derive(sources, fn) has no name and is recomputed at commit.
  self.resource(name, { version, deps?, exclusive?, open, close? }) (rev is version). self.task(fn). self.on(name,
  source, [select,] handler) replaces listen/listener. cell(), derivation(), resource(), listener() and the
  declaration maps are gone; a name declared twice in one factory run is an error.
- A listener on state receives the current value at start and after every reload: handlers must reconcile.
- method(args?, result?): an omitted result is void, omitted arguments make a no-argument method called as
  ref.name(ctx) and implemented as (ctx) => ...; the wire omits both.
2026-10-14 06:12:23 +00:00
Mario Zechner
2688aba7dc refactor(chord): name things after the concepts; object variants; link protocol version; object lifetimes
- References are Ref<C> (was Handle<C>), objects ObjectImpl (record.ts is object.ts), ImplementationOf<C> (was
  BehaviorOf<C>), ResourceDeclaration (was ResourceSpec); epochs are code versions (codeVersion in Ctx, RootOptions,
  Shape, Host, frames and reload reports). Docs say object and reference.
- A remote object's variant is named variant (was kind): factories return { variant }, references expose ref.variant.
- Links announce a protocol version (PROTOCOL_VERSION = 2) in hello; peers of different versions close the link with
  reason protocol-version <n> and report it through LinkOptions.onProtocolMismatch.
- ServeOptions.lifetime ("referenced" | "host") replaces eager: a host-lifetime object is local, has an empty input,
  exists once per host and lives until the host closes; awaitEager is awaitHostObjects. A reload that makes a type
  referenced releases the host's own reference.
2026-10-14 05:34:14 +00:00
Mario Zechner
6db4070f17 docs: describe only the current Chord, Durable, and coding-agent code
Chord: README rewritten for the new runtime; notes.md and requests.md removed, their open limits and
decisions folded into spec.md section 12; guide and spec status lines updated. Durable: drops the
pico-v5 handoff and delta findings, renames the Pico spec, and uses current Chord terms. Delta README,
issue categories, and PI_RADIUS_GATEWAY updated.
2026-10-12 10:47:09 +00:00
Mario Zechner
0104fef1e9 feat(chord): replace the facet and service runtime with the chord2 host runtime
Moves chord2 (contracts, objects, cells, roots, plugins, hot reload, memory/IPC/WebSocket links)
from p2 into packages/chord, with its tests (now on vitest) and docs. The root export is the new
runtime plus the shared base (Context, JsonValue, JsonRepresentation, Draft, copyJson, isJsonValue,
replicatedState and its types). ./node now holds the host manager, child control, process locks,
and nodeLoader. ./context and ./delta are unchanged; ./bundler is removed.

replicatedState stays a standalone module with the same exports. It no longer exposes the exact-op
subscription the removed service provider used, so listener failures of mutable states are always
reported asynchronously. The package depends on typebox instead of esbuild. The wire format stays JSON.
2026-10-12 10:41:59 +00:00
Mario Zechner
35180b9df8 feat(durable): add document states and buffered watches 2026-09-28 16:36:31 +02:00
Mario Zechner
d5cba1d97c feat(chord): make immutable delta tracker canonical
Replace the alternate tracker implementations with the optimized immutable overlay, validate draft placements as strict JSON, and document the ownership contract. Consolidate tracker tests and benchmarks around the canonical implementation.
2026-09-24 15:07:12 +02:00
Mario Zechner
10d1ad621f feat: add transactional replicated state 2026-09-21 16:07:59 +02:00
Mario Zechner
2c995acf44 feat(chord): replace delta tracker with operation log 2026-09-17 16:36:37 +02:00
Armin Ronacher
5dd8c01320 fix(chord): make post-cutover reload failures terminal 2026-09-01 16:16:12 +02:00
Armin Ronacher
c4b0e35abe feat(chord): keep services available during facet reloads 2026-09-01 10:47:08 +02:00
Armin Ronacher
fa5036836d feat: consolidate remote service adapters 2026-09-01 10:09:01 +02:00
Armin Ronacher
1a7bc80e7c feat: move service wire semantics into Chord
Keep protocol payloads transport-opaque and validate service calls, state snapshots, updates, and errors at Chord boundaries. Ensure successful OpenAI Responses messages remain strict JSON so terminal Transcript updates reach clients.
2026-09-01 00:53:40 +02:00
Armin Ronacher
86bac52f94 feat: add delta-backed replicated state
Track mutable provider state and publish immutable operation batches. Keep path codecs isolated per client subscription and state member.
2026-08-31 14:04:42 +02:00
Armin Ronacher
ae2cc5116f feat(coding-agent): replace lane RPC with Chord service 2026-08-31 12:52:25 +02:00
Mario Zechner
2865393cec docs(chord): clarify delta ownership and array costs 2026-08-31 04:30:01 +02:00
Mario Zechner
9af45be821 feat(chord): generate deltas at flush 2026-08-31 03:53:50 +02:00
Mario Zechner
f7079d5628 feat(chord): add JSON delta tracking 2026-08-31 02:04:46 +02:00
Armin Ronacher
65f77ecd6f feat: make Chord facet generations collectible 2026-08-31 00:13:44 +02:00
Armin Ronacher
429f4e756c feat: add package-based facet plugins
Bundle declarative plugin packages with Chord and load fresh generations. Scope client selections to persisted Session and TUI branches, and replace stale local server generations after protocol changes.
2026-08-30 22:05:12 +02:00
Armin Ronacher
0252dff886 feat: add bundled facet distribution and reload
Bundle content-addressed Chord facet entries, distribute server-selected presentation artifacts during the handshake, and reload Session and TUI generations through /reload.
2026-08-30 12:52:58 +02:00
Armin Ronacher
024439d151 feat: clarify Chord remote service APIs 2026-08-29 21:43:01 +02:00
Armin Ronacher
e6dc574d1b feat: isolate Chord context API 2026-08-29 20:01:32 +02:00
Armin Ronacher
6ec57e6e5f feat: consolidate Chord public API 2026-08-29 18:56:26 +02:00
Armin Ronacher
34dc9d055a feat: move facet services into Chord
Move service, replicated-state, facet host, and loader ownership into the standalone Chord package. Migrate consumers and leave runtime JSON enforcement to serializers.
2026-08-29 14:55:51 +02:00
Armin Ronacher
28b49a6b3b feat: add Chord runtime foundation 2026-08-29 14:54:19 +02:00