Commit graph

74 commits

Author SHA1 Message Date
Mario Zechner
e9c0ff4d84 feat(chord): Chord v2 is the package's root
src/next is src (the main export), src/next-node is src/node with short subpaths (/http, /workers, /plugins,
/plugins/runtime, /app, /surfaces, /web, /react, /tui, /tui/launcher); the foundations get /json and /watch;
Chord v1 moves to src/old (/old, /old/node) until p2 runs on v2; tests and docs move likewise (test/, docs/,
test/old, docs/old). Durable and env import the foundations from their subpaths.
2026-11-02 22:36:55 +00:00
Mario Zechner
5bc4145165 feat(chord): Chord v2 web and terminal surfaces (slice 11)
- Web: base bundles built from the app's entry at start (content-hashed, served
  forever), plugin scripts that register themselves and one combined file,
  WebScripts/TuiScripts showing the current set, startPage with the connection
  loop, in-place plugin reloads and a reload on a new base bundle.
- React bindings: HostProvider, useHold, useWatch, useRegistry, useRefState.
- Terminal: startTerminal, a launcher that signs in with a token, a token file or
  device pairing and always runs the app's terminal code in a sandboxed child
  process (pipes, empty environment, --permission granting nothing, no SQLite,
  a builtin allowlist and a frozen process inside); output filter, confirmed
  copy/open/attach actions; startTerminalRuntime and TuiScope inside.
- Themes: the Themes registry, ThemeChoice per principal, CSS variables on the
  page, terminal colours by role; CSS modules in plugin builds.
- App kit: startServer gains web and tui; startApp renamed startServer,
  exportAddress/importAddress renamed exportData/importData and export no longer
  stops the worker; ephemeral sessions end with their client's last link.
- Examples 56-web, 57-tui, 58-themes, 59-chat.
2026-11-02 21:31:59 +00:00
Mario Zechner
072e413bf3 feat(chord): Chord v2 slice 13, the app kit
chord/next/app: startApp and startWorker with the fixed app layout, worker kinds,
AppInfo and WorkerInfo; sessions (hashed tokens, rolling cookies, bearer tokens),
the one-time owner link, the owner token file, device pairing; Files per host
with ordered chunked uploads; export and import per address; plugin roots,
installs and applyChanges; optional users, invite and ephemeral login facets.
HTTP: deny by default (public routes and sockets), remoteAddress,
readSignedCookie. Examples 53-55.
2026-11-02 15:13:37 +00:00
Mario Zechner
9823079ad9 feat(chord): Chord v2 slice 12, the plugin model
Plugins import the app's versioned API (createPlugins api option: unlisted
versions, other kinds' modules, mixed versions and direct Chord imports refused),
each other by relative path within a root or by package name across
repositories (npm or git pins), with per-kind files checked by name.
chord.plugins lists a package's plugins. Required roots, single-file plugins,
installPlugin from git or npm, testPlugin with stub facets.
plugin-authoring.md for authors and agents. Examples 49-52.
2026-11-02 13:19:03 +00:00
Mario Zechner
6536f80129 feat(ai,agent,coding-agent): validate tool arguments with pi-schema, drop TypeBox
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
2026-11-01 10:08:42 +00:00
Mario Zechner
fab2c5353e feat(schema): add pi-schema, JSON Schema builders and validation without eval
@earendil-works/pi-schema compiles JSON Schema (drafts 2020-12 and 7) once into
plain closures: no eval or new Function, so it runs under a strict CSP. It
passes 2,222 of the 2,223 core tests of the official JSON-Schema-Test-Suite,
checks Chord-style argument schemas within 1.2x of TypeBox's JIT, and keeps
3,200 compiled schemas in 7.9 MB (TypeBox: 54.7 MB).

- compile(schema, options) returns a Validator: check() is the boolean check
  (a type guard for builder schemas), errors() reports each failure with a
  JSON Pointer path, missing and unknown fields at the field.
- Builders (str, int, obj, opt, union, ...) return plain JSON Schema typed with
  the values it accepts (Static<typeof schema>).
- Formats are an optional entry point (./formats), adapted from TypeBox (MIT).
- Tests run the vendored suite (drafts 2020-12 and 7, with optional and format
  tests); npm run bench compares conformance, speed, setup cost, memory and
  bundle size with TypeBox, @cfworker/json-schema and omptype.
2026-10-31 21:39:34 +00:00
Mario Zechner
8959ad6e68 feat(chord): Chord v2 slice 10, HTTP and workers
chord/next/http: a Web fetch handler built from the registry Http (routes, wraps,
sockets, hosts, relays, CSP sources) and Authenticators, responses as descriptions,
a security baseline on by default, Chord's own Node listener as a host-lifetime
object, nodeRequest for apps mounting it, links and plugin sockets.
chord/next/workers: a forked worker per address, started on demand, stopped when
idle, never restarted by itself, plugins built by the server and shipped.
Plugins: kinds with platform and host packages, createPluginRuntime, scripts(kind),
rollout results per host, imports of other plugins by name. Kernel: RouteHandle
(retry, suspend), no route backoff, anonymous host.root, origins only from relays.
ws is an optional peer dependency.
2026-10-31 20:40:36 +00:00
Mario Zechner
a14cdf90f0 feat(chord): v2 slice 7c, plugins component and per-plugin load and reload
Plugin discovery, esbuild builds (optional peer dependency), vm evaluation with
pluginFile and mapStack, dependency installs, a watcher with dirty marks and
rollout across hosts; host.load and host.reload per plugin with the closure rule;
examples 35-39 as folders with real plugin directories.
2026-10-30 18:18:23 +00:00
Mario Zechner
14b4c56e57 feat(chord,durable): Chord v2 slice 6b, Durable and jobs on the store
StoreStorage (Durable's storage on Chord v2's store, @earendil-works/pi-durable/storage/store) passes
Durable's storage conformance suite and runs Durable's harness tests (npm run test:store). The store
gets owner names and claims, one id allocator, Durable's Storage semantics for refused steps, history
none/latest/rewindable with reads at a point, archive revisions and applyBatch. self.follow follows
Durable's synchronous observers as Chord sources. Persisted jobs: records and state in the owner's
store, restarts as Durable's tasks, abort stored, deferred state. Chord's ./next export returns for
the adapter surface.
2026-10-28 19:18:30 +00:00
Armin Ronacher
2e0f175bd5
feat(ai): refactor provider definitions (#11368)
This builds the model catalog from per-provider definitions and
models.dev v2 data.

It eplacer `generate-models.ts` with one `defineProvider()` file per
built-in provider, shared model-family rules (Claude, GPT, DeepSeek,
Kimi, Google), and source adapters. models.dev is read through its v2
API. Outputs and CLI flags are unchanged.
2026-10-28 11:06:10 +01:00
Mario Zechner
d9025a5f5e feat: SQLite telemetry store in Chord, shared TelemetryStore and trace renderer
- pi-telemetry: StoredSpan/StoredTrace/TelemetryStore (findTraces with name, attribute, error, time-window filters and
  cursor paging; getTrace; findMetrics), trace ids carried in carriers, renderTrace in the /render subpath; the in-memory
  adapter implements the store; the adapter contract trusts typed inputs (malformed-input conformance cases removed)
- chord: @earendil-works/chord/telemetry-sqlite over chord/sqlite: settled spans with inline events and links, metrics
  aggregated per series per 10 s, 1 s batched flush with retention, bounded queue (dropped and counted), store queries
2026-10-19 16:50:11 +00:00
Mario Zechner
ea7c648430 feat(durable): workerd tests for Durable Object hosting
Runs the storage conformance suite, the schema migration, an alarm-woken sleep across an eviction, and a
turn recovered through the recovery alarm in workerd via @cloudflare/vitest-plugin (npm run test:workers).

fix(durable): decline background timers in Cloudflare Workers. With nodejs_compat, timers have an unref(),
so the idle-memory timers were armed and a pending one kept a Durable Object from hibernating or being evicted.
2026-10-17 23:02:48 +00:00
Mario Zechner
c268154ca1 feat(chord): move Durable's SQLite drivers into chord/sqlite 2026-10-17 16:48:29 +00:00
Mario Zechner
4a5d372915 feat(chord,durable): move OutputBuffer and Progress to @earendil-works/chord/output
OutputBuffer, Progress, boundOutput, sanitizeOutput, characterEnd and the
output types move from Durable's internal harness/output.ts to the new
Chord subpath, with OutputSkip (the shape of Durable's ShellOutputSkip) and
utf8ByteLength. Durable imports them from Chord; its truncate.ts re-exports
Chord's utf8ByteLength instead of keeping a second copy.
2026-10-15 09:59:46 +00:00
Mario Zechner
0104fef1e9 feat(chord): replace the facet and service runtime with the chord2 host runtime
Moves chord2 (contracts, objects, cells, roots, plugins, hot reload, memory/IPC/WebSocket links)
from p2 into packages/chord, with its tests (now on vitest) and docs. The root export is the new
runtime plus the shared base (Context, JsonValue, JsonRepresentation, Draft, copyJson, isJsonValue,
replicatedState and its types). ./node now holds the host manager, child control, process locks,
and nodeLoader. ./context and ./delta are unchanged; ./bundler is removed.

replicatedState stays a standalone module with the same exports. It no longer exposes the exact-op
subscription the removed service provider used, so listener failures of mutable states are always
reported asynchronously. The package depends on typebox instead of esbuild. The wire format stays JSON.
2026-10-12 10:41:59 +00:00
Mario Zechner
7472cda0d6 feat(coding-agent): remove the experimental server stack and the client, protocol, and server packages
Deletes packages/client, packages/protocol (CBOR codec), packages/server, coding-agent's
src/experimental (except source-resolver.ts, which source runs still preload), src/cli/experimental,
src/client, the example plugin, and their tests. coding-agent no longer depends on @earendil-works/chord.
The installer lock drops esbuild, so its platform-package sanity check goes too.
2026-10-12 10:41:59 +00:00
Armin Ronacher
748b351fe2 feat(codemode): run sandboxes on Cloudflare Workers via a remote Durable Object
Split the host into transport-independent execution logic and a VM driver, so the VM can run in a worker thread (CodemodeSandbox, unchanged), on the calling thread (InlineCodemodeSandbox), or in another isolate over RPC (RemoteCodemodeSandbox).

Add the /portable entry without Node imports and the /cloudflare entry with CodemodeSandboxDurableObject, meant to be deployed as its own Worker so scripts never block the caller's thread. Scripts that compute without awaiting a tool are bounded by an interrupt budget, now also polled while draining promise jobs. examples/cloudflare shows the setup.
2026-10-10 14:41:36 +02:00
Mario Zechner
ba03e03f2d feat(env): add @earendil-works/pi-env, remote execution environments over SSH
- RemoteExecutionEnv: a Durable ExecutionEnv on another machine whose
  results match NodeExecutionEnv there; Connection to the pi-env daemon
- packages/env/daemon: the Rust daemon for Linux, macOS, Android and
  Windows, following Node and libuv semantics
- env.yml runs Durable's env tests, the daemon's checks and pi-env's
  conformance and differential tests on Linux, macOS and Windows
2026-10-05 12:50:48 +02:00
Mario Zechner
7fd478a2e8 feat(agent): remove the experimental harness from pi-agent-core
pi-agent-core now contains only Agent, the agent loop, the proxy stream, and
their types. Removed the harness, sessions and session storage, pico3, harness
tools, compaction, skills, prompt templates, telemetry schemas, search types,
and the uuidv7 and pi-telemetry re-exports, plus the ./node, ./harness/* and
./experimental/pico3 subpath exports. Durable sessions live in
@earendil-works/pi-durable.

Also removed packages/session-backends and the experimental mini and micro
coding-agent frontends, with their scripts and build configuration.
2026-10-01 14:45:01 +02:00
Mario Zechner
5609b0d6c0 feat(coding-agent): experimental TUI coding agent on pi-durable
A small local agent in src/experimental/durable: pi's model runtime, settings,
system prompt, theme, and interactive components over a durable Harness with
SQLite storage. Streaming, tools, steer and follow-up, abort, model and
thinking selection, compaction, --continue with recovery of interrupted turns,
a subagent tool with /agents to switch to and talk to subagents, and a live
/tasks panel from the task graph view. The tsconfig path for
@earendil-works/pi-durable/* resolves subpaths to source.
2026-10-01 09:53:21 +02:00
Armin Ronacher
8562bcf66a feat(coding-agent): codemode and MCP
This adds codemode and MCP support to pi. Codemode (packages/codemode) runs
model-written JavaScript in a QuickJS wasm VM inside a worker. From there the
script calls pi's tools as async functions, uses a per-session store, and can
read the model catalog and run classifiers. The codemode tool in coding-agent
is compatible with Codex's exec tool, and codemode.mode works like Codex's
tool modes (on or only). Large results are written to a JSON temp file. MCP
(packages/mcp) is a standalone client for stdio and streamable HTTP servers,
with OAuth.

The core only gets general mechanisms; codemode, tool_search and MCP are
built-in extensions that use them. Other extensions can replace these
built-ins. Tools declare an exposure: direct, model-only, codemode, deferred
or hidden. prepareLoadout() lets an orchestrating tool change which tool
declarations the model sees. ctx.executeTool() runs nested calls through
validation and the tool_call/tool_result hooks. Those calls emit events with
parentToolCallId and are recorded as bounded nestedCalls on the parent
result, which compaction and HTML export use. Tools can also return
structuredContent with an outputSchema, and can report errors with isError
instead of throwing. MCP servers come from mcp.json (global, or per project
after the project is trusted) or from pi.registerMcpServer(). They are
managed with /mcp and pi mcp list|login|logout, and their runtimes load on
first use.

Closes #10040
2026-09-29 09:12:50 +02:00
Armin Ronacher
ca7460d16b feat: build with TypeScript 7 and run sources with plain node
Replace the tsgo native preview with typescript@7.0.2 and port the check scripts to the TS 7 API. Target ES2024, drop useDefineForClassFields: false and unused decorator options, and enable verbatimModuleSyntax. Replace tsx with node plus the source resolver hook, passed to --import as a file URL.

closes #9965
2026-09-25 13:14:33 +02:00
Christian Klotz
b45597504e
feat(durable): export scoped storage conformance suite (#9977)
* feat(durable): export scoped storage conformance suite

* docs(durable): clarify storage conformance example
2026-09-23 23:21:15 +02:00
Armin Ronacher
7fd564cbb7 feat: share model catalog protocol with pi.dev (#9763)
Move catalog layout, index validation, version selection, and request
negotiation into scripts/model-catalog-protocol.ts. pi.dev keeps an
identical copy, so the publisher and a test running the current client
exercise the same selection pi.dev performs. Replaces the cross-repository
compatibility dispatch.

Related to #9099.
2026-09-23 20:08:05 +02:00
Mario Zechner
0801601621 feat(durable): move Pico into dedicated package 2026-09-18 13:39:09 +02:00
Mario Zechner
7e19507681 feat(coding-agent): add experimental micro agent 2026-09-17 16:33:03 +02:00
Armin Ronacher
86bac52f94 feat: add delta-backed replicated state
Track mutable provider state and publish immutable operation batches. Keep path codecs isolated per client subscription and state member.
2026-08-31 14:04:42 +02:00
Armin Ronacher
429f4e756c feat: add package-based facet plugins
Bundle declarative plugin packages with Chord and load fresh generations. Scope client selections to persisted Session and TUI branches, and replace stale local server generations after protocol changes.
2026-08-30 22:05:12 +02:00
Armin Ronacher
0252dff886 feat: add bundled facet distribution and reload
Bundle content-addressed Chord facet entries, distribute server-selected presentation artifacts during the handshake, and reload Session and TUI generations through /reload.
2026-08-30 12:52:58 +02:00
Armin Ronacher
e6dc574d1b feat: isolate Chord context API 2026-08-29 20:01:32 +02:00
Armin Ronacher
6ec57e6e5f feat: consolidate Chord public API 2026-08-29 18:56:26 +02:00
Armin Ronacher
34dc9d055a feat: move facet services into Chord
Move service, replicated-state, facet host, and loader ownership into the standalone Chord package. Migrate consumers and leave runtime JSON enforcement to serializers.
2026-08-29 14:55:51 +02:00
Armin Ronacher
28b49a6b3b feat: add Chord runtime foundation 2026-08-29 14:54:19 +02:00
Mario Zechner
5507d76ee8 feat(agent,ai): add narrow subpath exports so clients skip the barrel
A barrel forces Node to evaluate its whole graph: importing reduceLaneSnapshot
(1 file, 6 KB) through pi-agent-core's index evaluated 106 files, and mini's
server loaded 112 files to enumerate JSONL headers. Types are free, so this only
ever affected value imports.

pi-ai gains ./utils/* alongside the existing ./providers/* and ./api/*
wildcards. pi-agent-core gains ./harness/context, ./harness/env/nodejs,
./harness/runtime/reducer and ./harness/session, and ./session/testing becomes
./harness/session/testing so every export mirrors its source path. The barrel
and ./node keep their batteries-included semantics.

The session layer takes uuidv7 from @earendil-works/pi-ai/utils/uuid rather than
the pi-ai barrel, which is what let ./harness/session drop from 42 files to 19.
Two imports there were written `import { type Usage } from` - not a type-only
statement, so it emitted a runtime import of the barrel.

scripts/check-entry-graphs.mjs walks each declared entry's value-import graph and
enforces per-entry budgets and forbidden paths, so a stray `export *` cannot
silently undo this. It caught the two `import { type ... }` regressions above.

Measured with mini: server 112 files and 98 MB to 28 files and 69 MB, presentation
204 files and 125 MB to 99 files and 110 MB. The worker still loads the provider
graph, correctly.
2026-08-26 22:02:31 +02:00
Armin Ronacher
8dce5da769 feat(coding-agent): coordinate replaceable servers and workers
Keep the stable socket in an opaque detached coordinator while server processes own detached worker discovery and lifecycle. Remove the public drain RPC in favor of server replacement through coordinator routing.
2026-08-15 11:49:53 +02:00
Christian Klotz
6b3fb49dd9
feat: add fail-fast server generation replacement 2026-08-14 13:40:55 +03:00
Christian Klotz
e2888c4a93
feat(coding-agent): add local list and attach demo 2026-08-14 00:36:33 +03:00
Christian Klotz
b3bf0a2a81
feat(agent): add storage backend conformance suite 2026-08-12 00:22:30 +03:00
Mario Zechner
5aeb06bb11 feat(agent): establish durable harness type contracts 2026-08-11 17:43:26 +02:00
Mario Zechner
35f5c265d3 feat: add telemetry reference adapter and span composition 2026-08-06 11:44:58 +02:00
Mario Zechner
6b461b75b3 feat: extract telemetry package 2026-08-05 23:23:12 +02:00
Cristina Poncela Cubeiro
a80008b96c chore: rename storage package to session-backends 2026-08-05 09:28:44 +02:00
Christian Klotz
79cc1ef00a
fix(agent): own SQLite backend tests in storage package (#7626)
* fix(agent): own SQLite backend tests in storage package

* fix(agent): use lexical disposal for SQLite tests
2026-08-04 22:54:15 +03:00
Mario Zechner
44289550aa feat(agent): promote durable harness API 2026-08-04 17:01:43 +02:00
Cristina Poncela Cubeiro
f119b01cb1
refactor: update sqlite for lanes (#7591)
* feat: align sqlite storage with session lanes

* fix: delete lane from entry, action

* fix: errors and types

* fix: tests

* fix: usage records instead of session materialized

* chore: 1 mig

* fix

* refactor(agent): move harness experimental changes to split branch

* refactor and cleanup

* fix: use assertJsonSerializable for session metadata

* fix(sqlite): reject corrupt lane leafs on open

* fix: rebuild cache

* refactor: branch cache

* feat: session stats

* fix: enforce leases

* feat(agent): enforce fenced SQLite writer leases

* fix(agent): keep SQLite transactions synchronous

* fix(agent): align SQLite storage with conformance suite

---------

Co-authored-by: Christian Klotz <hello@christianklotz.co.uk>
2026-08-04 15:56:56 +02:00
Christian Klotz
33bc0a7b83
feat(client): add runtime-neutral session client
Split harness shutdown initiation from settlement so callbacks can request shutdown without deadlocking active work.
2026-07-31 12:19:19 +03:00
Christian Klotz
56eb685b69
feat(protocol): add remote session wire protocol 2026-07-30 21:45:58 +03:00
Cristina Poncela Cubeiro
8495f9d0d6
chore: rename orchestrator to server (#6898) 2026-07-21 13:32:42 +02:00
Cristina Poncela Cubeiro
9e7582aa03
feat: sqlite session storage (#6594)
This PR:

- Adds retainedTail to compaction entries in the new agent harness so we don't have to walk up the tree for the 2000 tokens before compaction,
- Changes getPathToRoot to getPathToRootOrCompaction to only load until last compaction, as unnecessary to access all nodes where it is called,
- Adds a SQLite storage backend, in a separate packages/session-backend-sqlite, with a migration system and schemas as per on-site discussions: sessions to match session header messages (except for metadata, which I couldn't understand what it's used for or where it gets written, so I omitted it), session_entries for shared entry types as columns plus payload as a json for what remains, session_sequences to represent the append-only, serialized nature of the jsonl files, branch_entries to attribute nodes to branches (relationship one-to-many), and session_materialized with the session info (see /session in TUI) to act as a "cache" or quick-access for costs, message count, token info, labels, session name, and model-thinking-level config (e.g. for fast resume).
- This is compatible with the new agent harness Session abstraction.
2026-07-21 11:36:31 +02:00
Armin Ronacher
a9f6a3159a
feat(ai): separate generated model data (#6765) 2026-07-17 11:08:48 +02:00