src/next is src (the main export), src/next-node is src/node with short subpaths (/http, /workers, /plugins,
/plugins/runtime, /app, /surfaces, /web, /react, /tui, /tui/launcher); the foundations get /json and /watch;
Chord v1 moves to src/old (/old, /old/node) until p2 runs on v2; tests and docs move likewise (test/, docs/,
test/old, docs/old). Durable and env import the foundations from their subpaths.
Update the model catalog protocol to pi.dev after a change to stop
pi.dev redirecting released clients to `?pi-version=` URLs and serve
their catalog directly. The released CLI is unaffected.
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
@earendil-works/pi-schema compiles JSON Schema (drafts 2020-12 and 7) once into
plain closures: no eval or new Function, so it runs under a strict CSP. It
passes 2,222 of the 2,223 core tests of the official JSON-Schema-Test-Suite,
checks Chord-style argument schemas within 1.2x of TypeBox's JIT, and keeps
3,200 compiled schemas in 7.9 MB (TypeBox: 54.7 MB).
- compile(schema, options) returns a Validator: check() is the boolean check
(a type guard for builder schemas), errors() reports each failure with a
JSON Pointer path, missing and unknown fields at the field.
- Builders (str, int, obj, opt, union, ...) return plain JSON Schema typed with
the values it accepts (Static<typeof schema>).
- Formats are an optional entry point (./formats), adapted from TypeBox (MIT).
- Tests run the vendored suite (drafts 2020-12 and 7, with optional and format
tests); npm run bench compares conformance, speed, setup cost, memory and
bundle size with TypeBox, @cfworker/json-schema and omptype.
Replace build-time shell utilities with Node filesystem operations and retire the native sandbox example in favor of existing isolation options. Upgrade glob matching and worker-test dependencies and refresh the lockfiles.
A Session no longer holds its line while a commit is stored. Commits that
arrive while one storage call is in flight go out together in the next call,
each with its own seq and outcome; a job runs until it touches a key an
unpublished commit wrote, then waits for that commit's publication. Outcomes
are delivered in line order; publication is in line order, one commit per
macrotask. The scheduler's decisions run as regions that record the keys they
read and rerun after a wait, with side effects only after a clean check.
Storage.commit(batches, receive, context) replaces commit(writes, adopt,
context): prefix semantics (Durable's own checks refuse a batch and keep the
ones before it; any other failure rejects the call), receipts per stored
batch, optional publishing() scope. Memory, SQLite, JSONL, IndexedDB and
StoreStorage implement it natively; StoreStorage through the new
ChordStore.commitSeries (one store transaction per call), replacing
commitInBatch.
A normal run makes no storage reads: bounded record cache, conversation
tails, task-document index, kept context ranges. A tool slot's running status
and a generation's attempt are stored with the next commit (12 instead of 14
commits per run).
One Harness with 128 conversations over storage with a 4 ms round trip: about
5 -> 440 agent runs per second. Design: packages/durable/docs/group-commit.md.
Registries: shared tokens (defineRegistry), one provider per host (host.serve(R, { compose, accepts })),
contributions from facets and host-lifetime objects, plain values or followed Chord watchables, a table
sorted by plugin, lazy compose in the batch that changed the table, failures left out and reported with
the last good value kept; self.registry as a declared dependency, host.registry outside objects, older
readers mapped by accepts[old].value; keyedEntries and orderedEntries. Native sources: self.writable,
registry values and ref.state. self.retryOn wakes failed objects on registry values. host.root gives code
outside objects a call scope sealed at settle, abort and host.close.
Durable: registryReader(value, context); a Session close ends synchronous observers failed
("session closed"), watches still end ended.
This builds the model catalog from per-provider definitions and
models.dev v2 data.
It eplacer `generate-models.ts` with one `defineProvider()` file per
built-in provider, shared model-family rules (Claude, GPT, DeepSeek,
Kimi, Google), and source adapters. models.dev is read through its v2
API. Outputs and CLI flags are unchanged.
`scripts/check-entry-graphs.mjs` built the list of files each export
reaches with `path.relative()`, which returns paths with `\` separators
on Windows. The allow and forbid patterns are written with `/`, which
caused two problems on Windows:
- The allow entry `packages/telemetry/src/index.ts` never matched, so
the bare `index.ts` forbid pattern flagged it and the check failed even
though nothing was wrong.
- Forbid patterns that contain a `/` (`providers/`, `runtime/host.ts`,
...) never matched, so real violations went unreported.
The fix converts the relative paths to `/` before any comparison. On
Linux and macOS nothing changes.
Testing:
- The scripts have no unit tests, so no test was added.
- `node scripts/check-entry-graphs.mjs` and `npm run check` pass on
Linux.
- I simulated Windows path handling with `path.win32`, without running
Windows itself:
- Before the fix: `relative(...)` gives
`packages\telemetry\src\index.ts`, and
`endsWith("packages/telemetry/src/index.ts")` is false.
- After the fix: the path becomes `packages/telemetry/src/index.ts`, and
the suffix check is true.
Fixes#11333
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
- chord/delta: PendingOps (push, size, take -> PendingTake): raw pending ops, r drops everything
before it, a leaf index merges appends and primitive sets per path (cleared by any other op),
past 10,000 ops only "replace with the current value"; one batch is kept by reference and
received ops are never mutated
- chord: Watch queues frames in PendingOps while its listener is busy or not started and delivers
one call with the newest value, the folded ops (or [["r", newest]] after an overflow) and the
newest frame's context; replaces the 100-frame collapse. NativeWatch is unchanged and takes an
optional replace()
- durable: watchEvents keeps every event batch in order (NativeWatch with a snapshot replace);
document, view and task-graph watches fold; spec, README, example 04 and tests updated
- tests: fold fuzzers (tracker, appendText, splices/reorders, root arrays, diffRevisions) with
codec round trips, adversarial cases, cap, ownership, cost and GC guards
Follow-up to #9265.
- chord: Delta appendText(draft, key, text) / StringKeys<T> record an append op without the
O(length) prefix compare that a plain `draft[key] += text` costs in prepare()
- durable: partial commits append text and thinking deltas with appendText (assignJson plan,
checked by length), and apply tool-call argument changes from their own StreamingJsonParser
of the raw toolcall_delta text instead of reassigning the arguments snapshot
- durable "." entry-graph budget 73 -> 76 for the parser it now imports
* feat: publish preview packages to pkg.earendil.com
CI packs every public package after the build, with internal dependencies
pointing at the same commit's builds. A workflow_run job publishes them to
the earendil-pkg worker with the job's GitHub OIDC token, moves main or
pr-<n>, and comments install commands on the PR. Closing a PR deletes its
ref. pi-env is left out because its daemons are only built for releases.
* feat: publish preview packages only for main and same-repo PRs
pkg.earendil.com requires a consumer token, so fork authors could not
install their previews anyway. Fork PRs are no longer packed or
published, which keeps fork code out of the registry and removes the PR
lookup: the PR number now comes from the workflow_run event. The PR
comment is dropped; the job summary lists the install commands.
* fix: pick the PR into main and describe only the workflow in preview-packages.yml
- Read the PR number from the workflow_run pull request list with jq,
selecting the PR into main; a branch can also have PRs into other
branches, and the index expression tripped workflow linters.
- Comments describe what the workflow does and what it needs from
pkg.earendil.com, not how the service works.
* fix: gate preview publishing to main pushes and open PRs into main
- Publish only CI runs for pushes to main and for pull requests. A branch
can make its own copy of CI run on push; those runs are unreviewed and
were uploaded as push builds.
- Move pr-<number> only while the PR is open into main with this commit
as its head, checked live, so a late publish cannot recreate the ref of
a closed PR; otherwise publish without a ref and print a notice.
- Set concurrency per job, so skipped fork runs never wait in a group.
* docs: drop the service rule from the preview-packages.yml header
* docs: name the earendil-works/pkg worker in the preview script
Every built-in storage implements ReplayableStorage: head(), replay() at an explicit sequence, and export() of the current state as a log that replays to the same reads. SQLite schema version 3 adds sequence indexes so export streams. @earendil-works/pi-durable/sync adds ReplicatingStorage, which appends each committed batch to an outbox with fork copies resolved, compactRecords(), and versioned JSON log lines.
pi-ai gets a ClientIdentity request option and createModels() default that choose the User-Agent (detailed with version, short, or custom) and provider attribution headers for OpenRouter, NVIDIA NIM, and OpenCode. Attribution is off by default in pi-ai and on by default in pi-agent-core and pi-durable (HarnessOptions.identity). pi-coding-agent sets its identity from the telemetry setting instead of adding headers itself.
Breaking change: defineDoc and defineDocFamily now require an explicit checkpointWhen predicate; no default is supplied. Set policies on built-in documents, examples, tests, and triage state, and document the required API.
- pi-ai: AI_TELEMETRY_SCHEMA; pi.ai.request and pi.ai.poll spans around Models requests via a lazyStream observer
(no provider changes), first_chunk and block events (never per delta), usage/cost end attributes, provider_error
status with a bounded message, aborted stays ok; duration, first-chunk, token and cost metrics; option renamed
telemetryContext -> telemetry (a parent span)
- pi-telemetry: TelemetryContext carries readonly metrics and propagation capabilities on every span
- chord/context: Context.telemetry is always a TelemetrySpan (NOOP default); withTelemetry takes spans
- durable: DURABLE_TELEMETRY_SCHEMA; pi.durable.submit (inputs start their own trace linked to the caller) and
pi.durable.phase spans for every task phase; stored carriers on submissions and tasks (TaskOptions.trace, links);
entry, submission, checkpoint, replay and report events after commit; resumed tasks stay in their trace; phase
duration, active task and report metrics; failed phases carry fixed categories with bounded messages
pi.dev serves each published catalog to released pi >= MINIMUM_PI_VERSION.
Compile the latest release's pi-ai against its built-in models overlaid by
the new catalog, so a catalog it cannot handle (e.g. kimi-coding moving to
openai-completions) fails in CI instead of breaking released clients.
MINIMUM_PI_VERSION moves to scripts/model-catalog-minimum-version.mjs.
The pin script discovered the catalog with the checkout's version, which on
main is the last release, so pi.dev answered with the catalog for released
clients after MINIMUM_PI_VERSION was raised. It now pins the newest
published catalog and treats the pin as stale when its provider/type/API
model groups differ. The release script pins the newest catalog and runs
the Nix build steps before tagging.
closes#10985
Switch kimi-coding to OpenAI Chat Completions at api.kimi.com/coding/v1,
the protocol the official Kimi Code client uses. K3, K3-256K, and Kimi For
Coding accept tools declared in later system messages, so tools loaded
mid-conversation (e.g. by tool_search) keep the prompt cache. Highspeed
rejects tool-bearing system messages and keeps added tools top-level.
Re-lands b06c1f2 (reverted in 6a1f799b6) with MINIMUM_PI_VERSION bumped
to 1.2.1. Released clients implement only Anthropic Messages for
kimi-coding and ignore a model's api, so they must keep receiving the
last catalog published under 0.80.7.
closes#10720
Chord: README rewritten for the new runtime; notes.md and requests.md removed, their open limits and
decisions folded into spec.md section 12; guide and spec status lines updated. Durable: drops the
pico-v5 handoff and delta findings, renames the Pico spec, and uses current Chord terms. Delta README,
issue categories, and PI_RADIUS_GATEWAY updated.
Moves chord2 (contracts, objects, cells, roots, plugins, hot reload, memory/IPC/WebSocket links)
from p2 into packages/chord, with its tests (now on vitest) and docs. The root export is the new
runtime plus the shared base (Context, JsonValue, JsonRepresentation, Draft, copyJson, isJsonValue,
replicatedState and its types). ./node now holds the host manager, child control, process locks,
and nodeLoader. ./context and ./delta are unchanged; ./bundler is removed.
replicatedState stays a standalone module with the same exports. It no longer exposes the exact-op
subscription the removed service provider used, so listener failures of mutable states are always
reported asynchronously. The package depends on typebox instead of esbuild. The wire format stays JSON.
Deletes packages/client, packages/protocol (CBOR codec), packages/server, coding-agent's
src/experimental (except source-resolver.ts, which source runs still preload), src/cli/experimental,
src/client, the example plugin, and their tests. coding-agent no longer depends on @earendil-works/chord.
The installer lock drops esbuild, so its platform-package sanity check goes too.
Everything outside apps/ is the same in pi and pi-private; apps/ exists only
in pi-private, and these files now tolerate its absence:
- workspaces include apps/*
- biome.json leaves apps/ to its own config
- the pre-commit hook runs the apps/mobile check only when it exists
- the Android APK workflow runs only in earendil-works/pi-private
- scripts/biome-check.mjs runs native Biome, or Biome WASM on Android
(@biomejs/wasm-nodejs 2.3.5), for npm run check
From pi-private bac58de97, c2a8629da, 106714ebc.
Split the host into transport-independent execution logic and a VM driver, so the VM can run in a worker thread (CodemodeSandbox, unchanged), on the calling thread (InlineCodemodeSandbox), or in another isolate over RPC (RemoteCodemodeSandbox).
Add the /portable entry without Node imports and the /cloudflare entry with CodemodeSandboxDurableObject, meant to be deployed as its own Worker so scripts never block the caller's thread. Scripts that compute without awaiting a tool are bounded by an interrupt budget, now also polled while draining promise jobs. examples/cloudflare shows the setup.
read returns an image file as one image block (programs get an ImageContent) plus an info diagnostic,
instead of an unsupported_image error. Limits are the conversation model's inputLimits.images.resize,
by default 2000x2000 pixels and 4.5 MB of base64.
- Without a processor: PNG, JPEG, GIF, and WebP within the limits pass through undecoded; the byte limit
is checked before reading the file, dimensions from the header. Larger images and BMP are error results.
- With createCodingTools({ images }): the processor orients by EXIF, converts BMP to PNG, and shrinks to
fit. ImageProcessor.prepare is async.
- @earendil-works/pi-durable/images: createPhotonImages(wasm) on @silvia-odwyer/photon 0.3.3 (web build);
/images/node and /images/cloudflare load the wasm. The root and ./tools entries never reach it (enforced
in check-entry-graphs).
- An image must be read whole: one that changes while read is read again, unlike a growing text file.
- A model without image input gets a diagnostic that it sees a placeholder.
Nested tool calls: ToolExecutionApi.executeTool() runs another tool as its own pi.tool task owned by the caller,
keyed for replay-safe reattachment. Results for programs live in the caller's NestedResultDoc until it settles;
pi.live.nestedTools shows each call with the arguments it runs with and a summary. TaskOptions.abandonOnRestart
aborts work its creator awaited only in memory before it runs again after a restart.
Structured output: structuredOutputSchema and structuredOutput, validated; a tool without a schema gives programs its
bounded output (one text as a string, one image as itself, else the list). ToolExecutionResult.content is now output.
callers and modelTools decide who may call a tool.
Storage errors: any error a Storage method throws fails the Session, which closes itself; later calls and waits reject
with SessionFailed, watches end with session_failed, and Session.closed settles. Nothing retries in the Harness.
StorageRejected is removed; StorageRequestError marks invalid reads that fail only their call. Listener and onReport
errors are contained and reported.
Also updates scripts/issues/triage.ts and the vacation experiment for the content -> output rename.
Syncs issues with gh, triages new ones with a pi-durable agent (category, duplicates, summary, p0/p1/none suggestion), and serves a web UI for decisions.
- RemoteExecutionEnv: a Durable ExecutionEnv on another machine whose
results match NodeExecutionEnv there; Connection to the pi-env daemon
- packages/env/daemon: the Rust daemon for Linux, macOS, Android and
Windows, following Node and libuv semantics
- env.yml runs Durable's env tests, the daemon's checks and pi-env's
conformance and differential tests on Linux, macOS and Windows
Resolve the QuickJS wasm path once and spawn the bundled codemode worker
from an in-memory data: URL. Show a restart hint when errors occur after
the install changed on disk.
closes#10439
Assign each conversation its own UUIDv7 and forward it to model requests for prompt-cache affinity. Lazily backfill legacy conversations and cover real Codex cache reuse.
Closes#10424
The scheduled npm audit failed on GHSA-86w9-cpqp-85rv, which has no fixed node-forge release. node-forge is only reachable through gondolin in the private gondolin example extension, and gondolin only verifies certificates from its own CA with exponent 65537, so the low-exponent forgery does not apply. The audit now runs through scripts/npm-audit.mjs, which fails on any other advisory at or above moderate.
Add a Nix flake that builds Pi from the checkout on aarch64/x86_64 Linux
and macOS. The build is offline: provider model data is hydrated from a
pi.dev model catalog revision pinned in nix/model-catalog.json, and the
runtime dependency tree comes from packages/coding-agent/install-lock with
internal packages built from source.
The Nix workflow keeps the catalog pin current on main, builds release
tags on all systems, and fast-forwards the stable branch to each release
that builds. The release script refreshes the pin before tagging.
npm installs no longer pin transitive dependencies. Pinned installs come
from the pi.dev installer lockfile. pi update on global npm installs now
recommends migrating with the installer.
closes#5653
pi-agent-core now contains only Agent, the agent loop, the proxy stream, and
their types. Removed the harness, sessions and session storage, pico3, harness
tools, compaction, skills, prompt templates, telemetry schemas, search types,
and the uuidv7 and pi-telemetry re-exports, plus the ./node, ./harness/* and
./experimental/pico3 subpath exports. Durable sessions live in
@earendil-works/pi-durable.
Also removed packages/session-backends and the experimental mini and micro
coding-agent frontends, with their scripts and build configuration.
Package 22: runtime.now() and report() throw after the invocation ended; a
migration-only commit is published; a failed Harness.open closes without the
caller's context and rethrows the open error; ConversationWatch alias. Tests
for close joining non-cooperative code, cancelled close, queued operations at
the seal, progress calls and viewers on a paused Harness, the Chord guide, and
the spec's usage examples.
Package 23: Harness.taskGraph() and watchTaskGraph() mount every live task with
its owner edge, committed status, flags, and owned conversations, advanced from
commit publications. Example 24 prints the checkout's tree.
- Registry of named extensions (tools, sections, hooks, wraps, tasks) with install/uninstall;
defineExtension, defineTool, section, hook, wrapTool, wrapSection
- pi.agent document (model, thinking level, extensions, tools filter, instructions, cwd),
configure(), Conversation.agent()/configure(), agent option on root/createConversation/fork,
task-owned conversations copy their owner's agent
- Harness-wide HarnessSettings read at every use; env function built per use
- Hooks and the agent resolved per task phase; tool calls resolved with the tools filter applied
- ToolRegistration<TParameters, TDetails> types execute() args from parameters
- HarnessOptions.conversationCreated; Tx.submissionByRequest() and Tx.createSubmission()
- FileSystem.id; edit/write serialize per file system id and canonical path
- CodingTools extension (read, write, edit, bash)
- Examples 06-31 on the new API, README and CHANGELOG
Manual compact(), background and blocking threshold compaction, and overflow
compaction with one retry. Summaries are headed pi.compaction entries placed
through the inbox, or appended by a compaction the generation waits on; the
stale rule decides between concurrent summaries.
