Commit graph

7337 commits

Author SHA1 Message Date
Mario Zechner
b556cfe5b0 docs(chord): schemas are pi-schema's in the design
Some checks are pending
CI / build-check (push) Waiting to run
CI / test (coding-agent) (push) Waiting to run
CI / test (durable) (push) Waiting to run
CI / test (other) (push) Waiting to run
CI / mcp-conformance (push) Waiting to run
Environments / daemons (push) Waiting to run
Environments / check (push) Blocked by required conditions
Environments / release-daemons (darwin-arm64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (darwin-x64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (linux-arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Environments / release-daemons (linux-x64, ubuntu-latest) (push) Blocked by required conditions
Environments / release-daemons (windows-arm64, windows-11-arm) (push) Blocked by required conditions
Environments / release-daemons (windows-x64, windows-latest) (push) Blocked by required conditions
Environments / tests (macos-latest) (push) Waiting to run
Environments / tests (ubuntu-24.04-arm) (push) Waiting to run
Environments / tests (ubuntu-latest) (push) Waiting to run
Environments / tests (windows-11-arm) (push) Waiting to run
Environments / tests (windows-latest) (push) Waiting to run
2026-11-01 15:20:46 +00:00
Armin Ronacher
c09c98d7d2 fix(chord): admit a routed host even when the relay that spawned it is gone
A relayed hold released (or its principal refused or revoked) while the route resolved made #resolve throw before adding the connection, so the spawned host's link was never read and its own upstream references failed unavailable/ready_timeout after readyTimeout. The connection is now added right after resolve; the checks only fail the relay.
2026-11-01 14:47:36 +01:00
Armin Ronacher
0f21a3319d fix(coding-agent,tui): reuse syntax highlighting and rendered Markdown blocks
Codemode and write rows keep their last highlighting in a CodeHighlighter
that checks theme and grammar changes itself. Markdown reuses the top-level
blocks of its last render, held weakly, and assistant messages reuse their
Markdown instances while streaming. Text.setText() keeps its wrapped lines
for unchanged text.

closes #10971
2026-11-01 14:40:56 +01:00
Mario Zechner
6291b9bcc3 feat(agent): pi-schema's Static types TypeBox-built schemas too, without depending on TypeBox 2026-11-01 11:12:44 +00:00
Mario Zechner
a6bde34bb6 feat(schema): benchmark against ata-validator, Ajv and Zod 2026-11-01 10:22:20 +00:00
Mario Zechner
be039fb316 feat(chord): Chord v1 and v2 on pi-schema, drop TypeBox
Schema helpers are a thin layer over pi-schema's builders with the same JSON,
so fingerprints and stored data are unchanged (proven by schema-switch tests and
a store written by the TypeBox code). Checks through pi-schema; error messages
are pi-schema's; format is an annotation. v1 contracts take any JSON Schema.
map(bytes()) works.
2026-11-01 10:15:02 +00:00
Mario Zechner
6536f80129 feat(ai,agent,coding-agent): validate tool arguments with pi-schema, drop TypeBox
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
2026-11-01 10:08:42 +00:00
Mario Zechner
2c8d1feb14 docs(chord): the schema switch decisions 2026-11-01 00:36:33 +00:00
Mario Zechner
8ffea8a1bf feat(schema): faster object checks and compile, less memory
Some checks are pending
CI / build-check (push) Waiting to run
CI / test (coding-agent) (push) Waiting to run
CI / test (durable) (push) Waiting to run
CI / test (other) (push) Waiting to run
CI / mcp-conformance (push) Waiting to run
Environments / daemons (push) Waiting to run
Environments / check (push) Blocked by required conditions
Environments / release-daemons (darwin-arm64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (darwin-x64, macos-latest) (push) Blocked by required conditions
Environments / release-daemons (linux-arm64, ubuntu-24.04-arm) (push) Blocked by required conditions
Environments / release-daemons (linux-x64, ubuntu-latest) (push) Blocked by required conditions
Environments / release-daemons (windows-arm64, windows-11-arm) (push) Blocked by required conditions
Environments / release-daemons (windows-x64, windows-latest) (push) Blocked by required conditions
Environments / tests (macos-latest) (push) Waiting to run
Environments / tests (ubuntu-24.04-arm) (push) Waiting to run
Environments / tests (ubuntu-latest) (push) Waiting to run
Environments / tests (windows-11-arm) (push) Waiting to run
Environments / tests (windows-latest) (push) Waiting to run
- Declared-object checks walk the value's own keys with for...in, whose
  property reads V8 serves from the enum cache, instead of reading each
  declared name from a list: nested Chord-style arguments 535 -> 409 ns
  (TypeBox 474), recursive $ref schemas 1.1 us -> 753 ns.
- Schemas are indexed for references only on the first $ref; base URIs are
  derived from $id while compiling. Compiling a new schema 43 -> 21 us.
- Drop the keyed cache of leaf checks: on distinct schemas its entries cost
  more than sharing saved. Bare types still share module-level checks and
  patterns share their RegExp. 3,200 distinct schemas 7.9 -> 7.1 MB
  (cfworker 7.6 MB).
2026-10-31 21:59:21 +00:00
Mario Zechner
6d4b744363 docs(chord): the schema package is built 2026-10-31 21:47:22 +00:00
Mario Zechner
fab2c5353e feat(schema): add pi-schema, JSON Schema builders and validation without eval
@earendil-works/pi-schema compiles JSON Schema (drafts 2020-12 and 7) once into
plain closures: no eval or new Function, so it runs under a strict CSP. It
passes 2,222 of the 2,223 core tests of the official JSON-Schema-Test-Suite,
checks Chord-style argument schemas within 1.2x of TypeBox's JIT, and keeps
3,200 compiled schemas in 7.9 MB (TypeBox: 54.7 MB).

- compile(schema, options) returns a Validator: check() is the boolean check
  (a type guard for builder schemas), errors() reports each failure with a
  JSON Pointer path, missing and unknown fields at the field.
- Builders (str, int, obj, opt, union, ...) return plain JSON Schema typed with
  the values it accepts (Static<typeof schema>).
- Formats are an optional entry point (./formats), adapted from TypeBox (MIT).
- Tests run the vendored suite (drafts 2020-12 and 7, with optional and format
  tests); npm run bench compares conformance, speed, setup cost, memory and
  bundle size with TypeBox, @cfworker/json-schema and omptype.
2026-10-31 21:39:34 +00:00
Armin Ronacher
7ef8680abd fix: sync esbuild platform package dev flags in lockfile
esbuild is a non-dev package through chord's optional peer dependency, so its optional @esbuild/* platform packages are non-dev as well. npm install regenerates this; the committed lockfile had stale dev flags.
2026-10-31 22:25:03 +01:00
Mario Zechner
6af92c7950 docs(chord): the plugin model, a primer 2026-10-31 21:02:36 +00:00
Mario Zechner
20362c155e fix(chord): widen the origin-record heap bound for CI noise 2026-10-31 20:52:13 +00:00
Mario Zechner
cc08278a62 fix(chord): slice 10 http builds without DOM types 2026-10-31 20:46:26 +00:00
Mario Zechner
b22c5f6941 docs(chord): slice 10 as built (design 23, spec, explainer) 2026-10-31 20:43:29 +00:00
Mario Zechner
8959ad6e68 feat(chord): Chord v2 slice 10, HTTP and workers
chord/next/http: a Web fetch handler built from the registry Http (routes, wraps,
sockets, hosts, relays, CSP sources) and Authenticators, responses as descriptions,
a security baseline on by default, Chord's own Node listener as a host-lifetime
object, nodeRequest for apps mounting it, links and plugin sockets.
chord/next/workers: a forked worker per address, started on demand, stopped when
idle, never restarted by itself, plugins built by the server and shipped.
Plugins: kinds with platform and host packages, createPluginRuntime, scripts(kind),
rollout results per host, imports of other plugins by name. Kernel: RouteHandle
(retry, suspend), no route backoff, anonymous host.root, origins only from relays.
ws is an optional peer dependency.
2026-10-31 20:40:36 +00:00
Mario Zechner
81f3cd4127 docs(chord): a schema package beneath pi-ai and Chord 2026-10-31 20:06:14 +00:00
Mario Zechner
0baf611f04 docs(chord): API versions in import paths; the plugin model proposal 2026-10-31 19:07:10 +00:00
Mario Zechner
2b06c437ce docs(chord): settings are arguments, extensions are registries 2026-10-31 18:52:54 +00:00
Mario Zechner
83740ffa7a docs(chord): CSP nonce per response 2026-10-31 18:33:24 +00:00
Mario Zechner
d52e953fa8 docs(chord): slice 10 route overhead accepted 2026-10-31 17:47:31 +00:00
Petr Houška
e3da248ea9
fix(coding-agent): keep updating git packages when git clean fails (#11648)
Closes #11597

## Problem

On Windows, `pi update` stopped with `git clean -fdx failed with code 1`
for git extensions such as pi-lens.

1. Another pi session has the extension loaded, so a native module
(`@ast-grep/napi` `.node`) or `ast-grep.exe` is open in that process.
2. npm renames the in-use package dir to
`node_modules/@ast-grep/.napi-win32-x64-msvc-<hash>`. Windows allows the
rename, but npm cannot delete the dir afterwards.
3. The next `pi update` runs `git reset --hard`, then `git clean -fdx`.
git prints `warning: failed to remove ...: Invalid argument` and exits
1.
4. pi threw, leaving the checkout at the new commit with the old
dependencies.

The suggested `-ffdx` does not help. Reproduced locally (git 2.52,
Windows): with a running `.exe`, or a `.node` loaded by another node
process and its dir renamed, both `-fdx` and `-ffdx` fail with `Invalid
argument`. Once the process exits, `-fdx` succeeds.

## Fix

In `cleanAndInstallGitDependencies`, a failed `git clean` no longer
aborts the update. pi:

- prints a warning that files may be in use by another pi session
- still runs `npm install`, so dependencies match the new commit
- keeps the `.pi-update-incomplete` marker, so the next update retries
the clean

## Tests

- `test/git-update.test.ts`: new regression test. A failing clean still
updates and installs dependencies, warns, and keeps the marker. The next
update cleans and removes the marker.
- `test/package-manager.test.ts`: updated the test that expected the
throw.

`npm run check` reports `install-lock is out of date`, and
`package-manager.test.ts > should expand ~ in exclusion and exact
patterns` fails on Windows. Both also happen on `main` without this
change.
2026-10-31 18:19:44 +01:00
Petr Houška
70ca8e8960
docs(coding-agent): list external limitations as known issues with remedies (#11647)
Restructures external limitations in the tmux and Windows guides as
"Known issues" with a remedy for each entry.

- `tmux.md`: the tmux 3.6 color reply section becomes a Known issues
table (issue, affected versions, remedy).
- `windows.md`: the conhost problem table moves to a Known issues
section at the end of the page, with a Remedy column instead of the
cause column. The terminal table links to it. Issue links are reduced to
one per row.

Known issues are limited to things outside Pi's control or deliberate
limitations. Bugs Pi can fix stay in the issue tracker.
2026-10-31 17:55:58 +01:00
Armin Ronacher
b4c2e92acb fix: remove vulnerable dependency chains
Replace build-time shell utilities with Node filesystem operations and retire the native sandbox example in favor of existing isolation options. Upgrade glob matching and worker-test dependencies and refresh the lockfiles.
2026-10-31 17:54:29 +01:00
Petr Houška
6de14ec098
docs(coding-agent): note tmux 3.6 color reply leak in tmux guide (#11645)
Adds a troubleshooting section to the tmux guide
(`packages/coding-agent/docs/tmux.md`, published at
https://pi.dev/docs/latest/tmux) for the color-code text that tmux 3.6
types into the editor at startup.

tmux 3.6 forwards Pi's palette queries to the outer terminal. When a
reply reaches tmux split across reads, tmux treats its bytes as key
presses and sends them to Pi as typed text. tmux 3.7 fixes this
([tmux#4749](https://github.com/tmux/tmux/issues/4749),
[tmux#4793](https://github.com/tmux/tmux/issues/4793)). Pi cannot filter
the text because it arrives exactly like typing.

The section also notes that switching to a fixed theme does not help:
`applyFromSettings()` queries terminal colors regardless of the theme,
which reporters confirmed in #10679 (`--use-theme dark`) and #10256
(`"theme": "dark"`).

Related: #10250, #10679, #10256.
2026-10-31 17:29:36 +01:00
Mario Zechner
f0646a00fb docs(chord): slice 10 fix round small points 2026-10-31 15:57:13 +00:00
Petr Houška
25426f72ae
docs(coding-agent): document Windows terminal and conhost limitations (#11644)
Adds a "Choose a terminal" section to
`packages/coding-agent/docs/windows.md`
(https://pi.dev/docs/latest/windows), with two tables (last updated
October 2026):

- Terminals and which conhost they use: Windows Terminal (recommended),
VS Code, WezTerm nightly (bundled, up to date); WezTerm 20240203
(bundled, old); Alacritty (built-in unless `conpty.dll` +
`OpenConsole.exe` are provided); classic console window, mintty,
ConEmu/Cmder, Windows OpenSSH (built-in).
- Known problems by conhost type:
- Built-in: crash when switching fullscreen -> regular or using `Ctrl+G`
from fullscreen (#9186, microsoft/terminal#17515).
- Built-in behind another terminal (mintty, OpenSSH): query replies leak
into the prompt, BEL opens the external editor (#10256, #10362, #10742).
- Old bundled (WezTerm 20240203): reply fragments in the prompt (#10717,
#10970).
- Built-in, classic console: reported regular-mode redraw issue (#6300).
- Notes that the fixes exist upstream and reach the built-in conhost
with a future Windows update, timing unknown.

Sources: VS Code defaults `terminal.integrated.windowsUseConptyDll` to
`true` (bundled conpty v1.25). Alacritty loads `conpty.dll` only if
present (`alacritty_terminal/src/tty/windows/conpty.rs`) and its
installer ships only `alacritty.exe`.

The crash was verified locally in the classic console window on conhost
10.0.26100.9807: fullscreen -> regular crashed 3/3 runs and `Ctrl+G`
from fullscreen crashed 2/2 runs (`0xC0000005`).
2026-10-31 16:15:42 +01:00
Armin Ronacher
705d047cb5 fix(coding-agent): tighten system prompt 2026-10-31 16:11:43 +01:00
Mario Zechner
ed2bdde64b docs(chord): HTTP core as a fetch handler, Node listener as the easy path 2026-10-31 15:04:43 +00:00
Mario Zechner
56cae78234 docs(chord): plugins import each other by name 2026-10-31 14:46:28 +00:00
Mario Zechner
112a8b46f3 docs(chord): tutorial: base code vs plugins, shipped plugins, subpaths 2026-10-31 14:43:24 +00:00
Mario Zechner
bb36171aac docs(chord): tutorial: only the server builds 2026-10-31 14:28:28 +00:00
Mario Zechner
86338abbd8 docs(chord): tutorial: loading explained (build vs evaluate, host packages, why plugins.load) 2026-10-31 14:26:12 +00:00
Mario Zechner
fe6211607b docs(chord): tutorial: persisted documents and tables 2026-10-31 14:09:23 +00:00
Mario Zechner
dae7bd8fcb docs(chord): tutorial: an object's parts before they are used 2026-10-31 14:07:01 +00:00
Mario Zechner
436d99e04b docs(chord): tutorial: one schema for contract and document 2026-10-31 14:01:40 +00:00
Mario Zechner
a34444d6bd docs(chord): tutorial: overview figure, client hosts 2026-10-31 13:56:14 +00:00
Mario Zechner
081f28bad7 docs(chord): tutorial on hosts and plugins 2026-10-31 13:50:20 +00:00
Mario Zechner
e22468919b docs(chord): HTTP server as a native object; the opinionated app kit as the next design pass 2026-10-31 13:47:28 +00:00
Mario Zechner
b79f21095d docs(chord): slice 10 explainer (HTTP and workers) 2026-10-31 13:30:29 +00:00
Mario Zechner
07309b63eb docs(chord): slice 10 security decisions: origins only from relays, public worker errors, hosts by contribution, the HTTP baseline 2026-10-31 13:28:07 +00:00
Mario Zechner
44d9fb5f6e docs(chord): slice 10 spec wording from the build 2026-10-31 09:18:58 +00:00
Mario Zechner
9b7a44c48d docs(chord): slice 11 draft, P1-P2, one definition per facet kind 2026-10-31 08:22:49 +00:00
Mario Zechner
e4adba965e docs(chord): remove the stale v2 explainer (from before slice 1) 2026-10-31 08:02:27 +00:00
Mario Zechner
bc9a20f847 docs(chord): slice 10 approved 2026-10-31 07:56:35 +00:00
Mario Zechner
61f73f2a80 docs(chord): slice 10 spec review round 2 folded (Q1-Q6), ready to build 2026-10-31 07:56:28 +00:00
Mario Zechner
35dfa75737 fix(durable): idle checks and wake() without per-commit scans
After every commit that changed a task, the scheduler re-checked every
conversation with a kept context range (every conversation read in the last
contextRetentionMs) by walking its subtree, scanned them again for the next
expiry, and wake() scanned every live task. CPU per run grew linearly with the
number of conversations.

Now a commit re-checks only the conversations above tasks that became live or
ended (collected before the sweep; a chain load that resolves an unknown for a
live unloaded task falls back to a full check), idle context ranges expire
from an index ordered by idleSince (re-sorted when the clock goes back; zero
retention drops all idle ranges; reconcile passes still expire for hosts that
decline timers), and wake() takes the earliest sleep from the sleeping tasks
and scans live tasks only while nothing runs.

One Session, one-tool runs, no storage latency: 2048 conversations 100 -> 805
runs/s, 4096: 46 -> 772; local SQLite at 2048: 88 -> 459; CPU per run flat at
about 1.6 ms.
2026-10-30 23:17:56 +00:00
Mario Zechner
918c6ffc22 docs(chord): slice 10 spec review folded (R1-R6), no automatic worker restarts, app kit malleable by plugins 2026-10-30 22:57:59 +00:00
xu0o0
79006f0619
fix(durable,coding-agent): support Unicode word segmentation in tool search (#11037)
This PR uses unicode-aware segmentation and normalization to handle tool
search for multilingual text.

Fixes #10816

---------

Co-authored-by: Petr Houška <houskape@gmail.com>
2026-10-30 23:34:00 +01:00