Adds experimental session sync (`@earendil-works/pi-durable/sync`).
Devices upload and pull a session's commits through a remote, reading
them from the Chord store's outbox and applying them with `applyBatch`.
If two devices continue the same session, the one that uploads second
forks to a new session.
I took out the extra sequence indexes from the earlier version of this
branch, since the Chord outbox already persists every commit, so this no
longer touches storage.
---------
Co-authored-by: Armin Ronacher <armin.ronacher@active-4.com>
buildWorkerBase bundles with esbuild code splitting into one private directory: the entry
and its static chunks load at each fork, a lazily imported module and the packages it
imports only when imported. @earendil-works/chord/workers/child exports the worker's side
(workerMain, processLock) without esbuild.
- type-check subagent-extension.test.ts against the extension view like the other subagent test
- update the test to current ToolRenderContext and ExtensionToolContext types
- use a decodable PNG in the image-only steering test; the fake data was omitted by image processing
The shipped subagent example could say a child was done while it was
still working, lose the useful part of process failures, and return a
normal-looking tool result when the delegated work had actually failed.
Single and chain output could also exceed the normal tool limit.
This also adds a dedicated regression suite covering progress state,
cancellation, process failures, output limits, cwd-based agent
selection, and persisted tool-result error state.
After this change:
- queued, running, completed, failed, and aborted work stay distinct
throughout the run;
- interrupting the parent run preserves partial child results, stops
queued work from starting, and escalates when a child ignores SIGTERM;
- signal exits, spawn failures, provider errors, stderr, and assistant
output remain visible instead of collapsing to `(no output)`;
- failed subagents become failed tool results, so the parent model can
react instead of treating them as successful calls;
- single and chain results stay within the normal 50 KB / 2,000-line
limit;
- project agents are selected and confirmed from each task's actual
working directory rather than the dispatcher's directory.
## Further potential improvements
These are useful follow-ups, but they are separate product and storage
decisions rather than part of the failure-handling fixes in this PR.
<details>
<summary>Potential follow-ups</summary>
### Keep persisted transcripts bounded
Subagent tool results persist the full child message history in
`details`, which can make parent session files large and slow to reload.
A follow-up could compact that history or spill it to disk.
### Define an aggregate parallel-output budget
Parallel mode still gives every task its own 50 KB budget. A follow-up
could keep every task represented while enforcing one overall
tool-result limit.
### Report child usage to the calling session
Per-child token and cost usage is visible in result cards but is not
added to the calling session's totals.
### Isolate malformed agent definitions
One malformed YAML agent can still break discovery for every valid agent
in the same directory.
### Limit subagent recursion
A nesting limit would prevent accidental process and paid-request
multiplication when subagents can invoke the subagent tool again.
* https://github.com/earendil-works/pi/issues/8195
### Add a configurable timeout
Unattended runs would benefit from a generous opt-in timeout for
children that never complete.
</details>
## Related issues
### Spawn failures lose their diagnostics
This directly addresses the reported case where a nonexistent child
working directory produces `(no output)` instead of the actual spawn
error. The regression suite includes that same reproduction.
* https://github.com/earendil-works/pi/issues/8129
### Persisted child history can bloat sessions
This PR intentionally keeps full child history in tool-result details,
so session compaction or spill-to-disk remains follow-up work rather
than something fixed here.
* https://github.com/earendil-works/pi/issues/7452
## What
Hardens the plan-mode example extension's step-progress tracking.
Previously, completion markers were only recognized in the exact
`[DONE:n]` form inside visible text blocks, so steps frequently never
got checked off during execution.
## Changes
- **`getTextContent`** now reads `thinking` blocks as well as text
blocks, so `[DONE:n]` tags emitted during reasoning are honored.
- **Marker tolerance** (`extractDoneSteps`): accepts `[DONE: 1]`, `[DONE
1]`, `Done: n`, and prose phrasings like "Step n done / is complete /
finished" — all adjacency-anchored so negations ("Step 9 is not done")
never match.
- **Whole-plan fallback** (`allStepsComplete`): if the run's final
message says the plan is done (e.g. "All steps complete"), all remaining
steps are marked at `agent_end`.
- **Plan extraction dedup**: only the last `Plan:` section is extracted,
so a plan drafted in a thinking block no longer duplicates the steps
from the text block.
- **No more truncation**: step text is kept in full (previously cut at
50 chars); the widget truncates for display only. The execution prompt
now shows the complete step text.
- **No more dropped steps**: steps whose text starts with a backtick
(e.g. `1. \`index.ts\` — …`) are no longer skipped.
- **Widget refresh** happens on every turn instead of only when a marker
matched.
- **Plan-complete card** is delivered with `nextTurn` instead of
triggering an extra LLM turn.
## Tests
`packages/coding-agent/test/plan-mode-utils.test.ts` updated for the new
behavior (truncation, backtick steps, marker-count semantics) and
extended with coverage for the new cases (marker variants, prose
phrasings, negation safety, thinking-draft dedup, markdown titles,
whole-plan completion).
All 43 tests in `plan-mode-utils.test.ts` +
`plan-mode-extension.test.ts` pass (`npx vitest run`).
Note: the repo-wide type-check currently fails on `main` with
pre-existing `ModelCatalog`/`never` errors unrelated to this change;
this branch adds none (same count with and without the diff).
src/next is src (the main export), src/next-node is src/node with short subpaths (/http, /workers, /plugins,
/plugins/runtime, /app, /surfaces, /web, /react, /tui, /tui/launcher); the foundations get /json and /watch;
Chord v1 moves to src/old (/old, /old/node) until p2 runs on v2; tests and docs move likewise (test/, docs/,
test/old, docs/old). Durable and env import the foundations from their subpaths.
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
Replace build-time shell utilities with Node filesystem operations and retire the native sandbox example in favor of existing isolation options. Upgrade glob matching and worker-test dependencies and refresh the lockfiles.
Fixes#4748.
## Problem
Extensions that import `keyText`/`keyHint` from
`@earendil-works/pi-coding-agent` can resolve their own copy of the
package (their own `node_modules`, or the bundled `cli.js` vs
`dist/index.js`). That copy's keybindings are never set by the host, so
it falls back to a manager with only `tui.*` bindings, and
`keyHint("app.tools.expand", "to expand")` renders as ` to expand`.
## Change
The host passes its own `keyText` and `keyHint` to extension renderers,
the same way it passes `theme`:
- `ToolRenderContext` (`renderCall`/`renderResult` context):
`context.keyHint(...)`
- `MessageRenderOptions` and `EntryRenderOptions`:
`options.keyHint(...)`
All three extend a new exported `KeybindingHints` interface. The docs
and the `debug-provider` example now use the injected functions and warn
against importing them in renderers.
Not done:
- Moving the `pi-tui` keybindings singleton to `globalThis`. Per the
discussion in #7011, extension-facing state should be host-owned and
passed through the extension API instead of shared through process-wide
module state.
- `ctx.ui.keyText`/`keyHint`. No reported case needs them outside
renderers, RPC and print modes never install app keybindings, and adding
them to `ExtensionUIContext` would break SDK hosts that implement it.
## Tests
- `test/extension-keybinding-hints.test.ts`: tool, message, and entry
renderers format a user-overridden `app.tools.expand` binding.
- `npm run check` passes.
tui.select.pageUp/pageDown were configurable but ignored by SelectList,
SettingsList, editor autocomplete, and most coding-agent selectors. In
fullscreen mode, the default PageUp/PageDown also scrolled the transcript
instead of reaching the focused list.
- add getSelectPageIndex() and handle page keys in SelectList, SettingsList,
editor autocomplete, and the model, scoped models, auth, user message,
settings submenu, llama search, and /mcp selectors
- add Component.capturesViewportInput: a focused component that sets it
receives fullscreen viewport keys instead of the transcript, as a focused
overlay already does
- let page keys move an open autocomplete menu before app actions, so a
remapped Ctrl+V pages the menu instead of pasting an image
closes#7629
Co-authored-by: muyiyr <940955635@qq.com>
## Summary
The docs refresh (#9898) cut `docs/extensions.md` from about 3,100 lines
to 130, dropping API reference and tool guidance. #11039 restored part
of it. This PR restores the full guide and corrects it against current
source.
- **Structure:** the original sections, with the ASCII lifecycle diagram
replaced by three Mermaid diagrams (run, turn, session) and ordered hook
lists, plus a handler composition table.
- **Merged:** content added after the cut: tool exposure, calling other
tools, structured output, MCP servers, `registerToolRenderer`, virtual
models, `provider_stream_event`, `before_agent_start` `messageType`.
- **Corrected:** removed or renamed APIs (`modelRegistry.stream()`,
prompt sections are `<tools>`/`<rules>`, `builtin:read` paths,
`highlightCode`), wrong behavior claims (errors only by throwing,
`model_select` on restore, `sendUserMessage` always triggering a turn,
override warning, `truncateLine` bytes), examples that did not
type-check, links to removed `tui.md` patterns and `rpc.md` anchors, and
wrong API lists in the examples table.
- **Kept:** anchors that other docs and changelogs link to, as `<a id>`
tags.
- **Fixed example:** `reload-runtime.ts` tool queued `/reload-runtime`
with `deliverAs: "followUp"`. Queued text reaches the model as a plain
user message, so the command never ran and Pi never reloaded. The tool
now sets a flag and returns `terminate: true`; an `agent_settled`
handler sends `/reload-runtime` with `expandPromptTemplates: true`,
which runs after the settled handlers finish, while the agent is idle.
In the long run it may be better to revisit how API reference is,
provided to the agent, especially since more capable models do explore
the source without it.
## Validation
- `npm run check`
- Edited TypeScript examples type-check against current types
- All relative links and anchors in `extensions.md` resolve, including
inbound anchors from other docs
- `reload-runtime.ts`: faux-provider harness test (temporary, not
committed) shows one reload after the run settles and no extra model
request; the old version never reloads. Also verified in interactive
mode with a real model.
- Reviewed by GPT-6 Astra and GPT-5.6 Sol against source; all findings
applied
- ai: StreamingJsonParser / ToolCallArgumentsStream read each argument delta once (tolerant,
immutable snapshots, linear cost); every provider, pi-messages and the faux provider use it
and keep stream state (partialJson, partialArgs, index, customInput, streamIndex, Bedrock
reasoning bytes) off the content blocks; grammar tool input appends per delta
- agent: streamProxy uses the same parser
- chord: Delta append ops own their text (ownedSlice), so ops and replicas no longer keep the
whole growing string alive
- durable, ai, mcp: bounded error excerpts and summaries own their characters
- coding-agent: tool calls render streamed arguments once per frame, not once per delta
When a todo call fails schema validation, createErrorToolResult produces
details: {} (a truthy empty object), so renderResult skips both the
!details and details.error branches, then switch (details.action) has no
default and returns undefined. ToolExecutionComponent.updateDisplay()
then calls addChild(undefined), and the next Box.render pass crashes the
whole interactive TUI.
Add a default case that renders the validation error text.
Deletes packages/client, packages/protocol (CBOR codec), packages/server, coding-agent's
src/experimental (except source-resolver.ts, which source runs still preload), src/cli/experimental,
src/client, the example plugin, and their tests. coding-agent no longer depends on @earendil-works/chord.
The installer lock drops esbuild, so its platform-package sanity check goes too.
Runs started by pi.sendMessage(..., { triggerTurn: true }) skipped before_agent_start. After their first tool call, the next-turn refresh fell back to the base prompt options and patched away the sections handlers had added, changing the system prompt mid-run and breaking the provider prompt cache.
These runs now go through the same setup as prompt(): compaction check, before_agent_start, tool loadout, nextTurn messages, handler messages, and the system patch. BeforeAgentStartEvent is a union on messageType ("user" or "custom"); custom events carry the starting message.
The session counts as busy from the start of setup, so messages sent while handlers run go into the starting run instead of starting a second one, and an abort requested meanwhile aborts the run once it starts.
closes#10267, closes#5581, closes#10639, closes#10289
partly fixes#10606
* feat(coding-agent): use pi.dev configuration schemas
* refactor(coding-agent): separate schema URL from output path
* docs(coding-agent): document published schema URLs
Editor and Input wrap their drawn cursor in fake cursor markers. The TUI
resolves them to reverse video, or drops the focused one when the hardware
cursor is shown, so terminals that extend edge cell colors into the window
padding no longer stretch the drawn cursor at the first or last column.
Also keep CURSOR_MARKER when an overlay ends directly left of the cursor.
MCP codemode exposure no longer lists tools, tool counts, or server
instructions in the codemode description. Servers are listed by name and
an optional mcp.json description; scripts find tools with searchTools()
and read instructions with describeNamespace(). codemode-deferred is an
alias for codemode.
refs #10212
Entries of only +name/-name modify the inherited tool selection, so
"defaultTools": ["+codemode"] enables codemode without repeating the
defaults. Project entries layer on top of user settings. Document how to
enable codemode without MCP and how to use classifier models from it.
This adds codemode and MCP support to pi. Codemode (packages/codemode) runs
model-written JavaScript in a QuickJS wasm VM inside a worker. From there the
script calls pi's tools as async functions, uses a per-session store, and can
read the model catalog and run classifiers. The codemode tool in coding-agent
is compatible with Codex's exec tool, and codemode.mode works like Codex's
tool modes (on or only). Large results are written to a JSON temp file. MCP
(packages/mcp) is a standalone client for stdio and streamable HTTP servers,
with OAuth.
The core only gets general mechanisms; codemode, tool_search and MCP are
built-in extensions that use them. Other extensions can replace these
built-ins. Tools declare an exposure: direct, model-only, codemode, deferred
or hidden. prepareLoadout() lets an orchestrating tool change which tool
declarations the model sees. ctx.executeTool() runs nested calls through
validation and the tool_call/tool_result hooks. Those calls emit events with
parentToolCallId and are recorded as bounded nestedCalls on the parent
result, which compaction and HTML export use. Tools can also return
structuredContent with an outputSchema, and can report errors with isError
instead of throwing. MCP servers come from mcp.json (global, or per project
after the project is trusted) or from pi.registerMcpServer(). They are
managed with /mcp and pi mcp list|login|logout, and their runtimes load on
first use.
Closes#10040
This adds experimental support for virtual models. A virtual model is a catalog entry that an extension registers with pi.registerVirtualModel(). It does not talk to a provider itself; for every request it picks a physical model and thinking level. The motivation is to make routing policies pluggable: plan on a strong model and implement on a cheap one, pick a model with a classifier, or move to a larger window when the context grows. Today this needs model switching by hand or hacks in prepareRequest.
The selection stays virtual: model_change records it, resume restores it, and each response records the physical model and thinking level that produced it. Router state can be stored on the session branch, so it follows /tree and forks and survives compaction. Context limits, compaction and image sizing use the physical model a request was routed to. examples/extensions/jev-router.ts shows a full router that uses the Jev classifier to pick Sol or Terra for planning and hands off to Luna after the first edit. docs/virtual-models.md describes the API.
Replace the tsgo native preview with typescript@7.0.2 and port the check scripts to the TS 7 API. Target ES2024, drop useDefineForClassFields: false and unused decorator options, and enable verbatimModuleSyntax. Replace tsx with node plus the source resolver hook, passed to --import as a file URL.
closes#9965
This change makes system prompt text and tool changes part of the transcript rather than silently rewriting its starting conditions. This lets Pi record when instructions changed or tools became available, restore that state after resuming or navigating branches, and preserve cached prompt prefixes where the upstream supports it.
