Adds unsafe<T>(), recursive<T>(), Validator.validate() (check plus
errors, no coercion), bigint(), equal(), validation and coercion of
TypeBox's Type.BigInt/Type.Undefined, and Static for TypeBox's Enum,
Tuple, BigInt and Undefined. Also fixes a required property set to
undefined being rejected when its schema accepts undefined, as TypeBox
allows.
Ctrl+P did nothing when the scope held exactly one model and the active
model was a different one. It showed "Only one model in scope" and kept
the current model, because `_cycleScopedModel` returned early for any
list with one entry. `_cycleAvailableModel` had the same early return,
so it also failed when the active model was no longer available and one
other model was.
A second bug had the same cause. When the active model was not in the
list, the code treated it as if it were at index 0, so cycling forward
skipped the first scoped model. With `--models A,B` and another model
active, Ctrl+P went to B.
This change adds a small `getNextCycleIndex` helper that both functions
now use:
- When the active model is not in the list, cycling forward selects the
first entry and cycling backward selects the last.
- It returns `undefined` (the "Only one model…" message) only when the
list is empty, or when its single entry is already the active model.
**Testing**
- Added a regression test to
`test/suite/agent-session-model-extension.test.ts`. It covers a
one-model scope, a repeated cycle once that model is active, and forward
and backward cycling into a two-model scope from outside it. The test
fails without the change ("expected undefined to be 'faux-2'") and
passes with it.
- `npm run check` passes.
- Manual check: `pi --model anthropic/claude-sonnet-4-5 --models
anthropic/claude-opus-4-5`, then Ctrl+P.
- Before: "Only one model in scope".
- After: "Switched to Claude Opus 4.5".
- With `--models anthropic/claude-opus-4-5,anthropic/claude-haiku-4-5`,
Ctrl+P now selects opus instead of haiku.
Fixes#11806
Requested by @davidbrai
Co-authored-by: triagent <triagent@localhost>
Co-authored-by: David Brailovsky <davidbrai@gmail.com>
models.login loads the OAuth flow module through a URL import since 5c6ce2b32, which takes real I/O under vitest. A single 1000ms fake-timer advance ran before the device flow started, leaving the poll wait unadvanced and the test hanging.
Back-to-back cache warms update a single 'Cache warmed: $<total>' line with the cumulative cost instead of adding one line per warm. Each warm is still persisted as its own usage entry.
buildWorkerBase bundles with esbuild code splitting into one private directory: the entry
and its static chunks load at each fork, a lazily imported module and the packages it
imports only when imported. @earendil-works/chord/workers/child exports the worker's side
(workerMain, processLock) without esbuild.
registerLazyOAuthFlows (@earendil-works/pi-ai/lazy-oauth) and registerLazyBedrockProvider
(@earendil-works/pi-ai/lazy-bedrock-provider, built on setBedrockProviderLoader) load the
flows and Bedrock through literal dynamic imports, so a splitting bundler keeps them inside
the bundle as lazy chunks. The variable-specifier default is unchanged.
Chord's worker base inlines pi-ai into a bundle in a temporary directory. esbuild keeps the variable-specifier imports of the OAuth flows and the bedrock API, so Node resolved them against the bundle and failed (radius OAuth: Cannot find module .../chord-worker-*/radius.ts). Resolving against import.meta.url, which Chord defines per source module, finds the real file.
Fixes#11692.
## Problem
In RPC mode the startup catalog refresh runs in the background and also
refreshes an expired OAuth credential. If pi exits after the token
request went out but before the new token is written, the provider has
already spent the old single-use refresh token and auth.json keeps only
that one. The next request fails with `invalid_grant`.
Trace: `pi --mode rpc` starts, `get_state` is answered, catalog refresh
sends the token request, the client sends SIGTERM (or closes stdin),
`shutdown()` runs `runtimeHost.dispose()`, which does not know about the
refresh, and the process exits. Same for print and interactive mode, and
for a prompt-time refresh cut off by an exit.
## Fix
- `RuntimeCredentials.modify()` tracks every pending modification (same
idiom as the MCP `track()` / `McpAuthProvider.settled()`).
- `RuntimeCredentials.settled()` /
`ModelRuntime.settleCredentialWrites()` wait for them.
- `AgentSessionRuntime.dispose()` waits for them. All modes exit through
`dispose()`. No extra timeout: the OAuth refresh (15 s) and the
auth.json lock wait (30 s) are already bounded, and a second signal
still exits at once.
Not changed: the catalog refresh still refreshes expired OAuth
credentials, also for providers whose catalog does not use them (the
pi.dev catalog for `openai`, `anthropic`, ...). Skipping that would need
a per-provider "catalog needs auth" signal in pi-ai; it is optional now
that exits wait.
`docs/rpc.md` now says that pi waits for pending work before exiting.
## Testing
- `test/runtime-credentials.test.ts`: `settled()` waits until a pending
modification is written, and resolves after a failed one.
-
`test/suite/regressions/11692-dispose-waits-for-oauth-refresh.test.ts`:
`dispose()` waits until the token from a background catalog refresh is
saved. Fails without the fix.
- Real process: `pi --mode rpc` with an extension OAuth provider
(expired credential, 400 ms refresh), SIGTERM about 100 ms after the
token request started. Without the wait: 3 of 3 runs kept the spent
refresh token. With the fix: 3 of 3 saved the new one (exit 143).
Closing stdin instead of SIGTERM: new token saved, exit 0.
- `npm run check`; `test/rpc*`, `test/print-mode`,
`test/model-runtime*`, runtime and regression suites pass.
Fixes#11613.
## Problem
The bash tool waits for stdout/stderr to close, not only for the shell
to exit (#5303). A background process that inherited those pipes and
keeps writing keeps the tool waiting after the shell exited. When the
timeout then hits, the tool reported only `Command timed out after N
seconds`.
Example, timeout 3s:
```bash
bash -c 'while true; do echo tick; sleep 0.05; done' & sleep 1; echo boom >&2; exit 1
```
The shell exits with code 1 after ~1s. The tool still times out at the
deadline, because the loop is still writing to its output. Both facts
are true: the command failed with code 1, and something it started was
still running at the timeout. The old message dropped the first, so the
model read a failed command as a slow one and retried with a longer
timeout, which never helps here.
## Fix
The timeout stays a timeout, since the tool did hit its deadline while
background processes were still writing. When the shell had already
exited before the deadline, the message now also says so:
```
Command timed out after 3 seconds. The shell exited with code 1 after 1.1 seconds, but background processes it started kept writing output until the timeout.
```
A shell still running at the deadline keeps the plain `Command timed out
after N seconds`.
The local backend throws a module-private `ShellTimeoutError` with the
message `timeout:<seconds>` as before, so custom `BashOperations` that
throw `timeout:N` are unaffected.
## Behavior per platform
- Linux and macOS: the shell runs in its own process group (`detached:
true`). At the deadline `kill(-pgid)` still reaches the background
processes after the shell exited, the pipes close, and the tool returns
at the deadline. Tested on Linux; macOS uses the same mechanism and
matches the macOS repros in the issue.
- Windows: still not fully solved. At the deadline `taskkill /T /PID
<shell>` fails with `process not found`, because the shell pid is gone.
Walking parent pids does not help: Git's `bin\bash.exe` is a launcher
for `usr\bin\bash.exe`, and the orphaned loop's parent is that
already-exited inner bash, not the pid pi holds. The tool keeps reading
until the output pauses for 100ms. Measured with Git Bash: the message
is now correct, but the tool returned after 14s instead of 3s; with a
loop that never pauses it never returns. This PR only fixes the
reporting.
## Same pattern elsewhere
- `packages/durable/src/env/node.ts` (`NodeExecutionEnv.exec`): same
code, same false `timeout:N` after the shell exited. Same Windows
limitation.
- `packages/coding-agent/src/core/exec.ts` (`pi.exec` for extensions):
the timeout sends SIGTERM/SIGKILL to the direct child only, no process
group. A descendant that keeps writing keeps the call waiting past the
timeout on every platform, and the result has `killed: true` even when
the child had already exited with its own code.
- `packages/coding-agent/src/core/package-manager.ts` (capture commands
for git/npm): waits for `close` and kills only the direct child on
timeout, so a descendant holding the pipes makes the timeout ineffective
on every platform, even when the descendant writes nothing.
- `packages/mcp/src/transports/stdio.ts`: on Windows, closing a server
that already exited skips taskkill, so children holding its pipes
survive. The code documents this as a known limitation.
Possible follow-up for Windows and the cases above: once the deadline
passed, stop waiting for the pipes (destroy our ends). On Windows this
also ends most writers, which die from a broken pipe (tested with Git
Bash).
## Tests
- `test/suite/regressions/11613-bash-timeout-after-shell-exit.test.ts`
(Unix only; it would hang on Windows for the reason above): background
writer after the shell exits with code 3, and a shell still running at
the deadline. The first case fails without the fix.
- Checked manually on Windows 11 with Git Bash through the built bundle:
both messages are as above.
- `npm run check`, `test/tools.test.ts`, the #5303 and #5208
regressions, `bash-close-hang-windows`, `powershell-tool`,
`shell-kill-process-tree`.
This PR description is AI-generated.
## Problem
`read` loads the whole file before it applies `offset`/`limit`:
`ops.readFile` -> `toString("utf-8")` -> `split("\n")`. A one-line read
still holds the whole file three times: as a Buffer, as a string and as
an array of lines.
Measured on main with a 200 MB log (2.36M lines):
- `offset: 1, limit: 1` peaks at 638 MB RSS. With
`--max-old-space-size=256` it crashes in `Runtime_StringSplit` (OOM).
- Files over about 512 MB fail with `Cannot create a string longer than
0x1fffffe8 characters`. Files over 2 GB fail with
`ERR_FS_FILE_TOO_LARGE`.
- Aborting only rejects the promise; the read keeps running in the
background.
## Solution
- `ReadOperations` gets a new optional `createReadStream(absolutePath, {
signal })`, which returns an `AsyncIterable<Uint8Array>` (a Node
`Readable` works). The default local operations open the file once,
`fstat` the handle, and stream from it. Like `fs.readFile`, they stop a
regular file at its size when opened: text appended during the read is
not counted, and a file that keeps growing still finishes. A file that
reports size 0 (such as `/proc` files) and other file types (FIFOs,
devices) are read until EOF, as `fs.readFile` reads them. No existing
member changed.
- The text path decodes the chunks with a streaming UTF-8 `TextDecoder`
(non-fatal, `ignoreBOM: true`). That matches `Buffer.toString("utf-8")`,
including invalid bytes and a UTF-8 byte order mark kept as U+FEFF.
- It counts every line but stores only the selected lines `truncateHead`
can show. Storing stops once the stored lines are past
`DEFAULT_MAX_LINES` or `DEFAULT_MAX_BYTES`, plus one more line, and the
existing `truncateHead` runs on the stored lines.
`totalLines`/`totalBytes` are then patched from the counts, as grep and
find do on main.
- The extra line is needed because `truncateHead` drops a trailing empty
line. Example: 2001 short lines, then an empty line, then more text. If
storing stopped at the empty line, the stored text would end in `\n`.
`truncateHead` would count 2000 lines and report "not truncated", but
the full text is truncated by lines.
- A line over 50KB is never stored whole; only its size is tracked. A
first line of 1 GB therefore still gives today's `[Line 1 is ...,
exceeds 50.0KB limit ...]` notice.
- Abort: the signal is passed to the stream, and the loop also checks it
between chunks. Leaving the loop closes the stream.
- The whole-file path stays unchanged for custom operations without
`createReadStream`, and for fractional `offset`/`limit` or negative
`limit`. Array slicing defines what those odd ranges select; for
example, `limit: -1` selects all lines but the last.
- Images are unchanged.
Output is the same as main's, byte for byte, including line counts,
notices, details (including key order) and errors. A file appended to
during the read gives the same output too: both read a regular file up
to its size when opened. Out of scope: the trailing-newline off-by-one
in line counts from #7329 is kept as is.
## Testing
- `test/read-streaming.test.ts` keeps a copy of the old text algorithm
as the reference. It compares the streaming path against it on text,
details and error messages, with chunks of 1 and 4096 bytes and offsets
and limits around the boundaries. The cases are: empty file, trailing
newline or not, CRLF, multi-byte characters and a UTF-8 BOM, 2001 lines,
and a first line over 50KB. There are also edge cases where the line or
byte cap falls exactly on an empty line, and the 50KB + trailing newline
case that `truncateHead` reports as `truncatedBy: "lines"`. Further
tests:
- details key order
- multi-byte characters split across chunks, and invalid UTF-8
- the `readFile` fallback, including fractional and negative ranges
- local files through the default stream, including EISDIR
- abort: the stream is closed and no more chunks are pulled
- a local file appended to mid-read (a decode hook in the test appends
after the first chunk): `limit: 1` still says `[1 more lines ...]`. This
test fails with an unbounded `fs.createReadStream`.
- bounded memory: a child process with `--max-old-space-size=64` reads a
200 MB single line, generated in memory as 64 KB chunks from a custom
`createReadStream`, and must print the size notice. It takes about 0.8 s
and uses 17 MB of heap. If over-long lines are stored whole, it runs out
of memory. 64 KB chunks matter here: V8 keeps decoded strings of 1 MB
outside the heap limit, so with 1 MB chunks the test would not catch
that regression.
- With main's `read.ts`, 5 of the 8 tests fail: matrix, key order,
chunked decoding, bounded memory and abort. The fallback, local-file and
append tests pass on main because they check behavior that did not
change. Removing the one-line-past-the-cap rule, or the
trailing-empty-line adjustment of `totalLines`, makes the matrix fail.
- `test/tools.test.ts`, `read-line-range-display`,
`tool-execution-component`, `block-images` and `image-resize-callers`
pass. `npm run check` is clean.
Measured with an ad-hoc script on a 200 MB log. Baseline RSS after
loading modules is about 121 MB.
| Case | main | this PR |
| --- | --- | --- |
| offset 1, limit 1 | 1028 ms, 638 MB peak RSS | 478 ms, 131 MB |
| offset 2000000, limit 10 | 926 ms, 637 MB | 486 ms, 132 MB |
| no offset/limit (counts all lines) | 1561 ms, 1010 MB | 524 ms, 132 MB
|
| offset 1, limit 1, 256 MB heap | OOM crash | 131 MB |
| 1.1 GB single-line file | string length error | 2.2 s, 130 MB,
line-size notice |
Remaining difference: custom operations that only implement `readFile`
keep the whole-file memory use, and their reads continue after an abort,
as before.
This PR is AI-generated.
createWorkers bundles the worker entry once (esbuild, Node builtins and installed
packages external) and forks every worker from that snapshot, so an edit of the
app's source while the server runs reaches workers only through a restart. The kit
watches the bundle's inputs as base code. p2 creates its session workers with
SessionWorkers.create and keeps the worker base's inputs as a dev restart baseline.
Opening the selector starts a background catalog refresh. When it
finished, the selection jumped back to the current model, or to the top
row with a search query, discarding any browsing done in the meantime.
Pass the selected model into filterModels() so the refresh keeps it
selected by provider and ID. If it is gone, the existing fallback
applies. The index is chosen once and the list is rebuilt once.
Based on #11758.
closes#9109
---------
Co-authored-by: bermudi <github.igizp@dabg.uk>
Several places truncate text by UTF-16 code units with `slice()`. When
the cut lands between the two halves of a surrogate pair (an emoji or
other character outside the Basic Multilingual Plane), the result keeps
a lone high or low surrogate. That is invalid UTF-16.
Example: a grep match line with 499 ASCII characters followed by `😀` is
cut at 500 units, so the output ends in `\uD83D... [truncated]`.
The bundled providers strip lone surrogates before sending
(`sanitizeSurrogates` in pi-ai), so the usual effect is a dropped
character. But the broken text still reaches the session file, RPC and
JSON clients (strict UTF-8 encoders such as Rust's serde_json reject
lone surrogates), extensions, and custom providers.
This adds `sliceHead(text, n)` and `sliceTail(text, n)` to
`core/tools/truncate.ts`. They behave like `slice(0, n)` and `slice(-n)`
but return one unit fewer when the cut would split a pair, and are used
where truncated text reaches the model:
- `truncateLine` (grep match lines), the case reported in #7121
- codemode `truncateOutput` head and tail, and its argument and error
previews
- compaction `truncateForSummary` (tool results sent to the summarizer)
- MCP server summaries in the system prompt and stderr tails in
connection errors
The cost is two `charCodeAt` checks per truncation, and nothing on the
untruncated path.
The surrogate check follows the fix in #7122 by @IKEASven69, which
@cristinaponcela reviewed in a follow-up branch. That PR was closed
before merging.
Not changed: TUI-only previews (renderers, visual truncation) and
line-based slicing. pi-private's pi-durable codemode `output.ts` has the
same head/tail slicing and needs a separate change there.
**Testing**
- New `test/truncate-surrogates.test.ts`: an emoji at the cut in
`truncateLine` is dropped whole, an emoji ending exactly at the limit is
kept, `sliceHead`/`sliceTail` never leave a lone surrogate for any
length, and they match `slice` on text without surrogates.
- `test/tools.test.ts`, compaction, MCP extension, and codemode tests
pass.
- `npm run check` passes.
fixes#7121
This PR is AI-generated.
Commits queued behind a running commit step share the next step's backend
transaction: each keeps its own step, seq, outbox commit and adoption in
queue order, the metadata is written once. A refusal rolls the group back
and its admitted commits run again alone; series, exports, replays and empty
commits end a group; backend failures while closing are latched; any throw
after COMMIT stops the store with memory at the durable group.
SqliteStorage.commit(batches) reads the check state of all batches up front,
checks each batch in memory against what the earlier ones left (same
prefix-stop refusals, no savepoints), and writes the accepted batches per
table: small rows as one JSON array parameter (json_each), rows with much
text bound column by column within 100 parameters. A group commit over a
remote SQLite takes a few round trips instead of about eight per batch;
local groups are faster too. Randomized differential test against
MemoryStorage, raw-table and statement-count guards.
Servers with auth.provider, such as Radius, were reported as needing sign-in because the shell command sent no token. pi mcp now reads and refreshes the provider's /login token, and its hints and login/logout point to /login <provider>.
An end-to-end reload suite (a kit app, a worker, a real Chromium page; one test per row of design's reload table).
The kit gains timeouts and base (base bundles' inputs and worker entries mark base). Fixes: a call crossing a failed
or a removal gets the cause, not not_ready; a call sent with an old method index is decoded by the names its consumer
had until the new bound is acked; a page retries a failed script fetch; applyChanges runs one at a time.
A hold unbinding with loading (a restart, a parked swap, a live failure retrying) keeps its calls in flight; each
gets one answer from what it waits on (the instance it runs on, or the gate). Calls sent by the old method names
before the consumer learns of the loading wait at the gate too. failed and removal still reject.
The server drops a page's own chord.* attributes and writes chord.host and the principal hook's result itself; tests
for the hook returning nothing, the NOOP adapter and calls over a link.
The four telemetry gaps (rows on commits, chord.host, microsecond times, a read-only store) and: pages record and
send span batches over their link to the server's ingest object (checked, capped, rate-limited, stamped browser and
with the link's principal); workers send theirs too, so the server is the store's only writer; principalAttributes
lets an app choose what a span records about its principal. The tutorial shows a real page's click traced to the
worker, with the trace viewer.
- type-check subagent-extension.test.ts against the extension view like the other subagent test
- update the test to current ToolRenderContext and ExtensionToolContext types
- use a decodable PNG in the image-only steering test; the fake data was omitted by image processing
The shipped subagent example could say a child was done while it was
still working, lose the useful part of process failures, and return a
normal-looking tool result when the delegated work had actually failed.
Single and chain output could also exceed the normal tool limit.
This also adds a dedicated regression suite covering progress state,
cancellation, process failures, output limits, cwd-based agent
selection, and persisted tool-result error state.
After this change:
- queued, running, completed, failed, and aborted work stay distinct
throughout the run;
- interrupting the parent run preserves partial child results, stops
queued work from starting, and escalates when a child ignores SIGTERM;
- signal exits, spawn failures, provider errors, stderr, and assistant
output remain visible instead of collapsing to `(no output)`;
- failed subagents become failed tool results, so the parent model can
react instead of treating them as successful calls;
- single and chain results stay within the normal 50 KB / 2,000-line
limit;
- project agents are selected and confirmed from each task's actual
working directory rather than the dispatcher's directory.
## Further potential improvements
These are useful follow-ups, but they are separate product and storage
decisions rather than part of the failure-handling fixes in this PR.
<details>
<summary>Potential follow-ups</summary>
### Keep persisted transcripts bounded
Subagent tool results persist the full child message history in
`details`, which can make parent session files large and slow to reload.
A follow-up could compact that history or spill it to disk.
### Define an aggregate parallel-output budget
Parallel mode still gives every task its own 50 KB budget. A follow-up
could keep every task represented while enforcing one overall
tool-result limit.
### Report child usage to the calling session
Per-child token and cost usage is visible in result cards but is not
added to the calling session's totals.
### Isolate malformed agent definitions
One malformed YAML agent can still break discovery for every valid agent
in the same directory.
### Limit subagent recursion
A nesting limit would prevent accidental process and paid-request
multiplication when subagents can invoke the subagent tool again.
* https://github.com/earendil-works/pi/issues/8195
### Add a configurable timeout
Unattended runs would benefit from a generous opt-in timeout for
children that never complete.
</details>
## Related issues
### Spawn failures lose their diagnostics
This directly addresses the reported case where a nonexistent child
working directory produces `(no output)` instead of the actual spawn
error. The regression suite includes that same reproduction.
* https://github.com/earendil-works/pi/issues/8129
### Persisted child history can bloat sessions
This PR intentionally keeps full child history in tool-result details,
so session compaction or spill-to-disk remains follow-up work rather
than something fixed here.
* https://github.com/earendil-works/pi/issues/7452
use llama.cpp's default port and remove the stale `--jinja` note
same change as #11616 by @pcuenca with the original commit preserved
disclaimer: my julien-cto agent helped me write this
Co-authored-by: Pedro Cuenca <pedro@huggingface.co>
Chord's telemetry schema; spans for calls, methods, relays, holds, activations, natives, loads, reloads, resolves,
revocations, routines, jobs and HTTP requests; trace carriers on hold and call (bounded, printable ASCII) continued by
relays and workers; events and host-wide metrics; the app kit passes an adapter to its hosts. renderTrace escapes
control characters. Example 62 prints a trace across three hosts.
