## Problem
The edit tool's intra-line renderer can drop indentation from a removed
line when text is inserted before otherwise unchanged content.
For example, this input:
```diff
-83 await commitRecordingChunk({
+83 const result = await commitRecordingChunk({
```
was rendered as:
```diff
-83 await commitRecordingChunk({
+83 const result = await commitRecordingChunk({
```
`diffWords()` groups the shared leading spaces into the added segment
(`" const result = "`). The existing highlighting logic moved those
spaces outside inverse styling, but only appended them to the added
line.
## Solution
Extract whitespace shared by both lines before calling `diffWords()`,
then initialize both rendered lines with that prefix. This preserves
indentation without changing the existing word-level highlighting
behavior. The existing handling for whitespace that belongs to only one
side remains unchanged.
Added regression coverage for:
- insertion before indented content;
- replacement within indented content;
- keeping shared indentation outside inverse highlighting.
## Validation
- `npm run check`
- `./test.sh`
- targeted `packages/coding-agent/test/diff.test.ts` (3 tests)
Closes#10164.
A user `!` command can discard earlier chunks but report `truncated:
false` when the retained tail fits within the output limits. The model
then receives no truncation notice or full-log path.
Track discarded chunks and include that flag in the executor's result
for both completion and cancellation. This applies to the current shared
finalization path and preserves streaming ANSI handling. Output limits
and message format stay unchanged.
Related: #2852. Here the complete log is already saved; the incorrect
flag hides its path from the model.
Validation on main `f1b2e77f5`:
- Two deterministic 64 KiB + 44 KiB cases fail without the fix. All
three regressions pass with it, checking completion, cancellation, the
complete log, model-facing path, and short-output behavior.
- Regressions are in `test/bash-executor-truncation.test.ts`; that file
and `test/tools.test.ts` pass all 88 tests.
- `npm run check`, offline build, and `./test.sh` pass.
With this PR, we stop rendering every option.
Instead, we render a centered, clamped window of up to 10 options and
show `(selection/total)` when the list overflows.
This keeps the selected option visible in long extension-provided lists
without changing navigation, keybindings, callbacks, countdowns, or
short-list rendering.
Fixes#6688.
## Merge Danger
**Door:** Two-way
The change is localized to `ExtensionSelectorComponent.updateList()` and
can be reverted without migration or persisted-state concerns.
**Blast Radius:** Low
Only extension selectors with more than 10 options change appearance;
they now render a window and position indicator instead of every option.
---------
Co-authored-by: Pi via openai-codex (gpt-5.6-sol) <noreply@openai-codex>
`JsonRpcLineReader.readStdout` (used by the stdio transport and by
`PipeTransport`) appended every stdout chunk to the buffered bytes with
`Buffer.concat`, then searched the whole buffer from the start for a
newline. A large message arriving in many chunks was therefore copied
and scanned again on every chunk. A 12 MiB tool result in 64 KiB chunks
took about 350 ms just to frame.
Now the reader keeps the bytes of an incomplete message as a list of
chunks, with a running byte count, and searches only the new chunk for a
newline. When the newline arrives, the chunks are joined once. A line
that fits inside a single chunk is sliced directly. The rest is
unchanged: the `maxMessageBytes` limit (now checked against the running
count), CRLF and blank-line handling, errors for invalid JSON and
oversized messages, and the incomplete-message error in `end()`.
Testing:
- New `packages/mcp/test/lines.test.ts`:
- a 1 MiB message fed in 4 KiB chunks calls `Buffer.concat` once (fails
on the old code);
- a mixed input of multi-byte UTF-8, CRLF, blank lines, an invalid line
and several messages gives the same results at every chunk size from 1
byte to the whole input;
- reading recovers after an oversized message;
- a message without its newline is reported at the end.
- `test/lines.test.ts`, `test/stdio.test.ts` and `test/pipe.test.ts`
pass. `npm run check` passes.
- Micro-benchmark, one message fed in 64 KiB chunks:
- before: 1 MiB 5.4 ms, 4 MiB 49.9 ms, 12 MiB 348 ms;
- after: 2.2 ms, 7.9 ms, 21.6 ms.
Fixes#11761
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
`@earendil-works/pi-coding-agent` pins `minimatch@10.2.6`.
GHSA-g3hv-75wg-m249 (moderate, a ReDoS via globparts amplification)
covers every minimatch version below 11.0.1, and there is no patched
10.x release. As a result, `npm audit` fails in every project that
installs pi-coding-agent.
This change pins `minimatch` to `11.0.3` and regenerates
`package-lock.json` and `packages/coding-agent/install-lock/` with
`scripts/generate-coding-agent-install-lock.mjs`. minimatch 11 adds one
new transitive dependency, `@isaacs/nonrecursive-glob-match@1.0.3`,
which has no install scripts. It also moves brace-expansion to ^5.0.12.
pi uses minimatch in only two places, `model-resolver.ts` and
`package-manager.ts`. Both use the named `minimatch` export, which 11.x
keeps unchanged.
No new test was added: the change only bumps a dependency, and existing
tests cover both places that use it.
Testing:
- `npm run build` and `npm run check` pass, and the install-lock check
reports it is up to date.
- `test/model-resolver.test.ts`, `test/package-manager.test.ts` and
`test/package-manager-ssh.test.ts` pass (213 tests).
- Before: installing 1.4.0 and running `npm audit` reports `minimatch
<11.0.1 … GHSA-g3hv-75wg-m249`.
- After: `npm audit --package-lock-only` in
`packages/coding-agent/install-lock` reports `found 0 vulnerabilities`.
- At the monorepo root, `npm audit` still flags a `minimatch@3.1.5` that
comes in through the `shx` dev dependency. pi does not ship that copy,
and this change does not touch it.
Fixes#11646
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
`pi config` ignored `~/.pi/agent/keybindings.json`. `selectConfig()`
never called `setKeybindings()`, so the config selector's
`getKeybindings()` fell back to the default TUI bindings. Remaps such as
`"tui.select.down": ["down", "ctrl+n"]` did nothing there, although they
work in `/settings` and the `--resume` picker.
`selectConfig()` now installs
`KeybindingsManager.create(options.agentDir)` before it builds the UI,
the same way the session picker and startup UI do.
**Testing**
- Added `test/config-selector-keybindings.test.ts`. It runs
`selectConfig()` with a stubbed terminal and component and an agent dir
whose `keybindings.json` remaps `tui.select.up` and `tui.select.down`,
then checks that the active keybindings include `ctrl+p` and `ctrl+n`.
It fails without the change and passes with it.
- Manual check: ran `pi config` in tmux with that `keybindings.json`.
Before the change, Ctrl+N did nothing. After it, Ctrl+N moves the
selection like Down.
- `npm run check` reports only two errors, both missing modules in
`packages/schema/test/bench/candidates.ts`. They occur on `main` without
this change.
Fixes#11651
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
`FileAuthStorageBackend` locked `auth.json` with proper-lockfile and
`realpath: false`, so the lock directory was `<path as given>.lock`.
Reads and writes follow symlinks to the same real file, but each path
alias got its own lock. For example, an agent dir whose `auth.json` is a
symlink to a shared file used a different lock than the real file. Two
pi processes could then enter the credential-modification section at the
same time and refresh the same OAuth token concurrently.
This change drops `realpath: false` from the sync and async lock calls,
so proper-lockfile resolves the path and every alias of the file uses
one lock. Resolving the path needs the file to exist, which is already
the case: `ensureFileExists()` runs before either lock is taken.
The same `realpath: false` pattern is also in `settings-manager.ts`,
`trust-manager.ts`, and `package-manager-cli.ts`. This change leaves
those alone.
## Testing
- I added a test to `test/auth-storage.test.ts`. It holds the lock on
the real `auth.json`, then checks that a backend opened through a
symlink alias waits in `withLockAsync` until aborted, and gets `ELOCKED`
from `withLock`. Neither path runs the mutation. Without the fix the
test fails because the async call enters the lock; with the fix it
passes.
- `node ../../node_modules/vitest/dist/cli.js --run test/auth-storage
test/settings`: 151 tests pass.
- `npm run check` passes.
- With the reproduction script from the issue, current main prints
`symlink alias: ENTERED`; with this change it prints `symlink alias:
BLOCKED`. The same-path control prints `BLOCKED` in both cases.
Fixes#11655
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Co-authored-by: Mario Zechner <contact@badlogicgames.com>
`qwen-token-plan-individual` filters the models.dev `alibaba-token-plan`
catalog through a fixed list of model IDs. That list still stopped at
`glm-5.2`. models.dev now has `glm-5.3`, and `qwen-token-plan` and
`qwen-token-plan-cn` already list it. Individual did not, because
`glm-5.3` was not on its list.
This change adds `glm-5.3` to the Individual list and updates the
"verified" date in the comment above it.
Tests in `packages/ai/test/qwen-token-plan-models.test.ts`:
- `glm-5.3` is added to the expected list for the exact-list test.
- A new test checks that Individual's `glm-5.3` uses the international
Token Plan base URL. It also checks the thinking levels models.dev gives
for it: low, high and max.
How I tested it:
- `npm run generate-models`, then `test/qwen-token-plan-models.test.ts`:
all 90 tests pass. Without the change to the list, 3 tests fail.
- `npm run check` passes.
- Before: `pi --list-models qwen-token-plan-individual` listed `glm-5.2`
but not `glm-5.3`. After: it lists both.
Fixes#11658
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
When a request fails before the provider answers (connection refused or
reset, DNS, TLS or certificate, proxy), the `openai` and Anthropic SDKs
throw an error with the generic message `"Connection error."`. The
actual reason is in the `cause` chain, usually two levels down: undici's
`TypeError: fetch failed` wraps `connect ECONNREFUSED 127.0.0.1:8080`.
pi built `errorMessage` from `error.message` only, so users, print mode,
and session files saw only "Connection error." and had nothing to go on.
This change adds `withCauseDetail()` in `utils/error-body.ts`:
- It walks the `cause` chain, up to 5 levels, and appends the deepest
message, plus its `code` if the message doesn't already contain it. For
example: `Connection error.: connect ECONNREFUSED 127.0.0.1:8080`.
- `normalizeProviderError` applies it only when the error has no HTTP
status and no body. That covers every adapter that goes through
`formatProviderError`.
- The Anthropic adapter's catch block also uses it, since it read
`error.message` directly.
Errors without a cause come out unchanged. Retry classification still
matches, because the message still contains "Connection error".
**Testing**
- Added tests to `packages/ai/test/error-body.test.ts`:
- a synthetic nested cause chain;
- adding the code when the message lacks it;
- no cause leaves the message unchanged;
- a real `openai` SDK call against a closed local port, which now gives
`Connection error.: connect ECONNREFUSED 127.0.0.1:<port>`.
- Without the source change, the nested-chain test and the real SDK test
fail, because the message is just `Connection error.`.
- The full `packages/ai` test suite passes, and `npm run check` passes.
Fixes#11695
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Images in idle prompts go through `processImage`: they are resized to
the model's limits, and a coordinate hint is added to the text. Images
in messages queued while the agent was busy skipped this step. That
covers `steer()`, `followUp()`, `prompt()` with `streamingBehavior`, and
RPC `steer`/`follow_up`. Those images were sent and saved at full size.
For example, a 1200x9000 PNG stayed 1200x9000, which is over Anthropic's
8000 px limit, and because it is saved in the session it was replayed on
every later request.
What the change does:
- `_queueSteer`/`_queueFollowUp` now mark queued user messages that
carry images.
- When such a message is delivered (`message_start`), `AgentSession`
normalizes its images in place with `_normalizePromptImages`, the same
code the idle prompt path uses, and appends the resize hints to its
text. This runs before extensions and listeners see the message, before
it is saved, and before it is added to the request context.
- Doing it at delivery keeps enqueueing synchronous. A slow resize
therefore cannot leave a message queued after the run has ended. I hit
exactly that when I first tried normalizing at enqueue time. Delivery is
also when the resize uses the model current at that moment.
- The queue display still matches and removes the message, because
matching runs before the text is changed.
Testing:
- New `test/agent-session-queued-images.test.ts`: a 1200x9000 PNG is
queued while the agent is streaming, through `steer()`, `followUp()` and
`prompt(..., { streamingBehavior: "followUp" })`. The test checks three
things: each saved message has a resized image and the "original
1200x9000, displayed at 267x2000" hint; no request sent to the model
contains the original image; and the queue is empty afterwards. It fails
without the change and passes with it.
- `npm run check` passes. The `agent-session*` tests, the queue- and
image-related tests and `test/suite` pass.
- Before the change, the saved steer and follow-up messages had
`image/png 1200x9000` while the idle prompt had a resized JPEG. After
it, all three are resized.
Fixes#11738
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Azure's OpenAI models are copied from the direct `openai` provider. That
provider caps the GPT-6 family at 272K so requests stay in OpenAI's
short-context pricing tier.
The Azure provider overrides this cap through
`CONTEXT_WINDOW_OVERRIDES`, but that list only covered gpt-5.4, gpt-5.5
and gpt-5.6. As a result, `azure/gpt-6-astra`, `gpt-6-luna`,
`gpt-6-sol`, `gpt-6.1-sol` and `gpt-6.1-terra` showed a 272K context
window, and pi compacted sessions far earlier than needed. Azure Foundry
deploys these models with a 1,050,000-token context window.
This change adds the five models to the Azure override list.
OpenAI-direct is unchanged.
Testing:
- Added a test in `packages/ai/test/supports-xhigh.test.ts` that checks
each of these Azure models has a 1,050,000 context window. It fails on
all five models without the override (5 failed) and passes with it.
- After regenerating the model data, the generated Azure catalog lists
all five models at 1050000. Before the change they were at 272000.
- `npm run check` passes.
Fixes#11742
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
On Windows and WSL, `app.clipboard.pasteImage` was bound only to
`alt+v`. Most Windows terminals handle Ctrl+V themselves and send a
bracketed paste. Some terminals have no Ctrl+V paste action and pass
Ctrl+V to the program as the raw `0x16` key instead; Tern's embedded
terminal is one. In those terminals nothing in pi was bound to `ctrl+v`,
so pressing it did nothing and showed no message.
This change makes the Windows/WSL default `["alt+v", "ctrl+v"]`. `alt+v`
keeps working as before. In terminals that handle Ctrl+V themselves, pi
never receives the key, so the extra binding has no effect there. No
other default binding uses `ctrl+v`. `docs/keybindings.md` is updated to
match.
Testing:
- Added a test to `test/keybindings.test.ts`. It sets `WSL_DISTRO_NAME`,
re-imports the keybindings module, and checks that the paste defaults
are `["alt+v", "ctrl+v"]`. It fails without the change and passes with
it.
- Updated the existing defaults assertion in the same file to match the
new value.
- `node ../../node_modules/vitest/dist/cli.js --run
test/keybindings.test.ts`: 6 passed.
- `npm run check`: passes.
Fixes#11765
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
When `/login llama.cpp` could not connect to the server URL, pi showed
only `Error: Failed to save API key for llama.cpp: fetch failed`. That
happens on a refused connection, a wrong port, or `https://` against a
plain-HTTP server. Node's `fetch` keeps the real reason in
`error.cause`, and pi dropped it, so the user had nothing to go on.
`LlamaClient` now sends its requests, including the `/models/sse` watch,
through a small `fetchServer` helper. When `fetch` fails at the network
level, the helper throws `Could not reach llama.cpp server at <url>:
<cause>` and keeps the original error as `cause`. Aborts and timeouts
are rethrown unchanged. HTTP error responses are handled as before.
The same message also improves `/llama` and the model catalog refresh,
because they use the same client.
**Testing**
- Added a test in `test/llama-extension.test.ts` that calls
`LlamaClient.list()` against a closed local port and expects the server
URL and `connect ECONNREFUSED ...` in the error. It fails without the
change (`got 'fetch failed'`) and passes with it. The whole file passes
(16 tests).
- `npm run check` passes.
- Manual check, `/login llama.cpp` with `http://127.0.0.1:8099` (nothing
listening):
- Before: `Error: Failed to save API key for llama.cpp: fetch failed`
- After: `Error: Failed to save API key for llama.cpp: Could not reach
llama.cpp server at http://127.0.0.1:8099: connect ECONNREFUSED
127.0.0.1:8099`
Fixes#11745
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Hyperlink detection recognised Alacritty only through
`TERM_PROGRAM=alacritty`, and Alacritty never sets `TERM_PROGRAM`. So in
a default Alacritty setup, pi fell back to `hyperlinks: false` and
rendered Markdown links as `label (url)` instead of clickable OSC 8
links.
Alacritty does set two other variables, which this change now checks:
- `TERM=alacritty` when its terminfo entry is installed (otherwise
`TERM=xterm-256color`);
- `ALACRITTY_WINDOW_ID` for the shell on Unix and macOS. This covers the
common macOS case where the terminfo entry is missing.
The tmux and screen checks still run first, so behaviour inside
multiplexers is unchanged.
**Testing**
- Added a `detectCapabilities` test covering `TERM=alacritty` and
`TERM=xterm-256color` with `ALACRITTY_WINDOW_ID`. It fails without the
change and passes with it.
- `node --test test/*.test.ts` in `packages/tui`: 1121 passing tests.
- `npm run check` passes.
- Checked `detectCapabilities()` with clean environments:
- Before: `TERM=alacritty` and `ALACRITTY_WINDOW_ID=123` both gave
`hyperlinks: false`.
- After: both give `hyperlinks: true`.
- Plain `xterm-256color` and tmux with `ALACRITTY_WINDOW_ID` still give
`false`.
Fixes#11767
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
`consumeSseStream()` split SSE lines only on `\n` and then stripped one
trailing `\r`. That handled LF and CRLF. The SSE spec also allows a lone
CR as a line ending, and a stream framed that way never got split. At
EOF the whole stream was parsed as a single `id` line, so no event was
dispatched, and a tool call answered this way failed with "stream ended
without a response".
With this change, a line ends at CR, LF, or CRLF. A CR at the end of a
chunk ends the line right away, so CR-terminated events are delivered
while the stream is still open. A flag skips the LF that may start the
next chunk, so a CRLF split across chunks still counts as one line
ending. Size limits, dispatch, and error cleanup are unchanged.
Testing:
- Added two tests to `packages/mcp/test/streamable-http.test.ts`:
- LF, CRLF, and CR framing, each sent as one chunk and one character per
chunk (which splits every CRLF), plus mixed endings.
- A CR-terminated event is dispatched before the stream closes.
- Both tests fail without the fix and pass with it. All 127 tests in
`packages/mcp` pass.
- Feeding `id: r`, `data: {...}`, blank, blank joined by `\r` used to
produce no events. It now produces the same event as the LF and CRLF
versions.
- `npm run check` fails here, but only in `packages/ai` and
`packages/chord`, which this change does not touch. It reports nothing
in `packages/mcp`; biome and `tsc` are clean on the changed package.
Fixes#11764
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
A call arriving while an attempt activates waits for its outcome (bounded by readyTimeout) instead of failing
not_ready while its client sees live. Fates start in the reverse of the order objects went live, so a pure
dependent of a swapping provider swaps instead of parking.
Three limit problems in the find and ls tools.
**1. `limit: 0` and other values below 1.** find passed `limit` to fd
unchanged. fd treats `--max-results 0` as no limit, so `find({ pattern:
"*", limit: 0 })` returned every match, followed by `[0 results limit
reached. Use limit=0 for more, or refine pattern]`. ls with `limit: 0`
listed nothing and printed `(empty directory)`. A fractional limit such
as `1.5` printed `1.5 results limit reached. Use limit=3`.
Fix: both tools use `Math.max(1, Math.floor(limit ?? DEFAULT_LIMIT))`.
grep already clamps to at least 1. The fd path and the custom `glob`
path share this value.
**2. Notice at exactly N results (#7121).** With 3 matching files and
`limit: 3`, fd returned 3 results. find checked `results >= limit` and
printed `[3 results limit reached. Use limit=6 for more, or refine
pattern]`, even though there were no more files.
Fix: find asks fd for `limit + 1` results, shows the first `limit`, and
prints the notice only when the extra result exists. ls already worked
this way. Custom `FindOperations.glob` now also gets `limit + 1`. The
tool shows at most `limit` of the returned results and prints the notice
only when more were returned. The interface is unchanged. This follows
closed PR #7122 without its unrelated changes.
**3. Results past the 50KB cap.** find kept every fd line in memory.
`truncateHead` then dropped everything past 50KB. find now handles each
line as it arrives. It keeps lines up to the first one past the cap and
only counts the rest, as grep does in #11705. Because that first line
past the cap is kept, `truncateHead` makes the same truncation decision
as before. `details.truncation.totalLines` and `totalBytes` come from
the counts, so the notices and `details` are unchanged.
Behavior differences:
- A custom `glob` that ignores `options.limit` and returns more results
used to have all of them shown. Now the tool shows the first `limit`
results with the limit notice.
- Custom `glob` implementations get `limit + 1` instead of `limit`.
- In the fd path, output made only of blank lines with exit code 0 now
gives `No files found matching pattern` instead of empty text. fd does
not produce such output.
**Testing**
- New tests in `test/tools.test.ts`:
- find: exactly N matches gives no notice, and N+1 matches give the
notice with N lines (#7121). Fails on main.
- find: `limit` 0, -5, 0.5 and 1.9 each show 1 result with the `1
results` notice. Fails on main.
- find with custom `glob`: the operation gets `limit + 1`, and exact and
over-limit results give the right output. Fails on main.
- find past the byte cap: 300 names of 200 bytes each.
`details.truncation` equals `truncateHead` of the full output, except
for `content`. Passes on main too; it guards the counting. It fails if
the `totalLines` override is removed.
- ls: `limit` 0, -5, 0.5 and 1.9 each list 1 entry with the notice.
Fails on main.
- ls: exactly N entries gives no notice. Passes on main too; it guards
the existing behavior.
- `test/tools.test.ts` and the find regression suites 3302, 3303 and
6104 pass (110 tests). `npm run check` passes.
fixes#7121
This PR is AI-generated.
When `context` is set, the grep tool read every matched file whole and
split it into lines, only to print the lines around each match. Memory
grew with the size of the matched files, not the output. In the issue's
reproduction (8 log files of about 95 MB, one match each, `context: 2`,
`--max-old-space-size=1024`), the process ran out of heap after about 10
s.
ripgrep already supports context. This change passes `--context N` to rg
and formats the `context` events from its JSON output. The tool no
longer opens matched files.
Details:
- After the last allowed match (`limit`), rg still sends that match's
trailing context. The tool keeps those lines, then stops rg. A match
inside that window is shown as a context line, as before.
- Lines past the 50 KB output cap are counted but not stored, since
truncation drops them anyway. Match counts, the match-limit notice, and
`details.matchLimitReached` stay as before.
- rg merges overlapping context, so lines shared by nearby matches are
printed once instead of repeated per match. This is the one intended
output change.
- Context lines are decoded by rg, so UTF-16 files with a BOM show
correctly with context (#11423, closed as no-action). Non-UTF-8 lines
and paths, which rg sends base64-encoded, are decoded too.
Two related grep fixes in the same code:
- **Exact match limit:** with exactly N matches and `limit: N`, grep
said "N matches limit reached. Use limit=2N". It now sets the notice and
`details.matchLimitReached` only after rg sends one more match. Cost:
when no further match exists, rg finishes the search instead of being
stopped at the Nth match, the same as a search with fewer than N
matches.
- **Lone CR:** grep removed every `\r` in a line, so `one\rtwo needle`
showed as `onetwo needle` and text copied from grep failed to match in
edit. It now strips only the line terminator (`\n` or `\r\n`), like the
read tool. The TUI still hides CRs when rendering. This updates the
expectation in the #11398 test from `beforeprogress` to
`before\rprogress`.
Breaking: `GrepOperations.readFile` is removed because nothing uses it
anymore. rg always ran locally, so this hook never made grep work on a
remote backend. Docs now say to override `execute()` for remote search,
as the Gondolin example does. Code that passes `readFile` still runs,
and the value is ignored. Only code that type-checks an object literal
against `GrepOperations` fails to compile.
Known remaining differences and limits:
- Memory no longer grows with file size or match count, but a single
huge matching line (tens of MB) is still buffered whole before being cut
to 500 characters. That was already true for `context: 0` on main.
Bounding it would mean discarding oversized rg JSON records, which is a
separate change.
- In a directory search, rg stops reading a file once it finds a NUL
byte, so context past that point in a text-looking file that turns
binary later is omitted. main reread the file and printed it. Explicit
file paths are unaffected.
- `details.truncation.totalLines`/`totalBytes` count formatted rows.
With deduplicated context, or file names containing newlines, they can
differ from main. No renderer or notice displays them.
**Testing**
- Issue reproduction: before, `FATAL ERROR ... JavaScript heap out of
memory`. After, correct output in about 60 ms with 135 MB RSS.
- `pattern: "line", context: 3, limit: 100000` over the same 770 MB of
logs reports both limit notices and peaks at 154 MB RSS.
- New tests in `test/tools.test.ts`:
- Merged overlapping context, plus trailing context at the limit
including a match inside it. Fails on main.
- UTF-16 with context. Fails on main.
- Truncation reported when stored output ends exactly at 50 KB. Guards
the counting logic.
- Matches still counted past the byte cap. Passes on main too; it guards
against stopping rg early at the cap.
- Huge `context` value accepted.
- No limit notice with exactly `limit` matches (with and without
context), notice with more. Fails before this change.
- Lone CR kept and CRLF stripped, with and without context. Fails before
this change.
- All 94 tests in the file pass. `npm run check` passes.
fixes#9276
This PR is AI-generated.
A schema whose root `$id` was a relative path with a directory part,
such as `schemas/tool.json`, could not resolve its local `$ref`s. The
`Resolver` constructor applied the root `$id` to `DEFAULT_BASE` once to
compute `rootBase`. The compiler, the error explainer and the coercer
start from `rootBase` and call `baseFor(root, rootBase)`, which applied
the `$id` a second time. Local references therefore resolved against
`https://json-schema.invalid/schemas/schemas/tool.json`.
The effect was that `compile(schema)` threw `Unresolved $ref
#/$defs/count`. With `unresolvedRefs: "reject"`, which is how tool
arguments are validated (pi-ai's `validateToolArguments`, used by the
agent loop and durable, and durable's `structuredOutput` check), valid
arguments were rejected and the tool never ran. Absolute `$id`s and bare
names like `tool.json` were not affected, because resolving them twice
gives the same URI.
The fix:
- `rootBase` is now `DEFAULT_BASE`; walkers apply the root's own `$id`
through `baseFor`.
- `baseFor` indexes the schema before applying an `$id` and then returns
the indexed base, so a caller whose base already includes the `$id`
cannot apply it again. Without this,
`unevaluatedProperties`/`unevaluatedItems` still failed: `build()`
passes the already-based node to `annotating()`, which applied the `$id`
once more.
- The coercer resolves bases through `baseFor` instead of reading
`baseOf` directly. Otherwise the `rootBase` change broke the first
`parse`/`coerce` of a schema with `$dynamicRef`: the restarted dynamic
compiler is not indexed yet, so a relative `$ref` resolved against
`DEFAULT_BASE` and conversion was silently skipped.
- The explainer drops `$id` from the copy it makes for unevaluated
keywords, and resolves `anyOf`/`oneOf` branch `$ref`s against the
branch's own base (both threw in `errors()` on main for nested resources
with a relative `$id`).
Testing:
- Regression tests in `validator.test.ts` (relative root `$id` with
local `$ref`, with `unevaluatedProperties`/`unevaluatedItems`, error
reporting under nested relative resources) and `coerce.test.ts` (first
coercion of a `$dynamicRef` schema with a relative external `$ref`).
They fail without the change and pass with it.
- Full pi-schema suite: 8712 passed.
- Independently reviewed against edge cases: draft 7 (inert `$id` beside
`$ref`, `#anchor` ids), fragment ids, nested relative chains, anchors,
registered schemas referring back to the root, fresh
`parse`/`coerce`/`errors` after a dynamic restart.
Fixes#11697
Requested by @badlogic
---------
Co-authored-by: triagent <triagent@localhost>
Co-authored-by: Petr Houška <houskape@gmail.com>
Problem: `pi.registerMarkdownTransformer()` only runs on user, assistant
and thinking Markdown. Pi renders other Markdown that extensions cannot
reach: custom messages, skill blocks, branch and compaction summaries,
and the changelog notices (What's New, `/changelog`, Extension Updates,
the Update Available note). For example, a transformer cannot rewrite
text in an extension's changelog shown under "Extension Updates",
because that call site passes no transform.
Running every transformer everywhere would break existing ones. The
documented example only skips thinking blocks and replaces `-->` with
`→`; on a changelog it would turn `<!-- note -->` into `<!-- note →`.
Solution: an optional `{ messageTypes }` lists the surfaces a
transformer runs on. Without options it runs on the same three types as
before.
| `messageType` | Surfaces |
|---|---|
| `user`, `assistant`, `assistant-thinking` | transcript messages
(default) |
| `custom` (new) | custom messages without a message renderer; expanded
skill blocks |
| `summary` (new) | expanded branch and compaction summaries |
| `changelog` (new) | What's New, `/changelog`, Extension Updates, the
Update Available note |
```typescript
pi.registerMarkdownTransformer((markdown) => markdown.replaceAll("⚠️", "⚠"), {
messageTypes: ["changelog"],
});
```
Details:
- Filtering happens in one place,
`ExtensionRunner.getMarkdownTransformers()`: each transformer is wrapped
so Markdown of unlisted types passes through unchanged. Components still
take a plain list of transformers.
- Skill blocks and summaries pass only their content to transformers; pi
adds its bold header after the transform.
- Custom, skill and summary messages now get `messageId` and `timestamp`
when loaded from the session. Changelog notices have no session entry,
so both are `undefined`.
- The built-in Mermaid transformer renders in user and assistant text
and now also in summaries, which are model-written like assistant text.
It still skips thinking, custom messages and changelogs.
- `/hotkeys` does not run transformers.
- New exports: `MarkdownMessageType`, `MarkdownTransformerOptions`,
`MarkdownTransformerRegistration`.
Tests: runner type filtering (default vs. opted in), custom message,
skill and summary components (type, header excluded, `messageId`),
Mermaid on the new types, and startup notices running `changelog`
transformers on What's New and Extension Updates while default
transformers are skipped.
Related: #11710 (HTML comments in Extension Updates). #11755 already
fixed it in the renderer; this PR lets extensions transform those
notices in general.
The setting was saved but never read: thinking always started hidden.
It now sets the view whenever its effective value changes (unset = hidden);
the toggle overrides it until then and survives reloads.
Fixes#11710.
## Problem
The tui `Markdown` component printed `<!-- ... -->` comments verbatim.
marked lexes them as `html` tokens and both the block and inline `html`
cases pushed `token.raw` as text, so comments leaked into every Markdown
surface (Extension Updates, What's New, `/changelog`, messages).
```
## [1.0.0]
<!-- release-notes:start -->
Fixed a bug.
See <!-- todo: link --> the docs.
```
rendered both comments.
## Fix
- After lexing (next to `trimPartialClosingFences`, before tokens are
cached), remove block `html` tokens that contain only comments,
recursing into lists and blockquotes. If the removed comment sat between
two blank lines, or at the start/end, one blank line is dropped too, so
no extra gap remains.
- Block HTML with other content keeps rendering as text, with comments
stripped (`<!-- x --> text` renders `text`).
- Inline comments are skipped; if text before ended with a space, the
next text's leading space is dropped (`See <!-- c --> the docs.` renders
`See the docs.`).
- An unclosed block comment runs to the end of the block (CommonMark),
so streamed comments stay hidden.
- Other HTML still renders as text (`with <tag> markers`).
Parsing stays in marked; the change only filters tokens marked already
produced. Cost: one pass over top-level tokens per lex (cached), a regex
on `html` tokens only.
## Tests
`packages/tui/test/markdown.test.ts`, "HTML comments": compares
rendering with comments against the same Markdown without them (issue
repro, blank-line handling, multi-line/unclosed, blockquotes and lists,
other HTML kept).
This PR is AI-generated.
Follow-up to #11746 for the case @fcatuhe reported on #11599: a second
shutdown trigger cut off cleanup.
Problem: Paseo stops pi by closing stdin and sending SIGTERM in the same
tick (SIGKILL 2 s later); systemd sends SIGTERM and stdin closes after.
In RPC mode:
- stdin EOF, then SIGTERM: `shutdown()` removed pi's SIGTERM handler
when it started, so the signal killed pi (exit 143) mid-`dispose()`.
- SIGTERM, then stdin EOF: the second `shutdown()` call ran
`process.exit()`.
A pending MCP token refresh was lost: the server had rotated the refresh
token, the next pi sent the old one, and the server revoked the grant.
Rule after this change: once a shutdown has started, a stdin EOF or a
single signal does not interrupt it; a second signal exits at once.
SIGKILL still always works.
- **RPC mode** (`rpc-mode.ts`): signal handlers stay registered until
exit; `shutdown()` runs once and later calls do nothing; the signal
handler counts signals and exits only on a repeated one.
- **Print mode** (`print-mode.ts`): the final `finally` removed the
signal handlers before `disposeRuntime()`, so a signal during the final
disposal killed pi. Handlers are now removed after it,
`disposeRuntime()` returns the shared in-flight disposal (a signal then
waits for it), and a repeated signal exits at once, as before.
- **Interactive mode**: unchanged. It already lets a started shutdown
finish.
Docs (`docs/rpc.md`) and the CHANGELOG are updated.
## Testing
- `test/rpc-prompt-response-semantics.test.ts`: EOF then SIGTERM,
SIGTERM then EOF (both wait for a slow dispose, exit once with 0 and
143), and EOF then two SIGTERMs (exits at once with 143). All three fail
on main.
- `test/print-mode.test.ts`: SIGTERM during the final disposal waits for
it, a second SIGTERM exits with 143, handlers are removed afterwards.
Fails on main.
- Real process, `pi --mode rpc` with a `session_shutdown` handler that
takes 1 s and real signals:
- EOF then SIGTERM: cleanup done, exit 0 (main: no cleanup, exit 143)
- SIGTERM then EOF: cleanup done, exit 143
- EOF then two SIGTERMs: exit 143 at once
- `npm run check` passes; all `test/rpc*` and `test/print-mode` tests
pass.
With `PI_STARTUP_BENCHMARK=1`, pi initializes interactive mode, stops
the TUI, prints the timings and returns. It never disposed the session
runtime, so extensions never got `session_shutdown`. If an extension
cleans up a timer or a socket in that handler, the resource stays open
and the process never exits after printing the timings.
This change calls `await runtime.dispose()` right after
`interactiveMode.stop()` in the benchmark path. That is the same order a
normal interactive quit uses: stop the TUI, then dispose the runtime.
No test was added. Nothing in the test suite covers the startup
benchmark path, and that path needs a real TTY.
Testing:
- `npm run check` passes.
- I wrote an extension that starts a `setInterval` in `session_start`
and clears it in `session_shutdown`. Then I ran `PI_STARTUP_BENCHMARK=1
PI_TIMING=1 pi --offline -ne -e ./ext.ts` in a terminal, wrapped in
`timeout --foreground 15`:
- before the change: the timings print, then the process hangs until the
timeout (exit 124);
- after the change: it exits 0.
- Without extensions it exits 0 both before and after.
Fixes#11693
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Fixes#11703.
**Problem:** `getNewEntries()` parsed the saved version with
`split(".")`. `0.1.5-20261020.7` became `["0", "1", "5-20261020", "7"]`,
`Number("5-20261020")` is `NaN`, and `|| 0` turned it into `0.1.0`. The
`[0.1.1]` to `[0.1.5]` notes counted as new, were shown, and the same
prerelease string was saved again, so they repeated on every fresh
launch.
**Fix:** read only the leading `major.minor.patch` of the saved version,
the same way changelog headings are parsed. `0.1.5-20261020.7` and
`0.1.5+build.3` now count as `0.1.5`.
If the saved version has no `major.minor.patch` (for example `latest`),
nothing is shown and the installed version is saved in its place, like
on a fresh install. Before, it was read as `0.0.0` and the whole
changelog was shown. Saving the installed version matters because
callers only save when there are new entries, so an unreadable value
would otherwise silence the changelog for good.
Alternatives considered (noted in a code comment):
- plain string comparison: orders `0.10.0` before `0.9.0`
- `localeCompare` with `numeric: true`: works only because `0.1.5-x`
happens to sort after `0.1.5`, and depends on locale collation
- strict semver: orders `0.1.5-x` before `0.1.5`, so the `[0.1.5]` notes
would still repeat forever
Tests added in `test/changelog.test.ts` and
`test/interactive-mode-extension-changelog.test.ts`. `npm run check`
passes.
#11749 was opened before 1.5.0 was released and merged after. Its two
changelog entries stayed in the `### Fixed` section that became `##
[1.5.0]`, so the changelog now says 1.5.0 contains fixes it does not.
This moves the two entries (#11335 JSON `maxMessageBytes`, error-body
excerpt) to `## [Unreleased]` → `### Fixed`. No other changes. Comparing
every `packages/*/CHANGELOG.md` against the `v1.5.0` tag shows no other
entries misplaced.
The Streamable HTTP transport checked `maxMessageBytes` on SSE replies
but not on `application/json` replies. The JSON branch called
`response.json()`, which buffered and parsed the whole body with no size
check. A server could send an oversized message as plain JSON, and the
same content sent as SSE would have been rejected.
A new helper, `readBodyText(response, maxBytes)`, reads a body through a
reader and decodes each chunk as it arrives. Once the body goes over
`maxBytes`, it cancels the rest and returns only the first `maxBytes`
bytes, marked as truncated. Counting during the read also covers chunked
replies that have no `Content-Length`. `TextDecoder` strips a leading
BOM, as `response.json()` does.
Two places use it:
- JSON replies: if the body is truncated, `send()` throws `MCP JSON
response exceeds N bytes` (default 16 MiB). Bodies within the limit are
parsed as before.
- HTTP error bodies (`checkResponse`): before, `response.text()` read
the whole error body and then cut it to 8 KiB. A huge or endless error
body was fully buffered, or never finished. Now only the 8 KiB excerpt
is read and the rest is cancelled. The excerpt is decoded from the kept
bytes only, so it does not keep the whole body alive (#9265).
Testing (`packages/mcp/test/streamable-http.test.ts`):
- `applies maxMessageBytes to JSON responses`: with a 512-byte limit, a
small reply is accepted and a 2 KB reply is rejected. Fails without the
fix.
- `decodes JSON responses split inside a BOM and a multi-byte
character`: the body arrives one byte per chunk.
- `reads only an excerpt of HTTP error bodies and cancels the rest`: an
endless 500 body yields an 8 KiB excerpt and is cancelled. Without the
fix it never finishes.
- All MCP tests pass; `npm run check` passes.
Fixes#11335
Requested by @petrroll
---------
Co-authored-by: triagent <triagent@localhost>
Co-authored-by: Petr Houška <houskape@gmail.com>
In RPC mode, `shutdownHandler` only set a `shutdownRequested` flag. That
flag was checked in two places: after a stdin command was handled and on
`agent_settled`. If an extension called `ctx.shutdown()` while the
session was idle (for example from a timer), neither check ever ran. The
process stayed alive until the client sent another command. Interactive
mode already shuts down right away in this case.
Changes in `packages/coding-agent/src/modes/rpc/rpc-mode.ts`:
- **One check for all paths:** `checkShutdownRequested()` shuts down
only when a request is pending, input is attached, no command is being
handled, and the session is idle. Each of those ending calls it again:
command completion, prompt settle, `agent_settled`, and input attach.
- **Idle requests:** the shutdown handler schedules the check on the
next tick, so the requesting handler finishes first. The conditions are
checked when the tick runs, not when it is scheduled: a stdin command
that arrives in between is handled first, and shutdown follows when it
finishes.
- **Requests during startup:** a request made before input is attached
(for example a synchronous `ctx.shutdown()` in `session_start`) is
checked once the input reader is attached.
- **Requests during a command:** shutdown waits until the command
finishes, including extension commands. A `prompt` command returns
before `session.prompt()` settles, so it stays counted as in flight
until the prompt settles, and the shutdown check runs then. This also
fixes an existing ordering problem: an extension command that called
`ctx.shutdown()` could exit the process before its `prompt` response was
written.
- **Repeated requests:** `checkShutdownRequested()` does nothing once
shutdown has started, so a second `ctx.shutdown()` does not call
`process.exit` in the middle of async cleanup. A second stdin EOF or
signal still forces exit, as before.
Shutdown requested during a running turn is still handled on
`agent_settled`, as before. A request made while an extension-started
compaction runs (not idle, no run to settle) still waits for the next
command or run; `docs/rpc.md` and `docs/extensions.md` note this.
Docs (`docs/rpc.md`, `docs/extensions.md`) and the CHANGELOG are
updated.
## Testing
- Added three tests to `test/rpc-prompt-response-semantics.test.ts`:
- an async `ctx.shutdown()` on an idle session exits;
- an extension command that calls `ctx.shutdown()` gets its `prompt`
response written before exit;
- a command that arrives after an idle request but before the scheduled
check runs gets its response written before exit.
All three fail without the change and pass with it.
- `npm run check` passes. All `test/rpc*` tests pass.
- In `./test.sh`, all coding-agent tests pass. The only failures were
timeouts in durable/harness tests (`store-storage`, `tools-images`,
connection close), which this change does not touch.
- Reproduction from the issue: run `pi --mode rpc --no-session -ne -e
./shutdown-probe.mjs` with stdin held open and nothing sent.
- Before the change, the process stayed alive until a timeout killed it
(exit 124).
- After the change, `session_shutdown` fires and the process exits with
code 0.
- A synchronous `ctx.shutdown()` in `session_start` also exits with code
0.
Fixes#11599
Requested by @petrroll
---------
Co-authored-by: triagent <triagent@localhost>
Co-authored-by: Petr Houška <houskape@gmail.com>
defineMessages with typed t, t.message/t.render for messages as data, the Translations registry with fallbacks and
conflicts, a locale preference per principal with the device as default and a live switch, useMessages and
scope.messages, formatTime/formatNumber. The tutorial gains a chapter on languages.
self.query(table, q, limit) runs before the object goes live and again in the commit step of every commit that
touched the table, publishing only the diff; a reload swaps the object. The tutorial and 59-chat use it.
MCP tools are deferred and callable only by other tools, so code mode scripts call them and neither the model nor tool_search sees them. While code mode is off or its plugin is not loaded, tool_search loads them instead, as pi falls back. The mcp_servers prompt section, as pi's, names the servers and how their tools are reached.
The heartbeat no longer counts time the process did not run (a laptop's sleep, a blocked event loop) as silence: after such a pause each open link pings at once and closes only if its peer stays silent for close - ping.
A running worker's lost link is now a stop instead of a failure: worker_link_lost is reported and the held route starts a new worker. A second lost link within a minute of the last recovery still fails the address.
A worker killed by a signal left its lock's socket file, and every later start of its address failed closed on it, retry too. The workers component now removes a lock socket file nobody accepts on before each start and after a child's exit by a signal, in the address's queue. processLock also treats ENOTSOCK as stale instead of retrying until its context ended.
A second Ctrl+C or SIGHUP killed the server without its exit hooks, leaving the macOS lock socket file behind; the next start then refused with a stale lock. A close that does not finish within 8 s now exits too.
A manager on the p2 server owns every MCP connection and process. Session
workers hold one view each and lease the locations their conversations
work at; the manager runs the owner's mcp.json servers there (stdio per
session and location, or shared; HTTP shared by default) and offers their
tools as deferred Durable tools found through tool_search.
Stdio servers on an environment start through its pi-env daemon, one
connection per environment shared by its instances, through cmd.exe on
Windows. A project's .pi/mcp.json, read by the worker at each location,
may add stdio servers that run there and switch the owner's servers on or
off; the manager refuses everything else. A lease gets servers only on
environments its session works on.
pi-schema became a coding-agent runtime dependency, but the Nix package
neither packed it nor overrode its install-lock entry, which has no
integrity hash, so evaluation failed with "attribute 'integrity' missing".
p2 (Pi Mobile on Chord v2) moves from pi-private into pi as one commit. Root scripts that loop all workspaces
use --if-present; npm test, ./test.sh and the release script cover packages/* only (p2: cd apps/p2 && npm test);
sync-versions also updates apps' workspace pins.
When an SSE event went over `maxMessageBytes`, `consumeSseStream()`
threw the size-limit error, but its `finally` only released the reader
lock. Releasing the lock does not cancel the response body. If the
server kept the response open, for example with heartbeats, the failed
request held its HTTP connection until the server ended it or the whole
client was closed. The same applies to the GET notification stream,
which uses the same function.
On error, `consumeSseStream()` now cancels the reader before it rethrows
and releases the lock. A failure in `cancel()` is ignored, so the caller
still gets the original error.
Testing:
- New test in `packages/mcp/test/streamable-http.test.ts`: it feeds an
oversized event through a stream that stays open and checks that the
stream's `cancel` callback runs. It fails without the change and passes
with it.
- Full mcp package tests: 48/48 pass. `npm run check` is clean.
- Reproduction over loopback: a local `node:http` server sends a
512-byte event, then a heartbeat every 20 ms, and the client reads it
through `fetch` with `maxEventBytes: 128`, three times.
- Before: none of the three server responses closed, and the server made
28 more heartbeat writes.
- After: all three closed, with no further writes.
Fixes#11729
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Tokens from a Client ID Metadata Document sign-in through an https
callback route were refreshed with the loopback redirect URI, which the
documents do not list, so every expiry needed a browser sign-in.
createMcpAuthProvider takes the app's redirectUrl and startMcpSignIn
passes its own.
startMcpSignIn's wait for the refresh lock now ends when the signal
aborts. Only McpRefreshLockBusyError (coding-agent maps proper-lockfile's
ELOCKED to it) falls back to the browser step; other lock errors fail
the sign-in.
createMcpToolName, mcpNamespace, normalizeMcpInputSchema,
limitMcpContent (with a caller-provided saver) and validation of the
standard mcpServers shape move from coding-agent into pi-mcp, so other
applications name and limit MCP tools exactly like pi. Applications
validate their own fields on the tagged result; coding-agent keeps
exposure, auth.provider and its Client ID Metadata Document rule.
pi-mcp now has startMcpSignIn and completeMcpSignIn, so a server app can
start a sign-in in one request and complete it in another (a callback
route, a pasted redirect URL, or a loopback server via
listenForLoopbackSignIn/waitForLoopbackSignIn), plus createMcpAuthProvider.
Application specifics (client name, Client ID Metadata Document base URL
and listed redirect URIs) are options. coding-agent's signInMcpServer is a
thin wrapper over them.
authorizeMcp gains refreshOnly: a failed refresh no longer registers a
client or replaces a pending sign-in's PKCE verifier and state. Sign-in
refreshes with the stored client before dropping it for another redirect
URI.
When a program started by the process operation exits on its own, the
daemon now ends its process group (Unix) or its own job object (Windows)
before reaping it, so a wrapper like npx does not leave its server
running. The pid leaves the tracked groups before it is reaped, so a
racing close cannot signal a reused pid.
StdioTransport and PipeTransport share the newline-delimited framing and
the stderr tail (transports/lines.ts). PipeTransport accepts pi-env's
RemoteProcess as its McpPipe, so p2 can run stdio MCP servers on
execution environments through the daemon's process operation.
