From 4cf69332c3355c0023982d90650830a66617be60 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 09:36:54 +0100
Subject: [PATCH 01/88] fix(coding-agent): resize images in queued steering and
follow-up messages (#11782)
Images in idle prompts go through `processImage`: they are resized to
the model's limits, and a coordinate hint is added to the text. Images
in messages queued while the agent was busy skipped this step. That
covers `steer()`, `followUp()`, `prompt()` with `streamingBehavior`, and
RPC `steer`/`follow_up`. Those images were sent and saved at full size.
For example, a 1200x9000 PNG stayed 1200x9000, which is over Anthropic's
8000 px limit, and because it is saved in the session it was replayed on
every later request.
What the change does:
- `_queueSteer`/`_queueFollowUp` now mark queued user messages that
carry images.
- When such a message is delivered (`message_start`), `AgentSession`
normalizes its images in place with `_normalizePromptImages`, the same
code the idle prompt path uses, and appends the resize hints to its
text. This runs before extensions and listeners see the message, before
it is saved, and before it is added to the request context.
- Doing it at delivery keeps enqueueing synchronous. A slow resize
therefore cannot leave a message queued after the run has ended. I hit
exactly that when I first tried normalizing at enqueue time. Delivery is
also when the resize uses the model current at that moment.
- The queue display still matches and removes the message, because
matching runs before the text is changed.
Testing:
- New `test/agent-session-queued-images.test.ts`: a 1200x9000 PNG is
queued while the agent is streaming, through `steer()`, `followUp()` and
`prompt(..., { streamingBehavior: "followUp" })`. The test checks three
things: each saved message has a resized image and the "original
1200x9000, displayed at 267x2000" hint; no request sent to the model
contains the original image; and the queue is empty afterwards. It fails
without the change and passes with it.
- `npm run check` passes. The `agent-session*` tests, the queue- and
image-related tests and `test/suite` pass.
- Before the change, the saved steer and follow-up messages had
`image/png 1200x9000` while the idle prompt had a resized JPEG. After
it, all three are resized.
Fixes #11738
Requested by @badlogic
Co-authored-by: triagent
---
packages/coding-agent/CHANGELOG.md | 1 +
.../coding-agent/src/core/agent-session.ts | 36 +++++--
.../test/agent-session-queued-images.test.ts | 94 +++++++++++++++++++
3 files changed, 125 insertions(+), 6 deletions(-)
create mode 100644 packages/coding-agent/test/agent-session-queued-images.test.ts
diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md
index f9ca79449..39aaadcf0 100644
--- a/packages/coding-agent/CHANGELOG.md
+++ b/packages/coding-agent/CHANGELOG.md
@@ -14,6 +14,7 @@
### Fixed
+- Images in steering and follow-up messages, including `prompt()` while streaming, are now resized like images in idle prompts instead of being sent and saved at full size ([#11738](https://github.com/earendil-works/pi/issues/11738))
- A failed token refresh of an MCP connection no longer registers an OAuth client or replaces the PKCE verifier of a sign-in in progress, which made that sign-in fail. MCP sign-in now refreshes the stored tokens with the stored client even when the loopback callback port changed, instead of registering a new client and opening the browser. Ctrl+C in `/mcp` and the `--timeout` of `pi mcp login` now also stop a sign-in while it waits up to 25 seconds for another pi process's token refresh, and a refresh lock that fails for another reason than that wait now fails the sign-in instead of opening the browser.
- Fixed RPC mode ignoring an extension's `ctx.shutdown()` while idle, for example from a timer: the process kept running until the client sent another command. pi now shuts down right away when idle, and a shutdown requested by an extension command waits until the command's `prompt` response is written ([#11599](https://github.com/earendil-works/pi/issues/11599))
- Fixed changelog notes repeating on every fresh launch when the installed package version has a prerelease suffix, such as `0.1.5-20261020.7`: the saved version was read as `0.1.0`, so the `0.1.1` to `0.1.5` notes counted as new each time. A saved version without `major.minor.patch` now shows no notes and is replaced by the installed version, instead of showing the whole changelog ([#11703](https://github.com/earendil-works/pi/issues/11703))
diff --git a/packages/coding-agent/src/core/agent-session.ts b/packages/coding-agent/src/core/agent-session.ts
index dbe36c365..5d341c4d7 100644
--- a/packages/coding-agent/src/core/agent-session.ts
+++ b/packages/coding-agent/src/core/agent-session.ts
@@ -43,6 +43,7 @@ import type {
TextContent,
ToolResultMessage,
Usage,
+ UserMessage,
} from "@earendil-works/pi-ai/compat";
import {
clampThinkingLevel,
@@ -408,6 +409,8 @@ export class AgentSession {
private _steeringMessages: string[] = [];
/** Tracks pending follow-up messages for UI display. Removed when delivered. */
private _followUpMessages: string[] = [];
+ /** Queued user messages whose images are normalized when they are delivered. */
+ private _queuedMessagesWithImages = new WeakSet();
/** Messages queued to be included with the next user prompt as context ("asides"). */
private _pendingNextTurnMessages: CustomMessage[] = [];
/** Context-only custom messages queued during a run, flushed once the current turn's tool results are in. */
@@ -1153,6 +1156,12 @@ export class AgentSession {
}
}
}
+ // Normalize queued images at delivery, before events and persistence see the message,
+ // so they get the same resize as idle prompts with the current model's limits (#11738).
+ if (this._queuedMessagesWithImages.has(event.message)) {
+ this._queuedMessagesWithImages.delete(event.message);
+ await this._normalizeQueuedMessageImages(event.message);
+ }
}
// Emit to extensions first, then notify public listeners.
@@ -2327,11 +2336,9 @@ export class AgentSession {
if (images) {
content.push(...images);
}
- this.agent.steer({
- role: "user",
- content,
- timestamp: Date.now(),
- });
+ const message: AgentMessage = { role: "user", content, timestamp: Date.now() };
+ if (images?.length) this._queuedMessagesWithImages.add(message);
+ this.agent.steer(message);
}
/**
@@ -2344,7 +2351,24 @@ export class AgentSession {
if (images) {
content.push(...images);
}
- this.agent.followUp({ role: "user", content, timestamp: Date.now() });
+ const message: AgentMessage = { role: "user", content, timestamp: Date.now() };
+ if (images?.length) this._queuedMessagesWithImages.add(message);
+ this.agent.followUp(message);
+ }
+
+ /** Replace a queued user message's images in place with normalized ones, appending hints to its text. */
+ private async _normalizeQueuedMessageImages(message: UserMessage): Promise {
+ if (typeof message.content === "string") return;
+ const [first, ...rest] = message.content;
+ if (first?.type !== "text") return;
+ const images = rest.filter((block): block is ImageContent => block.type === "image");
+ const normalized = await this._normalizePromptImages(images);
+ const text = normalized.hints.length > 0 ? `${first.text}\n\n${normalized.hints.join("\n")}` : first.text;
+ message.content = [
+ { type: "text", text },
+ ...rest.filter((block) => block.type !== "image"),
+ ...normalized.images,
+ ];
}
/**
diff --git a/packages/coding-agent/test/agent-session-queued-images.test.ts b/packages/coding-agent/test/agent-session-queued-images.test.ts
new file mode 100644
index 000000000..800445bf3
--- /dev/null
+++ b/packages/coding-agent/test/agent-session-queued-images.test.ts
@@ -0,0 +1,94 @@
+import { deflateSync } from "node:zlib";
+import type { ImageContent, TextContent } from "@earendil-works/pi-ai";
+import { afterEach, describe, expect, it } from "vitest";
+import { createHarness, type Harness } from "./test-harness.ts";
+
+function crc32(data: Buffer): number {
+ let crc = 0xffffffff;
+ for (const byte of data) {
+ crc ^= byte;
+ for (let i = 0; i < 8; i++) crc = (crc >>> 1) ^ (0xedb88320 & -(crc & 1));
+ }
+ return (crc ^ 0xffffffff) >>> 0;
+}
+
+function pngChunk(type: string, data: Buffer): Buffer {
+ const length = Buffer.alloc(4);
+ length.writeUInt32BE(data.length);
+ const body = Buffer.concat([Buffer.from(type, "ascii"), data]);
+ const crc = Buffer.alloc(4);
+ crc.writeUInt32BE(crc32(body));
+ return Buffer.concat([length, body, crc]);
+}
+
+/** A solid gray RGB PNG of the given size, base64 encoded. */
+function createPng(width: number, height: number): string {
+ const header = Buffer.alloc(13);
+ header.writeUInt32BE(width, 0);
+ header.writeUInt32BE(height, 4);
+ header[8] = 8;
+ header[9] = 2;
+ const row = Buffer.alloc(1 + width * 3, 0x80);
+ row[0] = 0;
+ const raw = Buffer.concat(Array.from({ length: height }, () => row));
+ return Buffer.concat([
+ Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
+ pngChunk("IHDR", header),
+ pngChunk("IDAT", deflateSync(raw)),
+ pngChunk("IEND", Buffer.alloc(0)),
+ ]).toString("base64");
+}
+
+function persistedUserContent(harness: Harness, text: string): (TextContent | ImageContent)[] {
+ for (const entry of harness.sessionManager.getEntries()) {
+ if (entry.type !== "message" || entry.message.role !== "user") continue;
+ const content = entry.message.content;
+ if (typeof content === "string") continue;
+ if (content[0]?.type === "text" && content[0].text.split("\n")[0] === text) return content;
+ }
+ throw new Error(`No user message "${text}"`);
+}
+
+async function waitFor(condition: () => boolean): Promise {
+ for (let i = 0; i < 200 && !condition(); i++) await new Promise((resolve) => setTimeout(resolve, 25));
+}
+
+describe("AgentSession queued message images", () => {
+ let harness: Harness | undefined;
+
+ afterEach(() => {
+ harness?.cleanup();
+ harness = undefined;
+ });
+
+ // Regression test for #11738: queued messages skipped the image resize idle prompts get.
+ it("resizes images in steering, follow-up, and streaming prompt messages", async () => {
+ harness = await createHarness({ responses: [{ text: "first", delayMs: 300 }, "second", "third", "fourth"] });
+ const image: ImageContent = { type: "image", data: createPng(1200, 9000), mimeType: "image/png" };
+
+ const run = harness.session.prompt("idle");
+ await waitFor(() => harness!.session.isStreaming);
+ await harness.session.steer("steer", [image]);
+ await harness.session.followUp("follow up", [image]);
+ await harness.session.prompt("streaming prompt", { images: [image], streamingBehavior: "followUp" });
+ await run;
+ await waitFor(() => !harness!.session.isStreaming && harness!.faux.callCount >= 4);
+
+ for (const text of ["steer", "follow up", "streaming prompt"]) {
+ const content = persistedUserContent(harness, text);
+ const images = content.filter((block): block is ImageContent => block.type === "image");
+ expect(images).toHaveLength(1);
+ expect(images[0].data).not.toBe(image.data);
+ expect((content[0] as TextContent).text).toContain("original 1200x9000, displayed at 267x2000");
+ }
+ const sentImages = harness.faux.contexts
+ .flatMap((context) => context.messages)
+ .flatMap((message): readonly { type: string }[] =>
+ typeof message.content === "string" ? [] : message.content,
+ )
+ .filter((block): block is ImageContent => block.type === "image");
+ expect(sentImages.length).toBeGreaterThan(0);
+ expect(sentImages.some((block) => block.data === image.data)).toBe(false);
+ expect(harness.session.pendingMessageCount).toBe(0);
+ });
+});
From 4c992154d91a85f837b73374a86b8e73eefd0745 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 09:47:23 +0100
Subject: [PATCH 02/88] fix(ai): keep the underlying cause in connection error
messages (#11783)
When a request fails before the provider answers (connection refused or
reset, DNS, TLS or certificate, proxy), the `openai` and Anthropic SDKs
throw an error with the generic message `"Connection error."`. The
actual reason is in the `cause` chain, usually two levels down: undici's
`TypeError: fetch failed` wraps `connect ECONNREFUSED 127.0.0.1:8080`.
pi built `errorMessage` from `error.message` only, so users, print mode,
and session files saw only "Connection error." and had nothing to go on.
This change adds `withCauseDetail()` in `utils/error-body.ts`:
- It walks the `cause` chain, up to 5 levels, and appends the deepest
message, plus its `code` if the message doesn't already contain it. For
example: `Connection error.: connect ECONNREFUSED 127.0.0.1:8080`.
- `normalizeProviderError` applies it only when the error has no HTTP
status and no body. That covers every adapter that goes through
`formatProviderError`.
- The Anthropic adapter's catch block also uses it, since it read
`error.message` directly.
Errors without a cause come out unchanged. Retry classification still
matches, because the message still contains "Connection error".
**Testing**
- Added tests to `packages/ai/test/error-body.test.ts`:
- a synthetic nested cause chain;
- adding the code when the message lacks it;
- no cause leaves the message unchanged;
- a real `openai` SDK call against a closed local port, which now gives
`Connection error.: connect ECONNREFUSED 127.0.0.1:`.
- Without the source change, the nested-chain test and the real SDK test
fail, because the message is just `Connection error.`.
- The full `packages/ai` test suite passes, and `npm run check` passes.
Fixes #11695
Requested by @badlogic
Co-authored-by: triagent
---
packages/ai/src/api/anthropic-messages.ts | 3 +-
packages/ai/src/utils/error-body.ts | 29 ++++++++++++++-
packages/ai/test/error-body.test.ts | 43 +++++++++++++++++++++++
3 files changed, 73 insertions(+), 2 deletions(-)
diff --git a/packages/ai/src/api/anthropic-messages.ts b/packages/ai/src/api/anthropic-messages.ts
index ba25b59a3..a19a5d215 100644
--- a/packages/ai/src/api/anthropic-messages.ts
+++ b/packages/ai/src/api/anthropic-messages.ts
@@ -37,6 +37,7 @@ import type {
ToolResultMessage,
} from "../types.ts";
import { appendAssistantMessageDiagnostic } from "../utils/diagnostics.ts";
+import { withCauseDetail } from "../utils/error-body.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { shortHash } from "../utils/hash.ts";
import { headersToRecord } from "../utils/headers.ts";
@@ -1163,7 +1164,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
stream.end();
} catch (error) {
output.stopReason = options?.signal?.aborted ? "aborted" : "error";
- output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
+ output.errorMessage = error instanceof Error ? withCauseDetail(error) : JSON.stringify(error);
stream.push({ type: "error", reason: output.stopReason, error: output });
stream.end();
}
diff --git a/packages/ai/src/utils/error-body.ts b/packages/ai/src/utils/error-body.ts
index d35f51a0e..f94a029de 100644
--- a/packages/ai/src/utils/error-body.ts
+++ b/packages/ai/src/utils/error-body.ts
@@ -46,15 +46,42 @@ export function normalizeProviderError(error: unknown): NormalizedProviderError
const status = extractStatus(sdkError);
const body = extractBody(sdkError);
const messageCarriesBody = body === undefined || error.message.includes(body);
+ const message = status === undefined && body === undefined ? withCauseDetail(error) : error.message;
return {
status,
body,
- message: error.message,
+ message,
messageCarriesBody,
} satisfies NormalizedProviderError;
}
+const MAX_CAUSE_DEPTH = 5;
+
+/**
+ * Connection-level failures carry no status or body, and SDKs replace the
+ * reason with a generic message (`openai`: `"Connection error."`). The real
+ * reason sits in the `cause` chain, often two levels deep (undici's
+ * `"fetch failed"` wraps `connect ECONNREFUSED ...`). Append the deepest
+ * cause so callers that surface only the message keep the reason.
+ */
+export function withCauseDetail(error: Error): string {
+ let deepest: Error | undefined;
+ let current: unknown = error.cause;
+ for (let depth = 0; depth < MAX_CAUSE_DEPTH && current instanceof Error; depth++) {
+ deepest = current;
+ current = current.cause;
+ }
+ if (!deepest) return error.message;
+ let detail = (deepest.message || deepest.name).trim();
+ const code = (deepest as Error & { code?: unknown }).code;
+ if ((typeof code === "string" || typeof code === "number") && !detail.includes(String(code))) {
+ detail = detail ? `${detail} (${code})` : String(code);
+ }
+ if (!detail || error.message.includes(detail)) return error.message;
+ return error.message ? `${error.message}: ${detail}` : detail;
+}
+
/**
* Probe the HTTP status, first numeric hit wins, in SDK-field order:
* `statusCode` (Mistral) → `status` (`openai`, `@google/genai`) →
diff --git a/packages/ai/test/error-body.test.ts b/packages/ai/test/error-body.test.ts
index c2824e8be..b6e236770 100644
--- a/packages/ai/test/error-body.test.ts
+++ b/packages/ai/test/error-body.test.ts
@@ -5,13 +5,17 @@
// Bedrock ServiceException), plus the non-Error fallback, truncation, the empty
// parsed-body edge case, and the formatProviderError compose helper.
+import type { AddressInfo } from "node:net";
+import { createServer } from "node:net";
import { getHeapStatistics } from "node:v8";
+import OpenAI from "openai";
import { describe, expect, it } from "vitest";
import {
formatProviderError,
MAX_PROVIDER_ERROR_BODY_CHARS,
normalizeProviderError,
truncateErrorText,
+ withCauseDetail,
} from "../src/utils/error-body.ts";
import { collectGarbage } from "./memory-helpers.ts";
@@ -248,3 +252,42 @@ describe("truncateErrorText", () => {
expect(getHeapStatistics().used_heap_size - before).toBeLessThan(1_000_000);
});
});
+
+describe("connection error cause detail", () => {
+ it("appends the deepest cause of a connection error with no status or body", () => {
+ const socketError = Object.assign(new Error("connect ECONNREFUSED 127.0.0.1:8080"), { code: "ECONNREFUSED" });
+ const fetchError = new TypeError("fetch failed", { cause: socketError });
+ const error = new Error("Connection error.", { cause: fetchError });
+
+ expect(formatProviderError(normalizeProviderError(error))).toBe(
+ "Connection error.: connect ECONNREFUSED 127.0.0.1:8080",
+ );
+ });
+
+ it("adds the error code when the cause message lacks it", () => {
+ const cause = Object.assign(new Error("other side closed"), { code: "UND_ERR_SOCKET" });
+ const error = new Error("Connection error.", { cause });
+
+ expect(withCauseDetail(error)).toBe("Connection error.: other side closed (UND_ERR_SOCKET)");
+ });
+
+ it("leaves errors without a cause unchanged", () => {
+ expect(withCauseDetail(new Error("Connection error."))).toBe("Connection error.");
+ });
+
+ it("surfaces the reason from a real openai SDK connection failure", async () => {
+ const server = createServer();
+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
+ const port = (server.address() as AddressInfo).port;
+ await new Promise((resolve) => server.close(() => resolve()));
+
+ const client = new OpenAI({ apiKey: "test", baseURL: `http://127.0.0.1:${port}/v1`, maxRetries: 0 });
+ const error = await client.chat.completions
+ .create({ model: "m", messages: [{ role: "user", content: "hi" }] })
+ .catch((e: unknown) => e);
+
+ expect(formatProviderError(normalizeProviderError(error))).toBe(
+ `Connection error.: connect ECONNREFUSED 127.0.0.1:${port}`,
+ );
+ });
+});
From d53980fe11d910b1f77f577aa59077f04672ced7 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 09:54:48 +0100
Subject: [PATCH 03/88] fix(ai): add glm-5.3 to qwen-token-plan-individual
models (#11785)
`qwen-token-plan-individual` filters the models.dev `alibaba-token-plan`
catalog through a fixed list of model IDs. That list still stopped at
`glm-5.2`. models.dev now has `glm-5.3`, and `qwen-token-plan` and
`qwen-token-plan-cn` already list it. Individual did not, because
`glm-5.3` was not on its list.
This change adds `glm-5.3` to the Individual list and updates the
"verified" date in the comment above it.
Tests in `packages/ai/test/qwen-token-plan-models.test.ts`:
- `glm-5.3` is added to the expected list for the exact-list test.
- A new test checks that Individual's `glm-5.3` uses the international
Token Plan base URL. It also checks the thinking levels models.dev gives
for it: low, high and max.
How I tested it:
- `npm run generate-models`, then `test/qwen-token-plan-models.test.ts`:
all 90 tests pass. Without the change to the list, 3 tests fail.
- `npm run check` passes.
- Before: `pi --list-models qwen-token-plan-individual` listed `glm-5.2`
but not `glm-5.3`. After: it lists both.
Fixes #11658
Requested by @badlogic
Co-authored-by: triagent
---
.../providers/qwen-token-plan-individual.ts | 3 ++-
packages/ai/test/qwen-token-plan-models.test.ts | 17 +++++++++++++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/packages/ai/catalog/providers/qwen-token-plan-individual.ts b/packages/ai/catalog/providers/qwen-token-plan-individual.ts
index c172748d1..fa61c172a 100644
--- a/packages/ai/catalog/providers/qwen-token-plan-individual.ts
+++ b/packages/ai/catalog/providers/qwen-token-plan-individual.ts
@@ -2,7 +2,7 @@ import { assertExactModelIds } from "../../scripts/model-data.ts";
import { defineProvider } from "../define.ts";
import { INTERNATIONAL_BASE_URL, QWEN_COMPLETIONS_COMPAT, qwenTokenPlanModels } from "./qwen-token-plan.ts";
-// QwenCloud Token Plan Individual text-model allowlist, verified 2026-09-03.
+// QwenCloud Token Plan Individual text-model allowlist, verified 2026-11-01.
// It shares the international endpoint with a narrower catalog.
// https://docs.qwencloud.com/token-plan/personal/token-plan-personal-overview
const MODEL_IDS = new Set([
@@ -10,6 +10,7 @@ const MODEL_IDS = new Set([
"deepseek-v4-pro",
"deepseek-v4-pro-0813",
"glm-5.2",
+ "glm-5.3",
"qwen3.6-flash",
"qwen3.7-max",
"qwen3.7-plus",
diff --git a/packages/ai/test/qwen-token-plan-models.test.ts b/packages/ai/test/qwen-token-plan-models.test.ts
index a3d463684..eb5256d9a 100644
--- a/packages/ai/test/qwen-token-plan-models.test.ts
+++ b/packages/ai/test/qwen-token-plan-models.test.ts
@@ -63,6 +63,7 @@ const INDIVIDUAL_TEXT_MODELS = [
"deepseek-v4-pro",
"deepseek-v4-pro-0813",
"glm-5.2",
+ "glm-5.3",
"qwen3.6-flash",
"qwen3.7-max",
"qwen3.7-plus",
@@ -199,6 +200,22 @@ describe("Qwen Token Plan models", () => {
},
);
+ it("exposes glm-5.3 reasoning_effort levels for qwen-token-plan-individual", () => {
+ const model = getModels("qwen-token-plan-individual").find((candidate) => candidate.id === "glm-5.3");
+ expect(model).toBeDefined();
+ if (!model) throw new Error("Missing model: qwen-token-plan-individual/glm-5.3");
+
+ expect(model.baseUrl).toBe("https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1");
+ expect(model.thinkingLevelMap).toMatchObject({
+ minimal: null,
+ low: "low",
+ medium: null,
+ high: "high",
+ xhigh: null,
+ max: "max",
+ });
+ });
+
it.each(QWEN38_MODEL_CASES)(
"exposes qwen3.8 reasoning_effort levels for $provider/$modelId",
({ provider, modelId }) => {
From 081a6e87f69630aa45e18c0ec652f95820ae76cc Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 09:57:47 +0100
Subject: [PATCH 04/88] fix(coding-agent): share the auth.json lock across
symlinked paths (#11787)
`FileAuthStorageBackend` locked `auth.json` with proper-lockfile and
`realpath: false`, so the lock directory was `.lock`.
Reads and writes follow symlinks to the same real file, but each path
alias got its own lock. For example, an agent dir whose `auth.json` is a
symlink to a shared file used a different lock than the real file. Two
pi processes could then enter the credential-modification section at the
same time and refresh the same OAuth token concurrently.
This change drops `realpath: false` from the sync and async lock calls,
so proper-lockfile resolves the path and every alias of the file uses
one lock. Resolving the path needs the file to exist, which is already
the case: `ensureFileExists()` runs before either lock is taken.
The same `realpath: false` pattern is also in `settings-manager.ts`,
`trust-manager.ts`, and `package-manager-cli.ts`. This change leaves
those alone.
## Testing
- I added a test to `test/auth-storage.test.ts`. It holds the lock on
the real `auth.json`, then checks that a backend opened through a
symlink alias waits in `withLockAsync` until aborted, and gets `ELOCKED`
from `withLock`. Neither path runs the mutation. Without the fix the
test fails because the async call enters the lock; with the fix it
passes.
- `node ../../node_modules/vitest/dist/cli.js --run test/auth-storage
test/settings`: 151 tests pass.
- `npm run check` passes.
- With the reproduction script from the issue, current main prints
`symlink alias: ENTERED`; with this change it prints `symlink alias:
BLOCKED`. The same-path control prints `BLOCKED` in both cases.
Fixes #11655
Requested by @badlogic
Co-authored-by: triagent
Co-authored-by: Mario Zechner
---
packages/coding-agent/CHANGELOG.md | 1 +
.../coding-agent/src/core/auth-storage.ts | 5 ++--
.../coding-agent/test/auth-storage.test.ts | 29 ++++++++++++++++++-
3 files changed, 32 insertions(+), 3 deletions(-)
diff --git a/packages/coding-agent/CHANGELOG.md b/packages/coding-agent/CHANGELOG.md
index 39aaadcf0..5033453f6 100644
--- a/packages/coding-agent/CHANGELOG.md
+++ b/packages/coding-agent/CHANGELOG.md
@@ -91,6 +91,7 @@
- Fixed `tool_search` and codemode `searchTools()` ranking a tool below tools that only mention its name in their descriptions, for example `peekaboo app` ranking `mcp__peekaboo__app` 14th. A tool matching more of the query's words now ranks above one matching fewer, a query word in a tool's name scores more than any number of mentions in its description, a tool whose whole name is in the query gets a large bonus, and long server instructions no longer lower the scores of that server's tools ([#11331](https://github.com/earendil-works/pi/issues/11331))
- Fixed `tool_search` and codemode `searchTools()` finding nothing for Chinese, Japanese, Thai, and other non-ASCII queries, because the tokenizer kept only ASCII letters and digits. Words in any script now match, text without spaces between words such as `搜索记录` is split into words, and canonically equivalent text such as `café` and `cafe\u0301` matches ([#10816](https://github.com/earendil-works/pi/issues/10816), [#11037](https://github.com/earendil-works/pi/pull/11037) by [@haoqixu](https://github.com/haoqixu))
- Fixed `pi update` stopping with `git clean -fdx failed with code 1` on Windows when another pi session had a git extension's native module or executable open. Windows cannot delete such files, so the update left the new code with the old dependencies. pi now warns, installs the dependencies anyway, and retries the clean on the next update ([#11597](https://github.com/earendil-works/pi/issues/11597))
+- Fixed two pi processes refreshing OAuth credentials at the same time when they reach the same `auth.json` through different paths, such as a symlinked `auth.json` in a separate agent directory. The auth lock now uses the file's resolved path ([#11655](https://github.com/earendil-works/pi/issues/11655))
- Fixed tool rows and streamed answers highlighting the same code again on every redraw. A running codemode row highlighted its whole script on every progress update and timer tick, a complete `write` row its whole file on every update and expand toggle, a streamed `write` its first 50 lines on every delta, and a streamed answer every earlier code block on every delta. Rows now reuse their last highlighting until the code, theme, or loaded grammars change, and assistant messages keep their rendered Markdown blocks while streaming ([#10971](https://github.com/earendil-works/pi/issues/10971))
- Fixed resized photos being sent as multi-megabyte PNGs even when JPEG was much smaller, causing large requests to fail or time out. Image resizing now tries the smaller encoding first ([#11551](https://github.com/earendil-works/pi/issues/11551))
- Fixed `pi update --extensions` reporting success without warning when npm's stale registry metadata left a package at its old version. pi now reports the installed and expected versions ([#11536](https://github.com/earendil-works/pi/issues/11536))
diff --git a/packages/coding-agent/src/core/auth-storage.ts b/packages/coding-agent/src/core/auth-storage.ts
index 5a8fa417a..de4e57244 100644
--- a/packages/coding-agent/src/core/auth-storage.ts
+++ b/packages/coding-agent/src/core/auth-storage.ts
@@ -83,7 +83,8 @@ export class FileAuthStorageBackend implements AuthStorageBackend {
const delayMs = 20;
for (let attempt = 1; ; attempt++) {
try {
- return lockfile.lockSync(path, { realpath: false });
+ // Lock the resolved file so symlinked aliases of auth.json share one lock (#11655).
+ return lockfile.lockSync(path);
} catch (error) {
if (!getRetryableLockErrorCode(error) || attempt === maxAttempts) throw error;
const start = Date.now();
@@ -127,8 +128,8 @@ export class FileAuthStorageBackend implements AuthStorageBackend {
signal?.throwIfAborted();
let release: (() => Promise) | undefined;
try {
+ // Lock the resolved file so symlinked aliases of auth.json share one lock (#11655).
release = await lockfile.lock(this.authPath, {
- realpath: false,
retries: 0,
stale: staleMs,
onCompromised,
diff --git a/packages/coding-agent/test/auth-storage.test.ts b/packages/coding-agent/test/auth-storage.test.ts
index 8005bf978..35230c73b 100644
--- a/packages/coding-agent/test/auth-storage.test.ts
+++ b/packages/coding-agent/test/auth-storage.test.ts
@@ -1,4 +1,4 @@
-import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
+import { chmodSync, existsSync, mkdirSync, readFileSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { type CredentialStore, createModels, type Provider } from "@earendil-works/pi-ai";
@@ -366,6 +366,33 @@ describe("AuthStorage", () => {
expect(existsSync(authJsonPath)).toBe(false);
});
+ // #11655: aliases of the same auth.json must share one lock.
+ test("a symlinked auth.json path waits for the lock held on the real file", async () => {
+ writeAuthJson({ anthropic: { type: "api_key", key: "stored" } });
+ const aliasPath = join(tempDir, "alias.json");
+ symlinkSync(authJsonPath, aliasPath);
+ const release = await lockfile.lock(authJsonPath, { realpath: false });
+ const backend = new FileAuthStorageBackend(aliasPath);
+ const update = vi.fn(async () => ({ result: undefined, next: JSON.stringify({}) }));
+ const syncUpdate = vi.fn(() => ({ result: undefined, next: JSON.stringify({}) }));
+
+ try {
+ const controller = new AbortController();
+ const pending = backend.withLockAsync(update, { signal: controller.signal });
+ await new Promise((resolve) => setTimeout(resolve, 200));
+ controller.abort();
+ await expect(pending).rejects.toMatchObject({ name: "AbortError" });
+ expect(() => backend.withLock(syncUpdate)).toThrow(expect.objectContaining({ code: "ELOCKED" }));
+ } finally {
+ await release();
+ }
+ expect(update).not.toHaveBeenCalled();
+ expect(syncUpdate).not.toHaveBeenCalled();
+ expect(JSON.parse(readFileSync(authJsonPath, "utf8"))).toEqual({
+ anthropic: { type: "api_key", key: "stored" },
+ });
+ });
+
test("aborts while waiting for a held file lock without running the mutation later", async () => {
writeAuthJson({ anthropic: { type: "api_key", key: "stored" } });
const release = await lockfile.lock(authJsonPath, { realpath: false });
From b0c64f33c8c003212c487ee9a1d38fa4a9fc335d Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 09:59:49 +0100
Subject: [PATCH 05/88] fix(coding-agent): load user keybindings in pi config
(#11788)
`pi config` ignored `~/.pi/agent/keybindings.json`. `selectConfig()`
never called `setKeybindings()`, so the config selector's
`getKeybindings()` fell back to the default TUI bindings. Remaps such as
`"tui.select.down": ["down", "ctrl+n"]` did nothing there, although they
work in `/settings` and the `--resume` picker.
`selectConfig()` now installs
`KeybindingsManager.create(options.agentDir)` before it builds the UI,
the same way the session picker and startup UI do.
**Testing**
- Added `test/config-selector-keybindings.test.ts`. It runs
`selectConfig()` with a stubbed terminal and component and an agent dir
whose `keybindings.json` remaps `tui.select.up` and `tui.select.down`,
then checks that the active keybindings include `ctrl+p` and `ctrl+n`.
It fails without the change and passes with it.
- Manual check: ran `pi config` in tmux with that `keybindings.json`.
Before the change, Ctrl+N did nothing. After it, Ctrl+N moves the
selection like Down.
- `npm run check` reports only two errors, both missing modules in
`packages/schema/test/bench/candidates.ts`. They occur on `main` without
this change.
Fixes #11651
Requested by @badlogic
Co-authored-by: triagent
---
.../coding-agent/src/cli/config-selector.ts | 4 +-
.../test/config-selector-keybindings.test.ts | 75 +++++++++++++++++++
2 files changed, 78 insertions(+), 1 deletion(-)
create mode 100644 packages/coding-agent/test/config-selector-keybindings.test.ts
diff --git a/packages/coding-agent/src/cli/config-selector.ts b/packages/coding-agent/src/cli/config-selector.ts
index 261547acd..53c88f8da 100644
--- a/packages/coding-agent/src/cli/config-selector.ts
+++ b/packages/coding-agent/src/cli/config-selector.ts
@@ -2,7 +2,8 @@
* TUI config selector for `pi config` command
*/
-import { ProcessTerminal, type TUI, TuiMainScreen } from "@earendil-works/pi-tui";
+import { ProcessTerminal, setKeybindings, type TUI, TuiMainScreen } from "@earendil-works/pi-tui";
+import { KeybindingsManager } from "../core/keybindings.ts";
import type { SettingsManager } from "../core/settings-manager.ts";
import { ConfigSelectorComponent, type ScopedResolvedPaths } from "../modes/interactive/components/config-selector.ts";
import { initTheme, stopThemeWatcher } from "../modes/interactive/theme/theme.ts";
@@ -20,6 +21,7 @@ export interface ConfigSelectorOptions {
export async function selectConfig(options: ConfigSelectorOptions): Promise {
// Initialize theme before showing TUI
initTheme(options.settingsManager.getTheme(), true);
+ setKeybindings(KeybindingsManager.create(options.agentDir));
return new Promise((resolve) => {
const ui: TUI = new TuiMainScreen(
diff --git a/packages/coding-agent/test/config-selector-keybindings.test.ts b/packages/coding-agent/test/config-selector-keybindings.test.ts
new file mode 100644
index 000000000..b22805afa
--- /dev/null
+++ b/packages/coding-agent/test/config-selector-keybindings.test.ts
@@ -0,0 +1,75 @@
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { getKeybindings, setKeybindings } from "@earendil-works/pi-tui";
+import { afterEach, describe, expect, it, vi } from "vitest";
+
+vi.mock("@earendil-works/pi-tui", async (importOriginal) => {
+ const actual = await importOriginal();
+ class FakeTui {
+ terminal = { rows: 24 };
+ setClearOnShrink() {}
+ addChild() {}
+ setFocus() {}
+ start() {}
+ stop() {}
+ requestRender() {}
+ }
+ return { ...actual, ProcessTerminal: class {}, TuiMainScreen: FakeTui };
+});
+
+vi.mock("../src/modes/interactive/components/config-selector.ts", () => ({
+ ConfigSelectorComponent: class {
+ constructor(...args: unknown[]) {
+ const onClose = args[4] as () => void;
+ queueMicrotask(onClose);
+ }
+ getResourceList() {
+ return {};
+ }
+ },
+}));
+
+vi.mock("../src/modes/interactive/theme/theme.ts", () => ({
+ initTheme: () => {},
+ stopThemeWatcher: () => {},
+}));
+
+import { selectConfig } from "../src/cli/config-selector.ts";
+import { KeybindingsManager } from "../src/core/keybindings.ts";
+import type { SettingsManager } from "../src/core/settings-manager.ts";
+
+describe("selectConfig keybindings", () => {
+ let agentDir: string | undefined;
+
+ afterEach(() => {
+ if (agentDir) rmSync(agentDir, { recursive: true, force: true });
+ agentDir = undefined;
+ setKeybindings(new KeybindingsManager({}));
+ });
+
+ it("uses the user's keybindings.json", async () => {
+ agentDir = mkdtempSync(join(tmpdir(), "pi-config-keys-"));
+ writeFileSync(
+ join(agentDir, "keybindings.json"),
+ JSON.stringify({ "tui.select.up": ["up", "ctrl+p"], "tui.select.down": ["down", "ctrl+n"] }),
+ );
+ const settingsManager = {
+ getTheme: () => undefined,
+ getShowHardwareCursor: () => false,
+ getClearOnShrink: () => false,
+ } as unknown as SettingsManager;
+
+ await selectConfig({
+ resolvedPaths: {} as never,
+ settingsManager,
+ cwd: agentDir,
+ agentDir,
+ writeScope: "global",
+ projectModeAvailable: false,
+ });
+
+ expect(getKeybindings().getKeys("tui.select.down")).toContain("ctrl+n");
+ expect(getKeybindings().getKeys("tui.select.up")).toContain("ctrl+p");
+ });
+});
From 9bca21a0b8481f77ebabe614ba61c86f79ff02be Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 10:05:21 +0100
Subject: [PATCH 06/88] fix(coding-agent): update minimatch to 11.0.3 for
GHSA-g3hv-75wg-m249 (#11789)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
`@earendil-works/pi-coding-agent` pins `minimatch@10.2.6`.
GHSA-g3hv-75wg-m249 (moderate, a ReDoS via globparts amplification)
covers every minimatch version below 11.0.1, and there is no patched
10.x release. As a result, `npm audit` fails in every project that
installs pi-coding-agent.
This change pins `minimatch` to `11.0.3` and regenerates
`package-lock.json` and `packages/coding-agent/install-lock/` with
`scripts/generate-coding-agent-install-lock.mjs`. minimatch 11 adds one
new transitive dependency, `@isaacs/nonrecursive-glob-match@1.0.3`,
which has no install scripts. It also moves brace-expansion to ^5.0.12.
pi uses minimatch in only two places, `model-resolver.ts` and
`package-manager.ts`. Both use the named `minimatch` export, which 11.x
keeps unchanged.
No new test was added: the change only bumps a dependency, and existing
tests cover both places that use it.
Testing:
- `npm run build` and `npm run check` pass, and the install-lock check
reports it is up to date.
- `test/model-resolver.test.ts`, `test/package-manager.test.ts` and
`test/package-manager-ssh.test.ts` pass (213 tests).
- Before: installing 1.4.0 and running `npm audit` reports `minimatch
<11.0.1 … GHSA-g3hv-75wg-m249`.
- After: `npm audit --package-lock-only` in
`packages/coding-agent/install-lock` reports `found 0 vulnerabilities`.
- At the monorepo root, `npm audit` still flags a `minimatch@3.1.5` that
comes in through the `shx` dev dependency. pi does not ship that copy,
and this change does not touch it.
Fixes #11646
Requested by @badlogic
Co-authored-by: triagent
From f4427f8a3edd0de0332e44fed9e53d4f37ae0953 Mon Sep 17 00:00:00 2001
From: Mario Zechner
Date: Wed, 4 Nov 2026 09:07:23 +0000
Subject: [PATCH 07/88] docs(chord): the tutorial uses tx.update with its
integer key now that int keys type-check
---
packages/chord/docs/tutorial/book.md | 11 +++--------
packages/chord/docs/tutorial/index.html | 14 ++++----------
.../examples/61-tutorial/plugins/lists/worker.ts | 2 +-
3 files changed, 8 insertions(+), 19 deletions(-)
diff --git a/packages/chord/docs/tutorial/book.md b/packages/chord/docs/tutorial/book.md
index 9d559987f..f2e4b2f66 100644
--- a/packages/chord/docs/tutorial/book.md
+++ b/packages/chord/docs/tutorial/book.md
@@ -1626,14 +1626,9 @@ Await what you start: a write started with a context that ends is cancelled and
watch starts at the current value: read `handle.value`, then `start`; the listener gets changes only. The plain `t`
has no language of its own: components and screens render with `useMessages(t)` and `scope.messages(t)`.
-**Hard to explain, found while writing the example.** Each is a candidate for a fix in Chord rather than a paragraph in
-a tutorial.
-
-- **Integer table keys do not type-check.** `KeyOf` in `src/store/table.ts` still tests TypeBox's
- `"~kind": "Integer"`, which pi-schema's `int()` lacks, so the key of an `int` key column is typed `string`:
- `tx.get(items, { id })` with a number fails `tsc`. The store here uses `updateWhere` with `eq` instead of
- `tx.update`.
-Four more, found the same way, are fixed: a reloaded web plugin lost its own React state (now `usePageState`,
+**Found while writing the example, and fixed in Chord since.** Integer table keys were typed `string` (a leftover
+of the switch from TypeBox), so `tx.update(items, { id }, ...)` with a number failed `tsc`; the store now uses it. A
+reloaded web plugin lost its own React state (now `usePageState`,
chapter IX), a plugin's `contract.ts` could not be imported outside builds (the API is now a workspace package,
chapter I), plugins could not throw Chord's `invalid` (Chord exports `invalid` and `forbidden`, chapter II), and a
live window over a table needed a one-time read in `self.open`, which made the store restart at every reload of its
diff --git a/packages/chord/docs/tutorial/index.html b/packages/chord/docs/tutorial/index.html
index 912787e01..782e64b66 100644
--- a/packages/chord/docs/tutorial/index.html
+++ b/packages/chord/docs/tutorial/index.html
@@ -1455,19 +1455,13 @@ reference acts as whoever made it: ctx.get as the caller, hos
Await what you start: a write started with a context that ends is cancelled and reported transaction_cancelled. A
watch starts at the current value: read handle.value, then start; the listener gets changes only. The plain t
has no language of its own: components and screens render with useMessages(t) and scope.messages(t).
-Hard to explain, found while writing the example. Each is a candidate for a fix in Chord rather than a paragraph in
-a tutorial.
-
-- Integer table keys do not type-check.
KeyOf in src/store/table.ts still tests TypeBox's
-"~kind": "Integer", which pi-schema's int() lacks, so the key of an int key column is typed string:
-tx.get(items, { id }) with a number fails tsc. The store here uses updateWhere with eq instead of
-tx.update.
-Four more, found the same way, are fixed: a reloaded web plugin lost its own React state (now usePageState,
+Found while writing the example, and fixed in Chord since. Integer table keys were typed string (a leftover
+of the switch from TypeBox), so tx.update(items, { id }, ...) with a number failed tsc; the store now uses it. A
+reloaded web plugin lost its own React state (now usePageState,
chapter IX), a plugin's contract.ts could not be imported outside builds (the API is now a workspace package,
chapter I), plugins could not throw Chord's invalid (Chord exports invalid and forbidden, chapter II), and a
live window over a table needed a one-time read in self.open, which made the store restart at every reload of its
-plugin (now self.query, a live query Chord runs, chapters II and IX).
-
+plugin (now self.query, a live query Chord runs, chapters II and IX).
diff --git a/packages/chord/test/examples/61-tutorial/plugins/lists/worker.ts b/packages/chord/test/examples/61-tutorial/plugins/lists/worker.ts
index b592c7ff5..b098a12df 100644
--- a/packages/chord/test/examples/61-tutorial/plugins/lists/worker.ts
+++ b/packages/chord/test/examples/61-tutorial/plugins/lists/worker.ts
@@ -79,7 +79,7 @@ export default defineFacet((host) => {
return null;
},
tick: async ({ id, done }, ctx) => {
- await self.tx(async (tx) => tx.updateWhere(items, (item) => eq(item.id, id), { done }), ctx);
+ await self.tx(async (tx) => tx.update(items, { id }, { done }), ctx);
return null;
},
clear: async ({ by }, ctx) => {
From f6278a472e9b97579e9e9ef512303aa91a926a67 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 10:09:23 +0100
Subject: [PATCH 08/88] fix(mcp): read large stdio messages in linear time
(#11791)
`JsonRpcLineReader.readStdout` (used by the stdio transport and by
`PipeTransport`) appended every stdout chunk to the buffered bytes with
`Buffer.concat`, then searched the whole buffer from the start for a
newline. A large message arriving in many chunks was therefore copied
and scanned again on every chunk. A 12 MiB tool result in 64 KiB chunks
took about 350 ms just to frame.
Now the reader keeps the bytes of an incomplete message as a list of
chunks, with a running byte count, and searches only the new chunk for a
newline. When the newline arrives, the chunks are joined once. A line
that fits inside a single chunk is sliced directly. The rest is
unchanged: the `maxMessageBytes` limit (now checked against the running
count), CRLF and blank-line handling, errors for invalid JSON and
oversized messages, and the incomplete-message error in `end()`.
Testing:
- New `packages/mcp/test/lines.test.ts`:
- a 1 MiB message fed in 4 KiB chunks calls `Buffer.concat` once (fails
on the old code);
- a mixed input of multi-byte UTF-8, CRLF, blank lines, an invalid line
and several messages gives the same results at every chunk size from 1
byte to the whole input;
- reading recovers after an oversized message;
- a message without its newline is reported at the end.
- `test/lines.test.ts`, `test/stdio.test.ts` and `test/pipe.test.ts`
pass. `npm run check` passes.
- Micro-benchmark, one message fed in 64 KiB chunks:
- before: 1 MiB 5.4 ms, 4 MiB 49.9 ms, 12 MiB 348 ms;
- after: 2.2 ms, 7.9 ms, 21.6 ms.
Fixes #11761
Requested by @badlogic
Co-authored-by: triagent
---
packages/mcp/src/transports/lines.ts | 40 +++++++++++----
packages/mcp/test/lines.test.ts | 75 ++++++++++++++++++++++++++++
2 files changed, 105 insertions(+), 10 deletions(-)
create mode 100644 packages/mcp/test/lines.test.ts
diff --git a/packages/mcp/src/transports/lines.ts b/packages/mcp/src/transports/lines.ts
index 8d53d9c27..3a07be176 100644
--- a/packages/mcp/src/transports/lines.ts
+++ b/packages/mcp/src/transports/lines.ts
@@ -18,7 +18,9 @@ export interface JsonRpcLineReaderOptions {
*/
export class JsonRpcLineReader {
private readonly options: JsonRpcLineReaderOptions;
- private stdoutBuffer = Buffer.alloc(0);
+ /** The bytes of the incomplete message received so far. */
+ private stdoutChunks: Buffer[] = [];
+ private stdoutPendingBytes = 0;
private stderrBuffer = Buffer.alloc(0);
constructor(options: JsonRpcLineReaderOptions) {
@@ -30,21 +32,38 @@ export class JsonRpcLineReader {
return this.stderrBuffer.toString("utf8");
}
- /** Takes the next stdout bytes and reports every complete message, or the error parsing it. */
+ /**
+ * Takes the next stdout bytes and reports every complete message, or the error parsing it. Bytes of an
+ * incomplete message are kept as chunks and joined once when its newline arrives, so a large message that
+ * arrives in many chunks is copied and scanned only once.
+ */
readStdout(chunk: Uint8Array): void {
- this.stdoutBuffer = Buffer.concat([this.stdoutBuffer, chunk]);
+ const data = Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength);
const maxMessageBytes = this.options.maxMessageBytes ?? DEFAULT_MAX_MESSAGE_BYTES;
+ let start = 0;
while (true) {
- const newline = this.stdoutBuffer.indexOf(0x0a);
+ const newline = data.indexOf(0x0a, start);
if (newline < 0) {
- if (this.stdoutBuffer.length > maxMessageBytes) {
- this.stdoutBuffer = Buffer.alloc(0);
+ if (start < data.length) {
+ // Copied, since the caller may reuse the chunk's memory.
+ this.stdoutChunks.push(Buffer.from(data.subarray(start)));
+ this.stdoutPendingBytes += data.length - start;
+ }
+ if (this.stdoutPendingBytes > maxMessageBytes) {
+ this.stdoutChunks = [];
+ this.stdoutPendingBytes = 0;
this.options.onError(new Error(`MCP stdio message exceeds ${maxMessageBytes} bytes`));
}
return;
}
- const line = this.stdoutBuffer.subarray(0, newline);
- this.stdoutBuffer = this.stdoutBuffer.subarray(newline + 1);
+ let line = data.subarray(start, newline);
+ if (this.stdoutChunks.length > 0) {
+ this.stdoutChunks.push(line);
+ line = Buffer.concat(this.stdoutChunks, this.stdoutPendingBytes + line.length);
+ this.stdoutChunks = [];
+ this.stdoutPendingBytes = 0;
+ }
+ start = newline + 1;
if (line.length > maxMessageBytes) {
this.options.onError(new Error(`MCP stdio message exceeds ${maxMessageBytes} bytes`));
continue;
@@ -69,9 +88,10 @@ export class JsonRpcLineReader {
/** Ends stdout: a message without its newline is reported as an error and dropped. */
end(): void {
- if (this.stdoutBuffer.toString("utf8").trim()) {
+ if (Buffer.concat(this.stdoutChunks).toString("utf8").trim()) {
this.options.onError(new Error("MCP stdio server closed with an incomplete JSON-RPC message"));
}
- this.stdoutBuffer = Buffer.alloc(0);
+ this.stdoutChunks = [];
+ this.stdoutPendingBytes = 0;
}
}
diff --git a/packages/mcp/test/lines.test.ts b/packages/mcp/test/lines.test.ts
new file mode 100644
index 000000000..f911c063c
--- /dev/null
+++ b/packages/mcp/test/lines.test.ts
@@ -0,0 +1,75 @@
+import { afterEach, describe, expect, it, vi } from "vitest";
+import type { JsonRpcMessage } from "../src/index.ts";
+import { JsonRpcLineReader } from "../src/transports/lines.ts";
+
+function startReader(maxMessageBytes?: number) {
+ const messages: JsonRpcMessage[] = [];
+ const errors: string[] = [];
+ const reader = new JsonRpcLineReader({
+ maxMessageBytes,
+ onMessage: (message) => messages.push(message),
+ onError: (error) => errors.push(error instanceof Error ? error.message : String(error)),
+ });
+ return { reader, messages, errors };
+}
+
+function feed(reader: JsonRpcLineReader, bytes: Buffer, chunkSize: number): void {
+ for (let i = 0; i < bytes.length; i += chunkSize) reader.readStdout(bytes.subarray(i, i + chunkSize));
+}
+
+describe("JsonRpcLineReader", () => {
+ afterEach(() => {
+ vi.restoreAllMocks();
+ });
+
+ it("joins a message split across many chunks once, when its newline arrives", () => {
+ const { reader, messages, errors } = startReader();
+ const text = "x".repeat(1024 * 1024);
+ const bytes = Buffer.from(`${JSON.stringify({ jsonrpc: "2.0", id: 1, result: { text } })}\n`);
+ const concat = vi.spyOn(Buffer, "concat");
+ feed(reader, bytes, 4096);
+ expect(concat).toHaveBeenCalledTimes(1);
+ expect(errors).toEqual([]);
+ expect(messages).toEqual([{ jsonrpc: "2.0", id: 1, result: { text } }]);
+ });
+
+ it("handles multi-byte characters, CRLF, blank lines, and several messages per chunk at any split", () => {
+ const input = Buffer.from(
+ [
+ JSON.stringify({ jsonrpc: "2.0", id: 1, result: { text: "héllo 🌍" } }),
+ "",
+ `${JSON.stringify({ jsonrpc: "2.0", method: "notifications/x" })}\r`,
+ "not json",
+ JSON.stringify({ jsonrpc: "2.0", id: 2, result: {} }),
+ "",
+ ].join("\n"),
+ );
+ for (let chunkSize = 1; chunkSize <= input.length; chunkSize++) {
+ const { reader, messages, errors } = startReader();
+ feed(reader, input, chunkSize);
+ reader.end();
+ expect(messages).toEqual([
+ { jsonrpc: "2.0", id: 1, result: { text: "héllo 🌍" } },
+ { jsonrpc: "2.0", method: "notifications/x" },
+ { jsonrpc: "2.0", id: 2, result: {} },
+ ]);
+ expect(errors).toHaveLength(1);
+ }
+ });
+
+ it("drops an oversized message and keeps reading the next one", () => {
+ const { reader, messages, errors } = startReader(64);
+ const next = JSON.stringify({ jsonrpc: "2.0", id: 2, result: {} });
+ feed(reader, Buffer.from(`${"a".repeat(50)}`), 10);
+ feed(reader, Buffer.from(`${"a".repeat(50)}\n${next}\n`), 10);
+ expect(errors[0]).toBe("MCP stdio message exceeds 64 bytes");
+ expect(messages).toEqual([{ jsonrpc: "2.0", id: 2, result: {} }]);
+ });
+
+ it("reports a message without its newline at the end", () => {
+ const { reader, errors } = startReader();
+ feed(reader, Buffer.from('{"jsonrpc":"2.0",'), 4);
+ reader.end();
+ expect(errors).toEqual(["MCP stdio server closed with an incomplete JSON-RPC message"]);
+ });
+});
From 3b57ed7e278b63580eacd175786fbeb5b2fd70d6 Mon Sep 17 00:00:00 2001
From: Christian Tietze
Date: Wed, 4 Nov 2026 10:15:27 +0100
Subject: [PATCH 09/88] fix: limit selector window height (#11072)
With this PR, we stop rendering every option.
Instead, we render a centered, clamped window of up to 10 options and
show `(selection/total)` when the list overflows.
This keeps the selected option visible in long extension-provided lists
without changing navigation, keybindings, callbacks, countdowns, or
short-list rendering.
Fixes #6688.
## Merge Danger
**Door:** Two-way
The change is localized to `ExtensionSelectorComponent.updateList()` and
can be reverted without migration or persisted-state concerns.
**Blast Radius:** Low
Only extension selectors with more than 10 options change appearance;
they now render a window and position indicator instead of every option.
---------
Co-authored-by: Pi via openai-codex (gpt-5.6-sol)
---
.../components/extension-selector.ts | 16 +++-
.../test/extension-selector.test.ts | 82 +++++++++++++++++++
2 files changed, 97 insertions(+), 1 deletion(-)
create mode 100644 packages/coding-agent/test/extension-selector.test.ts
diff --git a/packages/coding-agent/src/modes/interactive/components/extension-selector.ts b/packages/coding-agent/src/modes/interactive/components/extension-selector.ts
index ed49c04a8..e6e40f923 100644
--- a/packages/coding-agent/src/modes/interactive/components/extension-selector.ts
+++ b/packages/coding-agent/src/modes/interactive/components/extension-selector.ts
@@ -84,13 +84,27 @@ export class ExtensionSelectorComponent extends Container {
private updateList(): void {
this.listContainer.clear();
- for (let i = 0; i < this.options.length; i++) {
+
+ const maxVisible = 10;
+ // Center the window on the selection, then clamp it to the list boundaries.
+ const startIndex = Math.max(
+ 0,
+ Math.min(this.selectedIndex - Math.floor(maxVisible / 2), this.options.length - maxVisible),
+ );
+ const endIndex = Math.min(startIndex + maxVisible, this.options.length);
+
+ for (let i = startIndex; i < endIndex; i++) {
const isSelected = i === this.selectedIndex;
const text = isSelected
? theme.fg("accent", "→ ") + theme.fg("accent", this.options[i])
: ` ${theme.fg("text", this.options[i])}`;
this.listContainer.addChild(new Text(text, 1, 0));
}
+
+ if (startIndex > 0 || endIndex < this.options.length) {
+ const scrollInfo = theme.fg("muted", ` (${this.selectedIndex + 1}/${this.options.length})`);
+ this.listContainer.addChild(new Text(scrollInfo, 0, 0));
+ }
}
handleInput(keyData: string): void {
diff --git a/packages/coding-agent/test/extension-selector.test.ts b/packages/coding-agent/test/extension-selector.test.ts
new file mode 100644
index 000000000..df5740f64
--- /dev/null
+++ b/packages/coding-agent/test/extension-selector.test.ts
@@ -0,0 +1,82 @@
+import { setKeybindings } from "@earendil-works/pi-tui";
+import { beforeAll, beforeEach, describe, expect, it } from "vitest";
+import { KeybindingsManager } from "../src/core/keybindings.ts";
+import { ExtensionSelectorComponent } from "../src/modes/interactive/components/extension-selector.ts";
+import { initTheme } from "../src/modes/interactive/theme/theme.ts";
+import { stripAnsi } from "../src/utils/ansi.ts";
+
+function render(selector: ExtensionSelectorComponent): string {
+ return stripAnsi(selector.render(80).join("\n"));
+}
+
+function visibleOptions(output: string): string[] {
+ return output
+ .split("\n")
+ .filter((line) => line.includes("option-"))
+ .map((line) => line.trim().replace(/^→ /, ""));
+}
+
+describe("extension selector", () => {
+ beforeAll(() => {
+ initTheme("dark");
+ });
+
+ beforeEach(() => {
+ setKeybindings(new KeybindingsManager());
+ });
+
+ // #6688: long extension selectors must keep the selected option visible.
+ it("windows long option lists around the selection", () => {
+ const options = Array.from({ length: 15 }, (_, index) => `option-${index + 1}`);
+ const selector = new ExtensionSelectorComponent(
+ "Choose",
+ options,
+ () => {},
+ () => {},
+ );
+
+ let output = render(selector);
+ expect(visibleOptions(output)).toEqual(options.slice(0, 10));
+ expect(output).toContain("→ option-1");
+ expect(output).toContain(" (1/15)");
+
+ for (let i = 0; i < 7; i++) selector.handleInput("\x1b[B");
+ output = render(selector);
+ expect(visibleOptions(output)).toEqual(options.slice(2, 12));
+ expect(output).toContain("→ option-8");
+ expect(output).toContain(" (8/15)");
+
+ for (let i = 0; i < 7; i++) selector.handleInput("\x1b[B");
+ output = render(selector);
+ expect(visibleOptions(output)).toEqual(options.slice(5, 15));
+ expect(output).toContain("→ option-15");
+ expect(output).toContain(" (15/15)");
+ });
+
+ it("renders short and empty option lists without a scroll indicator", () => {
+ const shortOptions = ["option-1", "option-2", "option-3"];
+ const shortOutput = render(
+ new ExtensionSelectorComponent(
+ "Choose",
+ shortOptions,
+ () => {},
+ () => {},
+ ),
+ );
+ expect(visibleOptions(shortOutput)).toEqual(shortOptions);
+ expect(shortOutput).toContain("→ option-1");
+ expect(shortOutput).not.toContain("(1/3)");
+
+ const emptyOutput = render(
+ new ExtensionSelectorComponent(
+ "Choose",
+ [],
+ () => {},
+ () => {},
+ ),
+ );
+ expect(emptyOutput).toContain("Choose");
+ expect(emptyOutput).toContain("↑↓ navigate");
+ expect(emptyOutput).not.toContain("→");
+ });
+});
From 66c6556771bcdbfbd37496e77f05515e99d1a2dd Mon Sep 17 00:00:00 2001
From: Gangqiang Yao
Date: Wed, 4 Nov 2026 17:16:53 +0800
Subject: [PATCH 10/88] fix(coding-agent): track discarded user bash output
(#10165)
Closes #10164.
A user `!` command can discard earlier chunks but report `truncated:
false` when the retained tail fits within the output limits. The model
then receives no truncation notice or full-log path.
Track discarded chunks and include that flag in the executor's result
for both completion and cancellation. This applies to the current shared
finalization path and preserves streaming ANSI handling. Output limits
and message format stay unchanged.
Related: #2852. Here the complete log is already saved; the incorrect
flag hides its path from the model.
Validation on main `f1b2e77f5`:
- Two deterministic 64 KiB + 44 KiB cases fail without the fix. All
three regressions pass with it, checking completion, cancellation, the
complete log, model-facing path, and short-output behavior.
- Regressions are in `test/bash-executor-truncation.test.ts`; that file
and `test/tools.test.ts` pass all 88 tests.
- `npm run check`, offline build, and `./test.sh` pass.
---
.../coding-agent/src/core/bash-executor.ts | 4 +-
.../test/bash-executor-truncation.test.ts | 59 +++++++++++++++++++
2 files changed, 62 insertions(+), 1 deletion(-)
create mode 100644 packages/coding-agent/test/bash-executor-truncation.test.ts
diff --git a/packages/coding-agent/src/core/bash-executor.ts b/packages/coding-agent/src/core/bash-executor.ts
index 679543ec4..9745ff217 100644
--- a/packages/coding-agent/src/core/bash-executor.ts
+++ b/packages/coding-agent/src/core/bash-executor.ts
@@ -53,6 +53,7 @@ export async function executeBashWithOperations(
): Promise {
const outputChunks: string[] = [];
let outputBytes = 0;
+ let discardedOutput = false;
const maxOutputBytes = DEFAULT_MAX_BYTES * 2;
let tempFilePath: string | undefined;
@@ -95,6 +96,7 @@ export async function executeBashWithOperations(
while (outputBytes > maxOutputBytes && outputChunks.length > 1) {
const removed = outputChunks.shift()!;
outputBytes -= removed.length;
+ discardedOutput = true;
}
// Stream to callback
@@ -140,7 +142,7 @@ export async function executeBashWithOperations(
output: truncationResult.truncated ? truncationResult.content : fullOutput,
exitCode: cancelled ? undefined : (exitCode ?? undefined),
cancelled,
- truncated: truncationResult.truncated,
+ truncated: discardedOutput || truncationResult.truncated,
fullOutputPath: tempFilePath,
};
}
diff --git a/packages/coding-agent/test/bash-executor-truncation.test.ts b/packages/coding-agent/test/bash-executor-truncation.test.ts
new file mode 100644
index 000000000..7c480fe46
--- /dev/null
+++ b/packages/coding-agent/test/bash-executor-truncation.test.ts
@@ -0,0 +1,59 @@
+import { readFile, rm } from "node:fs/promises";
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { executeBashWithOperations } from "../src/core/bash-executor.ts";
+import { bashExecutionToText } from "../src/core/messages.ts";
+
+describe("user bash output truncation (#10164)", () => {
+ const tempFiles: string[] = [];
+
+ afterEach(async () => {
+ await Promise.all(tempFiles.splice(0).map((path) => rm(path, { force: true })));
+ });
+
+ // #10164: the retained tail can fit even though earlier output was discarded.
+ it.each([false, true])("reports discarded chunks when cancelled=%s", async (cancelled) => {
+ const first = "error: missing build dependency\n".padEnd(64 * 1024, "x");
+ const last = "remaining build output\n".padEnd(44 * 1024, "y");
+ const controller = new AbortController();
+ const result = await executeBashWithOperations(
+ "cat build.log",
+ process.cwd(),
+ {
+ exec: async (_command, _cwd, { onData }) => {
+ onData(Buffer.from(first));
+ onData(Buffer.from(last));
+ if (cancelled) {
+ controller.abort();
+ throw new Error("aborted");
+ }
+ return { exitCode: 0 };
+ },
+ },
+ { signal: controller.signal },
+ );
+ if (result.fullOutputPath) tempFiles.push(result.fullOutputPath);
+ // The executor closes its file stream without awaiting its finish event.
+ await vi.waitFor(async () => {
+ expect(await readFile(result.fullOutputPath!, "utf-8")).toBe(first + last);
+ });
+
+ expect(result.cancelled).toBe(cancelled);
+ expect(result.output).toBe(last);
+ expect(result.truncated).toBe(true);
+ expect(
+ bashExecutionToText({ role: "bashExecution", command: "cat build.log", timestamp: 0, ...result }),
+ ).toContain(`[Output truncated. Full output: ${result.fullOutputPath}]`);
+ });
+
+ it("does not mark short output as truncated", async () => {
+ const result = await executeBashWithOperations("echo done", process.cwd(), {
+ exec: async (_command, _cwd, { onData }) => {
+ onData(Buffer.from("done\n"));
+ return { exitCode: 0 };
+ },
+ });
+ expect(result.output).toBe("done\n");
+ expect(result.truncated).toBe(false);
+ expect(result.fullOutputPath).toBeUndefined();
+ });
+});
From dca1f55527a2e044172e2f27f2c74b378382b7d4 Mon Sep 17 00:00:00 2001
From: Danila Poyarkov
Date: Wed, 4 Nov 2026 13:17:34 +0400
Subject: [PATCH 11/88] fix(coding-agent): preserve indentation in rendered
diffs (#9274)
## Problem
The edit tool's intra-line renderer can drop indentation from a removed
line when text is inserted before otherwise unchanged content.
For example, this input:
```diff
-83 await commitRecordingChunk({
+83 const result = await commitRecordingChunk({
```
was rendered as:
```diff
-83 await commitRecordingChunk({
+83 const result = await commitRecordingChunk({
```
`diffWords()` groups the shared leading spaces into the added segment
(`" const result = "`). The existing highlighting logic moved those
spaces outside inverse styling, but only appended them to the added
line.
## Solution
Extract whitespace shared by both lines before calling `diffWords()`,
then initialize both rendered lines with that prefix. This preserves
indentation without changing the existing word-level highlighting
behavior. The existing handling for whitespace that belongs to only one
side remains unchanged.
Added regression coverage for:
- insertion before indented content;
- replacement within indented content;
- keeping shared indentation outside inverse highlighting.
## Validation
- `npm run check`
- `./test.sh`
- targeted `packages/coding-agent/test/diff.test.ts` (3 tests)
---
.../src/modes/interactive/components/diff.ts | 18 +++++++---
packages/coding-agent/test/diff.test.ts | 35 +++++++++++++++++++
2 files changed, 48 insertions(+), 5 deletions(-)
create mode 100644 packages/coding-agent/test/diff.test.ts
diff --git a/packages/coding-agent/src/modes/interactive/components/diff.ts b/packages/coding-agent/src/modes/interactive/components/diff.ts
index 54e88273d..6c052df7b 100644
--- a/packages/coding-agent/src/modes/interactive/components/diff.ts
+++ b/packages/coding-agent/src/modes/interactive/components/diff.ts
@@ -20,14 +20,22 @@ function replaceTabs(text: string): string {
/**
* Compute word-level diff and render with inverse on changed parts.
- * Uses diffWords which groups whitespace with adjacent words for cleaner highlighting.
- * Strips leading whitespace from inverse to avoid highlighting indentation.
+ * Separates common indentation before diffing so a leading insertion cannot consume it.
*/
function renderIntraLineDiff(oldContent: string, newContent: string): { removedLine: string; addedLine: string } {
- const wordDiff = Diff.diffWords(oldContent, newContent);
+ let commonIndentLength = 0;
+ while (
+ commonIndentLength < oldContent.length &&
+ oldContent[commonIndentLength] === newContent[commonIndentLength] &&
+ /\s/.test(oldContent[commonIndentLength])
+ ) {
+ commonIndentLength++;
+ }
- let removedLine = "";
- let addedLine = "";
+ const commonIndent = oldContent.slice(0, commonIndentLength);
+ const wordDiff = Diff.diffWords(oldContent.slice(commonIndentLength), newContent.slice(commonIndentLength));
+ let removedLine = commonIndent;
+ let addedLine = commonIndent;
let isFirstRemoved = true;
let isFirstAdded = true;
diff --git a/packages/coding-agent/test/diff.test.ts b/packages/coding-agent/test/diff.test.ts
new file mode 100644
index 000000000..00403a84c
--- /dev/null
+++ b/packages/coding-agent/test/diff.test.ts
@@ -0,0 +1,35 @@
+import { beforeAll, describe, expect, it } from "vitest";
+import { renderDiff } from "../src/modes/interactive/components/diff.ts";
+import { initTheme, theme } from "../src/modes/interactive/theme/theme.ts";
+import { stripAnsi } from "../src/utils/ansi.ts";
+
+describe("renderDiff", () => {
+ beforeAll(() => initTheme("dark"));
+
+ it.each([
+ {
+ name: "text is inserted at the start of a line",
+ diff: "-83 await commitRecordingChunk({\n+83 const result = await commitRecordingChunk({",
+ expected: ["-83 await commitRecordingChunk({", "+83 const result = await commitRecordingChunk({"],
+ },
+ {
+ name: "indented text is replaced",
+ diff: "-12 const value = oldValue\n+12 const value = newValue",
+ expected: ["-12 const value = oldValue", "+12 const value = newValue"],
+ },
+ ])("preserves indentation when $name", ({ diff, expected }) => {
+ const rendered = renderDiff(diff);
+
+ expect(stripAnsi(rendered).split("\n")).toEqual(expected);
+ });
+
+ it("does not highlight shared indentation as an insertion", () => {
+ const rendered = renderDiff(
+ "-83 await commitRecordingChunk({\n+83 const result = await commitRecordingChunk({",
+ );
+ const inverseInsertion = theme.inverse("const result = ");
+
+ expect(rendered.indexOf("+83 ")).toBeLessThan(rendered.indexOf(inverseInsertion));
+ expect(rendered).toContain(inverseInsertion);
+ });
+});
From c3b938e3e9d320e0977173483c4f3620061dd73a Mon Sep 17 00:00:00 2001
From: Gangqiang Yao
Date: Wed, 4 Nov 2026 17:18:59 +0800
Subject: [PATCH 12/88] fix(coding-agent): reject reload while running or
compacting (#9222)
Fixes #9221.
An extension tool can lose its context when an RPC command or SDK caller
reloads the session during tool execution: the tool awaits work, reload
invalidates its runner, then `ctx.getSystemPrompt()` throws a
stale-context error. The failed tool result is stored in session
history.
Reject reload when `isStreaming` or `isCompacting` is already set,
before shutdown or runner invalidation. This gives RPC and direct
callers the same entry checks as the TUI. The production change remains
seven lines.
#9548 removed the wrapper access behind the original reproduction. The
regression exercises the tool's own context access after an await,
through both an RPC-style extension command and direct
`AgentSession.reload()`. Rejection preserves the successful result in
session history and the next provider request. Compaction, tree
navigation, and idle reload are also covered.
Scope: this guards operations already marked running or compacting. It
does not serialize reload with prompt preflight or prevent a new prompt
from starting during an ongoing reload.
Related: #5735 proposes a broader deferred-reload API. This patch only
adds the existing TUI entry checks to the core reload method.
Validation on main `f1b2e77f5`:
- Without the guard, four regression cases fail and idle reload passes.
With it, all five pass.
- `npm run check`, offline build, and `./test.sh` pass.
---
.../coding-agent/src/core/agent-session.ts | 7 +
.../9221-reload-during-tool.test.ts | 186 ++++++++++++++++++
2 files changed, 193 insertions(+)
create mode 100644 packages/coding-agent/test/suite/regressions/9221-reload-during-tool.test.ts
diff --git a/packages/coding-agent/src/core/agent-session.ts b/packages/coding-agent/src/core/agent-session.ts
index 5d341c4d7..0f5aa1105 100644
--- a/packages/coding-agent/src/core/agent-session.ts
+++ b/packages/coding-agent/src/core/agent-session.ts
@@ -3763,6 +3763,13 @@ export class AgentSession {
}
async reload(options?: { beforeSessionStart?: () => void | Promise }): Promise {
+ if (this.isStreaming) {
+ throw new Error("Wait for the current response to finish before reloading.");
+ }
+ if (this.isCompacting) {
+ throw new Error("Wait for compaction to finish before reloading.");
+ }
+
const oldRunner = this._extensionRunner;
const previousFlagValues = oldRunner.getFlagValues();
await emitSessionShutdownEvent(oldRunner, { type: "session_shutdown", reason: "reload" });
diff --git a/packages/coding-agent/test/suite/regressions/9221-reload-during-tool.test.ts b/packages/coding-agent/test/suite/regressions/9221-reload-during-tool.test.ts
new file mode 100644
index 000000000..4db68aba5
--- /dev/null
+++ b/packages/coding-agent/test/suite/regressions/9221-reload-during-tool.test.ts
@@ -0,0 +1,186 @@
+import { fauxAssistantMessage, fauxToolCall } from "@earendil-works/pi-ai";
+import type { ToolResultMessage } from "@earendil-works/pi-ai/compat";
+import { Type } from "typebox";
+import { afterEach, describe, expect, it, vi } from "vitest";
+import { assistantMsg, userMsg } from "../../utilities.ts";
+import { createHarness, getMessageText, type Harness } from "../harness.ts";
+
+function deferred(): { promise: Promise; resolve: () => void } {
+ let resolve!: () => void;
+ const promise = new Promise((done) => {
+ resolve = done;
+ });
+ return { promise, resolve };
+}
+
+// Regression for https://github.com/earendil-works/pi/issues/9221.
+describe("reload during an active session operation", () => {
+ let harness: Harness | undefined;
+
+ afterEach(() => {
+ harness?.cleanup();
+ vi.restoreAllMocks();
+ });
+
+ it.each(["command", "direct"] as const)(
+ "rejects %s reload without invalidating an active tool's context",
+ async (source) => {
+ const started = deferred();
+ const released = deferred();
+ const shutdown = vi.fn();
+ const errors: string[] = [];
+ let providerResults: ToolResultMessage[] = [];
+ const h = await createHarness({
+ extensionFactories: [
+ (pi) => {
+ pi.on("session_shutdown", shutdown);
+ pi.registerCommand("reload-runtime", {
+ description: "Reload extensions",
+ handler: async (_args, ctx) => ctx.reload(),
+ });
+ pi.registerTool({
+ name: "wait_then_succeed",
+ label: "Wait then succeed",
+ description: "Use the tool context after waiting",
+ parameters: Type.Object({}),
+ execute: async (_id, _params, _signal, _update, ctx) => {
+ started.resolve();
+ await released.promise;
+ // #9221: the tool itself can still use ctx after an async operation.
+ ctx.getSystemPrompt();
+ return { content: [{ type: "text", text: "tool succeeded" }], details: {} };
+ },
+ });
+ },
+ ],
+ });
+ harness = h;
+ await h.session.bindExtensions({
+ mode: "rpc",
+ onError: (error) => errors.push(error.error),
+ commandContextActions: {
+ waitForIdle: () => h.session.waitForIdle(),
+ newSession: async () => ({ cancelled: true }),
+ fork: async () => ({ cancelled: true }),
+ navigateTree: (targetId, options) => h.session.navigateTree(targetId, options),
+ switchSession: async () => ({ cancelled: true }),
+ // Match RPC's ctx.reload binding to the core reload entry point.
+ reload: () => h.session.reload(),
+ },
+ });
+ const runner = h.session.extensionRunner;
+ h.setResponses([
+ fauxAssistantMessage(fauxToolCall("wait_then_succeed", {}), { stopReason: "toolUse" }),
+ (context) => {
+ providerResults = context.messages.filter((message) => message.role === "toolResult");
+ return fauxAssistantMessage("done");
+ },
+ ]);
+
+ const prompt = h.session.prompt("run the tool");
+ await started.promise;
+ try {
+ if (source === "command") {
+ await h.session.prompt("/reload-runtime", { source: "rpc" });
+ } else {
+ await expect(h.session.reload()).rejects.toThrow(
+ "Wait for the current response to finish before reloading.",
+ );
+ }
+ } finally {
+ released.resolve();
+ await prompt;
+ }
+
+ const persistedResults = h.sessionManager
+ .getEntries()
+ .flatMap((entry) =>
+ entry.type === "message" && entry.message.role === "toolResult" ? [entry.message] : [],
+ );
+ expect(persistedResults).toHaveLength(1);
+ expect(persistedResults[0]).toMatchObject({ isError: false });
+ expect(getMessageText(persistedResults[0])).toBe("tool succeeded");
+ expect(providerResults).toEqual(persistedResults);
+ expect(errors).toEqual(
+ source === "command" ? ["Wait for the current response to finish before reloading."] : [],
+ );
+ expect(shutdown).not.toHaveBeenCalled();
+ expect(h.session.extensionRunner).toBe(runner);
+ expect(runner.getActiveTools()).toContain("wait_then_succeed");
+ },
+ );
+
+ it.each(["compaction", "tree navigation"] as const)("rejects reload during %s", async (operation) => {
+ const started = deferred();
+ const released = deferred();
+ const shutdown = vi.fn();
+ const h = await createHarness({
+ settings: { compaction: { keepRecentTokens: 1 } },
+ extensionFactories: [
+ (pi) => {
+ pi.on("session_shutdown", shutdown);
+ pi.on("session_before_compact", async (event) => {
+ started.resolve();
+ await released.promise;
+ return {
+ compaction: {
+ summary: "summary",
+ firstKeptEntryId: event.preparation.firstKeptEntryId,
+ tokensBefore: event.preparation.tokensBefore,
+ },
+ };
+ });
+ pi.on("session_before_tree", async () => {
+ started.resolve();
+ await released.promise;
+ });
+ },
+ ],
+ });
+ harness = h;
+ h.sessionManager.appendMessage(userMsg("first user"));
+ const targetId = h.sessionManager.appendMessage(assistantMsg("first assistant"));
+ h.sessionManager.appendMessage(userMsg("second user"));
+ h.sessionManager.appendMessage(assistantMsg("second assistant"));
+ h.session.agent.state.messages = h.sessionManager.buildSessionContext().messages;
+ const runner = h.session.extensionRunner;
+ const pending = operation === "compaction" ? h.session.compact() : h.session.navigateTree(targetId);
+ await started.promise;
+ try {
+ expect(h.session.isStreaming).toBe(false);
+ await expect(h.session.reload()).rejects.toThrow("Wait for compaction to finish before reloading.");
+ expect(shutdown).not.toHaveBeenCalled();
+ expect(h.session.extensionRunner).toBe(runner);
+ } finally {
+ released.resolve();
+ await pending;
+ }
+ expect(h.session.isIdle).toBe(true);
+ });
+
+ it("still reloads and replaces the runner when idle", async () => {
+ const shutdown = vi.fn();
+ const start = vi.fn();
+ const h = await createHarness({
+ extensionFactories: [
+ (pi) => {
+ pi.on("session_shutdown", shutdown);
+ pi.on("session_start", start);
+ },
+ ],
+ });
+ harness = h;
+ await h.session.bindExtensions({ onError: () => {} });
+ start.mockClear();
+ const runner = h.session.extensionRunner;
+ const reload = vi.spyOn(h.session.resourceLoader, "reload");
+
+ await h.session.reload();
+
+ expect(shutdown).toHaveBeenCalledOnce();
+ expect(reload).toHaveBeenCalledOnce();
+ expect(start).toHaveBeenCalledWith(expect.objectContaining({ reason: "reload" }), expect.anything());
+ expect(h.session.extensionRunner).not.toBe(runner);
+ expect(() => runner.getActiveTools()).toThrow("stale");
+ });
+});
From 2cfab7c269bb1fa12ab2704d05d18df3e5292e37 Mon Sep 17 00:00:00 2001
From: Sviatoslav Abakumov
Date: Wed, 4 Nov 2026 12:19:38 +0300
Subject: [PATCH 13/88] fix(coding-agent): render session tree search cursor
(#9157)
This PR adds the cursor block to the **Type to search** input in
**Session Tree**. This is similar to the search input in **Resume
Session**.
Before:
After:
---
.../interactive/components/tree-selector.ts | 30 +++++---
.../coding-agent/test/tree-selector.test.ts | 69 ++++++++++++++++++-
2 files changed, 88 insertions(+), 11 deletions(-)
diff --git a/packages/coding-agent/src/modes/interactive/components/tree-selector.ts b/packages/coding-agent/src/modes/interactive/components/tree-selector.ts
index 5bcdd846d..1a9d4b1f1 100644
--- a/packages/coding-agent/src/modes/interactive/components/tree-selector.ts
+++ b/packages/coding-agent/src/modes/interactive/components/tree-selector.ts
@@ -1,6 +1,7 @@
import {
type Component,
Container,
+ CURSOR_MARKER,
type Focusable,
getKeybindings,
Input,
@@ -1152,7 +1153,8 @@ class TreeList implements Component {
}
/** Component that displays the current search query */
-class SearchLine implements Component {
+class SearchLine implements Component, Focusable {
+ focused = false;
private treeList: TreeList;
constructor(treeList: TreeList) {
@@ -1162,11 +1164,16 @@ class SearchLine implements Component {
invalidate(): void {}
render(width: number): string[] {
+ if (width <= 0) return [""];
+
const query = this.treeList.getSearchQuery();
- if (query) {
- return [truncateToWidth(` ${theme.fg("muted", "Type to search:")} ${theme.fg("accent", query)}`, width)];
- }
- return [truncateToWidth(` ${theme.fg("muted", "Type to search:")}`, width)];
+ const line = query
+ ? ` ${theme.fg("muted", "Type to search:")} ${theme.fg("accent", query)}`
+ : ` ${theme.fg("muted", "Type to search:")} `;
+ if (!this.focused) return [truncateToWidth(line, width)];
+
+ const content = truncateToWidth(line, Math.max(0, width - 1));
+ return [`${content}${CURSOR_MARKER}\x1b[7m \x1b[27m`];
}
handleInput(_keyData: string): void {}
@@ -1325,20 +1332,21 @@ class LabelInput implements Component, Focusable {
*/
export class TreeSelectorComponent extends Container implements Focusable {
private treeList: TreeList;
+ private searchLine: SearchLine;
private labelInput: LabelInput | null = null;
private labelInputContainer: Container;
private treeContainer: Container;
private onLabelChangeCallback?: (entryId: string, label: string | undefined) => void;
public onCopy?: (text: string | undefined) => void;
- // Focusable implementation - propagate to labelInput when active for IME cursor positioning
+ // Focusable implementation - propagate to the active input for IME cursor positioning
private _focused = false;
get focused(): boolean {
return this._focused;
}
set focused(value: boolean) {
this._focused = value;
- // Propagate to labelInput when it's active
+ this.searchLine.focused = value && this.labelInput === null;
if (this.labelInput) {
this.labelInput.focused = value;
}
@@ -1364,6 +1372,7 @@ export class TreeSelectorComponent extends Container implements Focusable {
this.treeList.onCancel = onCancel;
this.treeList.onCopy = (text) => this.onCopy?.(text);
this.treeList.onLabelEdit = (entryId, currentLabel) => this.showLabelInput(entryId, currentLabel);
+ this.searchLine = new SearchLine(this.treeList);
this.treeContainer = new Container();
this.treeContainer.addChild(this.treeList);
@@ -1374,7 +1383,7 @@ export class TreeSelectorComponent extends Container implements Focusable {
this.addChild(new DynamicBorder());
this.addChild(new Text(theme.bold(" Session Tree"), 1, 0));
this.addChild(new TreeHelp());
- this.addChild(new SearchLine(this.treeList));
+ this.addChild(this.searchLine);
this.addChild(new DynamicBorder());
this.addChild(new Spacer(1));
this.addChild(this.treeContainer);
@@ -1396,8 +1405,8 @@ export class TreeSelectorComponent extends Container implements Focusable {
};
this.labelInput.onCancel = () => this.hideLabelInput();
- // Propagate current focused state to the new labelInput
- this.labelInput.focused = this._focused;
+ // Propagate current focused state
+ this.focused = this._focused;
this.treeContainer.clear();
this.labelInputContainer.clear();
@@ -1406,6 +1415,7 @@ export class TreeSelectorComponent extends Container implements Focusable {
private hideLabelInput(): void {
this.labelInput = null;
+ this.focused = this._focused;
this.labelInputContainer.clear();
this.treeContainer.clear();
this.treeContainer.addChild(this.treeList);
diff --git a/packages/coding-agent/test/tree-selector.test.ts b/packages/coding-agent/test/tree-selector.test.ts
index 579511b33..06d1ee541 100644
--- a/packages/coding-agent/test/tree-selector.test.ts
+++ b/packages/coding-agent/test/tree-selector.test.ts
@@ -1,5 +1,5 @@
import { stripVTControlCharacters } from "node:util";
-import { setKeybindings, visibleWidth } from "@earendil-works/pi-tui";
+import { CURSOR_MARKER, setKeybindings, visibleWidth } from "@earendil-works/pi-tui";
import { beforeAll, beforeEach, describe, expect, test } from "vitest";
import { KeybindingsManager } from "../src/core/keybindings.ts";
import type {
@@ -310,6 +310,73 @@ describe("TreeSelectorComponent", () => {
});
});
+ describe("search cursor", () => {
+ test("renders the cursor only while the selector is focused", () => {
+ const tree = buildTree([userMessage("user-1", null, "hello")]);
+ const selector = new TreeSelectorComponent(
+ tree,
+ "user-1",
+ 24,
+ () => {},
+ () => {},
+ );
+
+ expect(selector.render(80).join("\n")).not.toContain(CURSOR_MARKER);
+
+ selector.focused = true;
+ const focusedLines = selector.render(80);
+ const cursorLines = focusedLines.filter((line) => line.includes(CURSOR_MARKER));
+ expect(cursorLines).toHaveLength(1);
+ expect(stripVTControlCharacters(cursorLines[0]!.replace(CURSOR_MARKER, ""))).toBe(" Type to search: ");
+ expect(cursorLines[0]).toContain(`${CURSOR_MARKER}\x1b[7m \x1b[27m`);
+
+ selector.focused = false;
+ expect(selector.render(80).join("\n")).not.toContain(CURSOR_MARKER);
+ });
+
+ test("moves focus to the label input and restores it after cancellation", () => {
+ const tree = buildTree([userMessage("user-1", null, "hello")]);
+ const selector = new TreeSelectorComponent(
+ tree,
+ "user-1",
+ 24,
+ () => {},
+ () => {},
+ );
+ selector.focused = true;
+
+ selector.handleInput("L");
+ let rendered = selector.render(80);
+ let cursorLineIndex = rendered.findIndex((line) => line.includes(CURSOR_MARKER));
+ expect(cursorLineIndex).toBeGreaterThan(0);
+ expect(stripVTControlCharacters(rendered[cursorLineIndex - 1]!)).toContain("Label (empty to remove):");
+ expect(rendered.filter((line) => line.includes(CURSOR_MARKER))).toHaveLength(1);
+
+ selector.handleInput("\x1b");
+ rendered = selector.render(80);
+ cursorLineIndex = rendered.findIndex((line) => line.includes(CURSOR_MARKER));
+ expect(stripVTControlCharacters(rendered[cursorLineIndex]!)).toContain("Type to search:");
+ expect(rendered.filter((line) => line.includes(CURSOR_MARKER))).toHaveLength(1);
+ });
+
+ test("keeps the cursor within narrow terminal widths", () => {
+ const tree = buildTree([userMessage("user-1", null, "hello")]);
+ const selector = new TreeSelectorComponent(
+ tree,
+ "user-1",
+ 24,
+ () => {},
+ () => {},
+ );
+ selector.focused = true;
+ selector.handleInput("a long search query");
+
+ const cursorLine = selector.render(10).find((line) => line.includes(CURSOR_MARKER));
+ expect(cursorLine).toBeDefined();
+ expect(visibleWidth(cursorLine!)).toBeLessThanOrEqual(10);
+ });
+ });
+
describe("copy", () => {
test("copies the full selected message with ctrl+x", () => {
const message = `${"long message ".repeat(30)}\nsecond line`;
From 7a53c310a31bd7c04216add27df8bc9465145411 Mon Sep 17 00:00:00 2001
From: Gangqiang Yao
Date: Wed, 4 Nov 2026 17:21:16 +0800
Subject: [PATCH 14/88] fix(coding-agent): settle tool results before disposal
(#9126)
Fixes #9124.
Disposing a runtime during tool execution disconnects persistence before
the interrupted turn settles, leaving a stored assistant tool call
without its result. Await `session.abort()` before shutdown and disposal
so the interrupted tool result and final assistant message are
persisted.
This matches `teardownCurrent()`. `AgentSession.dispose()` remains
synchronous. Like the existing replacement path, the wait has no
timeout; handling tools that ignore cancellation is outside this change.
Related: #9986 concerns results for tool calls skipped during a
mid-session abort. This fix preserves the result of an already-running
tool during runtime shutdown.
Validation on main `f1b2e77f5`:
- The regression fails without the fix and passes with it, checking the
durable message order and `Operation aborted` result immediately after
`dispose()` returns, before awaiting the prompt.
- `npm run check`, offline build, and `./test.sh` pass.
---
.../src/core/agent-session-runtime.ts | 1 +
...24-runtime-dispose-tool-settlement.test.ts | 82 +++++++++++++++++++
2 files changed, 83 insertions(+)
create mode 100644 packages/coding-agent/test/suite/regressions/9124-runtime-dispose-tool-settlement.test.ts
diff --git a/packages/coding-agent/src/core/agent-session-runtime.ts b/packages/coding-agent/src/core/agent-session-runtime.ts
index 81c4264cf..cf06d5fce 100644
--- a/packages/coding-agent/src/core/agent-session-runtime.ts
+++ b/packages/coding-agent/src/core/agent-session-runtime.ts
@@ -402,6 +402,7 @@ export class AgentSessionRuntime {
}
async dispose(): Promise {
+ await this.session.abort();
await emitSessionShutdownEvent(this.session.extensionRunner, {
type: "session_shutdown",
reason: "quit",
diff --git a/packages/coding-agent/test/suite/regressions/9124-runtime-dispose-tool-settlement.test.ts b/packages/coding-agent/test/suite/regressions/9124-runtime-dispose-tool-settlement.test.ts
new file mode 100644
index 000000000..31f8713c2
--- /dev/null
+++ b/packages/coding-agent/test/suite/regressions/9124-runtime-dispose-tool-settlement.test.ts
@@ -0,0 +1,82 @@
+import type { AgentTool, AgentToolResult } from "@earendil-works/pi-agent-core";
+import { fauxAssistantMessage, fauxToolCall } from "@earendil-works/pi-ai";
+import { Type } from "typebox";
+import { afterEach, describe, expect, it } from "vitest";
+import {
+ AgentSessionRuntime,
+ type AgentSessionServices,
+ type CreateAgentSessionRuntimeFactory,
+} from "../../../src/core/agent-session-runtime.ts";
+import { createHarness, type Harness } from "../harness.ts";
+
+describe("regression #9124: runtime disposal during tool execution", () => {
+ const harnesses: Harness[] = [];
+
+ afterEach(() => {
+ while (harnesses.length > 0) {
+ harnesses.pop()?.cleanup();
+ }
+ });
+
+ async function createRuntime(tool: AgentTool): Promise<{ harness: Harness; runtime: AgentSessionRuntime }> {
+ const harness = await createHarness({ tools: [tool] });
+ harnesses.push(harness);
+ const services: AgentSessionServices = {
+ cwd: harness.tempDir,
+ agentDir: harness.tempDir,
+ modelRuntime: harness.session.modelRuntime,
+ settingsManager: harness.settingsManager,
+ resourceLoader: harness.session.resourceLoader,
+ diagnostics: [],
+ };
+ const unusedFactory: CreateAgentSessionRuntimeFactory = async () => {
+ throw new Error("runtime replacement is not used in this test");
+ };
+ return {
+ harness,
+ runtime: new AgentSessionRuntime(harness.session, services, unusedFactory),
+ };
+ }
+
+ it("persists the aborted tool turn before disconnecting the session", async () => {
+ let markToolStarted!: () => void;
+ const toolStarted = new Promise((resolve) => {
+ markToolStarted = resolve;
+ });
+ const tool: AgentTool = {
+ name: "wait",
+ label: "Wait",
+ description: "Wait until aborted",
+ parameters: Type.Object({}),
+ execute: (_toolCallId, _params, signal) =>
+ new Promise>((_resolve, reject) => {
+ markToolStarted();
+ signal?.addEventListener("abort", () => setTimeout(() => reject(new Error("Operation aborted")), 10), {
+ once: true,
+ });
+ }),
+ };
+ const { harness, runtime } = await createRuntime(tool);
+ harness.setResponses([fauxAssistantMessage(fauxToolCall("wait", {}), { stopReason: "toolUse" })]);
+
+ const prompt = runtime.session.prompt("run tool");
+ await toolStarted;
+ try {
+ await runtime.dispose();
+ // #9124: disposal itself must settle persistence; awaiting the prompt must not be necessary.
+ const messages = harness.sessionManager
+ .getEntries()
+ .filter((entry) => entry.type === "message")
+ .map((entry) => entry.message)
+ .filter((message) => message.role !== "system");
+ expect(messages.map((message) => message.role)).toEqual(["user", "assistant", "toolResult", "assistant"]);
+ expect(messages[2]).toMatchObject({
+ role: "toolResult",
+ content: [{ type: "text", text: "Operation aborted" }],
+ isError: true,
+ });
+ } finally {
+ await prompt;
+ }
+ });
+});
From f3a64af07b6f682d09f20dec5eaaab7a70c4e60a Mon Sep 17 00:00:00 2001
From: Sai Mallikarjun <106309863+Mallikarjun-0@users.noreply.github.com>
Date: Wed, 4 Nov 2026 14:53:35 +0530
Subject: [PATCH 15/88] fix(ai): preserve aborted stop reason during lazy setup
(#8635)
Fixes #8409
## Summary
- Pass the request abort signal through lazy stream setup wrappers.
- Report setup failures as aborted when the request signal is already
aborted.
- Added a regression test for aborting during a tool execution before
next auth setup.
---
packages/ai/src/api/lazy.ts | 39 +++++---
packages/ai/src/models.ts | 45 ++++++---
.../coding-agent/src/core/model-runtime.ts | 93 ++++++++++++-------
.../src/core/provider-composer.ts | 33 ++++---
.../8409-aborted-tool-auth-setup.test.ts | 53 +++++++++++
5 files changed, 187 insertions(+), 76 deletions(-)
create mode 100644 packages/coding-agent/test/suite/regressions/8409-aborted-tool-auth-setup.test.ts
diff --git a/packages/ai/src/api/lazy.ts b/packages/ai/src/api/lazy.ts
index d8db7440e..fad21164e 100644
--- a/packages/ai/src/api/lazy.ts
+++ b/packages/ai/src/api/lazy.ts
@@ -1,7 +1,12 @@
import type { Api, AssistantMessage, AssistantMessageEvent, Model, ProviderStreams } from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
-function createSetupErrorMessage(model: Model, error: unknown, timestamp: number): AssistantMessage {
+function createSetupErrorMessage(
+ model: Model,
+ error: unknown,
+ timestamp: number,
+ stopReason: "error" | "aborted",
+): AssistantMessage {
return {
role: "assistant",
content: [],
@@ -16,7 +21,7 @@ function createSetupErrorMessage(model: Model, error: unknown, timestamp: n
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
- stopReason: "error",
+ stopReason,
errorMessage: error instanceof Error ? error.message : String(error),
timestamp,
};
@@ -61,6 +66,7 @@ export function lazyStream(
model: Model,
setup: () => Promise>,
observer?: LazyStreamObserver,
+ signal?: AbortSignal,
): AssistantMessageEventStream {
const startedAt = Date.now();
const outer = new AssistantMessageEventStream();
@@ -74,8 +80,9 @@ export function lazyStream(
setup()
.then((inner) => forwardStream(outer, inner, observer, observeResult))
.catch((error) => {
- const message = createSetupErrorMessage(model, error, startedAt);
- const event: AssistantMessageEvent = { type: "error", reason: "error", error: message };
+ const reason = signal?.aborted ? "aborted" : "error";
+ const message = createSetupErrorMessage(model, error, startedAt, reason);
+ const event: AssistantMessageEvent = { type: "error", reason, error: message };
outer.push(event);
observer?.onEvent?.(event);
outer.end(message);
@@ -98,18 +105,28 @@ export interface LazyApiCapabilities {
export function lazyApi(load: () => Promise, capabilities?: LazyApiCapabilities): ProviderStreams {
const api: ProviderStreams = {
stream: (model, context, options) =>
- lazyStream(model, async () => (await load()).stream(model, context, options)),
+ lazyStream(model, async () => (await load()).stream(model, context, options), undefined, options?.signal),
streamSimple: (model, context, options) =>
- lazyStream(model, async () => (await load()).streamSimple(model, context, options)),
+ lazyStream(
+ model,
+ async () => (await load()).streamSimple(model, context, options),
+ undefined,
+ options?.signal,
+ ),
};
if (capabilities?.fetchDeferred) {
api.fetchDeferred = (model, handle, options) =>
- lazyStream(model, async () => {
- const implementation = await load();
- if (!implementation.fetchDeferred) throw new Error("API does not support deferred responses");
- return implementation.fetchDeferred(model, handle, options);
- });
+ lazyStream(
+ model,
+ async () => {
+ const implementation = await load();
+ if (!implementation.fetchDeferred) throw new Error("API does not support deferred responses");
+ return implementation.fetchDeferred(model, handle, options);
+ },
+ undefined,
+ options?.signal,
+ );
}
if (capabilities?.cancelDeferred) {
api.cancelDeferred = async (model, handle, options) => {
diff --git a/packages/ai/src/models.ts b/packages/ai/src/models.ts
index 3d26dcb5f..5842affce 100644
--- a/packages/ai/src/models.ts
+++ b/packages/ai/src/models.ts
@@ -915,6 +915,7 @@ class ModelsImpl implements MutableModels {
return provider.stream(requestModel, transcript, requestOptions as ApiStreamOptions);
},
observeModelRequest(model, options, false),
+ options?.signal,
);
}
@@ -936,6 +937,7 @@ class ModelsImpl implements MutableModels {
return provider.streamSimple(requestModel, transcript, requestOptions as SimpleStreamOptions);
},
observeModelRequest(model, options, false),
+ options?.signal,
);
}
@@ -963,6 +965,7 @@ class ModelsImpl implements MutableModels {
return provider.fetchDeferred(requestModel, handle, requestOptions as DeferredFetchOptions);
},
observeModelRequest(model, options, true),
+ options?.signal,
);
}
@@ -1108,12 +1111,18 @@ export function createProvider(input: CreateProviderOpti
const dispatch = (
model: Model,
run: (streams: ProviderStreams) => AssistantMessageEventStream,
+ signal?: AbortSignal,
): AssistantMessageEventStream => {
const streams = apiFor(model);
if (!streams) {
- return lazyStream(model, async () => {
- throw new ModelsError("stream", `Provider ${input.id} has no API implementation for "${model.api}"`);
- });
+ return lazyStream(
+ model,
+ async () => {
+ throw new ModelsError("stream", `Provider ${input.id} has no API implementation for "${model.api}"`);
+ },
+ undefined,
+ signal,
+ );
}
return run(streams);
};
@@ -1156,23 +1165,29 @@ export function createProvider(input: CreateProviderOpti
: undefined,
filterModels: input.filterModels,
filterAllModels: input.filterAllModels,
- stream: (model, context, options) => dispatch(model, (streams) => streams.stream(model, context, options)),
+ stream: (model, context, options) =>
+ dispatch(model, (streams) => streams.stream(model, context, options), options?.signal),
streamSimple: (model, context, options) =>
- dispatch(model, (streams) => streams.streamSimple(model, context, options)),
+ dispatch(model, (streams) => streams.streamSimple(model, context, options), options?.signal),
};
if (streams.some((entry) => entry.fetchDeferred !== undefined)) {
provider.fetchDeferred = (model, handle, options) =>
- lazyStream(model, async () => {
- const implementation = apiFor(model);
- if (!implementation?.fetchDeferred) {
- throw new ModelsError(
- "provider",
- `Provider ${input.id} does not support deferred responses for "${model.api}"`,
- );
- }
- return implementation.fetchDeferred(model, handle, options);
- });
+ lazyStream(
+ model,
+ async () => {
+ const implementation = apiFor(model);
+ if (!implementation?.fetchDeferred) {
+ throw new ModelsError(
+ "provider",
+ `Provider ${input.id} does not support deferred responses for "${model.api}"`,
+ );
+ }
+ return implementation.fetchDeferred(model, handle, options);
+ },
+ undefined,
+ options?.signal,
+ );
}
if (streams.some((entry) => entry.cancelDeferred !== undefined)) {
provider.cancelDeferred = async (model, handle, options) => {
diff --git a/packages/coding-agent/src/core/model-runtime.ts b/packages/coding-agent/src/core/model-runtime.ts
index 8e03f4d1f..6a5520d68 100644
--- a/packages/coding-agent/src/core/model-runtime.ts
+++ b/packages/coding-agent/src/core/model-runtime.ts
@@ -710,14 +710,19 @@ export class ModelRuntime implements Models {
options?: ModelsApiStreamOptions,
): AssistantMessageEventStream {
const transcript = normalizeContext(context);
- return lazyStream(model, async () => {
- assertChatModel(model);
- const prepared = await this.prepareRequest(
- model,
- options as (StreamOptions & ModelsRequestTransforms) | undefined,
- );
- return prepared.provider.stream(prepared.model, transcript, prepared.options as ApiStreamOptions);
- });
+ return lazyStream(
+ model,
+ async () => {
+ assertChatModel(model);
+ const prepared = await this.prepareRequest(
+ model,
+ options as (StreamOptions & ModelsRequestTransforms) | undefined,
+ );
+ return prepared.provider.stream(prepared.model, transcript, prepared.options as ApiStreamOptions);
+ },
+ undefined,
+ options?.signal,
+ );
}
complete(
@@ -733,27 +738,38 @@ export class ModelRuntime implements Models {
if (isVirtualModel(model)) {
// Requests outside the agent loop are routed here. Callers sized them before routing, so
// cap the output budget to the routed model.
- return lazyStream(model, async () => {
- const route = await this.resolveModel(model, transcript.messages, {
- reason: "direct",
- thinkingLevel: options?.reasoning ?? "off",
- signal: options?.signal,
- });
- const { maxTokens: limit } = route.model;
- const maxTokens = options?.maxTokens && limit > 0 ? Math.min(options.maxTokens, limit) : options?.maxTokens;
- const reasoning = route.thinkingLevel === "off" ? undefined : route.thinkingLevel;
- // Caller credentials were resolved for the virtual model's provider. Another provider
- // resolves its own, so they are not sent to the wrong vendor.
- const { apiKey, headers, env, ...rest } = options ?? {};
- const auth = route.model.provider === model.provider ? { apiKey, headers, env } : {};
- return this.streamSimple(route.model, context, { ...rest, ...auth, maxTokens, reasoning });
- });
+ return lazyStream(
+ model,
+ async () => {
+ const route = await this.resolveModel(model, transcript.messages, {
+ reason: "direct",
+ thinkingLevel: options?.reasoning ?? "off",
+ signal: options?.signal,
+ });
+ const { maxTokens: limit } = route.model;
+ const maxTokens =
+ options?.maxTokens && limit > 0 ? Math.min(options.maxTokens, limit) : options?.maxTokens;
+ const reasoning = route.thinkingLevel === "off" ? undefined : route.thinkingLevel;
+ // Caller credentials were resolved for the virtual model's provider. Another provider
+ // resolves its own, so they are not sent to the wrong vendor.
+ const { apiKey, headers, env, ...rest } = options ?? {};
+ const auth = route.model.provider === model.provider ? { apiKey, headers, env } : {};
+ return this.streamSimple(route.model, context, { ...rest, ...auth, maxTokens, reasoning });
+ },
+ undefined,
+ options?.signal,
+ );
}
- return lazyStream(model, async () => {
- assertChatModel(model);
- const prepared = await this.prepareRequest(model, options);
- return prepared.provider.streamSimple(prepared.model, transcript, prepared.options as SimpleStreamOptions);
- });
+ return lazyStream(
+ model,
+ async () => {
+ assertChatModel(model);
+ const prepared = await this.prepareRequest(model, options);
+ return prepared.provider.streamSimple(prepared.model, transcript, prepared.options as SimpleStreamOptions);
+ },
+ undefined,
+ options?.signal,
+ );
}
completeSimple(model: Model, context: Context, options?: ModelsSimpleStreamOptions): Promise {
@@ -765,14 +781,19 @@ export class ModelRuntime implements Models {
handle: DeferredHandle,
options?: ModelsDeferredFetchOptions,
): AssistantMessageEventStream {
- return lazyStream(model, async () => {
- assertChatModel(model);
- const prepared = await this.prepareRequest(model, options);
- if (!prepared.provider.fetchDeferred) {
- throw new ModelsError("provider", `Provider ${model.provider} does not support deferred responses`);
- }
- return prepared.provider.fetchDeferred(prepared.model, handle, prepared.options as DeferredFetchOptions);
- });
+ return lazyStream(
+ model,
+ async () => {
+ assertChatModel(model);
+ const prepared = await this.prepareRequest(model, options);
+ if (!prepared.provider.fetchDeferred) {
+ throw new ModelsError("provider", `Provider ${model.provider} does not support deferred responses`);
+ }
+ return prepared.provider.fetchDeferred(prepared.model, handle, prepared.options as DeferredFetchOptions);
+ },
+ undefined,
+ options?.signal,
+ );
}
async fetchDeferred(
diff --git a/packages/coding-agent/src/core/provider-composer.ts b/packages/coding-agent/src/core/provider-composer.ts
index f7f5c4c59..50940ec6a 100644
--- a/packages/coding-agent/src/core/provider-composer.ts
+++ b/packages/coding-agent/src/core/provider-composer.ts
@@ -591,21 +591,26 @@ export function composeModelProvider(
options: StreamOptions | undefined,
simple: boolean,
): AssistantMessageEventStream =>
- lazyStream(model, async () => {
- if (extension?.streamSimple && model.api === extension.api) {
- return extension.streamSimple(model, context, options as SimpleStreamOptions);
- }
- if (base && supportsBaseApi(model)) {
+ lazyStream(
+ model,
+ async () => {
+ if (extension?.streamSimple && model.api === extension.api) {
+ return extension.streamSimple(model, context, options as SimpleStreamOptions);
+ }
+ if (base && supportsBaseApi(model)) {
+ return simple
+ ? base.streamSimple(model, context, options as SimpleStreamOptions)
+ : base.stream(model, context, options);
+ }
+ const api = getApiProvider(model.api);
+ if (!api) throw new Error(`No API provider registered for api: ${model.api}`);
return simple
- ? base.streamSimple(model, context, options as SimpleStreamOptions)
- : base.stream(model, context, options);
- }
- const api = getApiProvider(model.api);
- if (!api) throw new Error(`No API provider registered for api: ${model.api}`);
- return simple
- ? api.streamSimple(model, context, options as SimpleStreamOptions)
- : api.stream(model, context, options);
- });
+ ? api.streamSimple(model, context, options as SimpleStreamOptions)
+ : api.stream(model, context, options);
+ },
+ undefined,
+ options?.signal,
+ );
const provider: Provider = {
id: providerId,
diff --git a/packages/coding-agent/test/suite/regressions/8409-aborted-tool-auth-setup.test.ts b/packages/coding-agent/test/suite/regressions/8409-aborted-tool-auth-setup.test.ts
new file mode 100644
index 000000000..b930bb7e7
--- /dev/null
+++ b/packages/coding-agent/test/suite/regressions/8409-aborted-tool-auth-setup.test.ts
@@ -0,0 +1,53 @@
+import type { AgentTool } from "@earendil-works/pi-agent-core";
+import { fauxAssistantMessage, fauxToolCall } from "@earendil-works/pi-ai";
+import { Type } from "typebox";
+import { afterEach, describe, expect, it } from "vitest";
+import { createHarness, type Harness } from "../harness.ts";
+
+describe("issue #8409 aborted tool auth setup", () => {
+ let harness: Harness | undefined;
+
+ afterEach(() => {
+ harness?.cleanup();
+ harness = undefined;
+ });
+
+ it("ends the turn as aborted when auth setup sees an already-aborted signal", async () => {
+ let markToolStarted = () => {};
+ const toolStarted = new Promise((resolve) => {
+ markToolStarted = resolve;
+ });
+ const blockingTool: AgentTool = {
+ name: "block",
+ label: "Block",
+ description: "Blocks until aborted",
+ parameters: Type.Object({}),
+ execute: async (_toolCallId, _params, signal) => {
+ markToolStarted();
+ await new Promise((resolve) => {
+ if (signal?.aborted) resolve();
+ else signal?.addEventListener("abort", () => resolve(), { once: true });
+ });
+ return { content: [{ type: "text", text: "tool aborted" }], details: {} };
+ },
+ };
+ harness = await createHarness({ tools: [blockingTool] });
+ const modelRuntime = harness.session.modelRuntime;
+ harness.session.agent.streamFunction = (model, context, options) =>
+ modelRuntime.streamSimple(model, context, options);
+ harness.setResponses([fauxAssistantMessage(fauxToolCall("block", {}), { stopReason: "toolUse" })]);
+
+ const prompt = harness.session.prompt("run the blocking tool");
+ await toolStarted;
+ await harness.session.abort();
+ await prompt;
+
+ const finalAssistant = harness.session.messages
+ .slice()
+ .reverse()
+ .find((message) => message.role === "assistant");
+ expect(finalAssistant?.role).toBe("assistant");
+ if (finalAssistant?.role !== "assistant") throw new Error("Expected final assistant message");
+ expect(finalAssistant.stopReason).toBe("aborted");
+ });
+});
From a63c4ec5c89ce30f8f144c2c1eba7842ffab974b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E6=A2=A7=E6=A1=90=E9=9B=A8?=
<147830929+wutongyuonce@users.noreply.github.com>
Date: Wed, 4 Nov 2026 17:24:43 +0800
Subject: [PATCH 16/88] fix(coding-agent): clear delivered image-only queue
entries (#8612)
## Summary
- remove delivered queue entries even when the user message has no text
- keep image-only steering and follow-up pending counts in sync with the
agent queue
- cover image-only steering delivery with a regression test
## Testing
- `npm run check`
- `./test.sh`
Fixes #8581
---
.../coding-agent/src/core/agent-session.ts | 22 +++++++-------
.../test/suite/agent-session-queue.test.ts | 29 +++++++++++++++++++
2 files changed, 39 insertions(+), 12 deletions(-)
diff --git a/packages/coding-agent/src/core/agent-session.ts b/packages/coding-agent/src/core/agent-session.ts
index 0f5aa1105..e4f53004b 100644
--- a/packages/coding-agent/src/core/agent-session.ts
+++ b/packages/coding-agent/src/core/agent-session.ts
@@ -1141,19 +1141,17 @@ export class AgentSession {
if (event.type === "message_start" && event.message.role === "user") {
this._overflowRecoveryAttempted = false;
const messageText = contentText(event.message.content, "");
- if (messageText) {
- // Check steering queue first
- const steeringIndex = this._steeringMessages.indexOf(messageText);
- if (steeringIndex !== -1) {
- this._steeringMessages.splice(steeringIndex, 1);
+ // Check steering queue first
+ const steeringIndex = this._steeringMessages.indexOf(messageText);
+ if (steeringIndex !== -1) {
+ this._steeringMessages.splice(steeringIndex, 1);
+ this._emitQueueUpdate();
+ } else {
+ // Check follow-up queue
+ const followUpIndex = this._followUpMessages.indexOf(messageText);
+ if (followUpIndex !== -1) {
+ this._followUpMessages.splice(followUpIndex, 1);
this._emitQueueUpdate();
- } else {
- // Check follow-up queue
- const followUpIndex = this._followUpMessages.indexOf(messageText);
- if (followUpIndex !== -1) {
- this._followUpMessages.splice(followUpIndex, 1);
- this._emitQueueUpdate();
- }
}
}
// Normalize queued images at delivery, before events and persistence see the message,
diff --git a/packages/coding-agent/test/suite/agent-session-queue.test.ts b/packages/coding-agent/test/suite/agent-session-queue.test.ts
index d198cbd2d..39a4470d2 100644
--- a/packages/coding-agent/test/suite/agent-session-queue.test.ts
+++ b/packages/coding-agent/test/suite/agent-session-queue.test.ts
@@ -437,6 +437,35 @@ describe("AgentSession queue characterization", () => {
expect(harness.session.pendingMessageCount).toBe(0);
});
+ it("removes an image-only steering message after delivery", async () => {
+ const waiting = await createWaitingHarness();
+ const { harness, waitForToolStart, promptPromise, releaseToolExecution } = waiting;
+ harnesses.push(harness);
+ let sawImage = false;
+
+ harness.setResponses([
+ fauxAssistantMessage(fauxToolCall("wait", {}), { stopReason: "toolUse" }),
+ (context) => {
+ const user = [...context.messages].reverse().find((message) => message.role === "user");
+ sawImage =
+ user?.role === "user" &&
+ typeof user.content !== "string" &&
+ user.content.some((part) => part.type === "image");
+ return fauxAssistantMessage("done");
+ },
+ ]);
+
+ await waitForToolStart;
+ await harness.session.steer("", [{ type: "image", mimeType: "image/png", data: "ZmFrZQ==" }]);
+ expect(harness.session.pendingMessageCount).toBe(1);
+ releaseToolExecution();
+ await promptPromise;
+
+ expect(sawImage).toBe(true);
+ expect(harness.session.agent.hasQueuedMessages()).toBe(false);
+ expect(harness.session.pendingMessageCount).toBe(0);
+ });
+
it("throws when queueing an extension command with steer", async () => {
const harness = await createHarness({
extensionFactories: [
From c5d3978350c7d32db5d8eb28400d4fa851624703 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 10:28:59 +0100
Subject: [PATCH 17/88] fix(ai): send session-id instead of session_id on
OpenAI Responses (#11793)
The `openai-responses` client sent the session-affinity header as
`session_id`. Proxies such as nginx with `underscores_in_headers off`
silently drop headers with underscores, and Envoy can be configured to
reject them. Codex now sends `session-id`, and the
`openai-codex-responses` path already followed it (#4967).
With the default `openai` session-affinity format, `openai-responses`
now sends `session-id`. `x-client-request-id` and `prompt_cache_key` are
unchanged. The `openai-nosession` and `openrouter` formats behave as
before.
The compat schema description for `sessionAffinityFormat`
(`openai-responses`) and `docs/models-configuration.md` are updated to
match.
Not changed: `openai-completions` still sends `session_id`. I did not
check whether xAI or other backends on the Responses API depend on the
underscored name. Their `prompt_cache_key` is still sent, so prompt
caching should not depend on the header.
Testing:
- `test/openai-responses-compat.test.ts` now captures `session-id`. It
also checks that `session_id` is no longer sent, and the
explicit-override test uses the new name. `test/xai-responses.test.ts`
expects `session-id` and no `session_id`.
- Without the source change, these tests fail (3 failures); with it,
both files pass (50 tests).
- `npm run check` passes.
- The full `packages/ai` suite passes, except
`generate-models-strict.test.ts`. That failure is unrelated: the test
depends on live model catalog data.
Fixes #11639
Requested by @badlogic
Co-authored-by: triagent
---
packages/ai/src/api/openai-responses.ts | 2 +-
packages/ai/src/providers/compat-schema.ts | 2 +-
packages/ai/test/openai-responses-compat.test.ts | 10 +++++++---
packages/ai/test/xai-responses.test.ts | 3 ++-
packages/coding-agent/docs/models-configuration.md | 2 +-
5 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/packages/ai/src/api/openai-responses.ts b/packages/ai/src/api/openai-responses.ts
index c29e6adfb..8d576a702 100644
--- a/packages/ai/src/api/openai-responses.ts
+++ b/packages/ai/src/api/openai-responses.ts
@@ -316,7 +316,7 @@ function createClient(
headers["x-session-id"] = sessionId;
} else {
if (compat.sessionAffinityFormat === "openai") {
- headers.session_id = sessionId;
+ headers["session-id"] = sessionId;
}
headers["x-client-request-id"] = sessionId;
}
diff --git a/packages/ai/src/providers/compat-schema.ts b/packages/ai/src/providers/compat-schema.ts
index 792b7a1b7..88defded8 100644
--- a/packages/ai/src/providers/compat-schema.ts
+++ b/packages/ai/src/providers/compat-schema.ts
@@ -364,7 +364,7 @@ const OpenAIResponsesCompatProperties = {
}),
sessionAffinityFormat: optionalSessionAffinityFormat({
description:
- "Session-affinity header format. openai sends session_id and x-client-request-id; openai-nosession sends x-client-request-id; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected.",
+ "Session-affinity header format. openai sends session-id and x-client-request-id; openai-nosession sends x-client-request-id; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected.",
}),
supportsLongCacheRetention: optionalCompatBoolean({
description:
diff --git a/packages/ai/test/openai-responses-compat.test.ts b/packages/ai/test/openai-responses-compat.test.ts
index ec7783313..2423da236 100644
--- a/packages/ai/test/openai-responses-compat.test.ts
+++ b/packages/ai/test/openai-responses-compat.test.ts
@@ -35,14 +35,17 @@ async function captureOpenAIResponseHeaders(
sessionId: string | null;
clientRequestId: string | null;
xSessionId: string | null;
+ legacySessionId: string | null;
}> {
const captured = {
sessionId: null as string | null,
clientRequestId: null as string | null,
xSessionId: null as string | null,
+ legacySessionId: null as string | null,
};
vi.spyOn(globalThis, "fetch").mockImplementation(async (_input, init) => {
- captured.sessionId = getHeader(init?.headers, "session_id");
+ captured.sessionId = getHeader(init?.headers, "session-id");
+ captured.legacySessionId = getHeader(init?.headers, "session_id");
captured.clientRequestId = getHeader(init?.headers, "x-client-request-id");
captured.xSessionId = getHeader(init?.headers, "x-session-id");
return new Response("data: [DONE]\n\n", {
@@ -295,6 +298,7 @@ describe("openai-responses provider defaults", () => {
const captured = await captureOpenAIResponseHeaders({ sessionId: "session-123" });
expect(captured.sessionId).toBe("session-123");
+ expect(captured.legacySessionId).toBeNull();
expect(captured.clientRequestId).toBe("session-123");
});
@@ -436,7 +440,7 @@ describe("openai-responses provider defaults", () => {
expect(capturedPayload?.prompt_cache_key).toBe("session-opencode");
});
- it("can omit OpenAI session_id header while preserving other affinity data", async () => {
+ it("can omit OpenAI session-id header while preserving other affinity data", async () => {
const proxyModel: Model<"openai-responses"> = {
...getModel("openai", "gpt-5.4"),
provider: "opencode",
@@ -463,7 +467,7 @@ describe("openai-responses provider defaults", () => {
const captured = await captureOpenAIResponseHeaders({
sessionId: "session-123",
headers: {
- session_id: "override-session",
+ "session-id": "override-session",
"x-client-request-id": "override-request",
},
});
diff --git a/packages/ai/test/xai-responses.test.ts b/packages/ai/test/xai-responses.test.ts
index 228ebe468..ca5084866 100644
--- a/packages/ai/test/xai-responses.test.ts
+++ b/packages/ai/test/xai-responses.test.ts
@@ -179,7 +179,8 @@ describe("xAI Responses provider", () => {
expect(captured.url).toBe("https://api.x.ai/v1/responses");
expect(captured.headers.get("authorization")).toBe("Bearer xai-test-token");
expect(captured.headers.get("user-agent")).toBe(PI_USER_AGENT);
- expect(captured.headers.get("session_id")).toBe("pi-session-123");
+ expect(captured.headers.get("session-id")).toBe("pi-session-123");
+ expect(captured.headers.get("session_id")).toBeNull();
expect(captured.body).toMatchObject({
model: "grok-4.5",
store: false,
diff --git a/packages/coding-agent/docs/models-configuration.md b/packages/coding-agent/docs/models-configuration.md
index 2a25bdb48..830f9bfdc 100644
--- a/packages/coding-agent/docs/models-configuration.md
+++ b/packages/coding-agent/docs/models-configuration.md
@@ -146,7 +146,7 @@ Also applies to `azure-openai-responses` and `openai-codex-responses`.
|---|---|---|---|
| `supportsDeveloperRole` | boolean | `true` | Whether the provider supports the developer role instead of system. |
| `supportsMidConvoSystemMessages` | boolean | `false` | Whether the exact model accepts developer or system messages after the conversation has started. When false, later system messages are folded into the leading system message. The generated catalog enables this for verified models. |
-| `sessionAffinityFormat` | `"openai" \| "openai-nosession" \| "openrouter"` | | Session-affinity header format. openai sends session_id and x-client-request-id; openai-nosession sends x-client-request-id; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected. |
+| `sessionAffinityFormat` | `"openai" \| "openai-nosession" \| "openrouter"` | | Session-affinity header format. openai sends session-id and x-client-request-id; openai-nosession sends x-client-request-id; openrouter sends x-session-id. Does not affect prompt_cache_key. Default: auto-detected. |
| `supportsLongCacheRetention` | boolean | `true` | Whether the provider supports long prompt cache retention. This uses prompt_cache_options.ttl: "30m" on GPT-5.6+ and prompt_cache_retention: "24h" on earlier models. |
| `supportsStrictMode` | boolean | | Whether the provider supports strict JSON-schema function tools. Defaults are API-specific; generated OpenAI models enable it explicitly. |
| `supportsOpenAIGrammarTools` | boolean | `false` | Whether to emit OpenAI custom tools with Lark or regex grammar formats. When false, grammar-constrained tools fall back to normal function tools. The generated catalog enables this for capable models. |
From c13f6acfbe0824b090851274bffa5aeb00c346d3 Mon Sep 17 00:00:00 2001
From: "robopipi[bot]" <336283572+robopipi[bot]@users.noreply.github.com>
Date: Wed, 4 Nov 2026 10:45:55 +0100
Subject: [PATCH 18/88] Highlight .dart, .nix, and .groovy files in read/write
views (#11790)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The `read` and `write` tool views find a file's language from its
extension with `getLanguageFromPath()`. That function's extension table
had no entries for Dart, Nix, or Groovy. Their highlight.js grammars are
loaded at startup, but the lookup returned `undefined`, so these files
were shown as plain uncolored text.
This change adds `dart → dart`, `nix → nix`, `groovy → groovy`, and
`gradle → groovy` to the table in
`packages/coding-agent/src/modes/interactive/theme/theme.ts`.
HTML export keeps its own separate extension map and a vendored
highlight bundle without a Dart grammar. This change does not touch
either.
## Testing
- Added a test to `test/syntax-highlight.test.ts` that checks the new
extension mappings. It fails without the fix and passes with it.
- `node ../../node_modules/vitest/dist/cli.js --run
test/syntax-highlight.test.ts` passes.
- Before, `getLanguageFromPath("a.dart")` (and `.nix`, `.groovy`,
`build.gradle`) returned `undefined` from the built dist. After, they
return `dart`, `nix`, `groovy`, and `groovy`.
- `npm run check` reports 6 type errors in `packages/ai/test` (model ID
types). Main without this change has the same 6 errors.
Fixes #11643
Requested by @badlogic
Co-authored-by: triagent
---
.../src/modes/interactive/theme/theme.ts | 4 ++++
.../coding-agent/test/syntax-highlight.test.ts | 16 +++++++++++++++-
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/packages/coding-agent/src/modes/interactive/theme/theme.ts b/packages/coding-agent/src/modes/interactive/theme/theme.ts
index 922b23d9f..8b43443f3 100644
--- a/packages/coding-agent/src/modes/interactive/theme/theme.ts
+++ b/packages/coding-agent/src/modes/interactive/theme/theme.ts
@@ -1008,6 +1008,10 @@ export function getLanguageFromPath(filePath: string): string | undefined {
java: "java",
kt: "kotlin",
swift: "swift",
+ dart: "dart",
+ groovy: "groovy",
+ gradle: "groovy",
+ nix: "nix",
c: "c",
h: "c",
cpp: "cpp",
diff --git a/packages/coding-agent/test/syntax-highlight.test.ts b/packages/coding-agent/test/syntax-highlight.test.ts
index 17fe3b2e4..f4ff51c2b 100644
--- a/packages/coding-agent/test/syntax-highlight.test.ts
+++ b/packages/coding-agent/test/syntax-highlight.test.ts
@@ -1,6 +1,12 @@
import { resetCapabilitiesCache, setCapabilities } from "@earendil-works/pi-tui";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
-import { getMarkdownTheme, highlightCode, initTheme, theme } from "../src/modes/interactive/theme/theme.ts";
+import {
+ getLanguageFromPath,
+ getMarkdownTheme,
+ highlightCode,
+ initTheme,
+ theme,
+} from "../src/modes/interactive/theme/theme.ts";
import {
highlight,
loadAllHighlightLanguages,
@@ -41,6 +47,14 @@ describe("syntax highlight renderer", () => {
expect(supportsLanguage("ada")).toBe(true);
});
+ it("maps file extensions of eagerly loaded languages to their language", () => {
+ expect(getLanguageFromPath("lib/main.dart")).toBe("dart");
+ expect(getLanguageFromPath("flake.nix")).toBe("nix");
+ expect(getLanguageFromPath("script.groovy")).toBe("groovy");
+ expect(getLanguageFromPath("build.gradle")).toBe("groovy");
+ expect(getLanguageFromPath("src/Main.kt")).toBe("kotlin");
+ });
+
it("renders highlighted spans with the provided theme", () => {
const rendered = renderHighlightedHtml('const value', {
keyword: (text) => `[keyword:${text}]`,
From ee962fb4db75fdc61e3745ceba5197d4485c1b79 Mon Sep 17 00:00:00 2001
From: Giannis Papakostas
Date: Wed, 4 Nov 2026 11:53:52 +0200
Subject: [PATCH 19/88] fix(plan-mode): make progress tracking robust and
tolerant (#7950)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
## What
Hardens the plan-mode example extension's step-progress tracking.
Previously, completion markers were only recognized in the exact
`[DONE:n]` form inside visible text blocks, so steps frequently never
got checked off during execution.
## Changes
- **`getTextContent`** now reads `thinking` blocks as well as text
blocks, so `[DONE:n]` tags emitted during reasoning are honored.
- **Marker tolerance** (`extractDoneSteps`): accepts `[DONE: 1]`, `[DONE
1]`, `Done: n`, and prose phrasings like "Step n done / is complete /
finished" — all adjacency-anchored so negations ("Step 9 is not done")
never match.
- **Whole-plan fallback** (`allStepsComplete`): if the run's final
message says the plan is done (e.g. "All steps complete"), all remaining
steps are marked at `agent_end`.
- **Plan extraction dedup**: only the last `Plan:` section is extracted,
so a plan drafted in a thinking block no longer duplicates the steps
from the text block.
- **No more truncation**: step text is kept in full (previously cut at
50 chars); the widget truncates for display only. The execution prompt
now shows the complete step text.
- **No more dropped steps**: steps whose text starts with a backtick
(e.g. `1. \`index.ts\` — …`) are no longer skipped.
- **Widget refresh** happens on every turn instead of only when a marker
matched.
- **Plan-complete card** is delivered with `nextTurn` instead of
triggering an extra LLM turn.
## Tests
`packages/coding-agent/test/plan-mode-utils.test.ts` updated for the new
behavior (truncation, backtick steps, marker-count semantics) and
extended with coverage for the new cases (marker variants, prose
phrasings, negation safety, thinking-draft dedup, markdown titles,
whole-plan completion).
All 43 tests in `plan-mode-utils.test.ts` +
`plan-mode-extension.test.ts` pass (`npx vitest run`).
Note: the repo-wide type-check currently fails on `main` with
pre-existing `ModelCatalog`/`never` errors unrelated to this change;
this branch adds none (same count with and without the diff).
---
.../examples/extensions/plan-mode/index.ts | 36 +++++++-----
.../examples/extensions/plan-mode/utils.ts | 58 ++++++++++++++-----
.../coding-agent/test/plan-mode-utils.test.ts | 57 +++++++++++++++---
3 files changed, 113 insertions(+), 38 deletions(-)
diff --git a/packages/coding-agent/examples/extensions/plan-mode/index.ts b/packages/coding-agent/examples/extensions/plan-mode/index.ts
index bbeb7ab13..8d718b613 100644
--- a/packages/coding-agent/examples/extensions/plan-mode/index.ts
+++ b/packages/coding-agent/examples/extensions/plan-mode/index.ts
@@ -16,7 +16,7 @@ import type { AgentMessage } from "@earendil-works/pi-agent-core";
import type { AssistantMessage, TextContent } from "@earendil-works/pi-ai";
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
import { Key } from "@earendil-works/pi-tui";
-import { extractTodoItems, isSafeCommand, markCompletedSteps, type TodoItem } from "./utils.ts";
+import { allStepsComplete, extractTodoItems, isSafeCommand, markCompletedSteps, type TodoItem } from "./utils.ts";
// Tools
const PLAN_MODE_TOOLS = ["read", "bash", "grep", "find", "ls", "questionnaire"];
@@ -36,11 +36,16 @@ function isAssistantMessage(m: AgentMessage): m is AssistantMessage {
return m.role === "assistant" && Array.isArray(m.content);
}
-// Extract text content from an assistant message
+// Extract text content from an assistant message.
+// Includes thinking blocks: with reasoning-max models the [DONE:n] tags and
+// plan drafts usually land in thinking, not the visible text.
function getTextContent(message: AssistantMessage): string {
return message.content
- .filter((block): block is TextContent => block.type === "text")
- .map((block) => block.text)
+ .map((block) => {
+ if (block.type === "text") return block.text;
+ if (block.type === "thinking") return (block as { thinking?: string }).thinking ?? "";
+ return "";
+ })
.join("\n");
}
@@ -70,12 +75,11 @@ export default function planModeExtension(pi: ExtensionAPI): void {
// Widget showing todo list
if (executionMode && todoItems.length > 0) {
const lines = todoItems.map((item) => {
+ const text = item.text.length > 60 ? `${item.text.slice(0, 57)}...` : item.text;
if (item.completed) {
- return (
- ctx.ui.theme.fg("success", "☑ ") + ctx.ui.theme.fg("muted", ctx.ui.theme.strikethrough(item.text))
- );
+ return ctx.ui.theme.fg("success", "☑ ") + ctx.ui.theme.fg("muted", ctx.ui.theme.strikethrough(text));
}
- return `${ctx.ui.theme.fg("muted", "☐ ")}${item.text}`;
+ return `${ctx.ui.theme.fg("muted", "☐ ")}${text}`;
});
ctx.ui.setWidget("plan-todos", lines);
} else {
@@ -255,10 +259,9 @@ After completing a step, include a [DONE:n] tag in your response.`,
if (!executionMode || todoItems.length === 0) return;
if (!isAssistantMessage(event.message)) return;
- const text = getTextContent(event.message);
- if (markCompletedSteps(text, todoItems) > 0) {
- updateStatus(ctx);
- }
+ markCompletedSteps(getTextContent(event.message), todoItems);
+ // Always refresh: cheap, keeps the widget consistent even when nothing matched.
+ updateStatus(ctx);
persistState();
});
@@ -266,11 +269,18 @@ After completing a step, include a [DONE:n] tag in your response.`,
pi.on("agent_end", async (event, ctx) => {
// Check if execution is complete
if (executionMode && todoItems.length > 0) {
+ // Fallback: the model wrapped up without [DONE:n] tags ("All steps complete").
+ const lastAssistant = [...event.messages].reverse().find(isAssistantMessage);
+ if (lastAssistant && allStepsComplete(getTextContent(lastAssistant))) {
+ for (const t of todoItems) t.completed = true;
+ }
if (todoItems.every((t) => t.completed)) {
const completedList = todoItems.map((t) => `~~${t.text}~~`).join("\n");
pi.sendMessage(
{ customType: "plan-complete", content: `**Plan Complete!** ✓\n\n${completedList}`, display: true },
- { triggerTurn: false },
+ // nextTurn: delivered with the next user prompt instead of triggering an
+ // extra (expensive, reasoning-max) LLM turn to acknowledge the card.
+ { deliverAs: "nextTurn" },
);
executionMode = false;
todoItems = [];
diff --git a/packages/coding-agent/examples/extensions/plan-mode/utils.ts b/packages/coding-agent/examples/extensions/plan-mode/utils.ts
index 62123f9e3..dc3c817aa 100644
--- a/packages/coding-agent/examples/extensions/plan-mode/utils.ts
+++ b/packages/coding-agent/examples/extensions/plan-mode/utils.ts
@@ -120,26 +120,30 @@ export function cleanStepText(text: string): string {
if (cleaned.length > 0) {
cleaned = cleaned.charAt(0).toUpperCase() + cleaned.slice(1);
}
- if (cleaned.length > 50) {
- cleaned = `${cleaned.slice(0, 47)}...`;
- }
+ // Keep full text: the exec message and widget display need the whole step.
+ // The widget truncates for display (see updateStatus in index.ts).
return cleaned;
}
export function extractTodoItems(message: string): TodoItem[] {
const items: TodoItem[] = [];
- const headerMatch = message.match(/\*{0,2}Plan:\*{0,2}\s*\n/i);
- if (!headerMatch) return items;
-
- const planSection = message.slice(message.indexOf(headerMatch[0]) + headerMatch[0].length);
- const numberedPattern = /^\s*(\d+)[.)]\s+\*{0,2}([^*\n]+)/gm;
+ // Take the LAST "Plan:" section: since getTextContent includes thinking blocks,
+ // a draft in the reasoning block can precede the user-visible plan in the text
+ // block — the text block comes last in content order.
+ const headers = [...message.matchAll(/\*{0,2}Plan:\*{0,2}\s*\n/gi)];
+ if (headers.length === 0) return items;
+ const lastHeader = headers[headers.length - 1];
+ const planSection = message.slice((lastHeader.index ?? 0) + lastHeader[0].length);
+ // Full-line capture: step titles may contain bold/italic/backticks ("1. **Step** \`x\` …").
+ // cleanStepText strips the markdown; / and - items are still skipped below.
+ const numberedPattern = /^\s*(\d+)[.)]\s+(.+)$/gm;
for (const match of planSection.matchAll(numberedPattern)) {
const text = match[2]
.trim()
.replace(/\*{1,2}$/, "")
.trim();
- if (text.length > 5 && !text.startsWith("`") && !text.startsWith("/") && !text.startsWith("-")) {
+ if (text.length > 5 && !text.startsWith("/") && !text.startsWith("-")) {
const cleaned = cleanStepText(text);
if (cleaned.length > 3) {
items.push({ step: items.length + 1, text: cleaned, completed: false });
@@ -151,18 +155,40 @@ export function extractTodoItems(message: string): TodoItem[] {
export function extractDoneSteps(message: string): number[] {
const steps: number[] = [];
- for (const match of message.matchAll(/\[DONE:(\d+)\]/gi)) {
- const step = Number(match[1]);
- if (Number.isFinite(step)) steps.push(step);
+ // Canonical and bracket variants: [DONE:1], [DONE: 1], [DONE 1], [Done:1]
+ for (const match of message.matchAll(/\[DONE\s*:?\s*(\d+)\]/gi)) {
+ steps.push(Number(match[1]));
}
- return steps;
+ // "Done: 1" without brackets
+ for (const match of message.matchAll(/\bDone\s*:\s*(\d+)\b/gi)) {
+ steps.push(Number(match[1]));
+ }
+ // "Step 1 done", "Step 2 is complete", "Step 3 finished" — adjacency-anchored,
+ // so "Step 9 is not done" / "Step 9 incomplete" never match.
+ for (const match of message.matchAll(/\bStep\s*(\d+)\s+(?:is\s+)?(?:done|complete|completed|finished)\b/gi)) {
+ steps.push(Number(match[1]));
+ }
+ return [...new Set(steps)];
}
export function markCompletedSteps(text: string, items: TodoItem[]): number {
const doneSteps = extractDoneSteps(text);
+ let marked = 0;
for (const step of doneSteps) {
- const item = items.find((t) => t.step === step);
- if (item) item.completed = true;
+ const item = items.find((t) => t.step === step && !t.completed);
+ if (item) {
+ item.completed = true;
+ marked++;
+ }
}
- return doneSteps.length;
+ return marked;
+}
+
+// Whole-plan completion phrasing, used as a fallback when the model summarizes
+// without [DONE:n] tags. Only consulted on the run's final message (agent_end).
+export function allStepsComplete(message: string): boolean {
+ return (
+ /(?:all|every)\s+steps?\s+(?:are|is\s+)?(?:done|complete|completed|finished)/i.test(message) ||
+ /\b(?:the\s+)?plan\s+(?:is\s+)?(?:complete|completed|done)\b/i.test(message)
+ );
}
diff --git a/packages/coding-agent/test/plan-mode-utils.test.ts b/packages/coding-agent/test/plan-mode-utils.test.ts
index db4b31ec1..22a3d623e 100644
--- a/packages/coding-agent/test/plan-mode-utils.test.ts
+++ b/packages/coding-agent/test/plan-mode-utils.test.ts
@@ -1,5 +1,6 @@
import { describe, expect, it } from "vitest";
import {
+ allStepsComplete,
cleanStepText,
extractDoneSteps,
extractTodoItems,
@@ -119,11 +120,10 @@ describe("cleanStepText", () => {
expect(cleanStepText("update config")).toBe("Config");
});
- it("truncates long text", () => {
+ it("keeps full text (truncation moved to the widget display)", () => {
const longText = "This is a very long step description that exceeds the maximum allowed length for display";
const result = cleanStepText(longText);
- expect(result.length).toBe(50);
- expect(result.endsWith("...")).toBe(true);
+ expect(result).toBe(longText);
});
it("normalizes whitespace", () => {
@@ -183,13 +183,31 @@ Plan:
expect(items[0].text).toContain("proper");
});
- it("filters out code-like items", () => {
+ it("keeps steps that start with backticks (code-file steps)", () => {
const message = `Plan:
1. \`npm install\`
2. Run the build process`;
const items = extractTodoItems(message);
- expect(items).toHaveLength(1);
+ expect(items).toHaveLength(2);
+ expect(items[0].text).toBe("Npm install");
+ });
+
+ it("extracts only the last Plan: section (thinking draft + text plan)", () => {
+ const message = `Some reasoning...\nPlan:\n1. Draft one\n2. Draft two\nNow the answer.\nPlan:\n1. Print 1\n2. Print 2\n3. Print 3`;
+
+ const items = extractTodoItems(message);
+ expect(items).toHaveLength(3);
+ expect(items.map((i) => i.text)).toEqual(["Print 1", "Print 2", "Print 3"]);
+ });
+
+ it("extracts markdown step titles fully", () => {
+ const message = `Plan:\n1. **Explore** the repo structure\n2. \`index.ts\` — fix getTextContent`;
+
+ const items = extractTodoItems(message);
+ expect(items).toHaveLength(2);
+ expect(items[0].text).toBe("Explore the repo structure");
+ expect(items[1].text).toBe("Index.ts — fix getTextContent");
});
});
@@ -209,11 +227,25 @@ describe("extractDoneSteps", () => {
expect(extractDoneSteps(message)).toEqual([1, 2, 3]);
});
- it("returns empty array with no markers", () => {
- const message = "No markers here";
+ it("extracts DONE marker variants", () => {
+ const message = "[DONE:1] [DONE: 2] [DONE3] [Done:4] Done: 5";
+ expect(extractDoneSteps(message)).toEqual([1, 2, 3, 4, 5]);
+ });
+
+ it("extracts prose completion phrasings", () => {
+ const message = "Step 6 done, Step 7 is complete, Step 8 finished";
+ expect(extractDoneSteps(message)).toEqual([6, 7, 8]);
+ });
+
+ it("does not match negated or unfinished phrasings", () => {
+ const message = "Step 9 is not done, Step 10 incomplete";
expect(extractDoneSteps(message)).toEqual([]);
});
+ it("collapses duplicate markers", () => {
+ expect(extractDoneSteps("[DONE:1] [DONE: 1]")).toEqual([1]);
+ });
+
it("ignores malformed markers", () => {
const message = "[DONE:abc] [DONE:] [DONE:1]";
expect(extractDoneSteps(message)).toEqual([1]);
@@ -248,8 +280,15 @@ describe("markCompletedSteps", () => {
const count = markCompletedSteps("[DONE:99]", items);
- expect(count).toBe(1); // Still counts the marker found
- expect(items[0].completed).toBe(false); // But doesn't mark anything
+ expect(count).toBe(0); // Counts only newly marked items
+ expect(items[0].completed).toBe(false);
+ });
+
+ it("marks all remaining steps for whole-plan completion phrasings", () => {
+ expect(allStepsComplete("All steps complete, wrapping up")).toBe(true);
+ expect(allStepsComplete("Every step is finished")).toBe(true);
+ expect(allStepsComplete("The plan is done")).toBe(true);
+ expect(allStepsComplete("Step 1 complete, moving on")).toBe(false);
});
it("doesn't double-complete already completed items", () => {
From 378d45a05b3a2c3249804c7dedc88a58ea424728 Mon Sep 17 00:00:00 2001
From: Mario Zechner
Date: Wed, 4 Nov 2026 10:02:47 +0000
Subject: [PATCH 20/88] feat(chord): telemetry, spans across hosts (slice 9)
Chord's telemetry schema; spans for calls, methods, relays, holds, activations, natives, loads, reloads, resolves,
revocations, routines, jobs and HTTP requests; trace carriers on hold and call (bounded, printable ASCII) continued by
relays and workers; events and host-wide metrics; the app kit passes an adapter to its hosts. renderTrace escapes
control characters. Example 62 prints a trace across three hosts.
---
packages/chord/CHANGELOG.md | 2 +
packages/chord/docs/benchmarks.md | 40 ++
packages/chord/docs/design.md | 88 ++-
.../chord/docs/guide-plugins-hosts/book.md | 56 +-
.../chord/docs/guide-plugins-hosts/index.html | 44 +-
packages/chord/docs/slices.md | 12 +-
packages/chord/src/data/principal.ts | 3 +-
packages/chord/src/data/validator.ts | 7 +-
packages/chord/src/documents/definition.ts | 3 +-
packages/chord/src/documents/documents.ts | 22 +-
packages/chord/src/documents/persistence.ts | 3 +-
packages/chord/src/documents/transaction.ts | 10 +
packages/chord/src/host/README.md | 4 +-
packages/chord/src/host/attempt.ts | 140 ++++-
packages/chord/src/host/ctx.ts | 99 +++-
packages/chord/src/host/host.ts | 154 +++++-
packages/chord/src/host/internal.ts | 2 +-
packages/chord/src/host/native.ts | 51 +-
packages/chord/src/host/object.ts | 60 +-
packages/chord/src/host/route.ts | 55 +-
packages/chord/src/host/routines.ts | 94 +++-
packages/chord/src/index.ts | 1 +
packages/chord/src/jobs/job.ts | 107 +++-
packages/chord/src/jobs/records.ts | 4 +-
packages/chord/src/jobs/run.ts | 6 +-
packages/chord/src/link/binary.ts | 26 +-
packages/chord/src/link/channel.ts | 16 +-
packages/chord/src/link/message.ts | 45 +-
packages/chord/src/link/part.ts | 7 +-
packages/chord/src/link/text.ts | 43 +-
packages/chord/src/node/app/app.ts | 9 +
packages/chord/src/node/app/worker.ts | 8 +
packages/chord/src/node/http/http.ts | 62 ++-
packages/chord/src/remote/consumer.ts | 17 +-
packages/chord/src/remote/hold.ts | 18 +-
packages/chord/src/remote/host-link.ts | 4 +-
packages/chord/src/remote/provider.ts | 87 ++-
packages/chord/src/remote/relay.ts | 119 +++-
packages/chord/src/remote/remote-object.ts | 94 +++-
packages/chord/src/state/batch.ts | 7 +-
packages/chord/src/store/archive.ts | 5 +-
packages/chord/src/store/json-column.ts | 3 +-
packages/chord/src/store/live-query.ts | 3 +-
packages/chord/src/store/store.ts | 13 +-
packages/chord/src/telemetry-sqlite/index.ts | 4 +
packages/chord/src/telemetry-sqlite/store.ts | 17 +-
packages/chord/src/telemetry.ts | 514 ++++++++++++++++++
packages/chord/test/bench/telemetry.ts | 195 +++++++
.../chord/test/documents/documents.test.ts | 43 +-
.../examples/62-telemetry/app/principals.ts | 7 +
.../test/examples/62-telemetry/app/v1.ts | 17 +
.../chord/test/examples/62-telemetry/main.ts | 79 +++
.../62-telemetry/plugins/tally/package.json | 5 +
.../62-telemetry/plugins/tally/worker.ts | 46 ++
.../chord/test/examples/62-telemetry/start.ts | 61 +++
.../test/examples/62-telemetry/worker.ts | 9 +
packages/chord/test/host/attempt.test.ts | 131 ++++-
packages/chord/test/host/ctx.test.ts | 103 +++-
packages/chord/test/host/host.test.ts | 76 ++-
packages/chord/test/host/native.test.ts | 86 +++
packages/chord/test/host/route.test.ts | 35 +-
packages/chord/test/host/routines.test.ts | 60 +-
packages/chord/test/jobs/job.test.ts | 101 +++-
packages/chord/test/jobs/persisted.test.ts | 42 +-
packages/chord/test/link/channel.test.ts | 2 +
packages/chord/test/link/message.test.ts | 103 ++++
packages/chord/test/node/http/http.test.ts | 89 +++
packages/chord/test/remote/provider.test.ts | 76 +++
.../chord/test/remote/remote-object.test.ts | 66 ++-
packages/chord/test/support.ts | 28 +-
packages/chord/test/telemetry-sqlite.test.ts | 14 +
packages/chord/test/telemetry.test.ts | 457 ++++++++++++++++
packages/telemetry/CHANGELOG.md | 4 +
packages/telemetry/src/render.ts | 27 +-
packages/telemetry/test/render.test.ts | 19 +
tsconfig.json | 1 +
76 files changed, 3929 insertions(+), 241 deletions(-)
create mode 100644 packages/chord/src/telemetry.ts
create mode 100644 packages/chord/test/bench/telemetry.ts
create mode 100644 packages/chord/test/examples/62-telemetry/app/principals.ts
create mode 100644 packages/chord/test/examples/62-telemetry/app/v1.ts
create mode 100644 packages/chord/test/examples/62-telemetry/main.ts
create mode 100644 packages/chord/test/examples/62-telemetry/plugins/tally/package.json
create mode 100644 packages/chord/test/examples/62-telemetry/plugins/tally/worker.ts
create mode 100644 packages/chord/test/examples/62-telemetry/start.ts
create mode 100644 packages/chord/test/examples/62-telemetry/worker.ts
create mode 100644 packages/chord/test/telemetry.test.ts
diff --git a/packages/chord/CHANGELOG.md b/packages/chord/CHANGELOG.md
index 55812b7ef..e6b0922b9 100644
--- a/packages/chord/CHANGELOG.md
+++ b/packages/chord/CHANGELOG.md
@@ -21,6 +21,8 @@
- Chord v2 gains live queries: `self.query(table, q, limit)` in the factory of a table's owner is a watchable of the query's rows (at most `limit`, in its order), usable as a state member and a derive source. Chord runs it before the object goes live and again inside the commit step of every commit of the object that writes the table, publishing only the diff in that commit's batch; a commit writing no table with a live query runs nothing (`chord.query.runs` counts the runs). Only the object's own tables; the limit is required (`invalid/args` at declaration). A run that fails (a row the code cannot read) is a failed member, reported once as the new `HostError` `query_failed { object, table, error }`, and heals at the next good run. A live query is no native, so an object showing table rows live swaps on reload instead of restarting, as reading them in `self.open` made it. Each touching commit runs the query once (100 rows: about 60 µs, 80 µs watched): not for tables written hundreds of times a second.
- Chord v2 gains messages and languages in the app kit (design §23): `defineMessages(namespace, { en: {...}, de: {...} })` returns a typed `t(key, ...args)` (the first catalog types the others: a missing key, an unknown one or a wrong argument is a type error; values are strings or functions), `t.message(key, ...args)` builds a message as data (`MessageData`, `{ namespace, key, params }`, for state members typed with `MessageSchema`), `t.render(message)` renders one (a foreign namespace or an unknown key from a link shows the key), `t.translations` lists its catalogs, and `MessagesOf` types a translation. The `Translations` registry takes a list of `{ namespace, locale, messages, override? }` per contributor (one catalog per namespace and locale: the first by plugin name wins, reported; `override: true` wins); a missing key falls back down the locale's chain (`pt-BR`, `pt`) to the first catalog, reported once. `LocaleChoice` keeps one locale per principal on the server (the theme choice's store, now shared as one preference); a client follows it with `followActiveLanguage`, defaulting to `navigator.languages` on a page and the system locale in the terminal (`startTerminal`'s `locales`), matched against the catalogs, switched live. `useMessages(t)` (`/react`) and `TuiScope.messages(t)` (`/tui`) render in the host's language and re-render or redraw on a change; `startPage` sets `lang` and `dir` on the page (`followLanguage`, `/web`); `formatTime` and `formatNumber` use `Intl` in the language being rendered. The kit serves `Translations` on the server, workers, pages and terminals and `LocaleChoice` on the server; client errors gain `{ kind: "messages", report }`.
+- Chord v2 gains telemetry across hosts (slice 9): `CHORD_TELEMETRY_SCHEMA` (root export) types every span, event and metric Chord records. With a host adapter (`createHost({ telemetry })`), calls (`chord.call`), methods (`chord.method`, local and remote), relays (`chord.relay`), holds (`chord.hold`), activations (`chord.activate`, ending `live`, `failed`, `parked` or `ended`, with `chord.retry` and `chord.park` events), natives' opens and closes, `load`, `reload` (a `chord.reload.fate` event per object), resolves (a worker start included), revocations, routine calls, job phases (`intent`, `run`, `finish`: a persisted run resumed in a new attempt or process stays under its stored carrier) and HTTP requests (`chord.http`: method, route pattern, status, never the path) are spans; failures carry a fixed category and the message cut at 500 characters, a cancel is no error; commits are `chord.commit { documents, ops, duration }` events of the span they ran under. `hold` and `call` carry the caller's trace carrier (`trace`, at most 8 keys, keys and values of at most 256 characters of printable ASCII, else the message fails the message schema and the link closes; the binary codec's kind byte flag `0x40`), relays forward their own span's, and the provider runs the method under it (an absent or unknown carrier: the adapter's root); a reference's carrier rides on its first hold only, a re-hold after a reconnect being a fresh root linked to it. A hold's `type` and its address's `kind` are bounded at 256 characters by the message schema, and span attributes a peer chose at 256 characters. `ctx.trace(name, attributes, fn)` runs `fn` in a child span with a child ctx. Host-wide metrics: `chord.link.bytes`, `chord.link.state_messages`, `chord.commit.duration`, `chord.calls.in_flight`, `chord.holds.active`, `chord.links.open`, `chord.park.duration`, `chord.backpressure.wait`. Without an adapter nothing is started, built or sent: a call over a link costs what it did before. `createHttp`, `startServer` and `startWorker` take a `telemetry` adapter (`createHttp` continues a W3C `traceparent` only from a request a contributed relay verified: an app's proxy must strip it from browsers). A host with an adapter counts the cost counters of its links' and root scopes' work on it also when the app passed a context without a span. The SQLite telemetry store keeps spans of one millisecond in the order they started.
+
### Changed
- Chord v2's HTTP component denies by default (slice 13): a request or plugin socket without a principal gets 401 before plugin code runs unless its route or socket is marked `public: true` (`{ handler, public: true }`; a `crossSite` route is public), and the `/chord` links endpoint refuses requests without a principal.
diff --git a/packages/chord/docs/benchmarks.md b/packages/chord/docs/benchmarks.md
index 4f5003322..749e26188 100644
--- a/packages/chord/docs/benchmarks.md
+++ b/packages/chord/docs/benchmarks.md
@@ -994,3 +994,43 @@ The diff depends on the reuse: with every row decoded anew, `diffRevisions` comp
previous 100 by content and the watched commit took 885 µs; with unchanged rows the same objects it compares them by
identity. "There may be dragons": at 100 rows a table written more than a few thousand times a second spends most of
its time in the query.
+
+## Slice 9: telemetry (2026-11-04)
+
+`test/bench/telemetry.ts`, Node 24.4.1, CPU 10, medians of nine bursts after a warm-up, three runs each; heap
+allocated per operation includes what the collections during a burst freed. "Before" is the same script (`--noop`,
+its imports pointed at a checkout of the commit before slice 9) run alternately with this one.
+
+A call over a memory link (a client host holding a remote counter on a server host; `bump` returns at once, `add`
+commits one document change):
+
+| Call | Before slice 9 | NOOP adapter | In-memory adapter on both hosts |
+| --- | --- | --- | --- |
+| over a link | 10.46-10.64 µs, 13,083 B | 10.24-10.30 µs, 12,766 B | 31.3 µs, 28,886 B |
+| that commits | 32.91-34.06 µs, 28,898 B | 31.44-31.61 µs, 28,598 B | 64.6 µs, 53,902 B |
+
+With the NOOP adapter the span code is within noise (the target): every span site is one identity check of its parent
+against the NOOP context's span starter, and nothing else runs (no span, no attribute object, no carrier). A call
+allocates 317 B less than before slice 9 (300 B less for one that commits), from the review's three code shapes (fix
+round): the call message takes its optional `trace` as a plain key (a key holding `undefined`, which both codecs and the
+schema treat as absent) instead of a conditional spread, `ChordObject.call`'s optional `parent` has no default
+expression (an async function with a non-simple parameter list allocated 48 B more), and `ProviderSide.#run` is a
+`.then` chain instead of an `async` function (its frame, about 85 B, held the extra argument). The first build's 206 B
+more per call were these shapes, not the `trace` field. Recording costs what the adapter does: the in-memory one keeps
+every span and measurement (each call records `chord.call`, `chord.method`, the commit's event, the host metrics,
+`chord.link.bytes` per message and, since the fix round, the links' cost counters on the host's adapter), about 21 µs
+and 16 KB per call here.
+
+Container width (R24, design §12): one item of a flat array changed per commit (a new text each time), alone and
+under a watched `mapShared` derive of the array (one mapper run per change: only the changed item is mapped again;
+the derive still visits every element once by identity and `diffRevisions` compares the arrays):
+
+| Width | One item changed | Under a watched `mapShared` derive |
+| --- | --- | --- |
+| 1,000 | 43.7-45.8 µs, 27.8 KB | 89.0-97.5 µs, 40.8 KB |
+| 10,000 | 84.5-88.7 µs, 131.7 KB | 299.8-310.0 µs, 262.8 KB |
+| 100,000 | 552.5-670.1 µs, 819.7 KB | 3,711-4,197 µs, 1,624.8 KB |
+
+A change costs O(width): the array's slots are copied (about 8 bytes per element) and a derive over it walks them. The
+documented limit: up to about 10,000 elements a list in one document stays below 0.3 ms per change with a derive; past
+that, keyed documents or a bounded window (design §12) keep commits O(change).
diff --git a/packages/chord/docs/design.md b/packages/chord/docs/design.md
index e272d9004..f3c165e54 100644
--- a/packages/chord/docs/design.md
+++ b/packages/chord/docs/design.md
@@ -912,8 +912,11 @@ Every handler gets a `ctx` (a `Context`): `telemetry` (the current span, section
cancelled or the object ended; there is no per-call deadline), `principal` (the caller as its link stamped it; on a
local object's handler the host's principal, none on a client), `incarnation` (the calling host's, section 4), and
`get(Type, params, { readyTimeout? })` (references as the caller sees them, released when the handler settles; a
-`get` after that throws `released`). Later: `ctx.trace(name, attributes, fn)` gives `fn` a child `ctx` under a child
-span (slice 9, cut C11 to decide); in a job's `run` also `jobId` and `state` (slice 5). Pass `ctx` to everything you wait
+`get` after that throws `released`), and `trace(name, attributes, fn)`: `fn` in a child span of `ctx.telemetry` with a
+child `ctx` (the same principal, incarnation, signal and `get`, its references the handler's), so calls inside are
+parented under it; a throw marks the span failed (`trace_failed`) and is rethrown; without an adapter that records,
+`fn` gets `ctx` itself (slice 9; round-3 cut C11 kept, as this section already stated it); in a job's `run` also `jobId`
+and `state` (slice 5). Pass `ctx` to everything you wait
on. Code outside objects gets a `ctx` from `host.root` (slice 8). Durable's `Context` is the same module
(`../context`); bridging is explicit. No code versions, no request ids.
@@ -2207,8 +2210,17 @@ validation, no clone of snapshots or ops.
The asynchronous inflate runs one part at a time; the queue behind it is bounded by the sending peer's window and N,
so only a misbehaving parent or route peer could grow it (accepted: those are the app's own hosts). The adapter reports "too large"
and "corrupt" as a tagged result. Limits and the threshold count bytes. No per-link compression.
-- **Trace carriers (R5; slice 9)** are an optional bounded field of `call` and `hold` (key count, string lengths);
- relays read them from the envelope.
+- **Trace carriers (R5; slice 9)** are an optional field `trace` of `call` and `hold`: an adapter's string record of at
+ most 8 keys, keys and values of at most 256 characters of printable ASCII (keys without spaces; W3C's headers and
+ Chord's adapters fit); past the bounds, with a control character or not a string record, the message fails `MESSAGE`
+ as any malformed header and the link closes (a trusted link skips the check; the adapter's `extract` then checks it).
+ A carrier names stored trace and parent ids, so no control character reaches a renderer (and `renderTrace` escapes
+ them anyway). Names a relay records unparsed are bounded the same way: a hold's `type` and its address's `kind` have
+ at most 256 characters. The
+ binary codec flags it in the kind byte (`0x40`, beside the origin's `0x80`) and writes it as JSON after the origin;
+ the text codec writes it as the last element (a hold's after `limit`, absent ones before it `null`; a call's after its
+ payload, whose two fields are then positional, `null` when absent). Relays read it from the envelope and forward
+ their own span's carrier (a relay recording nothing forwards it as it came). `PROTOCOL_VERSION` stays 1.
- **Adapted bindings get diffs** (slice 7b; decision "References are the only indirection" (5)): a consumer bound
through `accepts` attaches to the object instance's shared mapping for its version: per member one derive of the
instance's member through `(current) => old` (a changed member), of its whole state (a removed member), or the
@@ -2598,7 +2610,6 @@ this document):
| 16 | a Durable Object profile | 24 | when a DO target becomes concrete |
| 24 | a per-type drain override: a host option or fixed (no install watchdog: `reload` takes the app's context, decision "Slice 7b's open points" (4); the routine handover uses `closeTimeout`; a failing job `finish` backs off 1-60 s and blocks after five failures, decided) | 3 | if a real case needs it |
| 28 | Astra's circuit breaker (a hard transport-buffer cap plus a stall deadline); the heartbeat is built, the cap was never decided | 3 | if a real case needs it |
-| 29 | round-3 cut C11 (`ctx.trace`); C3 and C4 are decided (§14, §15) | 8 | slice 9 |
Apps (p3), from the functionality map, decided per domain when p3 is designed:
@@ -2736,7 +2747,72 @@ Chord reports traces, events and metrics through `@earendil-works/pi-telemetry`;
(`pi-telemetry/render`) prints links and never follows them. A web trace viewer in the playground reads the store.
- **Durable** (done): submit and phase spans, stored carriers on submissions and tasks (`TaskOptions.trace`, creation
links), events after commit, report events; pi-ai emits `pi.ai.request`/`pi.ai.poll` from one observer in
- `lazyStream`. Persisted Chord jobs store their start carrier.
+ `lazyStream`. Persisted Chord jobs store their start carrier. A worker's Harness gets its host's adapter
+ (`HarnessOptions.telemetry`) and a method that submits input passes its `ctx` (its `telemetry` the `chord.method`
+ span), so `pi.durable.submit` links to it and starts its own trace, as decided.
+
+**Built (slice 9, `slice-9.md`; `src/telemetry.ts`):**
+
+- **The schema:** `CHORD_TELEMETRY_SCHEMA` (root export, `defineTelemetrySchema`) declares every span, event and metric
+ Chord records with its attributes; the cost counters (`chord.data.*`, `chord.link.*`, `chord.store.*`,
+ `chord.query.runs`, `chord.cycle.visits`, `chord.derive.*`, `chord.registry.*`, `chord.contract.adapts`) are recorded
+ through it (`chordMetrics(metrics)`: typed instruments made once per adapter's metrics object and name, a `WeakMap`;
+ the NOOP metrics share one set). Ids (object labels, run ids, address keys) are span attributes only.
+- **Spans:** `chord.call` (a consumer's call of a remote member, until its reply), `chord.method` (a provider running a
+ method: under the call's carrier over a link, the caller's span on the same host; in `Attempt.call`, so local and
+ remote calls alike), `chord.relay` (a relayed hold until it is held upstream, a relayed call until its reply is
+ forwarded; the member is `hold` or `#`: a relay never knows names), `chord.hold` (a remote reference's hold
+ until its first `bound` or failure; its carrier rides on the reference's first hold only: a re-hold after a reconnect
+ is a fresh `chord.hold` root of the host's adapter linked to the first, and a relay's re-hold upstream after a drop a
+ fresh `chord.relay` root linked to its first, so a page kept open for days never adds children under a span long
+ ended), `chord.activate`,
+ `chord.open` (under the activation), `chord.close` (a root), `chord.load`, `chord.reload` (a `chord.reload.fate`
+ event per object), `chord.resolve`, `chord.revoke` (recorded after the synchronous revocation, its `count`),
+ `chord.routine` (a root per call), `chord.job` (`intent` under the starter, `run` and `finish` under the start's
+ carrier), `chord.http`. Outcomes are attributes; a failure is `status: "error"` with a fixed category (`call_failed`,
+ `method_failed`, `relay_failed`, `hold_failed`, `activation_failed`, `open_failed`, `close_failed`, `load_failed`,
+ `reload_failed`, `resolve_failed`, `routine_failed`, `job_failed`, `http_failed` for a status of 500 or more,
+ `trace_failed`) and the message cut at 500 characters; a cancel (the caller's signal, `aborted`, an open its attempt
+ abandoned) sets status ok explicitly (an adapter marks a span failed when its callback throws). String attributes a
+ peer chose (an object's label with its params key, an address key, a principal's kind, names from the wire) are cut
+ at 256 characters: stores keep them for days.
+- **Activations:** one span per attempt on the host's adapter, linked to the object's first trigger (the span that
+ created it, or a hold's carrier), a retry's to the attempt before it; it ends `live`, `failed` (a transient failure's
+ backoff a `chord.retry` event) or `ended`, or `parked` with a `chord.park { cause }` event, and a parked attempt's own
+ steps resume in a new span linked to the parked one (spans stay short); one that fails or ends while still parked
+ records it in a short span linked to the parked one. A getter binding to an object while it
+ activates links its span to the activation (shared work); a route's first hold links to its resolve and back.
+- **Parents across processes are carriers, not spans:** a settled span starts no recorded children, so whatever runs
+ later under work that has ended (a job's run and finish, a re-hold) goes through a carrier extracted with the host's
+ adapter. Every job run keeps its start's carrier (a persisted one stores it): a run resumed in a new attempt or
+ process stays in its trace.
+- **`ctx.trace`** gives `fn` a child of the same kind (a job run's keeps `jobId` and `state`), and a synchronous throw of
+ `fn` rejects whether or not anything records: recording changes nothing `fn` sees but its span.
+- **Routines:** each call's span links to the span of the commit that changed its source when the watch frame's
+ context names one (the newest folded frame's; the first call, with the current value, has none), and a call that
+ threw carries its backoff as a `chord.retry` event.
+- **Cost (§30):** a span site checks `untraced(parent)` (the parent starts spans as the NOOP context does: no adapter can
+ record) and then
+ starts no span, builds no attribute object and injects no carrier; with the NOOP adapter a call over a link is within
+ noise of the code before slice 9 (`benchmarks.md`). A recording host's call costs its spans (`OpenSpan`, a span kept
+ open until its work's promise settles).
+- **Events and metrics:** `chord.commit { documents, ops, duration }` on the span of the context a transaction ran
+ under (a method, a routine, a job phase, an HTTP request), only for a commit that changed something; the host-wide
+ metrics go to the host's adapter (`host.metrics`), not to a context: `chord.link.bytes { direction }` (every encoded
+ message of the host's links), `chord.link.state_messages`, `chord.commit.duration`, `chord.calls.in_flight`,
+ `chord.holds.active`, `chord.links.open`, `chord.park.duration`, `chord.backpressure.wait`. The cost counters still
+ count through the context of the work; a host with an adapter gives its links' and root scopes' work, when the app
+ passed a context without a span (`BACKGROUND_CONTEXT`, as the kit does), a span that records nothing but carries the
+ adapter's metrics (`countingOnly`), so they reach the adapter in an app too.
+- **Components:** `createHttp({ telemetry })` makes each routed request a `chord.http` root (method, route pattern,
+ status; never the path), its scope's `ctx` under it; a W3C `traceparent` (with its `tracestate`) is extracted (as the
+ carrier `{ traceparent, tracestate }`) only from a request a contributed relay verified, each header at most 256 and
+ 512 characters of printable ASCII (else ignored). A verified relay vouches for the proxy, not the browser behind it:
+ an app's proxy must strip `traceparent` and `tracestate` from browsers' requests (or set its own), or a browser
+ chooses the trace its requests join. `startServer({ telemetry })` passes the adapter to
+ its host and HTTP; `startWorker({ telemetry })` to its host. Each process exports to the store it is given; a shared
+ SQLite file per data directory is the app's choice (example 62). The SQLite store orders spans of one millisecond as
+ their recorder started them (by start time, then id length and id).
## 30. Cost model
diff --git a/packages/chord/docs/guide-plugins-hosts/book.md b/packages/chord/docs/guide-plugins-hosts/book.md
index a66d72075..6561d2ceb 100644
--- a/packages/chord/docs/guide-plugins-hosts/book.md
+++ b/packages/chord/docs/guide-plugins-hosts/book.md
@@ -13,7 +13,7 @@ note: |
`slice-10.md`, `slice-12.md`, `slice-13.md`, the decision log and the code in `packages/chord/src`. API names
are the current ones. The early chapters wire hosts by hand to show the parts; the last chapter shows the app kit
(`chord/app`), which does that wiring for an app.
-colophon: Hosts and Plugins · third edition · 2026-11-03 · Chord v2 slices 1-10, 12 and 13, messages and languages
+colophon: Hosts and Plugins · fourth edition · 2026-11-04 · Chord v2 slices 1-13, messages and languages, telemetry
---
# Before we begin {.preface}
@@ -809,6 +809,60 @@ writes them as one archive (a manifest, the store's export, the files) and stops
own store at one consistent point. `app.importData(path, Project({ id: "copy" }), ctx)` makes a new project from it and
refuses one that already has data (example 55).
+# Seeing a Request Across Hosts {#telemetry lede="One trace from a click in a client through the server's relay into a worker's method: adapters, carriers and spans."}
+
+A tab calls `notes.add(…)`. The call leaves the tab, the server relays it, the project's worker runs the method, the
+method commits. When it is slow, where did the time go? Each host is a different process, so no single log has the
+answer. Chord's telemetry makes the four steps one **trace**: a tree of **spans** (named, timed pieces of work), each
+host recording its own with its own **adapter**, joined by **carriers** (a few strings naming the caller's span) that
+ride on the wire with every `hold` and `call`.
+
+Give each host an adapter. With the app kit that is one option per process; the SQLite store
+(`@earendil-works/chord/telemetry-sqlite`) records settled spans and metrics, and here every process of the app opens
+the same file in the data directory (example 62):
+
+```ts
+// start.ts: the server
+const telemetry = await createSqliteTelemetry(await openNodeSqliteDatabase(join(dataDir, "telemetry.sqlite")));
+await startServer({ ...options, telemetry }, ctx);
+// worker.ts: each worker, its own recorder on the same file (the path in its environment)
+startWorker({ api, kinds, telemetry: await openTelemetry(process.env.TALLY_TELEMETRY!), onError });
+```
+
+Nothing else changes in plugin code. A method's `ctx.telemetry` is its `chord.method` span, so whatever it calls or
+commits is parented under it, on its host or the next one. `ctx.trace(name, attributes, fn)` adds a span of your own:
+
+```ts
+add: async (n, ctx) => {
+ await ctx.trace("tally.check", { n }, async () => {
+ if (!Number.isSafeInteger(n) || n <= 0) throw new Error("add a positive count");
+ });
+ return store.add(n, ctx);
+},
+```
+
+The store renders the click's trace across the three hosts (the client's spans, the server's relay, the worker's
+methods and the commit, an event of the method that made it):
+
+```
+tally.click
+ chord.hold type="tally.counter" object="tally.counter(t1)" address.kind="tally" outcome="bound"
+ chord.relay type="tally.counter" member="hold" upstream.kind="tally" outcome="ok"
+ → link to chord.resolve
+ chord.call type="tally.counter" member="add" object="tally.counter(t1)" address.kind="tally" outcome="ok"
+ chord.relay type="tally.counter" member="#0" upstream.kind="tally" outcome="ok"
+ chord.method type="tally.counter" member="add" object="tally.counter(t1)" principal.kind="owner" outcome="ok"
+ tally.check n=3
+ chord.method type="tally.store" member="add" object="tally.store" principal.kind="worker" outcome="ok"
+ · chord.commit documents=1 ops=1 duration=1.9
+```
+
+Work shared by several requests is its own trace, linked rather than nested: the worker's start (`chord.resolve`),
+an object's activation (`chord.activate`, linked to whoever asked first). Routines and job phases are roots too; a job's
+`run` and `finish` stay under the carrier its start stored, even when a restarted worker resumes the run. Metrics
+(`chord.link.bytes`, `chord.commit.duration`, `chord.calls.in_flight`, ...) are aggregated per 10 s; ids are never their
+attributes. Without an adapter (the default), Chord starts no span and sends no carrier: a call costs what it did before.
+
# Reference {.appendix}
| Call | What it does |
diff --git a/packages/chord/docs/guide-plugins-hosts/index.html b/packages/chord/docs/guide-plugins-hosts/index.html
index b96ccf5c3..7c4184fe6 100644
--- a/packages/chord/docs/guide-plugins-hosts/index.html
+++ b/packages/chord/docs/guide-plugins-hosts/index.html
@@ -131,6 +131,7 @@ are the current ones. The early chapters wire hosts by hand to show the parts; t
VIII. Changing a Plugin
IX. Base Code and Plugins
X. The App Kit
+XI. Seeing a Request Across Hosts
Appendix A: Reference
@@ -727,6 +728,47 @@ own store at one consistent point. app.importData(path, Project({ id: &quo
refuses one that already has data (example 55).
+
+A tab calls notes.add(…). The call leaves the tab, the server relays it, the project's worker runs the method, the
+method commits. When it is slow, where did the time go? Each host is a different process, so no single log has the
+answer. Chord's telemetry makes the four steps one trace: a tree of spans (named, timed pieces of work), each
+host recording its own with its own adapter, joined by carriers (a few strings naming the caller's span) that
+ride on the wire with every hold and call.
+Give each host an adapter. With the app kit that is one option per process; the SQLite store
+(@earendil-works/chord/telemetry-sqlite) records settled spans and metrics, and here every process of the app opens
+the same file in the data directory (example 62):
+// start.ts: the server
+const telemetry = await createSqliteTelemetry(await openNodeSqliteDatabase(join(dataDir, "telemetry.sqlite")));
+await startServer({ ...options, telemetry }, ctx);
+// worker.ts: each worker, its own recorder on the same file (the path in its environment)
+startWorker({ api, kinds, telemetry: await openTelemetry(process.env.TALLY_TELEMETRY!), onError });
+Nothing else changes in plugin code. A method's ctx.telemetry is its chord.method span, so whatever it calls or
+commits is parented under it, on its host or the next one. ctx.trace(name, attributes, fn) adds a span of your own:
+add: async (n, ctx) => {
+ await ctx.trace("tally.check", { n }, async () => {
+ if (!Number.isSafeInteger(n) || n <= 0) throw new Error("add a positive count");
+ });
+ return store.add(n, ctx);
+},
+The store renders the click's trace across the three hosts (the client's spans, the server's relay, the worker's
+methods and the commit, an event of the method that made it):
+tally.click
+ chord.hold type="tally.counter" object="tally.counter(t1)" address.kind="tally" outcome="bound"
+ chord.relay type="tally.counter" member="hold" upstream.kind="tally" outcome="ok"
+ → link to chord.resolve
+ chord.call type="tally.counter" member="add" object="tally.counter(t1)" address.kind="tally" outcome="ok"
+ chord.relay type="tally.counter" member="#0" upstream.kind="tally" outcome="ok"
+ chord.method type="tally.counter" member="add" object="tally.counter(t1)" principal.kind="owner" outcome="ok"
+ tally.check n=3
+ chord.method type="tally.store" member="add" object="tally.store" principal.kind="worker" outcome="ok"
+ · chord.commit documents=1 ops=1 duration=1.9
+Work shared by several requests is its own trace, linked rather than nested: the worker's start (chord.resolve),
+an object's activation (chord.activate, linked to whoever asked first). Routines and job phases are roots too; a job's
+run and finish stay under the carrier its start stored, even when a restarted worker resumes the run. Metrics
+(chord.link.bytes, chord.commit.duration, chord.calls.in_flight, ...) are aggregated per 10 s; ids are never their
+attributes. Without an adapter (the default), Chord starts no span and sends no carrier: a call costs what it did before.
+
+
@@ -798,7 +840,7 @@ refuses one that already has data (example 55).
-Hosts and Plugins · third edition · 2026-11-03 · Chord v2 slices 1-10, 12 and 13, messages and languages
+Hosts and Plugins · fourth edition · 2026-11-04 · Chord v2 slices 1-13, messages and languages, telemetry