diff --git a/packages/durable/CHANGELOG.md b/packages/durable/CHANGELOG.md index f66d732f4..d495f0ca4 100644 --- a/packages/durable/CHANGELOG.md +++ b/packages/durable/CHANGELOG.md @@ -2,6 +2,18 @@ ## [Unreleased] +### Added + +- `settings.compaction.modelOverrides` sets compaction thresholds per model, keyed by exact `"provider/modelId"`, so one harness can serve small- and large-window models; the threshold check, overflow recovery, and the compaction task use the entry of the model they act for ([#10942](https://github.com/earendil-works/pi/issues/10942)). + +### Changed + +- `ToolControl.terminate` is now typed `boolean` instead of `true`, matching `pi-agent-core` and `pi-coding-agent`; `false` does not end the run ([#10858](https://github.com/earendil-works/pi/issues/10858)). + +### Fixed + +- The README's Errors section now limits the `StorageRequestError` exemption to reads: a `StorageRequestError` thrown by `commit()` or `mintId()` fails the Harness like any other storage error ([#11014](https://github.com/earendil-works/pi/issues/11014)). + ## [1.3.0] - 2026-10-15 ### Added diff --git a/packages/durable/README.md b/packages/durable/README.md index 6907b4292..6e20295a8 100644 --- a/packages/durable/README.md +++ b/packages/durable/README.md @@ -475,10 +475,16 @@ settings: { reserveTokens: 16384, // above contextWindow - reserveTokens, the next request waits for a compaction keepRecentTokens: 20000, // roughly how much recent context stays verbatim backgroundTokens: 32768, // this far below that, a compaction starts in the background; 0 disables it + modelOverrides: { + // exact "provider/modelId"; each field set here replaces the one above for that model + "local/small-32k": { reserveTokens: 4096, keepRecentTokens: 8000, backgroundTokens: 4096 }, + }, }, } ``` +The fixed token counts rarely suit a 32k and a 1M context window alike; `modelOverrides` lets one harness serve both. The conversation's current model picks the entry, at the threshold check, after a context overflow, and when a compaction selects what to summarize. + When a provider rejects a request because the context is too long, generation compacts and retries once. A summary that would cut before the start of the current context settles as `stale` when it is placed, so when several are in flight, the furthest cut stays in effect. Summarization spend counts in `pi.usage`. A `beforeCompact` hook on `CompactionTask` can decline or supply its own summary. When a compaction that a generation waits for at the blocking threshold fails, the generation fails too, without sending a request: its inputs settle `unanswered` with `model_error` and detail `Compaction failed: `. One that finds nothing to compact or is declined lets the request go ahead. @@ -713,7 +719,7 @@ The package root loads TypeBox, because the tool task validates arguments with p Three rules decide what an error does: 1. **A storage error ends the Harness.** The call that hit it rejects with the error. Everything after it, and every pending wait, rejects with `SessionFailed` (its `cause` is that error); watches and event streams end with `session_failed`; running tasks get their abort signal; `onReport` gets the error once. The Harness then closes itself. Nothing is retried; a custom storage retries its own transient failures. Reopen to recover: commits are atomic, so the store is consistent, and resubmitting with the same `requestId` finds a submission that was already admitted. -2. **An invalid request fails only that call:** a storage throws `StorageRequestError` for a bad cursor, an unknown conversation, or history a document does not keep. A read you cancelled fails only itself too. +2. **An invalid read fails only that call:** a storage read throws `StorageRequestError` for a bad cursor, an unknown conversation, or history a document does not keep. A read you cancelled fails only itself too. The exemption covers reads only: any error from `commit()` or `mintId()`, `StorageRequestError` included, ends the Harness as in rule 1. 3. **Your code fails only its own unit:** a throwing task phase faults that task, a throwing tool gets an error result, a throwing hook is reported and skipped (`beforeTool` blocks the call), and a throwing commit callback rolls back that commit. Listeners that throw, or return a rejected promise, are reported and the others still run. The exception is a host `RegistryReader` whose `snapshot()` throws: the Harness cannot schedule without it, so it fails the Harness like a storage error. `harness.closed` settles once the Harness has closed, by `close()` or after a storage error, so a host restarts on failure like this: diff --git a/packages/durable/docs/spec.md b/packages/durable/docs/spec.md index efb1842c0..74fe6193c 100644 --- a/packages/durable/docs/spec.md +++ b/packages/durable/docs/spec.md @@ -408,6 +408,14 @@ type CompactionPolicy = { backgroundTokens: number; }; +/** + * `modelOverrides` is keyed by exact `"provider/modelId"`; each field it sets replaces the harness-wide one for that + * model. Readers resolve the policy for the model they act for (see the reader table below). + */ +type CompactionSettings = Partial & { + readonly modelOverrides?: Readonly>>; +}; + type ToolExecutionMode = "parallel" | "sequential"; /** How many queued items of one mode a boundary selects (section 6). */ @@ -419,7 +427,7 @@ type HarnessSettings = { readonly extensions?: readonly Extension[]; readonly stream?: ConversationStreamOptions; readonly retry?: Partial; - readonly compaction?: Partial; + readonly compaction?: CompactionSettings; readonly toolExecution?: ToolExecutionMode; readonly steeringMode?: QueueMode; readonly followUpMode?: QueueMode; @@ -434,7 +442,7 @@ type Settings = { readonly extensions?: readonly Extension[]; // default: every installed extension, in install order readonly stream: ConversationStreamOptions; // default {} readonly retry: ConversationRetryPolicy; // { enabled: true, maxRetries: 3, baseDelayMs: 2000, maxAgentDelayMs: 60000 } - readonly compaction: CompactionPolicy; // { enabled: true, reserveTokens: 16384, keepRecentTokens: 20000, backgroundTokens: 32768 } + readonly compaction: CompactionPolicy & Pick; // { enabled: true, reserveTokens: 16384, keepRecentTokens: 20000, backgroundTokens: 32768 } readonly toolExecution: ToolExecutionMode; // "parallel" readonly steeringMode: QueueMode; // "one-at-a-time" readonly followUpMode: QueueMode; // "one-at-a-time" @@ -2663,12 +2671,13 @@ await conversation.configure({ extensions: { add: [Venv] } }, context); | reader | resolves | when | |---|---|---| -| generation `prepare` | agent (tools, sections, model, thinking level), stream options, compaction thresholds | once per request; model, stream options, and offered tools are fixed in the request checkpoint (section 8.3) | +| generation `prepare` | agent (tools, sections, model, thinking level), stream options, compaction thresholds for that model | once per request; model, stream options, and offered tools are fixed in the request checkpoint (section 8.3) | | every task phase | extension selection for hooks; `runtime.agent()` | at most once per phase handler, at its first hook dispatch or `agent()` call, from that phase's snapshot and the committed `pi.agent`, off the Session line, with the invocation's context; fixed for the rest of the phase | | tool task | the implementation of an accepted call | at `call` and `execute`: the name among the phase agent's `tools`, the current selection after the tools filter; no such tool, for example one removed after preparation: `tool_unavailable`. Replay policy on reopen uses the same lookup (section 7.3) | | generation, when a tool round starts | tool execution mode | once per round (section 8.3) | | generation, compaction | retry policy | at each attempt's classification | -| compaction `select` | model, thinking level, stream options, compaction thresholds | once; the summary request stays pinned across its retries (section 8.7) | +| generation, on a context overflow | compaction policy for the request's model | at classification | +| compaction `select` | model, thinking level, stream options, compaction thresholds for that model | once; the summary request stays pinned across its retries (section 8.7) | | boundaries and idle admission, on the Session line | queue modes | per boundary (section 6) | | tool task, `prepare`, `runtime.env()` | environment | at each call (section 2.2) | diff --git a/packages/durable/src/harness/compaction.ts b/packages/durable/src/harness/compaction.ts index b05fdaae6..4cf1966ca 100644 --- a/packages/durable/src/harness/compaction.ts +++ b/packages/durable/src/harness/compaction.ts @@ -27,11 +27,13 @@ import { ensureProviderSessionId } from "./provider.ts"; import { admitSubmission } from "./submissions.ts"; import type { CompactionHooks, + CompactionPolicy, CompactionReason, CompactionResult, ContextView, ConversationStreamOptions, ModelRef, + Settings, } from "./types.ts"; import { recordUsage } from "./usage.ts"; @@ -119,7 +121,7 @@ export const CompactionTask = defineTask => { const live = await tx.doc(LiveDoc, conversationId); diff --git a/packages/durable/src/harness/types.ts b/packages/durable/src/harness/types.ts index 0f8c5f389..7a12804c0 100644 --- a/packages/durable/src/harness/types.ts +++ b/packages/durable/src/harness/types.ts @@ -136,7 +136,8 @@ export interface ConversationHandle { /** Post-tools controls requested by a tool result. */ export type ToolControl = { readonly addTools?: readonly string[]; - readonly terminate?: true; + /** End the run without another model request when every result of the round sets it to `true`; `false` is a no-op. */ + readonly terminate?: boolean; readonly handoff?: string; }; @@ -506,6 +507,14 @@ export type CompactionPolicy = { backgroundTokens: number; }; +/** + * Harness-wide compaction settings. `modelOverrides` is keyed by exact `"provider/modelId"`; each field it sets + * replaces the harness-wide one for that model, so small- and large-window models can share a harness. + */ +export type CompactionSettings = Partial & { + readonly modelOverrides?: Readonly>>; +}; + /** * How often running progress is committed. Each progress commit is a storage write; a host whose storage is remote can * commit less often, so live answers and tool output appear in larger steps. @@ -532,7 +541,7 @@ export type HarnessSettings = { readonly extensions?: readonly Extension[]; readonly stream?: ConversationStreamOptions; readonly retry?: Partial; - readonly compaction?: Partial; + readonly compaction?: CompactionSettings; readonly progress?: Partial; readonly toolExecution?: ToolExecutionMode; readonly steeringMode?: QueueMode; @@ -556,7 +565,8 @@ export type Settings = { readonly extensions?: readonly Extension[]; readonly stream: ConversationStreamOptions; readonly retry: ConversationRetryPolicy; - readonly compaction: CompactionPolicy; + /** Harness-wide policy over the defaults; per-model overrides apply where the model is known. */ + readonly compaction: CompactionPolicy & Pick; readonly progress: ProgressPolicy; readonly toolExecution: ToolExecutionMode; readonly steeringMode: QueueMode; diff --git a/packages/durable/src/index.ts b/packages/durable/src/index.ts index 083b8af00..546c6ffa6 100644 --- a/packages/durable/src/index.ts +++ b/packages/durable/src/index.ts @@ -68,6 +68,7 @@ export type { CompactionPolicy, CompactionReason, CompactionResult, + CompactionSettings, ContextView, Conversation, ConversationAbortOptions, diff --git a/packages/durable/test/harness-compaction.test.ts b/packages/durable/test/harness-compaction.test.ts index da5d21a51..ebea5e8a8 100644 --- a/packages/durable/test/harness-compaction.test.ts +++ b/packages/durable/test/harness-compaction.test.ts @@ -2390,3 +2390,52 @@ describe("context contributions", () => { await chat.harness.close(context); }); }); + +// #10942: per-model compaction settings. +describe("per-model compaction overrides", () => { + it("applies the current model's override to the blocking threshold", async () => { + for (const key of ["faux/faux-1", "faux/other"]) { + const chat = await open({ contextWindow: 1000 }); + await history(chat); + // Harness-wide: blocking only above the full window; the override for faux-1 blocks above 700. + chat.setup.settings.compaction = { + ...BLOCKING, + reserveTokens: 0, + modelOverrides: { [key]: { reserveTokens: 300 } }, + }; + chat.faux.summaries.push(summary()); + await turn(chat, text("u4", 200), "a4"); + expect(chat.faux.summaryRequests).toHaveLength(key === "faux/faux-1" ? 1 : 0); + await chat.harness.close(context); + } + }); + + it("applies the request model's override on a context overflow", async () => { + const chat = await open(); + await history(chat); + chat.setup.settings.compaction = { ...MANUAL, modelOverrides: { "faux/faux-1": { enabled: true } } }; + chat.faux.summaries.push(summary()); + chat.faux.agent.push(failure(OVERFLOW), answer("fits")); + const input = await chat.root.submit({ type: "input", content: text("u4", 100) }, context); + expect((await input.wait(context)).status).toBe("done"); + expect(chat.faux.summaryRequests).toHaveLength(1); + await chat.harness.close(context); + }); + + it("applies the model's override when a compaction selects and sizes its summary", async () => { + const chat = await open(); + await history(chat); + chat.setup.settings.compaction = { + ...MANUAL, + keepRecentTokens: 100_000, + modelOverrides: { "faux/faux-1": { keepRecentTokens: 150, reserveTokens: 500 } }, + }; + chat.faux.summaries.push(summary()); + expect((await result(chat, await chat.root.compact(undefined, context))).status).toBe("completed"); + // 0.8 * 500 from the override, below the model's 900. + expect(chat.faux.summaryRequests[0]!.options?.maxTokens).toBe(400); + const messages = (await chat.root.context(context)).messages; + expect(messages.slice(1).map(userText)).toEqual([text("u3", 100), text("a3", 100)]); + await chat.harness.close(context); + }); +}); diff --git a/packages/durable/test/harness-tools.test.ts b/packages/durable/test/harness-tools.test.ts index 75b8a9bd7..4c06ce805 100644 --- a/packages/durable/test/harness-tools.test.ts +++ b/packages/durable/test/harness-tools.test.ts @@ -673,6 +673,23 @@ describe("tool results", () => { ]); await first.harness.close(context); }); + + // #10858 + it("does not terminate on terminate: false", async () => { + const setup = chatSetup(); + const keep = tool("keep", async () => ({ output: [], control: { terminate: false } })); + addTool(setup.registry, keep); + const { harness, entries, status } = await run( + setup, + [calls(["keep", {}, "c1"]), DONE], + async (_harness, root) => { + await root.configure({ tools: [keep] }, context); + }, + ); + expect(status).toBe("done"); + expect(entries.at(-1)!.kind).toBe("pi.assistant"); + await harness.close(context); + }); }); describe("generation hooks", () => {