mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
parent
fb524dd2ea
commit
f6a9db1d83
15 changed files with 493 additions and 42 deletions
|
|
@ -95,7 +95,8 @@
|
|||
- `afterTool` runs on every result a tool task writes, not only on an executed call's: also on the error results the Harness writes itself, for a tool that does not resolve (`tool_unavailable`), invalid arguments (`invalid_arguments`, with `call.arguments` as the model sent them), a block (`blocked`), an interruption (`interrupted`), the restart limit (`restart_limit`), and an abort (`aborted`, `abandoned`); the diagnostic code tells them apart. Such a result is bounded with the tool's output limits after the hooks, never keeps a `structuredOutput`, and applies `control` only when the call completes; a nested call a hook turns into a success gets its bounded output as value unless the tool declares a schema. Once a tool task was cut off more than `settings.restartLimit` times its hooks no longer run, nor in the last attempt the scheduler gives its abort handler, and an abort's hooks delay `Conversation.abort()`. A non-replay-safe tool interrupted that often now ends with `restart_limit` too. Hooks that assumed the tool ran (cast `call.arguments`, read `structuredOutput`) check `result.diagnostics` first ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- A call to a tool its request did not offer gets a `pi.tool` task like any call, flagged `unoffered: true` in its input and its `pi.live.tools` slot, instead of a result the generation wrote without one: the task resolves nothing (a deferred tool the agent has still never runs), settles `tool_unavailable`, and runs `afterTool`, whose `control` then counts toward the round (one such call no longer prevents `terminate`). Its slot starts `pending` with a `taskId` instead of `done`, and its result entry lands in a later commit than the tool-calling answer. Controls are applied in call order. `pi.tool` is version 3 and `pi.generation` version 2 (a sequential round keeps such calls pending until their turn): an older Harness leaves both blocked (`task_too_old`) instead of resolving the flagged name ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- `SubmissionRecord` gains a `type: "continue"` variant (`placed` without `entry`, `done` with `answer`, `unanswered`), so narrowing a record by `status` alone no longer guarantees an `entry`: check `type === "input"` too. A generation whose context does not end in a user message or tool results, system messages aside, now ends its run `unanswered` with the new `RunFailureReason` `nothing_to_answer` without a request; an empty conversation no longer faults its generation. `ConversationHandle.submit()` takes any `SubmissionDraft` (was `InputSubmissionDraft`), as `Conversation.submit()` does ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- `RunFailureReason` (`pi.live.runFailure.reason`) gains `nothing_to_answer`, also an `unanswered` submission's `reason`: exhaustive switches over it need a case ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- `RunFailureReason` (`pi.live.runFailure.reason`) gains `nothing_to_answer` and `turn_limit`, also an `unanswered` submission's `reason`: exhaustive switches over it need cases ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- `pi.live.run` has a required `turns` field, the run's tool rounds so far; code that writes `run` itself sets it (`turns: 0`). The unreleased `pi.live` version 2 `migrate` adds `turns: 0` to a stored version 1 run; a database written by an unreleased build at version 2 without `run.turns` is not supported (its run counts `NaN` turns) ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
|
||||
### Added
|
||||
|
||||
|
|
@ -107,6 +108,7 @@
|
|||
- `onReport` gets `task_interrupted` (`reason` `restart_limit` or `interrupted`, `restarts`, and the task) when crashes end a built-in task: a tool call, a compaction, or a generation at the restart limit, or a run that ends `interrupted`. The telemetry report attributes gain the type `task_interrupted` and the reason `interrupted` ([#11139](https://github.com/earendil-works/pi/issues/11139)).
|
||||
- `HarnessOptions.readOnly` opens a Harness as a viewer of a session another process drives: open leaves running tasks as committed instead of recovering them, every commit with writes rejects with `ReadOnlySession` before it reaches Storage, and `resume()` and every call that would run tasks reject with it.
|
||||
- Continue submissions: `submit({ type: "continue" })` starts a run on the context as it is, without a user message, for example after a run that ended without an answer (`pi.live.runFailure`). Only an idle conversation admits one (a busy one throws `ConversationBusy`; it never queues); items a failed run left queued are placed first and join the run. A context that ends in an answer, also a text answer cut off at its output limit, has nothing to continue (`nothing_to_answer`); a cut-off answer with tool calls is continued, as its calls end in error results. Tasks and tools submit one through `ConversationHandle`. The `pi.durable.submit` span starts a root trace for it, as for an input ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
- Per-conversation turn limits: `AgentState.maxTurns` (`configure({ maxTurns })`, `null` clears; a `RangeError` for anything but an integer 1 or more) bounds the tool rounds one run takes. The round that reaches it runs its tools, then the run ends `unanswered` with the new `RunFailureReason` `turn_limit`, like a failed run, so `settings.afterFailedRun` decides the queued inputs and a continue resumes it on the round's results. The count is `pi.live.run.turns`; every run starts at 0, and an `onYield` continuation starts it over. Task-owned children and forks inherit the limit; `Agent.maxTurns` exposes it ([#11043](https://github.com/earendil-works/pi/issues/11043))
|
||||
|
||||
### Changed
|
||||
|
||||
|
|
|
|||
|
|
@ -217,7 +217,7 @@ The default `setTimer` declines the timers that only free memory in Workers, so
|
|||
|
||||
**Reports.** `onReport(report)` gets one `Report`: the `error` and what happened in `report.kind`: `storage_failed` for the error that failed the Harness (a Storage error, often a platform reset); `task_faulted`, `task_interrupted` (crashes ended it, with `report.reason` and `report.restarts`), `migration_failed`, `task_outdated` (`reason` `missing_task`, `task_too_old`, or `migration_failed`, the words a blocked task shows), or `task_error` for a task, with `report.task` (its `id`, `kind`, and `conversationId`) and the extension that installed it in `report.extension`; `extension_error` for hooks, sections, wrappers, and tools, with the extension's name; `host_error` for a host callback, named in `report.callback`; and `listener_error`. A host can feed `task_faulted` reports of one kind to its own breaker.
|
||||
|
||||
**Queued inputs after a failed run.** A run that fails with a model error, a missing model, or crashes (`interrupted`, `restart_limit`) leaves the inputs queued behind it waiting until the next input arrives. `settings.afterFailedRun` changes that: `continue` starts the next run with them, as after an answer, and `withdraw` settles them `unanswered` with reason `run_failed`.
|
||||
**Queued inputs after a failed run.** A run that fails with a model error, a missing model, nothing to answer, its turn limit, or crashes (`interrupted`, `restart_limit`) leaves the inputs queued behind it waiting until the next input arrives. `settings.afterFailedRun` changes that: `continue` starts the next run with them, as after an answer, and `withdraw` settles them `unanswered` with reason `run_failed`.
|
||||
|
||||
**Blocked tasks.** Tasks blocked on a missing or unmigratable definition show their reason in the task graph too, and a registry change that unblocks them makes `onWake` report the Session active, so a host need not poll them.
|
||||
|
||||
|
|
@ -391,6 +391,7 @@ await root.configure(
|
|||
modelTools: [readTool], // of those, offer the model only read; tools may still call bash
|
||||
instructions: "Only read; never edit files.",
|
||||
cwd: "/work/repo",
|
||||
maxTurns: 50, // tool rounds one run takes at most; unset: no limit
|
||||
},
|
||||
context,
|
||||
);
|
||||
|
|
@ -400,6 +401,8 @@ const agent = await root.agent(context); // resolved: model, extensions, tools,
|
|||
|
||||
Extensions and tools are passed as objects and stored by name, so a stored name outlives its code: after an extension is uninstalled, conversations that select it just stop getting it until it is installed again. `createConversation()`, `fork()`, and `root()` take the same change as `agent`. A task-owned conversation, such as a subagent's, starts as a copy of its owner's conversation's agent. A fork starts with the agent its parent had at the fork entry. The model, prompt, and offered tools of a request are fixed when it is prepared; a change applies from the next request. Tool calls and hooks use the agent as their task phase resolves it, and the environment is built from the current `cwd` at each use, so a `cwd` or extension change can reach calls the model already made.
|
||||
|
||||
**Turn limits.** `maxTurns` bounds the tool rounds one run takes: the round that reaches it still runs its tools, then the run ends `unanswered` with `turn_limit`, like a failed run (`runFailure`, and `settings.afterFailedRun` decides the inputs queued behind it). An answer always ends the run, so it is never cut off. The count is `docs["pi.live"].run.turns`; it survives restarts, and every run starts at 0, also one a continue starts (see [Busy Conversations](#busy-conversations)), so a host or tool that continues automatically gets a fresh budget each time and must bound that itself. An `onYield` continuation also starts the count over: an extension that keeps a run going bounds its own loop, reading `pi.live.run.turns` and `pi.agent.maxTurns` with `api.snapshot`. `configure()` throws a `RangeError` for anything but an integer 1 or more (or `null`); a change applies at the end of the current round, and children and forks inherit it like the rest of the agent.
|
||||
|
||||
## Settings
|
||||
|
||||
Run policy shared by every conversation is passed as `settings`. It is read at every use and never stored, so getters make it live, for example backed by a settings file:
|
||||
|
|
@ -542,7 +545,7 @@ await root.submit({ type: "write", entry: { kind: "app.note", data: "user opened
|
|||
|
||||
If a run fails, queued items stay in the inbox until the next input, queued write, or continue places them, oldest first.
|
||||
|
||||
When a run ends without an answer, `docs["pi.live"].runFailure` says why until the next run starts: `model_error` and `no_model` (with the error text or the missing model in `detail`), `aborted`, `reset`, `interrupted` (the host stopped while the model answered, with no retry left), `faulted`, `orphaned`, and `restart_limit` (the run's task could not go on), or `nothing_to_answer` (a continue, below, found nothing to answer). The run's inputs are in it too, settled `unanswered`; the settlement's reason is the same, except that an orphaned run's inputs carry why it was orphaned (`missing_task`, `task_too_old`, `migration_failed`) and `no_model` has no `detail` there (spec §8.2).
|
||||
When a run ends without an answer, `docs["pi.live"].runFailure` says why until the next run starts: `model_error` and `no_model` (with the error text or the missing model in `detail`), `aborted`, `reset`, `interrupted` (the host stopped while the model answered, with no retry left), `faulted`, `orphaned`, and `restart_limit` (the run's task could not go on), `turn_limit` (a tool round reached the conversation's `maxTurns`), or `nothing_to_answer` (a continue, below, found nothing to answer). The run's inputs are in it too, settled `unanswered`; the settlement's reason is the same, except that an orphaned run's inputs carry why it was orphaned (`missing_task`, `task_too_old`, `migration_failed`) and `no_model` has no `detail` there (spec §8.2).
|
||||
|
||||
To resume such a run without typing a message, submit a continue. It starts a run on the context as it is, so the model answers the user message or tool results the stopped run left:
|
||||
|
||||
|
|
|
|||
|
|
@ -548,6 +548,8 @@ type AgentState = {
|
|||
instructions?: string;
|
||||
/** Directory within the environment's file system, passed to `HarnessOptions.env`. */
|
||||
cwd?: string;
|
||||
/** Tool rounds one run takes at most (section 8.5); an integer, `1` or more; unset, no limit. */
|
||||
maxTurns?: number;
|
||||
};
|
||||
|
||||
/** Built-in rewindable agent document; see below. */
|
||||
|
|
@ -568,6 +570,8 @@ type AgentChange = {
|
|||
| null;
|
||||
readonly instructions?: string | null;
|
||||
readonly cwd?: string | null;
|
||||
/** `configure()` throws a `RangeError` unless it is an integer `1` or more, or `null`. */
|
||||
readonly maxTurns?: number | null;
|
||||
};
|
||||
|
||||
/** One change in any commit. A given field replaces the stored one; `null` clears it; `undefined` changes nothing. */
|
||||
|
|
@ -588,6 +592,7 @@ type Agent<Tool extends ToolRegistration = ToolRegistration> = {
|
|||
readonly sections: readonly PromptSection<Tool>[];
|
||||
readonly instructions?: string;
|
||||
readonly cwd?: string;
|
||||
readonly maxTurns?: number;
|
||||
};
|
||||
|
||||
/** Entry whose `data` has type `D`; `never` means the kind carries no data. */
|
||||
|
|
@ -2837,6 +2842,7 @@ Admission and terminal transitions:
|
|||
| boundary places write | `done`, with entry | append entry |
|
||||
| run answers | input or continue `done`, with required answer entry | remove `run`, or hand it to a successor |
|
||||
| run fails or its task aborts | input `unanswered`, with reason | remove `run`; inbox per `afterFailedRun` below |
|
||||
| tool round reaches `maxTurns` | input `unanswered`, reason `turn_limit` | as a failed run |
|
||||
| withdraw queued item | `unanswered`, reason `aborted` | remove inbox item |
|
||||
| stale head write | `unanswered`, reason `stale` | remove inbox item |
|
||||
|
||||
|
|
@ -2864,7 +2870,8 @@ run started in its commit kept) settles `nothing_to_answer`. Each continue start
|
|||
a run with its own turn budget.
|
||||
|
||||
`settings.afterFailedRun` decides what a run that fails with `model_error`,
|
||||
`no_model`, `interrupted`, `restart_limit`, or `nothing_to_answer` does with the inputs queued behind it, in the commit that settles its
|
||||
`no_model`, `interrupted`, `restart_limit`, `nothing_to_answer`, or
|
||||
`turn_limit` does with the inputs queued behind it, in the commit that settles its
|
||||
own inputs: `keep` (the default) leaves them queued; `continue` applies the
|
||||
final boundary, so the next run starts with them, as after an answer;
|
||||
`withdraw` settles them `unanswered` with reason `run_failed`, keeping queued
|
||||
|
|
@ -4326,8 +4333,11 @@ messages out of later requests, so no separate usage or notice kind exists.
|
|||
|
||||
```ts
|
||||
type LiveState = {
|
||||
/** Run control (section 6); present exactly while the conversation is busy. */
|
||||
run?: { taskId: TaskId; inputs: SubmissionId[] };
|
||||
/**
|
||||
* Run control (section 6); present exactly while the conversation is busy. `turns` counts the run's tool rounds,
|
||||
* which `AgentState.maxTurns` bounds (section 8.5); an `onYield` continuation starts it over (section 8.3).
|
||||
*/
|
||||
run?: { taskId: TaskId; inputs: SubmissionId[]; turns: number };
|
||||
/** Presentation of the current generation attempt. */
|
||||
generation?: {
|
||||
attempt: number;
|
||||
|
|
@ -4410,10 +4420,12 @@ type RunFailure = {
|
|||
| "orphaned"
|
||||
| "interrupted"
|
||||
| "restart_limit"
|
||||
| "nothing_to_answer";
|
||||
| "nothing_to_answer"
|
||||
| "turn_limit";
|
||||
/**
|
||||
* `model_error`: the error text; `no_model`: which model is missing; `faulted`: the error message; `orphaned`: why
|
||||
* the task could not run (`missing_task`, `task_too_old`, `migration_failed`).
|
||||
* the task could not run (`missing_task`, `task_too_old`, `migration_failed`); `interrupted` and `restart_limit`:
|
||||
* what was cut off, and how often; `turn_limit`: the tool rounds the run took, `"1 turn"` or `"N turns"`.
|
||||
*/
|
||||
detail?: string;
|
||||
};
|
||||
|
|
@ -4444,7 +4456,7 @@ type CompactionStatus = {
|
|||
| version | `2` |
|
||||
| scope/history/fork | conversation, `latest`, `initial` |
|
||||
| `initial()` | `{}` |
|
||||
| `migrate()` | from version 1: `reason` of each `compactions` status and of `compactionFailure` renamed `trigger` (section 3.6) |
|
||||
| `migrate()` | from version 1: `reason` of each `compactions` status and of `compactionFailure` renamed `trigger`, and `run.turns` set to `0` (section 3.6) |
|
||||
| checkpoint | complete base whenever nothing runs: `generation` absent and no `running` tool or nested slot |
|
||||
| view mount | `docs["pi.live"]` |
|
||||
| created | with every Harness conversation (section 2.2) |
|
||||
|
|
@ -4504,6 +4516,7 @@ that settles its inputs `unanswered`:
|
|||
| `interrupted` | the host stopped while the model was answering, with no retry left (section 8.3); `detail` names the attempt | `interrupted` with the text |
|
||||
| `restart_limit` | its generation was cut off `restartLimit` times in a row (section 5.4), also while an interrupted request was charged; `detail` says in which phase | `restart_limit` with the text |
|
||||
| `nothing_to_answer` | its generation found a context that does not end in a user message or tool results, system messages aside, such as a continue after an answer (section 8.3); no `detail` | `nothing_to_answer` |
|
||||
| `turn_limit` | a tool round reached the conversation's `maxTurns` and ran its tools (section 8.5); `detail` is the run's count, `"1 turn"` or `"N turns"` | `turn_limit` with the count |
|
||||
|
||||
The next commit that starts a run removes it. A run started in the same commit,
|
||||
the queue's next run under `afterFailedRun: "continue"` (section 6) or after a
|
||||
|
|
@ -4664,7 +4677,11 @@ The run's inputs live in `pi.live.run`, not in the task input.
|
|||
`{ continue }` wins. The commit appends the answer and applies the final
|
||||
boundary (section 6). With a continuation and no selected user item or
|
||||
reset, it appends a `pi.user` entry with the continuation content, creates a
|
||||
successor generation, and hands it `pi.live.run`, keeping the inputs open.
|
||||
successor generation, and hands it `pi.live.run`, keeping the inputs open and
|
||||
setting its `turns` to `0`: each stretch a continuation starts gets the whole
|
||||
turn budget, and bounding `onYield` loops is the extension's job (it reads
|
||||
`pi.live.run.turns` and `pi.agent.maxTurns` with `api.snapshot`). The chain runs
|
||||
whatever the count.
|
||||
Otherwise it settles the run's inputs `done`, removes `run` and
|
||||
`generation`, and starts a successor run for the selected user items, if
|
||||
any; a dropped continuation is not retried. Both complete with
|
||||
|
|
@ -4720,12 +4737,12 @@ A tool round starts in the commit that appends the tool-calling answer:
|
|||
`pending` shrinks.
|
||||
3. The generation commits `waiting` on its tool tasks with `allSettled` and the
|
||||
`tools` checkpoint (section 8.5); `pi.live.run` stays with it.
|
||||
4. `pi.live.tools` receives the round's slots (section 8.2), and `generation` is
|
||||
removed.
|
||||
4. `pi.live.tools` receives the round's slots (section 8.2), `generation` is
|
||||
removed, and `pi.live.run.turns` grows by one.
|
||||
|
||||
Input and continue submissions settle `unanswered` with one of these reasons:
|
||||
`no_model`, `model_error` (detail: provider error text), `nothing_to_answer`,
|
||||
`aborted`, `faulted` (detail: error message), or an orphaning blocked reason
|
||||
`turn_limit` (detail: the turn count, section 8.5), `aborted`, `faulted` (detail: error message), or an orphaning blocked reason
|
||||
(section 5.4). Fault and orphan
|
||||
settlement convert a committed partial into an aborted `pi.assistant` entry,
|
||||
like the abort handler.
|
||||
|
|
@ -4837,6 +4854,12 @@ and then commits once:
|
|||
- When every result of the round requests `terminate`, it settles the run's
|
||||
inputs `done` with the tool-calling answer, removes `run` and `tools`, and
|
||||
applies the final boundary.
|
||||
- Otherwise, when the run's `turns`, counted in the commit that started the
|
||||
round, reached the phase agent's `maxTurns`, the run ends as a failed run
|
||||
(section 6): its inputs settle `unanswered` with `turn_limit` (detail: the
|
||||
count), `runFailure` records it, `afterFailedRun` decides the queued items, and
|
||||
the generation still completes with `{ entryId }`. A continue resumes it on the
|
||||
round's results (section 6).
|
||||
- Otherwise it removes `tools` and applies the `postTools` boundary. When that
|
||||
boundary placed a queued reset, which wins over a handoff written before it,
|
||||
the run's inputs settle `unanswered` with `reset` and selected user items
|
||||
|
|
|
|||
|
|
@ -158,6 +158,11 @@ function applyChange(state: Draft<AgentState>, change: AgentChange): void {
|
|||
);
|
||||
set("instructions", change.instructions);
|
||||
set("cwd", change.cwd);
|
||||
const maxTurns = change.maxTurns;
|
||||
if (maxTurns !== undefined && maxTurns !== null && !(Number.isSafeInteger(maxTurns) && maxTurns >= 1)) {
|
||||
throw new RangeError(`maxTurns must be an integer 1 or more, got ${String(maxTurns)}`);
|
||||
}
|
||||
set("maxTurns", maxTurns);
|
||||
}
|
||||
|
||||
function isList<T>(value: readonly T[] | object): value is readonly T[] {
|
||||
|
|
@ -281,6 +286,7 @@ export function resolveAgent<Tool extends ToolRegistration>(
|
|||
sections: agentSections,
|
||||
...(instructions === undefined ? {} : { instructions }),
|
||||
...(state?.cwd === undefined ? {} : { cwd: state.cwd }),
|
||||
...(state?.maxTurns === undefined ? {} : { maxTurns: state.maxTurns }),
|
||||
};
|
||||
return agent;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ import type {
|
|||
} from "../types.ts";
|
||||
import { addTools, sectionExtension } from "./agent.ts";
|
||||
import { compactionPolicy, createCompaction, estimateContext, selectCut } from "./compaction.ts";
|
||||
import { applyBoundary, type Boundary, prepareBoundary, withdrawQueuedInputs } from "./inbox.ts";
|
||||
import { applyBoundary, type Boundary, hasQueuedReset, prepareBoundary, withdrawQueuedInputs } from "./inbox.ts";
|
||||
import {
|
||||
cutOffCount,
|
||||
cutOffTooOften,
|
||||
|
|
@ -796,10 +796,13 @@ async function answer(runtime: Runtime, message: AssistantMessage, context: Cont
|
|||
const live = await tx.doc(LiveDoc, conversationId);
|
||||
const entry = await appendAssistant(tx, conversationId, message);
|
||||
const result: Next = { status: "terminal", outcome: { status: "completed", result: { entryId: entry.id } } };
|
||||
const { users, reset } = await applyBoundary(tx, boundary, "final", runtime.now());
|
||||
if (continuation !== undefined && users.length === 0 && !reset) {
|
||||
const user = { role: "user", content: continuation, timestamp: runtime.now() } as const;
|
||||
const continues = takesContinuation(continuation, boundary) ? continuation : undefined;
|
||||
const { users } = await applyBoundary(tx, boundary, "final", runtime.now());
|
||||
if (continues !== undefined) {
|
||||
const user = { role: "user", content: continues, timestamp: runtime.now() } as const;
|
||||
await tx.appendEntry(UserEntry, conversationId, { model: [user] });
|
||||
// The continuation starts the run's turn count over: bounding `onYield` loops is the extension's job.
|
||||
if (live.run?.taskId === runtime.taskId) live.run.turns = 0;
|
||||
handOver(live, runtime.taskId, await createGeneration(tx, conversationId));
|
||||
delete live.generation;
|
||||
return result;
|
||||
|
|
@ -810,6 +813,20 @@ async function answer(runtime: Runtime, message: AssistantMessage, context: Cont
|
|||
}, context);
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether an answer's `onYield` continuation is taken (spec §8.3), decided from the inbox before the final boundary
|
||||
* places anything: only when it holds no user item, of which the boundary would select one, and no reset, which would
|
||||
* otherwise run first. A continuation starts the turn count over.
|
||||
*/
|
||||
function takesContinuation(continuation: UserInput | undefined, boundary: Boundary): continuation is UserInput {
|
||||
const items = boundary.inbox.items;
|
||||
return (
|
||||
continuation !== undefined &&
|
||||
!items.some((item) => item.mode === "steer" || item.mode === "followUp") &&
|
||||
!hasQueuedReset(items)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Append the tool-calling answer and start its tool round in one commit (spec §8.3). Every call gets a tool task owned by
|
||||
* the generation, in call order, only the first one now when the round is sequential; a call to a tool the request did
|
||||
|
|
@ -854,6 +871,8 @@ async function startToolRound(
|
|||
}
|
||||
delete live.generation;
|
||||
live.tools = slots;
|
||||
// A tool round is one turn of the run (`AgentState.maxTurns`).
|
||||
if (live.run?.taskId === runtime.taskId) live.run.turns += 1;
|
||||
const checkpoint = { phase: "tools", assistant: entry.id, tools, pending } as const;
|
||||
return { status: "waiting", checkpoint, on: tools, policy: "allSettled" };
|
||||
}, context);
|
||||
|
|
@ -861,8 +880,9 @@ async function startToolRound(
|
|||
|
||||
/**
|
||||
* The round's tools are terminal: apply their controls, append a `handoff`'s reset, and then end the run at the final
|
||||
* boundary (`terminate`), end it at a queued reset, or hand it to the next generation at the `postTools` boundary, in
|
||||
* the handoff's new context when there is one (spec §8.5).
|
||||
* boundary (`terminate`), end it like a failure at the conversation's turn limit (`turn_limit`), end it at a queued
|
||||
* reset, or hand it to the next generation at the `postTools` boundary, in the handoff's new context when there is one
|
||||
* (spec §8.5).
|
||||
*/
|
||||
async function finishToolRound(
|
||||
runtime: Runtime,
|
||||
|
|
@ -891,6 +911,8 @@ async function finishToolRound(
|
|||
const added = ordered.flatMap((control) => control?.addTools ?? []);
|
||||
// The last handoff in call order wins.
|
||||
const handoff = ordered.findLast((control) => control?.handoff !== undefined)?.handoff;
|
||||
// Read from the phase's agent, as every agent field; a `configure()` during the round applies at its end.
|
||||
const maxTurns = (await runtime.agent(context)).maxTurns;
|
||||
await runtime.commit(async (tx): Promise<Next> => {
|
||||
const boundary = await prepareBoundary(tx, conversationId, runtime.settings);
|
||||
if (added.length > 0) await addTools(tx, conversationId, added);
|
||||
|
|
@ -902,10 +924,21 @@ async function finishToolRound(
|
|||
const entry = await tx.appendEntry(ResetEntry, conversationId, { head: "self", model: [message] });
|
||||
boundary.head = entry.id;
|
||||
}
|
||||
const turns = live.run?.taskId === runtime.taskId ? live.run.turns : 0;
|
||||
if (terminate) {
|
||||
const { users } = await applyBoundary(tx, boundary, "final", now);
|
||||
endRun(tx, live, runtime.taskId, { status: "done", answer: assistant });
|
||||
if (users.length > 0) await startRun(tx, conversationId, live, users);
|
||||
} else if (maxTurns !== undefined && turns >= maxTurns) {
|
||||
// The turn limit ends the run like a failure: `afterFailedRun` decides the queued items, and a continue resumes
|
||||
// it on the round's results. The table was read for the boundary already, before any write.
|
||||
const policy = runtime.settings.afterFailedRun;
|
||||
const failure = { policy, ...(policy === "continue" ? { boundary } : {}) };
|
||||
await endFailedRun(tx, runtime, live, failure, {
|
||||
status: "unanswered",
|
||||
reason: "turn_limit",
|
||||
detail: turns === 1 ? "1 turn" : `${turns} turns`,
|
||||
});
|
||||
} else {
|
||||
const { users, reset } = await applyBoundary(tx, boundary, "postTools", now);
|
||||
if (reset) {
|
||||
|
|
@ -947,7 +980,7 @@ export async function startRun(
|
|||
inputs: SubmissionId[],
|
||||
options?: { readonly keepFailure?: boolean },
|
||||
): Promise<void> {
|
||||
live.run = { taskId: await createGeneration(tx, conversationId, { submissionIds: inputs }), inputs };
|
||||
live.run = { taskId: await createGeneration(tx, conversationId, { submissionIds: inputs }), inputs, turns: 0 };
|
||||
if (options?.keepFailure !== true) delete live.runFailure;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -102,7 +102,8 @@ export type CompactionFailure = {
|
|||
* - `restart_limit`: the run's generation was cut off `settings.restartLimit` times in a row; a request or poll it was
|
||||
* cut off during counts as a failed attempt first, so this ends one whose charge itself keeps failing;
|
||||
* - `nothing_to_answer`: the context did not end in a user message or tool results (empty, or ending in an answer), so
|
||||
* no request was sent, as for a continue after an answer.
|
||||
* no request was sent, as for a continue after an answer;
|
||||
* - `turn_limit`: a tool round reached the conversation's `maxTurns`; a continue resumes on its results.
|
||||
*/
|
||||
export type RunFailureReason =
|
||||
| "model_error"
|
||||
|
|
@ -113,7 +114,8 @@ export type RunFailureReason =
|
|||
| "orphaned"
|
||||
| "interrupted"
|
||||
| "restart_limit"
|
||||
| "nothing_to_answer";
|
||||
| "nothing_to_answer"
|
||||
| "turn_limit";
|
||||
|
||||
/** The newest run that ended without an answer (spec §8.2). */
|
||||
export type RunFailure = {
|
||||
|
|
@ -125,15 +127,18 @@ export type RunFailure = {
|
|||
/**
|
||||
* `model_error`: the error text; `no_model`: which model is missing; `faulted`: the error message; `orphaned`: why
|
||||
* the task could not run (`missing_task`, `task_too_old`, `migration_failed`); `interrupted` and `restart_limit`: what
|
||||
* was cut off, and how often.
|
||||
* was cut off, and how often; `turn_limit`: the tool rounds the run took, `"1 turn"` or `"N turns"`.
|
||||
*/
|
||||
detail?: string;
|
||||
};
|
||||
|
||||
/** Built-in live conversation state: run control and presentation of the current generation and tool round. */
|
||||
export type LiveState = {
|
||||
/** Run control: the task that settles the run's inputs, and those inputs; present exactly while busy. */
|
||||
run?: { taskId: TaskId; inputs: SubmissionId[] };
|
||||
/**
|
||||
* Run control: the task that settles the run's inputs, those inputs, and the run's tool rounds so far, which
|
||||
* `AgentState.maxTurns` bounds (an `onYield` continuation starts the count over); present exactly while busy.
|
||||
*/
|
||||
run?: { taskId: TaskId; inputs: SubmissionId[]; turns: number };
|
||||
/** Presentation of the current generation attempt. */
|
||||
generation?: {
|
||||
attempt: number;
|
||||
|
|
@ -173,12 +178,14 @@ export const LiveDoc = defineDoc<LiveState>({
|
|||
history: "latest",
|
||||
fork: "initial",
|
||||
initial: () => ({}),
|
||||
// Version 1 named a compaction's trigger `reason`, in `compactions` and `compactionFailure`.
|
||||
// Version 1 named a compaction's trigger `reason`, in `compactions` and `compactionFailure`, and had no `run.turns`:
|
||||
// a run in flight across the upgrade counts its turns from 0.
|
||||
migrate: (value) => {
|
||||
const live = value as { compactions?: JsonObject[]; compactionFailure?: JsonObject };
|
||||
const { compactions, compactionFailure } = live;
|
||||
const live = value as { run?: JsonObject; compactions?: JsonObject[]; compactionFailure?: JsonObject };
|
||||
const { run, compactions, compactionFailure } = live;
|
||||
return {
|
||||
...value,
|
||||
...(run === undefined ? {} : { run: { ...run, turns: 0 } }),
|
||||
...(compactions === undefined ? {} : { compactions: compactions.map(renameReason) }),
|
||||
...(compactionFailure === undefined ? {} : { compactionFailure: renameReason(compactionFailure) }),
|
||||
} as LiveState;
|
||||
|
|
|
|||
|
|
@ -455,6 +455,11 @@ export type AgentState = {
|
|||
instructions?: string;
|
||||
/** Directory within the environment's file system, passed to `HarnessOptions.env`. */
|
||||
cwd?: string;
|
||||
/**
|
||||
* Tool rounds one run takes at most (`pi.live.run.turns`): the round that reaches it runs its tools, then the run ends
|
||||
* `unanswered` with `turn_limit`. An integer, `1` or more; unset, no limit. Task-owned children and forks inherit it.
|
||||
*/
|
||||
maxTurns?: number;
|
||||
};
|
||||
|
||||
/** A change to `pi.agent`: a given field replaces the stored one, `null` clears it, `undefined` changes nothing. */
|
||||
|
|
@ -473,6 +478,8 @@ export type AgentChange = {
|
|||
| null;
|
||||
readonly instructions?: string | null;
|
||||
readonly cwd?: string | null;
|
||||
/** `configure()` throws a `RangeError` unless it is an integer `1` or more (or `null`). */
|
||||
readonly maxTurns?: number | null;
|
||||
};
|
||||
|
||||
/** A conversation's agent resolved against a registry snapshot and the settings. */
|
||||
|
|
@ -490,6 +497,11 @@ export type Agent<Tool extends ToolRegistration = ToolRegistration> = {
|
|||
readonly sections: readonly PromptSection<Tool>[];
|
||||
readonly instructions?: string;
|
||||
readonly cwd?: string;
|
||||
/**
|
||||
* Tool rounds one run takes at most (`pi.live.run.turns`): the round that reaches it runs its tools, then the run ends
|
||||
* `unanswered` with `turn_limit`. An integer, `1` or more; unset, no limit. Task-owned children and forks inherit it.
|
||||
*/
|
||||
readonly maxTurns?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
|
|
@ -596,8 +608,8 @@ export type HarnessSettings = {
|
|||
*/
|
||||
readonly documentRetentionMs?: number;
|
||||
/**
|
||||
* What a run that fails (`model_error`, `no_model`, `interrupted`, `restart_limit`, `nothing_to_answer`) does with the
|
||||
* inputs queued behind it: `keep` leaves them queued until the next input or queued write, `continue` starts the
|
||||
* What a run that fails (`model_error`, `no_model`, `interrupted`, `restart_limit`, `nothing_to_answer`,
|
||||
* `turn_limit`) does with the inputs queued behind it: `keep` leaves them queued until the next input or queued write, `continue` starts the
|
||||
* next run with them, as an answer would, and `withdraw` settles them `unanswered` with reason `run_failed`. Default
|
||||
* `keep`.
|
||||
*/
|
||||
|
|
@ -613,7 +625,10 @@ export type HarnessSettings = {
|
|||
readonly restartLimit?: number;
|
||||
};
|
||||
|
||||
/** What a failed run does with the inputs queued behind it (`HarnessSettings.afterFailedRun`). */
|
||||
/**
|
||||
* What a failed run does with the inputs queued behind it (`HarnessSettings.afterFailedRun`): a run that ends with
|
||||
* `model_error`, `no_model`, `interrupted`, `restart_limit`, `nothing_to_answer`, or `turn_limit`.
|
||||
*/
|
||||
export type AfterFailedRun = "keep" | "continue" | "withdraw";
|
||||
|
||||
/** Resolved settings: every field over its built-in default, object fields merged. */
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ import {
|
|||
type LiveState,
|
||||
MemoryStorage,
|
||||
ProviderDoc,
|
||||
type SubmissionId,
|
||||
type TaskId,
|
||||
UsageDoc,
|
||||
} from "@earendil-works/pi-durable";
|
||||
|
|
@ -2473,6 +2474,7 @@ describe("pi.live version 2", () => {
|
|||
);
|
||||
await session.commit(async (tx) => {
|
||||
const live = await tx.doc(LiveV1, conversationId);
|
||||
live.run = { taskId: 9, inputs: [4] };
|
||||
live.compactions = [{ taskId: 7, reason: "threshold", blocking: true, attempt: 2 }];
|
||||
live.compactionFailure = { taskId: 5, reason: "manual", error: "no model" };
|
||||
}, context);
|
||||
|
|
@ -2486,6 +2488,8 @@ describe("pi.live version 2", () => {
|
|||
return (await storage.document(record!.id, "current", context))!;
|
||||
};
|
||||
const migrated: LiveState = {
|
||||
// A run in flight across the upgrade counts its turns from 0 (#11043).
|
||||
run: { taskId: 9 as TaskId, inputs: [4 as SubmissionId], turns: 0 },
|
||||
compactions: [{ taskId: 7 as TaskId<CompactionResult>, trigger: "threshold", blocking: true, attempt: 2 }],
|
||||
compactionFailure: { taskId: 5 as TaskId<CompactionResult>, trigger: "manual", error: "no model" },
|
||||
};
|
||||
|
|
|
|||
|
|
@ -197,7 +197,7 @@ describe("generation recovery", () => {
|
|||
await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
expect(setup.faux.state.callCount).toBe(1);
|
||||
expect(await opened.harness.snapshot(LiveDoc, opened.root.id, context)).toEqual({
|
||||
run: { taskId, inputs: [id] },
|
||||
run: { taskId, inputs: [id], turns: 0 },
|
||||
generation: { attempt: 1, retry: { at: 61_000, error: "503 Service Unavailable" } },
|
||||
});
|
||||
now = 61_000;
|
||||
|
|
|
|||
|
|
@ -717,7 +717,7 @@ describe("generation", () => {
|
|||
{},
|
||||
{ ownership: { kind: "conversation" }, conversationId: root.id },
|
||||
);
|
||||
(await tx.doc(LiveDoc, root.id)).run = { taskId, inputs: [submission.id] };
|
||||
(await tx.doc(LiveDoc, root.id)).run = { taskId, inputs: [submission.id], turns: 0 };
|
||||
return { taskId, submissionId: submission.id };
|
||||
}, context);
|
||||
harness.resume();
|
||||
|
|
|
|||
|
|
@ -131,13 +131,14 @@ describe("pi.live deltas", () => {
|
|||
const result = entries.items.find((entry) => entry.kind === "pi.tool-result")!.id;
|
||||
expect(commits).toEqual([
|
||||
// submission
|
||||
[["s", ["run"], { taskId: firstGeneration, inputs: [expect.any(Number)] }]],
|
||||
[["s", ["run"], { taskId: firstGeneration, inputs: [expect.any(Number)], turns: 0 }]],
|
||||
// request
|
||||
[["s", ["generation"], { attempt: 1 }]],
|
||||
// the generation starts its tool round and keeps the run
|
||||
// the generation starts its tool round and keeps the run, counting the round as one turn (#11043)
|
||||
expect.arrayContaining([
|
||||
["d", ["generation"]],
|
||||
["s", ["tools"], [{ callId: "c1", name: "noop", taskId: tool, status: "pending" }]],
|
||||
["s", ["run", "turns"], 1],
|
||||
]),
|
||||
// intent
|
||||
[["s", ["tools", 0, "status"], "running"]],
|
||||
|
|
@ -158,7 +159,7 @@ describe("pi.live deltas", () => {
|
|||
["d", ["generation"]],
|
||||
]),
|
||||
]);
|
||||
expect(commits[2]).toHaveLength(2);
|
||||
expect(commits[2]).toHaveLength(3);
|
||||
expect(commits[4]).toHaveLength(2);
|
||||
expect(commits[5]).toHaveLength(2);
|
||||
await harness.close(context);
|
||||
|
|
@ -456,7 +457,7 @@ describe("pi.live deltas", () => {
|
|||
const base = (value: LiveState) =>
|
||||
LiveDoc.definition.checkpointWhen!(value, [], { deltasSinceBase: 1000 } as never);
|
||||
const slot = (status: "pending" | "running" | "done") => ({ callId: "c", name: "n", status });
|
||||
const run = { taskId: 1 as never, inputs: [] };
|
||||
const run = { taskId: 1 as never, inputs: [], turns: 0 };
|
||||
expect(base({})).toBe(true);
|
||||
expect(base({ run, generation: { attempt: 1 } })).toBe(false);
|
||||
expect(base({ run, tools: [slot("pending"), slot("done")] })).toBe(true);
|
||||
|
|
|
|||
|
|
@ -174,7 +174,7 @@ async function waitUntil(check: () => Promise<boolean>): Promise<void> {
|
|||
/** Mark a conversation busy with `task` standing in for its run, so submissions queue. */
|
||||
async function busy(conversation: Conversation, task: TaskId): Promise<void> {
|
||||
await conversation.commit(async (tx) => {
|
||||
(await tx.doc(LiveDoc, conversation.id)).run = { taskId: task, inputs: [] };
|
||||
(await tx.doc(LiveDoc, conversation.id)).run = { taskId: task, inputs: [], turns: 0 };
|
||||
}, context);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -267,7 +267,7 @@ describe("scheduler retention", () => {
|
|||
const c = (await harness.conversation(tree.c, context))!;
|
||||
// Busy, so the input queues instead of starting a run.
|
||||
await c.commit(async (tx) => {
|
||||
(await tx.doc(LiveDoc, tree.c)).run = { taskId: tree.keep, inputs: [] };
|
||||
(await tx.doc(LiveDoc, tree.c)).run = { taskId: tree.keep, inputs: [], turns: 0 };
|
||||
}, context);
|
||||
const queued = (await c.submit({ type: "input", content: "later" }, context)).id;
|
||||
await harness.abortTask(tree.p, context);
|
||||
|
|
@ -452,7 +452,7 @@ describe("scheduler retention", () => {
|
|||
await harness.waitForTask(tree.b, context);
|
||||
const c = (await harness.conversation(tree.c, context))!;
|
||||
await c.commit(async (tx) => {
|
||||
(await tx.doc(LiveDoc, tree.c)).run = { taskId: tree.keep, inputs: [] };
|
||||
(await tx.doc(LiveDoc, tree.c)).run = { taskId: tree.keep, inputs: [], turns: 0 };
|
||||
}, context);
|
||||
// Queued while nothing cancels: no withdrawal scan.
|
||||
const queued = (await c.submit({ type: "input", content: "later" }, context)).id;
|
||||
|
|
@ -529,7 +529,7 @@ describe("scheduler retention", () => {
|
|||
context,
|
||||
);
|
||||
await root.commit(async (tx) => {
|
||||
(await tx.doc(LiveDoc, root.id)).run = { taskId: busy, inputs: [] };
|
||||
(await tx.doc(LiveDoc, root.id)).run = { taskId: busy, inputs: [], turns: 0 };
|
||||
}, context);
|
||||
const before = scans;
|
||||
await root.submit({ type: "input", content: "queued" }, context);
|
||||
|
|
|
|||
357
packages/durable/test/harness-turn-limit.test.ts
Normal file
357
packages/durable/test/harness-turn-limit.test.ts
Normal file
|
|
@ -0,0 +1,357 @@
|
|||
// Per-conversation turn limits (`AgentState.maxTurns`) and continuing a run that reached one (#11043).
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { type AssistantMessage, fauxAssistantMessage, fauxText, fauxToolCall, Type } from "@earendil-works/pi-ai";
|
||||
import {
|
||||
type AfterFailedRun,
|
||||
type Conversation,
|
||||
defineTool,
|
||||
GenerationTask,
|
||||
LiveDoc,
|
||||
MemoryStorage,
|
||||
ResetEntry,
|
||||
type Storage,
|
||||
type ToolRegistration,
|
||||
ToolResultEntry,
|
||||
ToolTask,
|
||||
} from "@earendil-works/pi-durable";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
|
||||
import { allEntries, type ChatSetup, chatSetup, openChat, textOf } from "./chat-support.ts";
|
||||
import { addHooks, addTool } from "./harness-support.ts";
|
||||
import { context } from "./session-support.ts";
|
||||
import { aborted, deferred } from "./task-support.ts";
|
||||
|
||||
const directories = new Set<string>();
|
||||
const killed = new Set<Storage>();
|
||||
|
||||
afterEach(async () => {
|
||||
for (const storage of killed) await storage.close(context).catch(() => {});
|
||||
killed.clear();
|
||||
for (const directory of directories) await rm(directory, { recursive: true, force: true });
|
||||
directories.clear();
|
||||
});
|
||||
|
||||
const Parameters = Type.Object({});
|
||||
|
||||
function tool(
|
||||
name: string,
|
||||
execute: ToolRegistration<typeof Parameters>["execute"],
|
||||
): ToolRegistration<typeof Parameters> {
|
||||
return defineTool({ name, description: name, parameters: Parameters, execute });
|
||||
}
|
||||
|
||||
/** A tool-calling answer with one `noop` call. */
|
||||
function round(id: string): AssistantMessage {
|
||||
return fauxAssistantMessage([fauxToolCall("noop", {}, { id })], { stopReason: "toolUse" });
|
||||
}
|
||||
|
||||
const DONE = fauxAssistantMessage([fauxText("done")]);
|
||||
|
||||
/** A chat with a `noop` tool and the given turn limit. */
|
||||
async function limited(maxTurns: number | undefined, configure?: (setup: ChatSetup) => void) {
|
||||
const setup = chatSetup();
|
||||
addTool(
|
||||
setup.registry,
|
||||
tool("noop", async () => ({ output: [] })),
|
||||
);
|
||||
configure?.(setup);
|
||||
const opened = await openChat(new MemoryStorage(), setup);
|
||||
if (maxTurns !== undefined) await opened.root.configure({ maxTurns }, context);
|
||||
return { setup, ...opened };
|
||||
}
|
||||
|
||||
async function lastKind(root: Conversation): Promise<string | undefined> {
|
||||
return (await allEntries(root)).at(-1)?.kind;
|
||||
}
|
||||
|
||||
describe("turn limit", () => {
|
||||
// #11043
|
||||
it("ends a run after the tool round that reaches the limit, like a failure", async () => {
|
||||
const { setup, harness, root } = await limited(1);
|
||||
setup.faux.setResponses([round("c1"), DONE]);
|
||||
const submission = await root.submit({ type: "input", content: "go" }, context);
|
||||
expect(await submission.wait(context)).toMatchObject({
|
||||
status: "unanswered",
|
||||
reason: "turn_limit",
|
||||
detail: "1 turn",
|
||||
});
|
||||
expect(setup.faux.state.callCount).toBe(1);
|
||||
// The transcript ends with the round's results; the generation completed, so nothing is reported.
|
||||
expect(await lastKind(root)).toBe(ToolResultEntry.kind);
|
||||
expect((await harness.snapshot(LiveDoc, root.id, context))?.runFailure).toMatchObject({
|
||||
reason: "turn_limit",
|
||||
inputs: [submission.id],
|
||||
});
|
||||
const generations = await harness.commit((tx) => tx.scanTasks({ conversationId: root.id }, 20), context);
|
||||
const generation = generations.items.find((task) => task.kind === GenerationTask.definition.name);
|
||||
expect(generation?.state).toMatchObject({ status: "terminal", outcome: { status: "completed" } });
|
||||
expect(setup.reports).toEqual([]);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("counts tool rounds only: a run answers below its limit, and an answer at the limit settles done", async () => {
|
||||
const { setup, harness, root } = await limited(2);
|
||||
setup.faux.setResponses([round("c1"), DONE, round("c2"), round("c3"), DONE]);
|
||||
expect((await (await root.submit({ type: "input", content: "one" }, context)).wait(context)).status).toBe("done");
|
||||
// A new run starts with a fresh budget.
|
||||
const second = await (await root.submit({ type: "input", content: "two" }, context)).wait(context);
|
||||
expect(second).toMatchObject({ status: "unanswered", reason: "turn_limit", detail: "2 turns" });
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("runs onYield at the limit, and a continuation starts the count over", async () => {
|
||||
let yields = 0;
|
||||
const { setup, harness, root } = await limited(2, (chat) =>
|
||||
addHooks(chat.registry, GenerationTask, {
|
||||
onYield: () => (yields++ === 0 ? { continue: "keep going" } : undefined),
|
||||
}),
|
||||
);
|
||||
// Round (1 turn), answer: the continuation resets the count; then two rounds reach the limit.
|
||||
setup.faux.setResponses([round("c1"), DONE, round("c2"), round("c3"), DONE]);
|
||||
const settled = await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
|
||||
expect(settled).toMatchObject({ status: "unanswered", reason: "turn_limit" });
|
||||
expect(yields).toBe(1);
|
||||
expect(setup.faux.state.callCount).toBe(4);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("settles a steer that joined the run at an earlier round with turn_limit too", async () => {
|
||||
const gate = deferred();
|
||||
const started = deferred();
|
||||
const setup = chatSetup();
|
||||
let calls = 0;
|
||||
addTool(
|
||||
setup.registry,
|
||||
tool("noop", async () => {
|
||||
if (calls++ === 0) {
|
||||
started.resolve();
|
||||
await gate.promise;
|
||||
}
|
||||
return { output: [] };
|
||||
}),
|
||||
);
|
||||
const { harness, root } = await openChat(new MemoryStorage(), setup);
|
||||
await root.configure({ maxTurns: 2 }, context);
|
||||
setup.faux.setResponses([round("c1"), round("c2"), DONE]);
|
||||
const first = await root.submit({ type: "input", content: "go" }, context);
|
||||
await started.promise;
|
||||
const steer = await root.submit({ type: "input", content: "also this", whenBusy: "steer" }, context);
|
||||
gate.resolve();
|
||||
for (const submission of [first, steer]) {
|
||||
expect(await submission.wait(context)).toMatchObject({
|
||||
status: "unanswered",
|
||||
reason: "turn_limit",
|
||||
detail: "2 turns",
|
||||
});
|
||||
}
|
||||
expect((await harness.snapshot(LiveDoc, root.id, context))?.runFailure?.inputs).toEqual([first.id, steer.id]);
|
||||
expect(setup.faux.state.callCount).toBe(2);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("leaves items queued during the last round to afterFailedRun", async () => {
|
||||
for (const policy of ["keep", "continue", "withdraw"] as const satisfies readonly AfterFailedRun[]) {
|
||||
const gate = deferred();
|
||||
const started = deferred();
|
||||
const setup = chatSetup();
|
||||
setup.settings.afterFailedRun = policy;
|
||||
addTool(
|
||||
setup.registry,
|
||||
tool("noop", async () => {
|
||||
started.resolve();
|
||||
await gate.promise;
|
||||
return { output: [] };
|
||||
}),
|
||||
);
|
||||
setup.faux.setResponses([round("c1"), DONE]);
|
||||
const { harness, root } = await openChat(new MemoryStorage(), setup);
|
||||
await root.configure({ maxTurns: 1 }, context);
|
||||
const first = await root.submit({ type: "input", content: "go" }, context);
|
||||
await started.promise;
|
||||
const steer = await root.submit({ type: "input", content: "also", whenBusy: "followUp" }, context);
|
||||
gate.resolve();
|
||||
expect((await first.wait(context)).status).toBe("unanswered");
|
||||
await harness.waitForIdle(context);
|
||||
const queued = await steer.status(context);
|
||||
if (policy === "keep") expect(queued.status).toBe("queued");
|
||||
if (policy === "continue") expect(queued.status).toBe("done");
|
||||
if (policy === "withdraw") expect(queued).toMatchObject({ status: "unanswered", reason: "run_failed" });
|
||||
await harness.close(context);
|
||||
}
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("applies a changed limit at the end of the round; null removes it", async () => {
|
||||
const setup = chatSetup();
|
||||
let root!: Conversation;
|
||||
let rounds = 0;
|
||||
addTool(
|
||||
setup.registry,
|
||||
tool("noop", async () => {
|
||||
rounds++;
|
||||
// Lowered during the second round: that round ends the run.
|
||||
if (rounds === 2) await root.configure({ maxTurns: 1 }, context);
|
||||
return { output: [] };
|
||||
}),
|
||||
);
|
||||
setup.faux.setResponses([round("c1"), round("c2"), round("c3"), DONE]);
|
||||
const opened = await openChat(new MemoryStorage(), setup);
|
||||
root = opened.root;
|
||||
await root.configure({ maxTurns: 5 }, context);
|
||||
const settled = await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
|
||||
expect(settled).toMatchObject({ status: "unanswered", reason: "turn_limit", detail: "2 turns" });
|
||||
await root.configure({ maxTurns: null }, context);
|
||||
expect((await root.agent(context)).maxTurns).toBeUndefined();
|
||||
setup.faux.setResponses([round("c4"), round("c5"), DONE]);
|
||||
expect((await (await root.submit({ type: "input", content: "again" }, context)).wait(context)).status).toBe(
|
||||
"done",
|
||||
);
|
||||
await opened.harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("rejects a limit that is not an integer 1 or more", async () => {
|
||||
const { harness, root } = await limited(undefined);
|
||||
for (const value of [0, -1, 1.5, Number.NaN, Number.POSITIVE_INFINITY]) {
|
||||
await expect(root.configure({ maxTurns: value }, context)).rejects.toBeInstanceOf(RangeError);
|
||||
}
|
||||
await root.configure({ maxTurns: 3 }, context);
|
||||
expect((await root.agent(context)).maxTurns).toBe(3);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("lets terminate settle done at the limit, and writes a handoff's reset before ending", async () => {
|
||||
const { setup, harness, root } = await limited(1, (chat) =>
|
||||
addHooks(chat.registry, ToolTask, {
|
||||
afterTool: (call, result) =>
|
||||
call.id === "t"
|
||||
? { ...result, control: { terminate: true } }
|
||||
: { ...result, control: { handoff: "fresh" } },
|
||||
}),
|
||||
);
|
||||
setup.faux.setResponses([round("t"), round("h"), DONE]);
|
||||
expect((await (await root.submit({ type: "input", content: "one" }, context)).wait(context)).status).toBe("done");
|
||||
const handedOff = await (await root.submit({ type: "input", content: "two" }, context)).wait(context);
|
||||
expect(handedOff).toMatchObject({ status: "unanswered", reason: "turn_limit" });
|
||||
const resets = (await allEntries(root)).filter((entry) => ResetEntry.is(entry));
|
||||
expect(resets.map((entry) => (entry.model?.[0] as { content: string }).content)).toEqual(["fresh"]);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("is resumed by a continue, with a fresh budget, on the round's results", async () => {
|
||||
const { setup, harness, root } = await limited(1);
|
||||
let continued: string | undefined;
|
||||
setup.faux.setResponses([
|
||||
round("c1"),
|
||||
(request) => {
|
||||
continued = request.messages.at(-1)?.role;
|
||||
return fauxAssistantMessage([fauxText("finished")]);
|
||||
},
|
||||
]);
|
||||
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
|
||||
const resumed = await root.submit({ type: "continue" }, context);
|
||||
expect((await harness.snapshot(LiveDoc, root.id, context))?.run?.turns).toBe(0);
|
||||
const settled = await resumed.wait(context);
|
||||
if (settled.status !== "done" || settled.type === "write") throw new Error(`Unexpected ${settled.status}`);
|
||||
const answer = (await allEntries(root)).find((entry) => entry.id === settled.answer);
|
||||
expect(textOf(answer?.model?.[0])).toBe("finished");
|
||||
expect(continued).toBe("toolResult");
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("is inherited by forks and children, which may set their own", async () => {
|
||||
const setup = chatSetup();
|
||||
let child: number | undefined;
|
||||
addTool(
|
||||
setup.registry,
|
||||
tool("spawn", async (_args, api, callContext) => {
|
||||
child = await api.commit(
|
||||
async (tx) => (await tx.createConversation({ ownership: { kind: "task", taskId: api.taskId } })).id,
|
||||
callContext,
|
||||
);
|
||||
return { output: [] };
|
||||
}),
|
||||
);
|
||||
setup.faux.setResponses([
|
||||
fauxAssistantMessage([fauxToolCall("spawn", {}, { id: "c1" })], { stopReason: "toolUse" }),
|
||||
DONE,
|
||||
]);
|
||||
const { harness, root } = await openChat(new MemoryStorage(), setup);
|
||||
await root.configure({ maxTurns: 4 }, context);
|
||||
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
|
||||
const spawned = await harness.conversation(child as never, context);
|
||||
expect((await spawned!.agent(context)).maxTurns).toBe(4);
|
||||
const at = (await allEntries(root)).at(-1)!.id;
|
||||
const fork = await root.fork(at, { ownership: { kind: "ownerless" } }, context);
|
||||
expect((await fork.agent(context)).maxTurns).toBe(4);
|
||||
const own = await harness.createConversation(
|
||||
{ ownership: { kind: "ownerless" }, agent: { maxTurns: 2 } },
|
||||
context,
|
||||
);
|
||||
expect((await own.agent(context)).maxTurns).toBe(2);
|
||||
await harness.close(context);
|
||||
});
|
||||
|
||||
// #11043
|
||||
it("keeps the stored count across a restart", async () => {
|
||||
const directory = await mkdtemp(join(tmpdir(), "pi-durable-turns-"));
|
||||
directories.add(directory);
|
||||
const path = join(directory, "session.sqlite");
|
||||
const setup = chatSetup();
|
||||
setup.settings.progress = { outputIntervalMs: 0, persistWithinMs: 0 };
|
||||
const started = deferred();
|
||||
let runs = 0;
|
||||
addTool(
|
||||
setup.registry,
|
||||
defineTool({
|
||||
name: "noop",
|
||||
description: "noop",
|
||||
parameters: Parameters,
|
||||
replay: "safe",
|
||||
execute: async (_args, _api, callContext) => {
|
||||
runs++;
|
||||
if (runs === 2) {
|
||||
started.resolve();
|
||||
return aborted(callContext.abortSignal!);
|
||||
}
|
||||
return { output: [] };
|
||||
},
|
||||
}),
|
||||
);
|
||||
setup.faux.setResponses([round("c1"), round("c2"), DONE]);
|
||||
const inner = await openNodeSqliteStorage(path);
|
||||
let dead = false;
|
||||
const storage = new Proxy(inner, {
|
||||
get(target, property) {
|
||||
const value = Reflect.get(target, property, target);
|
||||
if (typeof value !== "function") return value;
|
||||
return (...args: unknown[]) => (dead ? new Promise(() => {}) : value.apply(target, args));
|
||||
},
|
||||
});
|
||||
const first = await openChat(storage, setup);
|
||||
await first.root.configure({ maxTurns: 2 }, context);
|
||||
const id = (await first.root.submit({ type: "input", content: "go" }, context)).id;
|
||||
await started.promise;
|
||||
expect((await first.harness.snapshot(LiveDoc, first.root.id, context))?.run?.turns).toBe(2);
|
||||
dead = true;
|
||||
killed.add(inner);
|
||||
|
||||
// The replay-safe call runs again after reopen; the stored count still ends the run at this round.
|
||||
runs = 0;
|
||||
const second = await openChat(await openNodeSqliteStorage(path), setup);
|
||||
second.harness.resume();
|
||||
expect((await second.harness.snapshot(LiveDoc, second.root.id, context))?.run?.turns).toBe(2);
|
||||
const settled = await (await second.harness.submission(id, context))!.wait(context);
|
||||
expect(settled).toMatchObject({ status: "unanswered", reason: "turn_limit", detail: "2 turns" });
|
||||
await second.harness.close(context);
|
||||
});
|
||||
});
|
||||
|
|
@ -103,7 +103,7 @@ describe("a run as the conversation view shows it", () => {
|
|||
expect(partialsIn(states).map(textOf)).toEqual(partials.map(textOf));
|
||||
// The run shows from the input's commit until the commit that appends the answer and clears the partial.
|
||||
const started = states.findIndex((state) => state.run !== undefined);
|
||||
expect(states[started]!.run).toEqual({ taskId: expect.any(Number), inputs: [submission.id] });
|
||||
expect(states[started]!.run).toEqual({ taskId: expect.any(Number), inputs: [submission.id], turns: 0 });
|
||||
expect(history.views[started]!.entries.at(-1)!.kind).toBe("pi.user");
|
||||
const ended = states.findIndex((state, index) => index > started && state.run === undefined);
|
||||
expect(states[ended]!.generation).toBeUndefined();
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue