mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
docs(agent): consolidate harness runtime plan
This commit is contained in:
Mario Zechner
parent
a5c013c0c6
commit
34bad4471a
7 changed files with 2532 additions and 972 deletions
348
packages/agent/docs/assistant-durability.md
Normal file
348
packages/agent/docs/assistant-durability.md
Normal file
|
|
@ -0,0 +1,348 @@
|
|||
# Assistant partial durability — implementation handoff
|
||||
|
||||
This document specifies durable partial assistant messages for ordinary assistant generation and deferred-response polling. It builds on:
|
||||
|
||||
- bound typed value/list addresses from `registers.md`;
|
||||
- `AssistantMessageFrame`, `assistantMessageEventToFrame()`, and `reduceAssistantMessageFrames()` from `@earendil-works/pi-ai`;
|
||||
- the assistant intent/effect/settlement state machine in `harness.md`.
|
||||
|
||||
The design persists compact replayable stream frames without making them operation-state authority and without storing a growing full partial message on every update.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Reconstruct the latest committed partial assistant message after process loss.
|
||||
2. Preserve provider stream order without repeated full-snapshot write amplification.
|
||||
3. Avoid provider backpressure from storage commits.
|
||||
4. Reuse pi-ai's canonical frame conversion and reduction semantics.
|
||||
5. Preserve current public assistant event ordering.
|
||||
6. Keep operation `effect_pending` state authoritative for recovery.
|
||||
7. Delete all partial frames atomically with normal or synthetic response settlement.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Inferring provider completion from frames.
|
||||
- Persisting terminal `done`/`error` events separately from response settlement.
|
||||
- Exactly-once provider requests.
|
||||
- Public frame cursors or a frame-persistence event.
|
||||
- Generic batching, timers, coalescing, or flush APIs.
|
||||
- Persisting structural summary-generation streams.
|
||||
- Persisting arbitrary provider SDK events or repeated full `partial` snapshots.
|
||||
|
||||
## Storage
|
||||
|
||||
Built-in address constructor in `session/values.ts`:
|
||||
|
||||
```ts
|
||||
export const pendingAssistantFrames = (
|
||||
operationId: string,
|
||||
responseEntryId: string,
|
||||
) => list<AssistantMessageFrame>(
|
||||
"pi.pending.assistant_frame",
|
||||
`${operationId}:${responseEntryId}`,
|
||||
);
|
||||
```
|
||||
|
||||
The procedure binds one exact address for ordinary generation or a deferred poll:
|
||||
|
||||
```ts
|
||||
const frames = pendingAssistantFrames(operationId, responseEntryId);
|
||||
```
|
||||
|
||||
`responseEntryId` is already reserved in assistant/deferred `effect_pending` state. No frame count, cursor, or list identity is stored in operation state, and later list operations receive only `frames`—never another key.
|
||||
|
||||
Each list element is one `AssistantMessageFrame`. The storage transaction's global write sequence orders frames. The list is auxiliary:
|
||||
|
||||
- missing is valid;
|
||||
- it does not prove request admission, completion, success, or failure;
|
||||
- it never selects the restart point;
|
||||
- base restore does not read it.
|
||||
|
||||
## Frame contract
|
||||
|
||||
For each pi-ai stream event:
|
||||
|
||||
```ts
|
||||
const frame = assistantMessageEventToFrame(event);
|
||||
```
|
||||
|
||||
- `start` and non-terminal block events produce a frame;
|
||||
- terminal `done` and `error` return `undefined` because final response settlement is separate;
|
||||
- later frames omit repeated growing `partial` snapshots;
|
||||
- text/thinking/tool end frames may contain one authoritative completed block value because some providers, including OpenAI Responses, can supply or correct content only at the end;
|
||||
- unfinished tool-call JSON is reconstructed by pi-ai's reducer;
|
||||
- completed tool-call arguments are parsed but remain unvalidated against a tool schema.
|
||||
|
||||
Do not define a second harness frame codec or reducer. Persistence stores the exported pi-ai value directly; hydration calls `reduceAssistantMessageFrames()`.
|
||||
|
||||
Provider event blocks may interleave. Storage and reduction rely on `contentIndex`, never block contiguity.
|
||||
|
||||
## Fresh stream scheduling
|
||||
|
||||
The simplest scheduling is deliberate: one list-append transaction per converted frame, enqueued without awaiting storage in the provider loop.
|
||||
|
||||
For every `start` or update event:
|
||||
|
||||
```text
|
||||
convert event to frame
|
||||
→ synchronously enqueue invocation-fenced appendList(frames, frame)
|
||||
→ attach the ordinary harness-fault observer to the returned promise
|
||||
→ replace process-local latestFrameWrite promise reference
|
||||
→ emit and await the existing message_start/message_update event
|
||||
→ consume the next provider event
|
||||
```
|
||||
|
||||
`appendList()` is called synchronously, so all frame mutations enter the lane line in provider-event order. The procedure does not await each write. Every promise is immediately observed for fault propagation before the latest reference replaces its predecessor. Model output limits bound the number and total size of queued frames; a response may queue at most its bounded output plus constant frame/transaction overhead.
|
||||
|
||||
The event side retains existing behavior: `AssistantStreamObserver.start/update` awaits `events.emit()` for every event. There is no separate latest event-delivery promise because no assistant event delivery remains outstanding when the provider loop advances.
|
||||
|
||||
When the provider stream settles:
|
||||
|
||||
```text
|
||||
stop frame admission
|
||||
→ await latestFrameWrite when present
|
||||
→ run after_response
|
||||
→ emit message_end
|
||||
→ classify and commit final response settlement
|
||||
```
|
||||
|
||||
The lane mutation line is FIFO, so completion of `latestFrameWrite` implies every earlier append completed. There is no array of promises, timer, batcher, active/waiting state, coalescer, or public/internal flush method.
|
||||
|
||||
A failed frame append faults the harness before `after_response` starts. The complete final response remains process-local and does not commit after a storage fault.
|
||||
|
||||
## Normal settlement
|
||||
|
||||
Every assistant response settlement that used a frame list deletes that exact list in the same transaction as the immutable response entry, usage, leaf, and next operation state:
|
||||
|
||||
```text
|
||||
TX[
|
||||
insert response entry R,
|
||||
insert usage U,
|
||||
setValue(laneLeaf(lane), R),
|
||||
deleteList(frames),
|
||||
setValue(operationState(operationId), classified next state)
|
||||
]
|
||||
```
|
||||
|
||||
This applies to:
|
||||
|
||||
- successful assistant responses;
|
||||
- provider `error`/`aborted` responses;
|
||||
- valid deferred responses;
|
||||
- deferred poll responses.
|
||||
|
||||
`after_response` may transform the final response before settlement. Frames preserve provider-stream observation, while the immutable entry remains the post-hook canonical result.
|
||||
|
||||
A crash after the final frame append but before settlement still restores `effect_pending`; frames do not turn a complete-looking draft into a settled response.
|
||||
|
||||
## Unknown-outcome generation recovery
|
||||
|
||||
An orphaned assistant generation `effect_pending` has no surviving provider stream. Activation:
|
||||
|
||||
1. constructs `frames = pendingAssistantFrames(operationId, responseEntryId)` from current typed state and reads bounded pages from that exact address;
|
||||
2. reduces frame values with `reduceAssistantMessageFrames()`;
|
||||
3. constructs a harness-owned synthetic assistant response under the already-reserved response ID;
|
||||
4. preserves reconstructed partial content and safe message identity metadata when frames exist;
|
||||
5. sets `stopReason: "error"`, zero usage, and an explicit interruption/unknown-outcome `errorMessage`/diagnostic;
|
||||
6. commits the synthetic response, zero-usage row, frame-list deletion, leaf, and ordinary retry/failure state atomically.
|
||||
|
||||
Required warning meaning:
|
||||
|
||||
```text
|
||||
The provider request was interrupted. The preceding content is the latest
|
||||
committed partial response; newer live output may be missing, and the external
|
||||
request outcome is unknown.
|
||||
```
|
||||
|
||||
If no start frame committed, the harness constructs the same synthetic error with empty content from the captured model/API identity.
|
||||
|
||||
The synthetic response follows ordinary assistant error classification:
|
||||
|
||||
- attempts remain → insert the error response and enter the ordinary retry-wait/next-attempt path;
|
||||
- cap reached → insert it and enter `failure_drain`.
|
||||
|
||||
Error responses remain durable transcript history but are omitted from later provider context by the existing projection rule. Partial tool calls inside the interrupted error response never execute.
|
||||
|
||||
Recovery does not run `after_response`: there is no trustworthy complete provider result to transform.
|
||||
|
||||
## Cancellation
|
||||
|
||||
A live cancelled provider stream settles through its ordinary final `aborted` response. All accepted frames are awaited first, and normal settlement deletes the list.
|
||||
|
||||
For restored cancelled assistant/deferred `effect_pending`, cancellation reconciliation:
|
||||
|
||||
- reduces available frames;
|
||||
- constructs a synthetic `aborted` response preserving the committed partial content;
|
||||
- uses zero usage and the existing reserved IDs;
|
||||
- atomically inserts the response and deletes the frame list;
|
||||
- starts no provider request and runs no response hook.
|
||||
|
||||
Cancellation wins classification even if reduced content appears complete.
|
||||
|
||||
## Deferred polling
|
||||
|
||||
A deferred poll that returns an `AssistantMessageEventStream` uses the same `pendingAssistantFrames(operationId, responseEntryId)` address constructor.
|
||||
|
||||
- normal pending/ready/error settlement deletes the poll's frame list;
|
||||
- restored cancellation synthesizes an aborted response from frames;
|
||||
- an unknown restored poll without a permit remains suspended and may expose its durable partial in snapshots;
|
||||
- when a poll permit replaces the unknown poll with fresh reserved response/usage IDs, that intent transaction deletes the abandoned old frame-list address;
|
||||
- the replacement poll starts a fresh list under its fresh response ID.
|
||||
|
||||
Frames never change deferred-handle equality or poll-number rules.
|
||||
|
||||
## Structural generation scope
|
||||
|
||||
Structural summary-generation streams remain process-local. They do not emit public assistant-message lifecycle and may span multiple nested provider requests before one structural publication. Existing attempt-level retry and usage recovery remains authoritative.
|
||||
|
||||
Do not store their intermediate text at a `pendingAssistantFrames(...)` address in this slice. If structural partial diagnostics become a requirement, add a separate explicitly scoped consumer rather than silently reusing transcript-assistant semantics.
|
||||
|
||||
## Snapshots and reconnect
|
||||
|
||||
`LaneSnapshot.streamingMessage` means the latest observed partial assistant message, not proof that a provider stream is currently attached.
|
||||
|
||||
Precedence:
|
||||
|
||||
1. newest process-local partial while a live stream is owned;
|
||||
2. otherwise, for assistant/deferred `effect_pending`, the value reduced from committed frame pages;
|
||||
3. otherwise absent.
|
||||
|
||||
A restored lane may therefore be `suspended` with a non-undefined `streamingMessage`. The field remains outside `transcript`; only `entry_added` moves a complete response into transcript history and clears the partial.
|
||||
|
||||
Snapshot hydration constructs the exact bound frame address from trusted typed operation state and enforces the assistant consumer's total frame/page budget. It does not scan arbitrary lists or perform a broad semantic restore audit.
|
||||
|
||||
Reconnect replays no historical `message_start` or `message_update` events. The snapshot carries the durable partial. Recovery later emits the ordinary recovery-tagged synthetic message lifecycle for the response it actually settles.
|
||||
|
||||
## Events
|
||||
|
||||
Live event ordering remains:
|
||||
|
||||
```text
|
||||
message_start
|
||||
→ message_update* each listener delivery awaited by provider loop
|
||||
→ await latest frame write
|
||||
→ after_response
|
||||
→ message_end
|
||||
→ atomic response settlement + frame-list delete
|
||||
→ entry_added
|
||||
→ usage
|
||||
```
|
||||
|
||||
Frame commits emit ordinary storage telemetry only. There is no public frame event and no claim that a `message_update` was durable. `entry_added` remains the only proof that the final assistant entry committed.
|
||||
|
||||
A crash may occur after a live update event but before its asynchronously queued frame append commits. Reconnect then shows the latest committed frame prefix, which may be older than the last live event.
|
||||
|
||||
## Close, fault, and external finalization
|
||||
|
||||
Close is a controlled crash:
|
||||
|
||||
- no synthetic response is written;
|
||||
- already-enqueued frame commits are ordinary admitted session work and may finish under the close barrier;
|
||||
- process loss may discard mutations that had not committed;
|
||||
- reopen restores the latest committed frame prefix under unchanged `effect_pending` state.
|
||||
|
||||
A frame-commit storage failure faults the harness. No later response settlement commits in that process.
|
||||
|
||||
Authorized external finalization deletes the operation-owned frame-list address in its terminal transaction. Every append mutation verifies current operation/response ownership on the lane line:
|
||||
|
||||
- append first → external terminal cleanup deletes the list;
|
||||
- finalization first → stale append rejects without recreating state.
|
||||
|
||||
## Terminal cleanup, forks, and migrations
|
||||
|
||||
Normal/synthetic response settlement should already delete its exact frame address. The operation terminal transaction also defensively constructs and deletes the current operation-owned frame address when state is assistant/deferred `effect_pending`.
|
||||
|
||||
Idle forks never copy lists in the `pi.pending.assistant_frame` address family. Precise rewrites and migrations page frame lists and preserve element sequences when retaining them.
|
||||
|
||||
A migration changing `AssistantMessageFrame` shape must map every surviving element or explicitly delete the whole list and leave `effect_pending` recovery with no partial. It must never infer completion from legacy frames.
|
||||
|
||||
JSONL retains deleted frame bytes until snapshot compaction. Logical deletion is immediate.
|
||||
|
||||
## Races
|
||||
|
||||
| Race | Required result |
|
||||
|---|---|
|
||||
| frame append vs next frame | synchronous lane enqueue preserves provider-event order |
|
||||
| frame append vs stream settlement | settlement awaits the latest promise; all accepted appends finish first |
|
||||
| live update event vs frame commit | either may finish first; event is observation and reconnect uses only committed frames |
|
||||
| append vs external finalization | append first is deleted by cleanup; finalization first fences the append |
|
||||
| process loss with queued writes | only the committed prefix restores |
|
||||
| final frame vs response settlement | frame commits first; settlement atomically deletes the list and inserts final entry |
|
||||
| activation vs snapshot | both reduce the same committed sequence prefix; activation may then settle and clear it |
|
||||
| unknown generation vs retry | synthetic partial error commits under old reserved IDs before later attempt starts |
|
||||
| unknown deferred poll vs replacement | old list is deleted with fresh replacement intent; new response ID gets a new list |
|
||||
|
||||
## Invariants
|
||||
|
||||
1. Scalar assistant/deferred state is the sole restart authority.
|
||||
2. One effect-pending response ID constructs exactly one assistant frame-list address.
|
||||
3. Every stored element is an exported pi-ai `AssistantMessageFrame`.
|
||||
4. Terminal `done`/`error` events are never stored as frames.
|
||||
5. Frame order equals provider event order.
|
||||
6. Awaiting the latest frame-write promise at stream settlement implies all accepted appends completed.
|
||||
7. Frames never establish provider completion or suppress unknown-outcome recovery.
|
||||
8. Final or synthetic response settlement atomically deletes the exact frame list.
|
||||
9. A restored partial may appear in `streamingMessage` but never in `transcript` before settlement.
|
||||
10. Interrupted partial tool calls never produce a tool plan because the synthetic response stops with `error`/`aborted`.
|
||||
11. Structural generation never writes a `pendingAssistantFrames(...)` list in this slice.
|
||||
12. Terminal cleanup, external finalization, and idle forks leave no operation-owned assistant frame list.
|
||||
|
||||
## Required tests
|
||||
|
||||
### Frame integration
|
||||
|
||||
- each convertible pi-ai event appends exactly one frame;
|
||||
- `done`/`error` append nothing;
|
||||
- interleaved content indexes preserve sequence;
|
||||
- provider loop does not await individual frame writes;
|
||||
- frame appends enqueue synchronously before the next provider event;
|
||||
- only the latest promise reference is retained;
|
||||
- awaiting the latest promise implies all earlier writes completed;
|
||||
- bounded output bounds queued frame memory;
|
||||
- storage failure prevents `after_response` and faults the harness.
|
||||
|
||||
### Settlement and recovery
|
||||
|
||||
- every ordinary response class deletes the frame list atomically;
|
||||
- crash at every frame/settlement boundary;
|
||||
- no frames and partial text/thinking/tool-call frames;
|
||||
- authoritative end-frame content survives;
|
||||
- interrupted generation commits partial synthetic error then retries or fails at cap;
|
||||
- interrupted partial tool calls never execute;
|
||||
- recovery uses zero usage and existing reserved IDs;
|
||||
- cancellation preserves committed partial content in synthetic aborted response;
|
||||
- `after_response` never runs for restored synthetic settlement.
|
||||
|
||||
### Deferred, snapshots, and lifecycle
|
||||
|
||||
- deferred poll frame persistence and normal cleanup;
|
||||
- unknown poll snapshot without permit;
|
||||
- replacement intent deletes abandoned poll frames;
|
||||
- live partial takes precedence over durable reduction;
|
||||
- reopen exposes reduced `streamingMessage` on suspended effect-pending lane;
|
||||
- no historical update event replay;
|
||||
- recovery settlement clears partial on `entry_added`;
|
||||
- structural generation writes no assistant frame list.
|
||||
|
||||
### Storage lifecycle
|
||||
|
||||
- Memory/JSONL/SQLite reduce identical frame sequences;
|
||||
- frame list is absent after normal, synthetic, cancellation, and external terminal transitions;
|
||||
- idle fork excludes frames;
|
||||
- JSONL compaction removes deleted frame bytes;
|
||||
- migration maps or explicitly discards every legacy frame;
|
||||
- instrumentation records append/delete ordering without frame content in telemetry.
|
||||
|
||||
## Implementation map
|
||||
|
||||
Expected runtime areas:
|
||||
|
||||
- built-in `pendingAssistantFrames(operationId, responseEntryId)` address constructor in `session/values.ts`;
|
||||
- bound value/list storage implementation from `registers.md`;
|
||||
- assistant execution observer and generation procedure;
|
||||
- deferred polling procedure;
|
||||
- activation and cancellation recovery;
|
||||
- snapshot hydration;
|
||||
- terminal cleanup, forks, and migrations;
|
||||
- instrumented writer and backend conformance tests.
|
||||
|
||||
Implement bound typed value/list addresses first, then frame enqueue/settlement, then recovery/snapshots. Update `harness.md` with this complete lifecycle before implementing runtime2 assistant parity.
|
||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
792
packages/agent/docs/runtime2.md
Normal file
792
packages/agent/docs/runtime2.md
Normal file
|
|
@ -0,0 +1,792 @@
|
|||
# AgentHarness `runtime2/` implementation status
|
||||
|
||||
## Goal
|
||||
|
||||
Build a second AgentHarness runtime from the bottom up under:
|
||||
|
||||
```text
|
||||
packages/agent/src/harness/runtime2/
|
||||
```
|
||||
|
||||
It must eventually export:
|
||||
|
||||
```ts
|
||||
export async function createAgentHarness<
|
||||
TContext extends object | undefined = object | undefined,
|
||||
>(
|
||||
options: AgentHarnessOptions<TContext>,
|
||||
): Promise<{
|
||||
harness: AgentHarness<TContext>;
|
||||
suspended: SuspendedOperation[];
|
||||
}>;
|
||||
```
|
||||
|
||||
Runtime2 replaces runtime1 through [WP00](work-packages/00-runtime1-removal.md), before further execution behavior is added. [WP01](work-packages/01-bound-values-lists.md) then converts the retained Session/storage surface across all existing backends. Do not release while the public runtime2 execution path is incomplete.
|
||||
|
||||
`harness.md` Part 8 and concrete `work-packages/*.md` files are the only active package plan. The R2.x sections below record historical checkpoints and design evidence; they are not executable work packages and must not override a handoff.
|
||||
|
||||
## Current baseline
|
||||
|
||||
Current pushed `dev` head:
|
||||
|
||||
```text
|
||||
03f576f54 feat(agent): inspect restored runtime2 operations
|
||||
├── 5030a5f53 feat(agent): add runtime2 harness config
|
||||
├── d5d656d4e feat(agent): add runtime2 lane creation
|
||||
├── 0697589ac fix(agent): tighten runtime2 harness boundaries
|
||||
└── earlier runtime2 restore/facade/close commits
|
||||
```
|
||||
|
||||
Runtime2 currently consists only of concrete `Harness` and `Lane` implementations plus restore/types. There is no separate runtime/controller abstraction. The oversized R2.4 acceptance/drive attempt was deliberately reverted in full; no TypeScript or test changes from that attempt remain. The focused runtime2 suite is back to the committed 44 tests across:
|
||||
|
||||
- `runtime2/restore.test.ts`
|
||||
- `runtime2/lane.test.ts`
|
||||
- `runtime2/harness.test.ts`
|
||||
|
||||
Current invariants:
|
||||
|
||||
- one restore constructs authoritative owned `LaneState`;
|
||||
- all lane-control writes route through `Lane.command()` and `Session.mutate()`;
|
||||
- command planners make state-dependent decisions under the lane line and return explicit `commit`, `return`, or `reject` outcomes;
|
||||
- commit commands publish memory after commit and synchronously materialize from `CommitResult`;
|
||||
- committed values and owned state are immutable snapshots updated only by replacement;
|
||||
- providers, tools, hooks, timers, events, and task waits stay outside the lane line;
|
||||
- the Lane-backed `sessionTree` routes leaf/branch reads and idle/run appends through owned state;
|
||||
- close/fault seal lanes before queued work can proceed, while admitted successful commits still publish;
|
||||
- harness creation never writes durable suspension; `suspension` is process-local inspection metadata and clears when committed ownership moves to another operation or idle;
|
||||
- durable control `running` means open/permitted; public `running` will mean a real process-local drive owns an actual continuation;
|
||||
- do not install a process owner, join promise, or `running` status around an effect-free classification shell; ownership lands with the first real drive procedure;
|
||||
- missing identity payloads are `{ tools: string[]; model?: string }`; one lane configuration can reference only one model;
|
||||
- runtime1 remains production-selected only until WP00 switches the factory and deletes it.
|
||||
|
||||
Latest committed validation before the reverted experiment: 44 focused runtime2 tests, 156 focused tests, full TypeScript, `git diff --check`, and `npm run check` passed. Revalidate after implementing the redesigned contract below.
|
||||
|
||||
## Slice completeness rule
|
||||
|
||||
A slice must close the entire named concern at parity, not only its immediately callable methods. Before editing, inventory all related constructor inputs, owned state, defaults, validation, public accessors, events, lifecycle behavior, recovery implications, and tests. Do not defer adjacent behavior merely because no current procedure consumes it yet. If scope must be deferred, name that boundary before implementation and get explicit agreement.
|
||||
|
||||
## Reading policy
|
||||
|
||||
`packages/agent/docs/harness.md` is the normative source. Until their designs are folded into it, `registers.md`, `tool-durability.md`, and `assistant-durability.md` are binding handoffs for bound typed value/list addresses, tool outcomes/checkpoints/memos, and assistant frames. Runtime1 is implementation inspiration only: do not preserve its abstractions, internal bookkeeping, or accidental behavior unless the normative contract independently requires them. For each slice, read its listed sections completely. If any handoff or `harness.md` changed since the previous slice, inspect that diff and reread affected sections.
|
||||
|
||||
Read a source or test file completely before editing that file. For dependencies that will not be edited, begin with the relevant exported types or functions and expand only when a concrete question requires it.
|
||||
|
||||
Do **not** pre-read later runtime phases. In particular, R2.0 must not load assistant procedures, tool procedures, or the complete R2–R4 behavioral suites. The purpose of the slices is to keep both implementation and reasoning local.
|
||||
|
||||
This plan is implementation guidance, not source authority. `harness.md` remains normative; correct this file whenever it disagrees with committed behavior or that contract.
|
||||
|
||||
### R2.0 — Restore one lane
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§0.2–0.3, 0.7–0.8, 1.3–1.4, 3.1–3.3, and 4.4;
|
||||
- the lane/configuration/operation definitions in `packages/agent/src/harness/session/types.ts` and built-in address constructors in `session/values.ts`;
|
||||
- repository/session helpers needed by the focused restore test;
|
||||
- only the exported shape of the existing `restoreLane()` when comparing terminology.
|
||||
|
||||
Do not read runtime1 procedures or R2–R4 behavior tests. Runtime2 intentionally trusts committed internal values and does not copy runtime1's semantic restore audit.
|
||||
|
||||
### R2.1 — Restore a configured session
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§2.3, 2.8, and the lane inventory portions of 4.4;
|
||||
- completed `runtime2/types.ts`, `runtime2/lane.ts`, and `runtime2/restore.ts`;
|
||||
- address-based `Session.getValue()`, `Session.scanValues()`, `Session.readList()`, and `Session.mutate()` signatures in `session/types.ts`;
|
||||
- focused session tests only when a repository helper is needed.
|
||||
|
||||
Do not read runtime1 initialization, provider, assistant, tool, or full harness-facade implementations.
|
||||
|
||||
### R2.1a — Lane owner and transition kernel
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§1.4, 3.4, 4.3, 4.7–4.8, and invariants 2, 21–23;
|
||||
- completed runtime2 restore/session-loader files;
|
||||
- `packages/agent/src/harness/session/lane-mutations.ts`;
|
||||
- `SessionMutator`, `Transaction`, and `CommitResult` definitions in `session/types.ts`;
|
||||
- focused mutation-line and instrumented-storage tests only when needed for test helpers.
|
||||
|
||||
Do not read provider, assistant, tool, or full harness-facade implementations.
|
||||
|
||||
### R2.2 — Main seeding and runtime shell
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§2.3, 2.8, 4.4–4.5, and 5.2;
|
||||
- completed runtime2 restore/session-loader files;
|
||||
- main-lane creation and configuration-seeding portions of `session/session.ts`;
|
||||
- only constructor/seeding cases in `agent-harness-runtime.test.ts`.
|
||||
|
||||
Do not read live assistant or tool procedures, and do not reintroduce runtime1's semantic restore audit.
|
||||
|
||||
### R2.3 — In-memory reads, config, lanes, and close
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§2.3–2.4, 4.3, 4.7–4.8, 5.1–5.5;
|
||||
- config, lane registry, facade delegation, close, and fault portions of `runtime/agent-harness-runtime.ts`;
|
||||
- query/configuration/public-session-view portions of `runtime/lane-runtime.ts`;
|
||||
- `packages/agent/src/harness/events.ts` and `hooks.ts` only for surfaces this slice instantiates;
|
||||
- corresponding shell/configuration/close tests in `agent-harness-runtime.test.ts` and session tests.
|
||||
|
||||
Do not read operation execution procedures.
|
||||
|
||||
### R2.4 — Contract reconciliation and atomic run acceptance
|
||||
|
||||
Read completely before editing:
|
||||
|
||||
- this document's “Superseding hook and execution design” and R2.4 section;
|
||||
- the current diff of `harness.md`, then §§3.1–3.6, 4.1–4.4, 5.1–5.3, 5.5–5.6, build rows R1–R2, and matching invariants/races; these sections still contain superseded hook semantics and must be reconciled first;
|
||||
- current `runtime2/harness.ts`, `lane.ts`, `types.ts`, `restore.ts`, and all focused runtime2 tests;
|
||||
- public request/result/event/hook types in `agent-harness.ts`;
|
||||
- neutral `hooks.ts`, `skills.ts`, `prompt-templates.ts`, bound value/list addresses and transaction helpers, and pending-entry reads;
|
||||
- runtime1 acceptance only to inventory production/public-contract consequences, never as architecture;
|
||||
- acceptance and restore cases in `agent-harness-r2.test.ts` and `restore.test.ts`.
|
||||
|
||||
Do not read provider/tool procedures or implement ownership in a no-work shell. Neutral harness imports remain allowed; `runtime2/` must never import `harness/runtime/*`.
|
||||
|
||||
### R2.5 — Minimal no-tool run parity
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§3.2, 3.7, 3.12–3.13, 4.1–4.2, 5.5–5.8;
|
||||
- `registers.md` assistant-frame consumer and `assistant-durability.md` fresh-stream/settlement sections;
|
||||
- pi-ai's exported assistant-message frame converter/reducer;
|
||||
- `execution/assistant.ts`;
|
||||
- checkpoint, ordinary-ready/intent/settlement, and terminal paths in:
|
||||
- `runtime/run-driver.ts`;
|
||||
- `runtime/checkpoint-procedure.ts`;
|
||||
- `runtime/assistant-procedure.ts`;
|
||||
- `runtime/transitions.ts`;
|
||||
- `execution-assistant.test.ts`;
|
||||
- `agent-harness-r2.test.ts`;
|
||||
- only R2-relevant restore cases in `restore.test.ts`.
|
||||
|
||||
Do not read R3 recovery/retry or R4 tool cases except where they share a type already established in R2.0.
|
||||
|
||||
### R2.6 — Generation recovery and retry parity
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: generation/deferred portions of §§3.2, 3.7, 4.4–4.6, and R3 rows in Parts 8–9;
|
||||
- `assistant-durability.md` recovery, cancellation, deferred, snapshot, and race sections;
|
||||
- assistant recovery, retry-wait, missing-identity, and deferred-suspension paths in `runtime/assistant-procedure.ts` and `runtime/run-driver.ts`;
|
||||
- retry/deferred classifiers in `runtime/transitions.ts`;
|
||||
- `agent-harness-r3.test.ts`;
|
||||
- R3-relevant cases in `restore.test.ts`.
|
||||
|
||||
Do not read tool-batch implementation or R4 tests.
|
||||
|
||||
### R2.7 — Tool parity
|
||||
|
||||
Read:
|
||||
|
||||
- `harness.md`: §§3.8, 4.5–4.6 tool portions, 5.6–5.8 tool portions, and the R4 rows in Parts 8–9;
|
||||
- `tool-durability.md` completely;
|
||||
- `execution/tools.ts`;
|
||||
- `runtime/tool-batch-procedure.ts`;
|
||||
- the tool transition helpers used by that procedure;
|
||||
- `execution-tools.test.ts`;
|
||||
- `agent-harness-r4.test.ts`;
|
||||
- tool-batch cases in `restore.test.ts`.
|
||||
|
||||
Read assistant code only in the slice where assistant settlement creates a tool plan.
|
||||
|
||||
### Expanding a slice's reads
|
||||
|
||||
If implementation exposes a concrete dependency not listed above:
|
||||
|
||||
1. state the specific unresolved question;
|
||||
2. locate the smallest defining source or test with `rg`;
|
||||
3. read that file or relevant section;
|
||||
4. do not recursively read its entire dependency graph;
|
||||
5. add it to this plan's slice list if future agents will need it.
|
||||
|
||||
## Value/list and durability prerequisites
|
||||
|
||||
Before further assistant/tool runtime slices, implement `registers.md` repo-wide:
|
||||
|
||||
- replace global namespace/value maps and register tokens with `value<T>(namespace, key?)` and `list<T>(namespace, key?)`;
|
||||
- put direct built-in address constructors in `session/values.ts`, using the universal `value()`/`list()` constructors and the `pi.*` namespaces/key grammars documented by `registers.md`;
|
||||
- expose `getValue`/`setValue`/`deleteValue`/`scanValues` and `readList`/`appendList`/`deleteList` without a second key argument; core lane inventory and cleanup use only the five exported `*Prefix` constructors as `scanValues` inputs;
|
||||
- add backend-conformant append/read/whole-list-delete storage;
|
||||
- add `pendingAssistantFrames(operationId, responseEntryId)` as a `ValueList<AssistantMessageFrame>` constructor.
|
||||
|
||||
Namespace `pi` and every `pi.*` namespace are reserved for built-ins by contract. Core and applications use the same `value()`/`list()` constructors; application misuse is a trusted-programming defect, not a runtime rejection. Runtime2 imports built-in addresses directly; do not add a private constructor, privilege token, bundle, catalog, runtime registry, `ValuePrefix` type, or runtime-specific storage abstraction. Every call receives an already-bound address. Stable physical namespace/key pairs preserve existing durable data; the new list table/records are a storage schema addition.
|
||||
|
||||
R2.5 owns fresh assistant frame append and cleanup. R2.6 owns frame-based generation/deferred recovery and snapshots. R2.7 owns `outcome_ready`, invocation memos, and tool checkpoint values exactly as specified by `tool-durability.md`.
|
||||
|
||||
## Non-goals until parity
|
||||
|
||||
Do not implement or mix in:
|
||||
|
||||
- R5 inbox/configuration/write behavior (`pendingNextRun` placement during R2.4 acceptance is the explicit exception; enqueue/cancel/consume behavior remains R5);
|
||||
- checkpoint hook;
|
||||
- caller-reserved queue IDs;
|
||||
- explicit owner-requested failure;
|
||||
- application-state semantics beyond migration to application-defined bound addresses;
|
||||
- durable state-shape normalization unrelated to the specified tool lifecycle;
|
||||
- summary/navigation slices beyond currently implemented R1–R4 behavior;
|
||||
- structural-summary partial-frame persistence;
|
||||
- compatibility shims between runtime1 and runtime2 internals.
|
||||
|
||||
## Core principle
|
||||
|
||||
Each lane has one in-memory owner while the harness instance exists.
|
||||
|
||||
```text
|
||||
create harness
|
||||
→ restore each lane once
|
||||
→ lane owns current leaf/config/queues/operation/last-result in memory
|
||||
→ all lane-affecting commands serialize through that owner
|
||||
→ compute private next state + exact transaction
|
||||
→ commit
|
||||
→ replace in-memory state
|
||||
→ publish observations
|
||||
```
|
||||
|
||||
External effects never hold the lane command line:
|
||||
|
||||
```text
|
||||
lane command: commit intent and publish pending state
|
||||
→ provider/tool/hook/timer effect outside lane owner
|
||||
→ lane command: settle against latest in-memory state
|
||||
```
|
||||
|
||||
A crash discards memory. The next `createAgentHarness()` restores the committed state again.
|
||||
|
||||
### Commit ordering
|
||||
|
||||
Never mutate the owned state before storage commit succeeds.
|
||||
|
||||
Correct:
|
||||
|
||||
```text
|
||||
current S0
|
||||
→ private candidate S1
|
||||
→ commit durable writes for S1
|
||||
→ lane.current = S1
|
||||
```
|
||||
|
||||
Incorrect:
|
||||
|
||||
```text
|
||||
lane.current = S1
|
||||
→ commit
|
||||
→ attempt rollback on failure
|
||||
```
|
||||
|
||||
The lane command remains serialized through commit and the memory replacement. A queued command sees either S0 before the transition or S1 after it, never an uncommitted state.
|
||||
|
||||
## Target directory rule
|
||||
|
||||
Runtime2 source must never import `../runtime/*.ts`. Copy the small primitive it needs into `runtime2/`, or move genuinely shared code to a neutral harness module. Runtime2 owns `Config`; do not reintroduce `RuntimeSettings`, a Runtime controller, or settings-line terminology.
|
||||
|
||||
Create files only as a slice needs them. Do not scaffold the final directory with empty modules.
|
||||
|
||||
Likely eventual shape—not a mandate:
|
||||
|
||||
```text
|
||||
runtime2/
|
||||
index.ts eventual stable runtime2 export
|
||||
types.ts lane/config types
|
||||
state.ts exhaustive refinements/invariants
|
||||
lane.ts owner + commit-then-publish kernel
|
||||
restore.ts one-time durable restore into lane memory
|
||||
harness.ts facade, registries, config, close
|
||||
operation.ts drive ownership/fencing
|
||||
run.ts direct phase dispatch
|
||||
assistant.ts intent/effect/settlement/recovery
|
||||
tools.ts batch intent/effect/outcome/recovery
|
||||
checkpoint.ts checkpoint/terminal procedure
|
||||
```
|
||||
|
||||
Merge modules when that is easier to read. Do not create helper files solely to reduce line counts.
|
||||
|
||||
## Public/test switching strategy
|
||||
|
||||
Runtime2 currently exports `createAgentHarness()` from `runtime2/harness.ts`. Add `runtime2/index.ts` only when constructor aliasing/test-selection work begins, then make it the stable runtime2 export. Until then, unimplemented runtime2 methods use the local `SliceNotImplemented` frontier from `runtime2/types.ts`.
|
||||
|
||||
Do not change the production import in:
|
||||
|
||||
```text
|
||||
packages/agent/src/harness/agent-harness.ts
|
||||
```
|
||||
|
||||
until parity.
|
||||
|
||||
### Preferred test selection
|
||||
|
||||
Add a runtime2 Vitest config or test alias that redirects only:
|
||||
|
||||
```text
|
||||
./runtime/agent-harness-runtime.ts
|
||||
```
|
||||
|
||||
as imported by `agent-harness.ts`, to:
|
||||
|
||||
```text
|
||||
./runtime2/harness.ts
|
||||
```
|
||||
|
||||
This allows existing tests to continue importing public `AgentHarness` unchanged. First prove the alias with a tiny constructor-selection test; do not trust an unverified Vite alias.
|
||||
|
||||
If exact aliasing is unreliable, use a test-only constructor:
|
||||
|
||||
```ts
|
||||
import type { AgentHarnessConstructor } from "../../src/harness/agent-harness.ts";
|
||||
import { createAgentHarness } from "../../src/harness/runtime2/harness.ts";
|
||||
|
||||
const Runtime2AgentHarness: AgentHarnessConstructor = {
|
||||
create: createAgentHarness,
|
||||
};
|
||||
```
|
||||
|
||||
Then parameterize only the scenarios runtime2 currently supports. Do not add a mutable production-global runtime selector.
|
||||
|
||||
When constructor aliasing begins, add `runtime2/index.ts`, point tests at that stable export, and verify selection with a tiny constructor test. At final parity, switch the one production import to `runtime2/index.ts`, run every existing test unchanged, then remove runtime1 in a separate reviewed change.
|
||||
|
||||
## Size and readability gates
|
||||
|
||||
Measure after every slice:
|
||||
|
||||
```bash
|
||||
find packages/agent/src/harness/runtime2 -name '*.ts' -print0 | xargs -0 wc -l
|
||||
```
|
||||
|
||||
Targets are diagnostic, not an excuse for compressed code:
|
||||
|
||||
- lane transition kernel: target ≤ 250 lines;
|
||||
- restore projection: target ≤ 250 lines;
|
||||
- facade/settings/close at R1 parity: target ≤ 450 lines;
|
||||
- no procedure file above 700 lines without explicit review;
|
||||
- R2 minimal no-tool parity: target total ≤ 2,000 lines;
|
||||
- R3 recovery/retry parity: target total ≤ 2,800 lines;
|
||||
- R4 tool parity: target total ≤ 3,400 lines.
|
||||
|
||||
Stop and review if:
|
||||
|
||||
- runtime2 exceeds a gate by more than 20%;
|
||||
- `restoreLane()` appears in an ordinary live transition;
|
||||
- the same ownership/phase check appears in three procedures;
|
||||
- a generic context object accumulates unrelated runtime methods;
|
||||
- a module cannot explain its protocol in one sentence;
|
||||
- code size is moving toward runtime1 without additional behavior.
|
||||
|
||||
Do not reduce lines by hiding transitions behind a generic reducer or callback framework.
|
||||
|
||||
# Historical runtime2 checkpoints
|
||||
|
||||
These sections explain how the current runtime2 shell was produced and preserve later design discoveries. They are not the active sequence. Execute only a concrete work-package handoff linked from `harness.md` Part 8.
|
||||
|
||||
## R2.0 — Restore one lane
|
||||
|
||||
### Goal
|
||||
|
||||
Load the current committed control values for one configured lane into a small process-local `Lane` without starting work or interpreting the state.
|
||||
|
||||
### Files
|
||||
|
||||
Create only:
|
||||
|
||||
```text
|
||||
runtime2/types.ts
|
||||
runtime2/lane.ts
|
||||
runtime2/restore.ts
|
||||
packages/agent/test/harness/runtime2/restore.test.ts
|
||||
```
|
||||
|
||||
### Contract
|
||||
|
||||
`LaneState` is the direct in-memory projection of:
|
||||
|
||||
```ts
|
||||
laneLeaf(lane)
|
||||
laneConfig(lane)
|
||||
laneState(lane)
|
||||
laneLastResult(lane) // when present
|
||||
operationMeta(operationId) // when currentOperationId is non-null
|
||||
operationState(operationId) // when currentOperationId is non-null
|
||||
```
|
||||
|
||||
`Lane` owns its name and one `LaneState`. It does not implement `AgentLane` yet.
|
||||
|
||||
Restore trusts committed internal values. It checks only that values required to construct the projection exist. It does not:
|
||||
|
||||
- validate referenced entries or semantic phase relationships;
|
||||
- hydrate entries, pending payloads, preparations, or tool arguments;
|
||||
- infer state from the transcript;
|
||||
- enumerate lanes or seed main configuration;
|
||||
- create hooks, events, effects, timers, drives, or public facade stubs.
|
||||
|
||||
Exact durable payload reads remain available to the later procedure that needs them. Live operation/phase/effect checks remain required when asynchronous work settles; they are concurrency fences, not restore validation.
|
||||
|
||||
### Tests
|
||||
|
||||
- idle configured lane;
|
||||
- optional latest result;
|
||||
- open operation metadata and state;
|
||||
- no interpretation of referenced payload ids;
|
||||
- missing required lane value;
|
||||
- missing current operation metadata/state.
|
||||
|
||||
### Stop condition
|
||||
|
||||
Report source and test line counts, focused test result, TypeScript result, and whether any additional runtime2 abstraction was required. Do not add a lane mutation kernel or session-wide loader yet.
|
||||
|
||||
## R2.1 — Restore a configured session
|
||||
|
||||
### Goal
|
||||
|
||||
Load every configured lane in one existing session into memory without starting work.
|
||||
|
||||
### Contract
|
||||
|
||||
Add one small session loader:
|
||||
|
||||
```ts
|
||||
restoreSession(session: Session): Promise<Map<string, Lane>>
|
||||
```
|
||||
|
||||
It calls `scanValues(laneLeafInventoryPrefix())` for the trusted lane inventory, requires `main`, and calls runtime2's one-lane restore exactly once for each listed lane under that lane's session mutation line. It does not seed configuration, scan other namespaces for orphaned values/lists, validate semantic state, or construct a public harness.
|
||||
|
||||
### Required tests
|
||||
|
||||
- configured main-only session;
|
||||
- multiple configured lanes;
|
||||
- open operations remain loaded and inert;
|
||||
- missing `main` rejects;
|
||||
- no hooks, providers, tools, timers, or writes;
|
||||
- each lane is restored exactly once.
|
||||
|
||||
### Stop condition
|
||||
|
||||
Report code size and focused tests. Do not add main seeding, a public factory, or lane mutation behavior.
|
||||
|
||||
## R2.1a — Lane owner and transition kernel
|
||||
|
||||
### Goal
|
||||
|
||||
Prove commit-then-publish on the real session mutation line without AgentHarness, providers, tools, hooks, events, or lifecycle machinery.
|
||||
|
||||
### Kernel contract
|
||||
|
||||
`Lane.transition()` accepts a synchronous planner. Inside `Session.mutate()` it:
|
||||
|
||||
1. plans a transaction, candidate state, and result from current memory;
|
||||
2. commits the transaction;
|
||||
3. replaces lane memory only after commit succeeds;
|
||||
4. returns the planned result before releasing the lane line.
|
||||
|
||||
External async effects cannot be passed through the planner. Storage-assigned entry metadata and observation publication are added only when a concrete transition requires them.
|
||||
|
||||
### Required tests
|
||||
|
||||
1. Memory remains unchanged while commit is pending.
|
||||
2. Commit failure leaves memory and storage unchanged.
|
||||
3. A queued transition is not planned until the prior commit publishes memory, then sees that memory.
|
||||
|
||||
### Stop condition
|
||||
|
||||
Report line count and focused tests. Do not add lifecycle faulting, close, observations, or execution behavior.
|
||||
|
||||
## R2.2 — Main seeding and runtime shell
|
||||
|
||||
### Goal
|
||||
|
||||
Seed an unconfigured `main`, call `restoreSession()`, and place the resulting lane map under one runtime2 harness shell without starting work.
|
||||
|
||||
### Behavior
|
||||
|
||||
- validate only constructor options needed to build the immutable lane seed;
|
||||
- commit the seed when `main` has no configuration;
|
||||
- call runtime2's configured-session loader;
|
||||
- create no drive task/provider/tool/hook/timer;
|
||||
- expose the exact `createAgentHarness()` signature only when the shell can satisfy it without casts or a speculative abstraction framework.
|
||||
|
||||
### Required tests
|
||||
|
||||
- fresh session/main seeding;
|
||||
- configured lanes are never overwritten by the seed;
|
||||
- idle and open multi-lane restore;
|
||||
- last result loaded once;
|
||||
- no effects during create;
|
||||
- no full restore after construction.
|
||||
|
||||
### Stop condition
|
||||
|
||||
The runtime2 shell owns the restored lane map. Do not implement operation acceptance or execution.
|
||||
|
||||
## R2.3 — In-memory reads, config, lanes, and close
|
||||
|
||||
### Goal
|
||||
|
||||
Make non-execution surfaces use owned memory.
|
||||
|
||||
### Completed
|
||||
|
||||
- `Harness` and `Lane` are the concrete `AgentHarness` and `AgentLane` implementations; there is no separate runtime/controller layer.
|
||||
- `Lane.command()` is the central effect-free serialized primitive with explicit `commit`, `return`, and `reject` outcomes.
|
||||
- Commit commands support bounded reads, one durable commit, memory publication, and synchronous `CommitResult` materialization.
|
||||
- Lane configuration getters/setters derive queued updates from current owned memory.
|
||||
- Lane lookup/listing and leaf/last-result reads use owned memory.
|
||||
- The `sessionTree` facade routes leaf/branch reads and idle/run appends through `Lane`; application values and lists remain Session-owned.
|
||||
- Close and fault synchronously seal every lane; close drains admitted session work without durable cancellation.
|
||||
- `Harness.createLane()` calls `Session.createLane()` directly with the captured seed, publishes a known idle candidate after commit, maps expected Session errors, and emits `lane_created` after map publication.
|
||||
- Harness-global config is one replace-only process-local `Config` snapshot. It captures mutable registries/policies plus immutable execution inputs (`systemPrompt`, `toolContext`, provider conversion, projectors, telemetry, drive mode, and tool-execution mode). Setters validate, publish synchronously, then emit `config_update`; no second mutation line exists.
|
||||
- Shared config defaults, validators, and missing-identity classification live in `packages/agent/src/harness/config.ts` and are used by both implementations.
|
||||
- Restored suspension descriptors hydrate run prompts and deferred handles once, classify captured missing identities, and stay owned by their lanes.
|
||||
- `Lane.inspectExecution()` and `Harness.lanes()` read the same owned state without storage access.
|
||||
- Application-defined values and lists pass directly through the Session facade while lanes are idle or active.
|
||||
- Focused tests cover ordering, commit failure, fault publication, close, no-commit promise boxing, expected rejection, appends, restore, lane creation, config, inspection, suspension classification, and application values and lists.
|
||||
|
||||
### Remaining R2.3 work
|
||||
|
||||
None. Watch surfaces have no runtime1 implementation and remain a later separate concern.
|
||||
|
||||
### Session boundary decision
|
||||
|
||||
Harness-owned lane/tree/operation writes route through `Lane.command()`. Application value/list reads and writes remain direct Session operations because they do not mutate lane control state.
|
||||
|
||||
## Superseding hook and execution design
|
||||
|
||||
The earlier R2.4 plan above was wrong. It put `before_run` inside acceptance, which forced an admission reservation, waiting appends, provisional ownership, two acceptance commands, and close/fault races before any durable operation existed. The rejected implementation then rebuilt runtime1's context/procedure shape through `OperationHost`, `OperationLocalState`, arbitration unions, and a fake `ActiveDrive` around a command that had no work. All of that code was reverted.
|
||||
|
||||
### Contract decisions
|
||||
|
||||
1. **`accept()` is one atomic lane command.** It normalizes and validates state-independent input outside the line, then under `Lane.command()` checks `state.operation`, captures current `pendingNextRun`, preflights identities, and commits caller/captured entries, leaf, `pi.op.meta`, `pi.op.state`, and `pi.lane.state`. There is no hook, provider, timer, process reservation, or process owner in acceptance. Concurrent accepts serialize naturally; the loser sees the committed operation and returns `LaneBusy`. An empty caller request with no captured entry still rejects; a later hook cannot rescue it.
|
||||
2. **A run is accepted in durable `{ kind: "starting" }` phase.** This payload-free phase is the only applied-once marker for initial durable extension context. It replaces acceptance-time hook output and avoids another stored value/flag.
|
||||
3. **`before_run` moves to the driver and returns only durable messages.** On the first real drive of `starting`, after ownership and deadline/cancellation checks, run the hook outside the line. Its consuming command appends returned messages and commits `starting → checkpoint` atomically. Crash before that commit may rerun the hook; crash after it cannot. Hook invocation is at-least-once, while its core-interpreted durable output is applied once. Injected messages become a stable conversation prefix and preserve provider KV-cache ordering.
|
||||
4. **Add `before_drive` as a pass prerequisite.** It runs once per newly installed real drive owner before recovery or ordinary effects; same-operation joiners do not rerun it. It runs again after yield, retry wait, deferred/missing-identity suspension, and process restart. Under `cancel_requested`, reconciliation wins and ordinary prerequisites do not run. A prerequisite failure rejects that drive pass without faulting the harness or writing durable progress; close/abort must interrupt or win through the normal gate.
|
||||
5. **Delete `before_resume`.** Extensions should not depend on process-origin categories. Durable phase decides dispatch/recovery. There is no public or internal `fresh | continue | restore` taxonomy. An unowned `effect_pending` phase is orphaned by construction once real owners span every admitted effect.
|
||||
6. **Delete `resumeData` and stable hook-ID routing.** Extension-private durable state belongs in extension-owned bound value/list addresses or audited custom entries keyed by lane/operation ID. Such writes are separate transactions and may orphan/replay; extensions own idempotency and cleanup, matching the exactly-once non-goal. Do not add transactional hook value writes until extension namespace, lane/session scope, version, and fork policy are designed explicitly.
|
||||
7. **Delete hook-produced durable `systemPromptOverride`.** Remove it from `before_run`, `BeforeResumePrepared`, and `OperationMeta`. Widen request-time `transform_context` to receive/return `{ messages, systemPrompt }`. Repeating the same effective system prompt preserves its cache key; changing it or earlier messages invalidates the prefix by design. Use `before_run` for durable message injection, not per-request `transform_context`.
|
||||
8. **Install a process owner only around real asynchronous work.** `before_drive`/`before_run` are real work, so the slice that implements them may install one simple Lane-owned work promise before invoking them. The model is direct: `accept()` checks durable `state.operation`; `drive()` joins the matching promise or installs one promise and starts actual work after the line. Do not add a controller, host adapter, local-state wrapper, action interpreter, or fake frontier pass.
|
||||
|
||||
### Hook API target
|
||||
|
||||
```ts
|
||||
before_run: {
|
||||
event: { prompt: AgentMessage[]; resources: Resources };
|
||||
result: { messages?: AgentMessage[] } | undefined;
|
||||
};
|
||||
before_drive: {
|
||||
event: { operation: "run" | "compaction" | "navigation" };
|
||||
result: void;
|
||||
};
|
||||
transform_context: {
|
||||
event: { messages: AgentMessage[]; systemPrompt: string };
|
||||
result: { messages?: AgentMessage[]; systemPrompt?: string } | undefined;
|
||||
};
|
||||
```
|
||||
|
||||
`HookInvocation` continues to add `lane` and `runId`. `before_run` is an at-least-once durable-settlement hook. `before_drive` is a fail-closed pass prerequisite. `transform_context` is request-local and replayed per provider request. Stable registration IDs are optional metadata only, not a durability protocol.
|
||||
|
||||
### Durable transitions
|
||||
|
||||
```text
|
||||
idle accept
|
||||
→ TX[ captured nextRun entries, caller entries, pending deletes,
|
||||
pi.lane.leaf, pi.op.meta(run prompt ids only),
|
||||
pi.op.state(run starting, running control, captured run settings, empty inbox),
|
||||
pi.lane.state(currentOperationId, captured ids removed) ]
|
||||
|
||||
starting + running control
|
||||
→ before_drive
|
||||
→ before_run
|
||||
→ TX[ injected message entries, pi.lane.leaf when injected,
|
||||
pi.op.state(checkpoint need_assistant(false), trigger=newest entry,
|
||||
skipInboxOnce=true, preserve concurrent inbox/control) ]
|
||||
|
||||
starting + cancel_requested
|
||||
→ cancellation reconciliation and aborted terminal transaction;
|
||||
invoke neither before_drive nor before_run
|
||||
```
|
||||
|
||||
Restoring `starting` reports the open run with caller prompt only and starts nothing. Public appends after acceptance already see an active run and enter its inbox; there is no pre-acceptance waiting window.
|
||||
|
||||
### Request reconstruction and cache discipline
|
||||
|
||||
DeepSeek Harness's useful precedent is that model-visible injected context becomes durable before dispatch, dynamic context snapshots are appended only when changed, and effective request headers can reconstruct requests. Pi should not copy Cordis or event sourcing, but R2.5 must decide before implementation whether to persist the effective model identity, stream options, system prompt, transformed message ordering, and tool schema identity before provider dispatch, or explicitly require deterministic/idempotent request transforms and accept weaker reconstruction. Do not casually use `transform_context` for timestamps, workspace notices, policy state, or other durable run context; place those through `before_run`.
|
||||
|
||||
### Broader hook audit to resolve before later slices
|
||||
|
||||
- Keep `before_request` for provider-neutral request options.
|
||||
- Reconsider `before_payload`: provider-specific wire mutation likely belongs in provider-adapter middleware.
|
||||
- Split tool policy from mutation: a monotonic allow/deny/ask guard cannot be overridden by listener order; an around-tool seam owns timeout/retry/tracing/cache/signal lifetime; a result transform precedes commit; a passive final-result event follows normalization and commit.
|
||||
- Reconsider `before_tool.args`: rewritten effective arguments currently disappear when operation-owned values are cleaned while the assistant entry retains original arguments, weakening auditability. Either remove rewriting or preserve effective arguments durably.
|
||||
- Prefer one configured compaction service and one branch-summary service over first-result-wins provider hooks; retain separate veto policy only if needed.
|
||||
- Replace latest-follow-up-wins `before_run_end` with an explicit `before_finish` continuation contract if multiple handlers can contribute.
|
||||
- Extension registrations eventually need scoped install/dispose and namespaced values/lists, but do not import a DI/plugin framework.
|
||||
|
||||
### Exactly-once statement
|
||||
|
||||
No external hook is globally exactly once. Core guarantees only that a hook's interpreted output commits before dependent durable progress. A crash before the consuming transaction may repeat the hook. External side effects require extension-owned idempotency keyed by stable operation/invocation IDs. `before_drive` is intentionally repeated per owned pass.
|
||||
|
||||
## R2.4 — Contract reconciliation and atomic run acceptance
|
||||
|
||||
### Required preparation
|
||||
|
||||
The current `harness.md`, shared `HookMap`, `OperationMeta`, runtime1 behavior, and parity tests still describe the superseded pre-acceptance hook/resumeData/systemPromptOverride contract. Reconcile the normative document and public type strategy before implementation. This is a real contract change. Because runtime1 remains production-selected and shares the public types, explicitly decide whether to update runtime1 to the new contract, isolate the experimental runtime2 contract temporarily, or move the production switch earlier. Do not silently preserve compatibility shims.
|
||||
|
||||
### Goal
|
||||
|
||||
Implement the smallest complete concern: the revised public/durable contract plus one-command prompt/skill/template acceptance into `RunPhase.starting`. Do not install drive ownership unless this same slice also implements real `before_drive`/`before_run` work and the `starting → checkpoint` transition.
|
||||
|
||||
### Required behavior
|
||||
|
||||
- prompt, skill, and prompt-template normalization;
|
||||
- write-nothing `Closed`, `LaneBusy`, missing-identity, pending-assistant, unknown-resource, and empty-input results;
|
||||
- captured `pendingNextRun` placement/deletion before caller entries;
|
||||
- exact acceptance transaction above;
|
||||
- owned memory publication after commit;
|
||||
- `run_start`, message lifecycle, `entry_added`, and trailing `queue_update` after publication;
|
||||
- accepted-undriven and accept→reopen `starting` inspection/restore;
|
||||
- no hook, breakpoint, process reservation, process owner, provider, tool, timer, or runtime1 import.
|
||||
|
||||
### Required tests
|
||||
|
||||
- exact transaction and durable-owned equality;
|
||||
- concurrent accepts serialize and only one commits;
|
||||
- append-vs-accept has only the natural two orders: append first becomes an earlier tree entry; accept first routes the append into the run inbox;
|
||||
- config-vs-accept snapshots one complete order;
|
||||
- all expected rejections write nothing;
|
||||
- commit failure leaves memory idle and faults;
|
||||
- admitted commit wins close and publishes;
|
||||
- close-first returns `Closed`;
|
||||
- pendingNextRun ordering and events;
|
||||
- reopen restores `starting` with caller prompt and starts no work.
|
||||
|
||||
Stop after this concern unless the user explicitly approves combining it with the real starting-phase drive slice.
|
||||
|
||||
## R2.5 — Minimal no-tool run parity
|
||||
|
||||
### Goal
|
||||
|
||||
Implement the first real drive owner together with all work it owns; never land an owner-only shell.
|
||||
|
||||
Implement direct flow:
|
||||
|
||||
```text
|
||||
drive arbitration installs one Lane-owned promise under the lane line
|
||||
→ same-id drives join; stale ids fence
|
||||
→ cancellation/deadline precheck
|
||||
→ before_drive prerequisite outside the line
|
||||
→ starting: before_run outside the line
|
||||
→ commit injected messages + starting → checkpoint
|
||||
→ generation.ready commit
|
||||
→ request-local context/system-prompt transform
|
||||
→ before_request/input preparation
|
||||
→ effect_pending intent commit
|
||||
→ provider effect; synchronously enqueue each converted assistant frame without awaiting storage
|
||||
→ await latest frame-write promise before after_response
|
||||
→ assistant settlement commit atomically deleting the frame list
|
||||
→ finish terminal commit
|
||||
→ remove owner only after final publication
|
||||
```
|
||||
|
||||
The owner is one ordinary Lane field, not a runtime/controller/context abstraction. Close/fault rejects it; effects and hooks remain outside `Lane.command()`. Reuse execution-only assistant mechanics where useful, never runtime1 orchestration. Before provider work, settle the request-reconstruction decision described above.
|
||||
|
||||
### Required tests
|
||||
|
||||
Run the complete R2 suite against runtime2, including exact writes, events, telemetry, breakpoints, close boundaries, convenience/primitive equivalence, one frame append per convertible event, no per-frame storage await, latest-promise settlement fencing, and atomic frame-list deletion.
|
||||
|
||||
### Size review
|
||||
|
||||
Target total runtime2 implementation ≤ 2,000 lines. If larger, stop before R3 and identify why rather than introducing context/framework abstractions.
|
||||
|
||||
## R2.6 — Generation recovery and retry parity
|
||||
|
||||
### Goal
|
||||
|
||||
Pass committed R3 behavior.
|
||||
|
||||
Implement:
|
||||
|
||||
- orphaned assistant `effect_pending` recovery when no live owner exists, reducing committed frames into the synthetic partial error before ordinary retry/failure classification;
|
||||
- repeated `before_drive` prerequisites on every newly installed pass;
|
||||
- synthetic cap settlement;
|
||||
- retry wait/wake/deadline policy;
|
||||
- deferred suspension/poll frame persistence, replacement cleanup, and snapshot foundation;
|
||||
- missing-identity suspension;
|
||||
- activation-only unknown-outcome policy.
|
||||
|
||||
Recovery reads only the already-restored in-memory state. A same-process safe yield/hosted wake must not pretend a crash occurred.
|
||||
|
||||
Run the complete R3 suite and scratch example against runtime2.
|
||||
|
||||
### Size review
|
||||
|
||||
Target total ≤ 2,200 lines.
|
||||
|
||||
## R2.7 — Tool parity
|
||||
|
||||
### Goal
|
||||
|
||||
Pass the complete target R4 contract, including `tool-durability.md`.
|
||||
|
||||
Implement:
|
||||
|
||||
- complete durable tool plans and stable invocation ID = result entry ID;
|
||||
- bound typed tool-argument, invocation-memo, progress-checkpoint, and staged-result addresses;
|
||||
- planned → effect_pending → outcome_ready → completed;
|
||||
- invocation-scoped `getMemo`/`setMemo` and Flue-style memoization support;
|
||||
- sequential and parallel execution with completion-order outcome staging;
|
||||
- source-ordered result materialization;
|
||||
- immediate/synthetic outcomes;
|
||||
- safe replay and unsafe interruption using the latest checkpoint value;
|
||||
- every selected checkpoint synchronously enqueued, latest promise retained, and latest awaited before `after_tool`;
|
||||
- hooks/events/usage/telemetry/breakpoints;
|
||||
- termination and cancellation handling at every tool state.
|
||||
|
||||
Parallel effects remain ordinary local promises. Intent, checkpoint, outcome-staging, and materialization commands queue through the lane owner. Current call state is read from memory, never via full restore. Do not reintroduce one-active/one-waiting checkpoint coalescing.
|
||||
|
||||
Run the complete R4 suite and scratch example against runtime2.
|
||||
|
||||
### Size review
|
||||
|
||||
Target total ≤ 3,000 lines; stretch ≤ 2,500. If runtime2 approaches runtime1 size, do not continue to parity cleanup without architectural review.
|
||||
|
||||
## Superseded parity/switch roadmap
|
||||
|
||||
The former R2.8 parity-first switch plan was superseded by the approved deletion strategy. Execute [WP00 — Runtime1 removal](work-packages/00-runtime1-removal.md), then [WP01 — Bound values and lists](work-packages/01-bound-values-lists.md). Those handoffs are the sole source for deletion lists, retained tests, validation, non-goals, and stop conditions. Do not recreate a dual-runtime parity gate or legacy smoke suite.
|
||||
|
||||
# Comparison discipline
|
||||
|
||||
The target contract and transaction tables in `harness.md`, plus the scoped redesign in this document until it is folded back, are the authority. Compare runtime2 behavior against those documents:
|
||||
|
||||
- transaction write arrays and durable value/entry state;
|
||||
- events and ordering;
|
||||
- telemetry spans/attributes;
|
||||
- breakpoints/details;
|
||||
- public results/errors and suspended descriptors;
|
||||
- storage read counts and source line counts.
|
||||
|
||||
Do not require parity with runtime1 on redesigned acceptance, hooks, FSM, restore validation, bound value/list storage, assistant partial durability, or tool durability. Until deletion, compare it only for scenarios whose documented contract is explicitly unchanged. After deletion, use the harvested scenario inventory and ordinary git history for archaeology; do not recreate a dual-runtime harness.
|
||||
|
||||
# Coding rules
|
||||
|
||||
- No `any`.
|
||||
- No dynamic/inline imports.
|
||||
- Erasable TypeScript syntax only.
|
||||
- Keep phase/ID checks beside the transition they protect.
|
||||
- No generic reducer/action framework.
|
||||
- No external effect inside the lane command line.
|
||||
- No mutation of published memory before commit.
|
||||
- No full restore in live procedures.
|
||||
- Inline one-use helpers.
|
||||
- Check external API types from source/node_modules.
|
||||
- Do not modify runtime1 except as required by WP00 deletion; never change the target contract to preserve it.
|
||||
- Do not touch unrelated coding-agent worktree files.
|
||||
- Do not commit unless the user asks.
|
||||
|
||||
# Validation
|
||||
|
||||
Use the active work-package handoff; it owns the exact test list. Every code package also runs agent/root TypeScript, `git diff --check`, and `npm run check`. Never run unrestricted Vitest, `npm test`, or paid-provider tests.
|
||||
|
||||
# Next assignment only
|
||||
|
||||
Execute [WP00 — Runtime1 removal](work-packages/00-runtime1-removal.md) and stop at its boundary. Then execute [WP01 — Bound values and lists](work-packages/01-bound-values-lists.md) as a separate package. Do not resume the historical R2.4–R2.8 sequence or begin provider execution without a new concrete handoff linked from `harness.md` Part 8.
|
||||
|
||||
The completed Fable and DeepSeek reviews are summarized in this document; no subagent session or temporary review checkout is expected to survive compaction.
|
||||
|
|
@ -28,7 +28,7 @@ The solution separates two orders:
|
|||
1. **outcome durability:** actual completion order;
|
||||
2. **entry materialization:** assistant source order.
|
||||
|
||||
A complete finalized result becomes durable immediately in `pending.entry`; the call becomes `outcome_ready`; placement happens later when every earlier source position is complete or ready.
|
||||
A complete finalized result becomes durable immediately in `pi.pending.entry`; the call becomes `outcome_ready`; placement happens later when every earlier source position is complete or ready.
|
||||
|
||||
## Goals
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ A complete finalized result becomes durable immediately in `pending.entry`; the
|
|||
- A nested durable state machine for each `step.do` call.
|
||||
- Inferring tool completion from progress output.
|
||||
- Treating partial output as the canonical final result.
|
||||
- Giving tools raw `Session`, `SessionMutator`, scalar-register, or list-register access.
|
||||
- Giving tools raw `Session`, `SessionMutator`, or bound storage-address access.
|
||||
- Finalizing the public harness-native tool type in this document.
|
||||
|
||||
## Durable identities
|
||||
|
|
@ -69,55 +69,52 @@ The surrounding operation state continues to provide:
|
|||
|
||||
## Storage
|
||||
|
||||
### Existing scalar namespaces
|
||||
|
||||
```text
|
||||
op.tool_args/{operationId}:{turnId}:{sourceIndex}
|
||||
Effective validated arguments, persisted before effect admission.
|
||||
|
||||
pending.entry/{resultEntryId}
|
||||
Complete finalized ToolResultMessage while outcome_ready awaits placement.
|
||||
```
|
||||
|
||||
### Invocation facts
|
||||
|
||||
Add an operation-owned scalar namespace:
|
||||
### Existing bound values
|
||||
|
||||
```ts
|
||||
interface RegisterValues {
|
||||
"op.tool_fact": JsonValue;
|
||||
}
|
||||
operationToolArgs(operationId, turnId, sourceIndex)
|
||||
// Effective validated arguments, persisted before effect admission.
|
||||
|
||||
pendingEntry(resultEntryId)
|
||||
// Complete finalized ToolResultMessage while outcome_ready awaits placement.
|
||||
```
|
||||
|
||||
Physical key:
|
||||
### Invocation memos
|
||||
|
||||
```text
|
||||
{operationId}:{invocationId}:{factName}
|
||||
Define the operation-owned address constructor in `session/values.ts`:
|
||||
|
||||
```ts
|
||||
export const operationToolMemo = (
|
||||
operationId: string,
|
||||
invocationId: string,
|
||||
memoName: string,
|
||||
) => value<JsonValue>(
|
||||
"pi.op.tool_memo",
|
||||
`${operationId}:${invocationId}:${memoName}`,
|
||||
);
|
||||
```
|
||||
|
||||
`factName` must be non-empty and contain no `:`. Names may use dots or slashes for application-local grouping. `setFact(name, undefined)` deletes the physical register.
|
||||
`memoName` must be non-empty and contain no `:`. Names may use dots or slashes for application-local grouping. `setMemo(name, undefined)` deletes the exact bound value.
|
||||
|
||||
The operation prefix permits defensive terminal cleanup. The invocation prefix permits atomic cleanup when one outcome becomes ready.
|
||||
`scanValues(operationToolMemoPrefix(operationId))` permits defensive operation cleanup. `scanValues(operationToolMemoPrefix(operationId, invocationId))` permits atomic invocation cleanup when one outcome becomes ready. Core cleanup uses these owner-defined prefix constructors rather than repeating raw reserved namespace/key grammar.
|
||||
|
||||
### Partial tool output
|
||||
|
||||
The durable recovery value is the latest complete bounded progress snapshot selected by the tool. That is total current state, so use a scalar namespace:
|
||||
The durable recovery value is the latest complete bounded progress snapshot selected by the tool. That is total current state, so use one bound value address:
|
||||
|
||||
```ts
|
||||
interface RegisterValues {
|
||||
"pending.tool_output": AgentToolResult<unknown>;
|
||||
}
|
||||
```
|
||||
|
||||
Physical key:
|
||||
|
||||
```text
|
||||
{operationId}:{invocationId}
|
||||
export const pendingToolOutput = (
|
||||
operationId: string,
|
||||
invocationId: string,
|
||||
) => value<AgentToolResult<unknown>>(
|
||||
"pi.pending.tool_output",
|
||||
`${operationId}:${invocationId}`,
|
||||
);
|
||||
```
|
||||
|
||||
This is auxiliary observation data. It never proves the effect succeeded or completed. The stored value has exactly the same content/details/usage shape as the live `partialResult`; recovery does not need a tool-specific progress codec.
|
||||
|
||||
The tool owns snapshot bounding, cadence, and duplicate suppression. The harness owns persistence, drop-coalescing, invocation fencing, and cleanup. The first API has no generic byte cap and does not truncate or reinterpret typed tool data; in-process tools are trusted to honor the bounded-snapshot contract. There is no tool-visible `flush()` method and no durable list of progress updates.
|
||||
The tool owns snapshot bounding, checkpoint cadence, and duplicate suppression. The harness owns synchronous enqueue, promise tracking, invocation fencing, and cleanup. The first API has no generic byte cap and does not truncate or reinterpret typed tool data; in-process tools are trusted to honor the bounded-snapshot contract. There is no tool-visible `flush()` method and no durable list of progress updates.
|
||||
|
||||
A crash may lose live updates newer than the latest committed checkpoint. JSONL physical growth is proportional to the size and frequency of distinct requested checkpoints until compaction; Memory and SQLite retain one current value. At 50 KiB every two seconds, JSONL's uncompacted worst case is approximately 15 MiB per ten minutes of continuously changing output.
|
||||
|
||||
|
|
@ -141,14 +138,15 @@ export type AgentHarnessToolUpdateCallback<TDetails> = (
|
|||
|
||||
Every callback invocation remains an immediate live update. The callback is synchronous and returns `void`; `checkpoint: true` additionally requests persistence of that complete snapshot. It is not a durability acknowledgement. Internally, the harness retains the latest `events.emit(tool_update)` promise so existing listener delivery completes before `after_tool`; tools do not await or receive that promise.
|
||||
|
||||
Checkpoint requests are drop-coalesced. The synchronous request path either immediately enqueues the active lane-line mutation or replaces the in-memory waiting value:
|
||||
Every `checkpoint:true` call synchronously enqueues one scalar replacement on the lane mutation line, attaches the ordinary harness-fault observer to that promise, and replaces the process-local `latestCheckpointWrite` reference. Writes themselves are neither dropped nor coalesced, and replacing the reference never leaves an earlier rejection unobserved:
|
||||
|
||||
- at most one checkpoint commit is active per invocation;
|
||||
- at most one newer requested snapshot waits;
|
||||
- a newer requested snapshot replaces the waiting one;
|
||||
- an admitted commit verifies the same call remains `effect_pending`;
|
||||
- tool-promise settlement closes checkpoint admission and drops the waiting, not-yet-enqueued snapshot; an active checkpoint is already ordered on the lane mutation line before outcome staging;
|
||||
- a failed admitted commit faults the harness under the ordinary storage-fault rule.
|
||||
- lane-line FIFO preserves request order;
|
||||
- each mutation verifies the same call remains `effect_pending`;
|
||||
- completion of the latest promise implies completion of every earlier checkpoint write;
|
||||
- tool-promise settlement stops accepting updates and awaits that latest promise before `after_tool`;
|
||||
- a failed checkpoint commit faults the harness under the ordinary storage-fault rule.
|
||||
|
||||
A tool that requests checkpoints faster than storage can queue work in memory. Under the trusted-tool contract, cadence is the tool's responsibility. The built-in bash policy bounds this queue in ordinary use.
|
||||
|
||||
Tools should compare against their last requested checkpoint when duplicate suppression matters. Storage does not read and deep-compare the current scalar as part of every checkpoint.
|
||||
|
||||
|
|
@ -197,8 +195,8 @@ type ToolCall =
|
|||
`outcome_ready` means:
|
||||
|
||||
- execution, error normalization, and `after_tool` have finished, or the harness has constructed a final synthetic result;
|
||||
- the complete final `ToolResultMessage` exists in `pending.entry/{resultEntryId}`;
|
||||
- invocation facts and partial-output storage are gone;
|
||||
- the complete final `ToolResultMessage` exists at `pendingEntry(resultEntryId)`;
|
||||
- invocation memos and partial-output storage are gone;
|
||||
- the tool must never execute again;
|
||||
- the immutable result entry may not exist yet because an earlier call has not materialized.
|
||||
|
||||
|
|
@ -232,7 +230,7 @@ Unchanged effect sandwich:
|
|||
planned
|
||||
→ prepare arguments, run before_tool, validate replacements
|
||||
→ TX[
|
||||
set op.tool_args,
|
||||
set pi.op.tool_args,
|
||||
set call = effect_pending(replay)
|
||||
]
|
||||
→ admit tool execution
|
||||
|
|
@ -246,11 +244,11 @@ Every `onUpdate(partialResult, options)` publishes the live update through the e
|
|||
|
||||
```text
|
||||
TX[
|
||||
set pending.tool_output/{operationId}:{invocationId} = partialResult
|
||||
setValue(pendingToolOutput(operationId, invocationId), partialResult)
|
||||
]
|
||||
```
|
||||
|
||||
The mutation verifies the same operation, turn, source position, and invocation remain `effect_pending`. It does not rewrite `op.state`. A late checkpoint after settlement returns without committing.
|
||||
The mutation verifies the same operation, turn, source position, and invocation remain `effect_pending`. It does not rewrite `pi.op.state`. A late checkpoint after settlement returns without committing.
|
||||
|
||||
The tool must checkpoint bounded complete snapshots, not growing unbounded values. Bash uses the same bounded `ShellCaptureProgress` snapshot it already sends live. Clients may render and locally retain live updates newer than the durable checkpoint, but those updates are explicitly process-local.
|
||||
|
||||
|
|
@ -258,25 +256,27 @@ The tool must checkpoint bounded complete snapshots, not growing unbounded value
|
|||
|
||||
When the tool promise settles:
|
||||
|
||||
1. synchronously stop accepting updates, expire the invocation capability, and close checkpoint admission;
|
||||
2. drop the waiting checkpoint; any active checkpoint or pre-return fact mutation is already ahead of staging on the lane mutation line;
|
||||
3. await the latest tracked `tool_update` delivery, which implies delivery of every earlier update;
|
||||
4. run `after_tool` when this is a real fresh or safely replayed result and cancellation did not prevent the hook;
|
||||
5. construct the complete final `ToolResultMessage`;
|
||||
6. emit and await `tool_end` for a real execution;
|
||||
7. commit the result as `outcome_ready`.
|
||||
1. synchronously stop accepting updates and expire the invocation capability;
|
||||
2. await the latest tracked `tool_update` delivery and latest checkpoint-write promise; each implies completion of its preceding queue;
|
||||
3. run `after_tool` when this is a real fresh or safely replayed result and cancellation did not prevent the hook;
|
||||
4. construct the complete final `ToolResultMessage`;
|
||||
5. emit and await `tool_end` for a real execution;
|
||||
6. commit the result as `outcome_ready`.
|
||||
|
||||
`setFact()` returns a promise and tools must await it; `step.do` always does. An unawaited pre-return mutation is still enqueued before staging and is deleted by staging. Calls begun after capability expiry reject. No separate invocation-write drain exists.
|
||||
`setMemo()` returns a promise and tools must await it; `step.do` always does. An unawaited pre-return mutation is still enqueued before staging and is deleted by staging. Calls begun after capability expiry reject. No separate invocation-write drain exists.
|
||||
|
||||
Transaction:
|
||||
|
||||
```text
|
||||
TX[
|
||||
set pending.entry/{resultEntryId}
|
||||
= { type: "message", payload: finalizedToolResultMessage },
|
||||
delete pending.tool_output/{operationId}:{invocationId},
|
||||
delete op.tool_fact/{operationId}:{invocationId}:*,
|
||||
set op.state call = outcome_ready(terminate)
|
||||
setValue(
|
||||
pendingEntry(resultEntryId),
|
||||
{ type: "message", payload: finalizedToolResultMessage },
|
||||
),
|
||||
deleteValue(pendingToolOutput(operationId, invocationId)),
|
||||
deleteValue(memo.address) for every memo returned before commit by
|
||||
scanValues(operationToolMemoPrefix(operationId, invocationId)),
|
||||
setValue(operationState(operationId), call = outcome_ready(terminate))
|
||||
]
|
||||
```
|
||||
|
||||
|
|
@ -309,17 +309,17 @@ Before the placement transaction, emit and await each finalized result's `messag
|
|||
|
||||
```text
|
||||
TX[
|
||||
insert result entry i from pending.entry/i,
|
||||
delete pending.entry/i,
|
||||
insert result entry i from pendingEntry(i),
|
||||
deleteValue(pendingEntry(i)),
|
||||
insert tool usage row i if reported,
|
||||
|
||||
insert result entry i+1 with parent = result i,
|
||||
delete pending.entry/i+1,
|
||||
deleteValue(pendingEntry(i+1)),
|
||||
insert tool usage row i+1 if reported,
|
||||
|
||||
set lane.leaf = newest result,
|
||||
set calls i..i+1 = completed,
|
||||
set next checkpoint in the same state write when the whole batch completed
|
||||
setValue(laneLeaf(lane), newest result),
|
||||
setValue(operationState(operationId),
|
||||
calls i..i+1 = completed and, when complete, next checkpoint)
|
||||
]
|
||||
```
|
||||
|
||||
|
|
@ -327,7 +327,7 @@ Each inserted entry uses its already-reserved `resultEntryId`. Writes construct
|
|||
|
||||
Tool-reported usage remains durable in the staged message until placement. The initial implementation writes its ledger row atomically with entry materialization, matching the current entry/usage ordering and avoiding a ledger row that references an entry not yet present. No usage ID reservation is needed because a failed placement transaction writes neither row nor completed state.
|
||||
|
||||
When the final call materializes, the same transaction deletes the batch's `op.tool_args` keys and transitions to the correct checkpoint:
|
||||
When the final call materializes, the same transaction calls `scanValues(operationToolArgsPrefix(operationId, turnId))` and deletes every returned address, and transitions to the correct checkpoint:
|
||||
|
||||
- every result terminates → `may_finish`, no final assistant required;
|
||||
- otherwise → `need_assistant(false)`.
|
||||
|
|
@ -360,13 +360,13 @@ The durable invariant changes from “completed calls form a source-ordered pref
|
|||
- after that prefix, parallel calls may be `planned`, `effect_pending`, or `outcome_ready` in any mixture;
|
||||
- only source-ordered materialization extends the completed prefix.
|
||||
|
||||
Sequential execution normally has at most one non-planned call after the completed prefix, but restore should validate this from the captured execution mode as it does today.
|
||||
Sequential execution constructs at most one non-planned call after the completed prefix. Committed call state is trusted on restore; the owning procedure enforces this shape while creating and consuming transitions rather than through a broad restore audit.
|
||||
|
||||
## Unsafe recovery with partial output
|
||||
|
||||
For orphaned `effect_pending` with `replay: "never"`:
|
||||
|
||||
1. read `pending.tool_output/{operationId}:{invocationId}` when present;
|
||||
1. read `pendingToolOutput(operationId, invocationId)` when present;
|
||||
2. preserve its bounded content and serializable details;
|
||||
3. append a mandatory human-readable interruption marker;
|
||||
4. construct a harness-owned `ToolResultMessage` with `isError: true`;
|
||||
|
|
@ -385,16 +385,16 @@ Rules:
|
|||
- do not run `after_tool` for this synthetic result;
|
||||
- preserve checkpoint `usage` when present, but ignore checkpoint `addedToolNames` and `terminate` because progress never has final-result authority;
|
||||
- set `terminate: false` and add no tools;
|
||||
- no partial register is also valid and yields only the interruption result;
|
||||
- an absent checkpoint value is also valid and yields only the interruption result;
|
||||
- never infer completion from an apparent success line in partial output;
|
||||
- cleanup of partial output and invocation facts is atomic with staging the synthetic result.
|
||||
- cleanup of partial output and invocation memos is atomic with staging the synthetic result.
|
||||
|
||||
## Safe recovery
|
||||
|
||||
For orphaned `effect_pending` where both the stored and current declarations are `replay: "safe"`:
|
||||
|
||||
1. retain invocation facts;
|
||||
2. atomically delete the previous `pending.tool_output` key;
|
||||
1. retain invocation memos;
|
||||
2. atomically delete `pendingToolOutput(operationId, invocationId)`;
|
||||
3. emit/reset process-local progress observation as needed;
|
||||
4. rerun the tool with persisted arguments and the same `invocationId`;
|
||||
5. completed `step.do` calls return their memoized values;
|
||||
|
|
@ -405,7 +405,7 @@ Deleting old progress prevents duplicate chunks when replayed code emits progres
|
|||
|
||||
If the current tool declaration is missing, suspend before replay. If it is present but no longer safe, use unsafe interruption recovery.
|
||||
|
||||
## Invocation facts
|
||||
## Invocation memos
|
||||
|
||||
The harness-native tool call receives a purpose-built invocation capability conceptually equivalent to:
|
||||
|
||||
|
|
@ -415,37 +415,37 @@ interface AgentHarnessToolInvocation {
|
|||
readonly operationId: string;
|
||||
readonly turnId: string;
|
||||
|
||||
getFact(key: string): Promise<JsonValue | undefined>;
|
||||
setFact(key: string, value: JsonValue | undefined): Promise<void>;
|
||||
getMemo(key: string): Promise<JsonValue | undefined>;
|
||||
setMemo(key: string, value: JsonValue | undefined): Promise<void>;
|
||||
}
|
||||
```
|
||||
|
||||
Each operation:
|
||||
|
||||
1. validates the fact name and checks process-local capability expiry;
|
||||
1. validates the memo name and checks process-local capability expiry;
|
||||
2. synchronously enqueues work on the lane mutation line before returning its promise;
|
||||
3. verifies the operation, turn, source position, and invocation are still the same `effect_pending` call when that job executes;
|
||||
4. reads or writes only the invocation's physical prefix;
|
||||
4. constructs and reads or writes only `operationToolMemo(operationId, invocationId, name)`;
|
||||
5. rejects after capability expiry or durable ownership loss.
|
||||
|
||||
The durable check matters for authorized external finalization. In ordinary execution, a fact mutation initiated before the tool returns is FIFO-ordered before outcome staging; one initiated afterward fails the expired-capability check.
|
||||
The durable check matters for authorized external finalization. In ordinary execution, a memo mutation initiated before the tool returns is FIFO-ordered before outcome staging; one initiated afterward fails the expired-capability check.
|
||||
|
||||
A late zombie callback can neither recreate facts after `outcome_ready` nor write into a later operation.
|
||||
A late zombie callback can neither recreate memos after `outcome_ready` nor write into a later operation.
|
||||
|
||||
Facts are immediate durable memo state, not application-visible settlement state. They survive close/crash while the call remains `effect_pending` and are deleted when any real or synthetic outcome becomes ready.
|
||||
Memos are immediate durable replay state, not application-visible settlement state. They survive close/crash while the call remains `effect_pending` and are deleted when any real or synthetic outcome becomes ready.
|
||||
|
||||
Terminal cleanup defensively deletes:
|
||||
Terminal cleanup defensively scans and deletes the operation-owned families:
|
||||
|
||||
```text
|
||||
op.tool_fact/{operationId}:*
|
||||
pending.tool_output/{operationId}:*
|
||||
```ts
|
||||
scanValues(operationToolMemoPrefix(operationId))
|
||||
scanValues(pendingToolOutputPrefix(operationId))
|
||||
```
|
||||
|
||||
in addition to existing operation-owned registers.
|
||||
in addition to other operation-owned addresses. Each returned `StoredValue` supplies its exact bound address for `deleteValue`; no later operation receives a raw key.
|
||||
|
||||
## Flue-style `step.do`
|
||||
|
||||
Build `step.do` over invocation facts; it does not need its own harness state union:
|
||||
Build `step.do` over invocation memos; it does not need its own harness state union:
|
||||
|
||||
```ts
|
||||
interface ToolSteps {
|
||||
|
|
@ -460,10 +460,10 @@ Algorithm:
|
|||
|
||||
```text
|
||||
validate deterministic unique name
|
||||
→ getFact("step/" + name)
|
||||
→ getMemo("step/" + name)
|
||||
→ present: return stored value
|
||||
→ absent: run effect
|
||||
→ setFact("step/" + name, value)
|
||||
→ setMemo("step/" + name, value)
|
||||
→ await durability
|
||||
→ return value
|
||||
```
|
||||
|
|
@ -487,9 +487,9 @@ Within one live execution, calling the same step name twice is an invariant erro
|
|||
|
||||
## Application persistent state
|
||||
|
||||
Flue-style application state is distinct from invocation facts.
|
||||
Flue-style application state is distinct from invocation memos.
|
||||
|
||||
Invocation fact:
|
||||
Invocation memo:
|
||||
|
||||
```text
|
||||
step completed → memo becomes visible immediately
|
||||
|
|
@ -503,14 +503,14 @@ tool stages state change
|
|||
→ outcome_ready: state and finalized result become visible together
|
||||
```
|
||||
|
||||
Do not implement application state by calling public `setCustomFact()` directly during execution.
|
||||
Do not implement application state by calling `Session.setValue()` directly during execution.
|
||||
|
||||
Two valid implementation stages:
|
||||
|
||||
1. Flue keeps its state externally and atomically stores application state plus a full result memo keyed by `invocationId`.
|
||||
2. A later harness API accepts staged application fact writes and promotes them in the `outcome_ready` transaction.
|
||||
2. A later harness API accepts staged application value writes and promotes them in the `outcome_ready` transaction.
|
||||
|
||||
The second option is required only if Flue's `usePersistentState` moves into harness-owned session facts. Its public typing and conflict semantics remain an open design item for the harness-native tool discussion.
|
||||
The second option is required only if Flue's `usePersistentState` moves into harness-owned session values. Its public typing and conflict semantics remain an open design item for the harness-native tool discussion.
|
||||
|
||||
## Cancellation
|
||||
|
||||
|
|
@ -520,7 +520,7 @@ Cancellation reconciliation never replays a restored tool.
|
|||
- a live started call may finalize its real local result under cancelled control, then become `outcome_ready` with `terminate: false`;
|
||||
- restored `effect_pending` calls use an interrupted synthetic result, optionally including partial output, regardless of safe replay declaration;
|
||||
- existing `outcome_ready` calls are preserved and materialized in source order;
|
||||
- invocation facts and partial output are deleted with each staged cancellation outcome;
|
||||
- invocation memos and partial output are deleted with each staged cancellation outcome;
|
||||
- no `before_tool` or `after_tool` starts during restored synthetic reconciliation.
|
||||
|
||||
The aborted terminal transaction runs only after every call outcome has materialized and accepted deferred writes have drained under the existing cancellation rules.
|
||||
|
|
@ -529,45 +529,36 @@ The aborted terminal transaction runs only after every call outcome has material
|
|||
|
||||
Close is still a controlled crash:
|
||||
|
||||
- fact/checkpoint mutations already enqueued under the admission barrier may finish;
|
||||
- memo/checkpoint mutations already enqueued under the admission barrier may finish;
|
||||
- live output newer than the latest committed tool-requested checkpoint may be lost;
|
||||
- no synthetic outcome or cancellation marker is written;
|
||||
- durable state remains at `effect_pending` or `outcome_ready`.
|
||||
|
||||
External finalization deletes operation-owned arguments, invocation facts, partial output, staged pending outcomes, and other pending entries in its terminal transaction. A live task that later tries to stage an outcome fails the ownership fence and stops through `OperationEnded`.
|
||||
External finalization deletes operation-owned arguments, invocation memos, partial output, staged pending outcomes, and other pending entries in its terminal transaction. A live task that later tries to stage an outcome fails the ownership fence and stops through `OperationEnded`.
|
||||
|
||||
## Restore and validation
|
||||
## Restore and consumption-time reads
|
||||
|
||||
Scalar restore remains authoritative. Extend direct hydration and semantic checks:
|
||||
Base restore constructs the trusted lane/operation projection from required owner values. It does not hydrate or semantically audit tool arguments, invocation memos, progress checkpoints, staged outcomes, completed entries, completed-prefix shape, or captured execution-mode relationships.
|
||||
|
||||
The procedure responsible for the current typed state performs only its exact consumption-time reads:
|
||||
|
||||
### `planned`
|
||||
|
||||
- result ID is reserved but has no entry or `pending.entry`;
|
||||
- no `op.tool_args` is required;
|
||||
- no partial output or invocation facts are required.
|
||||
Clearance needs no auxiliary restore read. It prepares the call and writes arguments before effect admission.
|
||||
|
||||
### `effect_pending`
|
||||
|
||||
- matching `op.tool_args` exists;
|
||||
- result entry and `pending.entry/{resultEntryId}` do not exist;
|
||||
- invocation facts may exist but are not required for base validation;
|
||||
- a partial-output scalar may exist but is auxiliary and must satisfy the trusted tool's bounded-snapshot contract.
|
||||
Activation reads `operationToolArgs(operationId, turnId, sourceIndex)` and optionally reads `pendingToolOutput(operationId, invocationId)`. Missing required arguments are an invariant defect at consumption. Invocation memos are read only through the scoped capability. Safe replay, unsafe interruption, and snapshots use no broad prefix scan.
|
||||
|
||||
### `outcome_ready`
|
||||
|
||||
- `pending.entry/{resultEntryId}` exists;
|
||||
- it contains a `ToolResultMessage` matching the source call's provider call ID and tool name;
|
||||
- immutable result entry does not yet exist;
|
||||
- invocation facts and partial output should be absent after the atomic staging transaction;
|
||||
- no tool identity is needed to materialize it.
|
||||
Materialization reads `pendingEntry(resultEntryId)`. Its absence or wrong trusted message relationship is an invariant defect when materialization consumes it. No tool identity or effect recovery is needed.
|
||||
|
||||
### `completed`
|
||||
|
||||
- immutable result entry exists and matches the source call;
|
||||
- no staged pending entry exists;
|
||||
- no invocation facts or partial output exist.
|
||||
Ordinary dispatch performs no restore-time entry audit. Context/tree reads later consume the immutable entry through their normal typed paths.
|
||||
|
||||
Restore validates the completed-prefix and execution-mode shape described above. Partial output does not participate in semantic validation. Activation or snapshot hydration may read the exact scalar afterward.
|
||||
Every live mutation still verifies current operation, turn, source position, invocation, and status on the lane line. Those checks fence concurrent settlement, cancellation, and external finalization; they are not historical restore validation. Terminal prefix cleanup remains defensive and does not make orphan scans part of restore.
|
||||
|
||||
## Snapshots and reconnect
|
||||
|
||||
|
|
@ -596,43 +587,44 @@ Instrumented-storage tests assert this ordering. A crash after `tool_end` but be
|
|||
|
||||
| Race | Required result |
|
||||
|---|---|
|
||||
| checkpoint vs tool settlement | an active checkpoint was enqueued first and staging later deletes it; the waiting checkpoint is dropped when the tool promise settles; a late update is ignored |
|
||||
| fact write vs `outcome_ready` | an awaited or pre-return-enqueued write precedes staging and is then deleted; a post-return call rejects; external finalization first causes the durable ownership check to reject |
|
||||
| checkpoint vs tool settlement | every accepted checkpoint was enqueued first; settlement awaits the latest promise, then staging deletes the checkpoint value; a late update is ignored |
|
||||
| memo write vs `outcome_ready` | an awaited or pre-return-enqueued write precedes staging and is then deleted; a post-return call rejects; external finalization first causes the durable ownership check to reject |
|
||||
| B outcome vs earlier A settlement | B stages independently; placement waits for A |
|
||||
| crash after outcome staging | tool never replays; pending result later materializes |
|
||||
| crash during source-prefix placement | transaction exposes either none or all of that placement prefix |
|
||||
| safe replay vs old partial output | old key is deleted before replay emits new progress |
|
||||
| safe replay vs old partial output | the old bound checkpoint value is deleted before replay emits new progress |
|
||||
| cancellation vs real settlement | lane-line order chooses real cancelled-control result or synthetic reconciliation; at most one outcome stages |
|
||||
| terminal finalization vs late result | terminal ownership wins or outcome stages first; late task never recreates operation data |
|
||||
| external finalization vs fact/checkpoint mutation | mutation first is removed by terminal cleanup; finalization first makes the mutation's durable ownership check reject |
|
||||
| external finalization vs memo/checkpoint mutation | mutation first is removed by terminal cleanup; finalization first makes the mutation's durable ownership check reject |
|
||||
|
||||
## Invariants
|
||||
|
||||
1. `invocationId` equals the reserved result entry ID and is stable across safe replay.
|
||||
2. A call in `outcome_ready` or `completed` never executes again.
|
||||
3. Every `outcome_ready` call has exactly one complete matching `pending.entry` value.
|
||||
3. Every `outcome_ready` call has exactly one complete matching `pi.pending.entry` value.
|
||||
4. Completed calls form a source-ordered prefix.
|
||||
5. Only source-ordered materialization extends that prefix.
|
||||
6. Parallel calls after the completed prefix may mix planned, effect-pending, and outcome-ready states.
|
||||
7. Invocation facts exist only while their call is `effect_pending`.
|
||||
7. Invocation memos exist only while their call is `effect_pending`.
|
||||
8. Partial output is auxiliary and never establishes effect completion.
|
||||
9. Unsafe synthetic results explicitly state that captured output is incomplete and the external outcome is unknown.
|
||||
10. Staging an outcome atomically deletes its invocation facts and partial output.
|
||||
10. Staging an outcome atomically deletes its invocation memos and partial output.
|
||||
11. Materialization atomically inserts the immutable entry and deletes its staged pending value.
|
||||
12. A late invocation capability cannot write after outcome settlement or operation loss.
|
||||
13. `step.do` values are memoized only after their fact write commits; effects remain at-least-once.
|
||||
14. Operation terminal cleanup leaves no tool args, invocation facts, partial output, or staged outcomes.
|
||||
13. `step.do` values are memoized only after their memo write commits; effects remain at-least-once.
|
||||
14. Operation terminal cleanup leaves no tool args, invocation memos, partial output, or staged outcomes.
|
||||
|
||||
## Required tests
|
||||
|
||||
### State and restore
|
||||
|
||||
- every planned/effect-pending/outcome-ready/completed state;
|
||||
- missing or mismatched pending result for `outcome_ready`;
|
||||
- pending result incorrectly coexisting with completed entry;
|
||||
- invocation facts and partial output absent after staging;
|
||||
- restore validates without reading partial output;
|
||||
- activation then hydrates the exact bounded scalar when needed.
|
||||
- every trusted planned/effect-pending/outcome-ready/completed projection restores without auxiliary reads;
|
||||
- base restore does not audit completed-prefix or execution-mode relationships;
|
||||
- effect-pending consumption reads exact required arguments and optional bounded checkpoint;
|
||||
- outcome-ready consumption reads the exact staged result;
|
||||
- missing required arguments or staged result fails at the consuming procedure, not base restore;
|
||||
- invocation memos and partial output are absent after staging;
|
||||
- snapshot/activation hydrate only the exact bounded checkpoint value when needed.
|
||||
|
||||
### Parallel ordering
|
||||
|
||||
|
|
@ -646,23 +638,23 @@ Instrumented-storage tests assert this ordering. A crash after `tool_end` but be
|
|||
### Replay and interruption
|
||||
|
||||
- safe replay uses persisted arguments and the same invocation ID;
|
||||
- safe replay preserves step facts but clears old partial output;
|
||||
- safe replay preserves step memos but clears old partial output;
|
||||
- current declaration downgrade from safe to never interrupts;
|
||||
- unsafe recovery with no checkpoint and with a complete bounded checkpoint;
|
||||
- synthetic result is error/incomplete/unknown and never runs `after_tool`;
|
||||
- cancellation never safely replays a restored call.
|
||||
|
||||
### Invocation facts and `step.do`
|
||||
### Invocation memos and `step.do`
|
||||
|
||||
- set/get/delete fact with physical invocation scoping;
|
||||
- set/get/delete memo with invocation-address scoping;
|
||||
- completed step skips effect after reopen;
|
||||
- crash before memo commit reruns effect;
|
||||
- crash after memo commit returns memo;
|
||||
- several completed steps followed by one interrupted step;
|
||||
- duplicate live step names reject;
|
||||
- fact write racing outcome staging;
|
||||
- memo write racing outcome staging;
|
||||
- capability write after expiry, cancellation outcome staging, and external finalization rejects;
|
||||
- terminal cleanup removes crash-leaked facts.
|
||||
- terminal cleanup removes crash-leaked memos.
|
||||
|
||||
### Partial output
|
||||
|
||||
|
|
@ -670,8 +662,8 @@ Instrumented-storage tests assert this ordering. A crash after `tool_end` but be
|
|||
- `checkpoint: true` writes the complete bounded snapshot;
|
||||
- tool-selected checkpoint cadence and duplicate suppression;
|
||||
- bash emits live snapshots at 100 ms and requests distinct checkpoints at most every two seconds;
|
||||
- drop-coalescing permits at most one active and one waiting checkpoint, and tool-promise settlement drops the waiting value;
|
||||
- a checkpoint after outcome staging cannot recreate the key;
|
||||
- every selected checkpoint enqueues one write, and awaiting the latest promise at tool settlement implies all earlier writes completed;
|
||||
- a checkpoint after outcome staging cannot recreate the address's value;
|
||||
- Memory, JSONL, and SQLite restore the same checkpoint value;
|
||||
- JSONL growth follows checkpoint cadence rather than raw bash output volume;
|
||||
- terminal compaction reclaims superseded/deleted checkpoint snapshots according to the existing dead-byte policy.
|
||||
|
|
@ -679,7 +671,7 @@ Instrumented-storage tests assert this ordering. A crash after `tool_end` but be
|
|||
### Atomicity and instrumentation
|
||||
|
||||
- exact intent, synchronous update acceptance, asynchronous update-delivery, `after_tool`, `tool_end`, outcome-ready, source-ordered message lifecycle, and materialization order;
|
||||
- outcome staging is atomic with fact/output cleanup;
|
||||
- outcome staging is atomic with memo/output cleanup;
|
||||
- materialization is atomic with pending deletion, usage, leaf, and state;
|
||||
- crash at every boundary;
|
||||
- no effect starts before intent;
|
||||
|
|
@ -691,20 +683,20 @@ Instrumented-storage tests assert this ordering. A crash after `tool_end` but be
|
|||
Expected runtime areas:
|
||||
|
||||
- operation-state types in `packages/agent/src/harness/session/types.ts`;
|
||||
- restore key collection and semantic validation;
|
||||
- trusted restore projection and exact consumption-time address reads;
|
||||
- tool-batch procedure and source-ordered materialization;
|
||||
- terminal cleanup and cancellation reconciliation;
|
||||
- harness-specific update options, snapshots, and events;
|
||||
- invocation-scoped capability implementation on the lane mutation line;
|
||||
- backend conformance through the already-implemented scalar/list register APIs;
|
||||
- backend conformance through the bound value/list APIs;
|
||||
- instrumented-storage transaction assertions.
|
||||
|
||||
Concrete namespace additions:
|
||||
Concrete built-in address constructors in `session/values.ts`:
|
||||
|
||||
```text
|
||||
scalar: op.tool_fact
|
||||
scalar: pending.tool_output
|
||||
existing scalar staging: pending.entry
|
||||
operationToolMemo(operationId, invocationId, name) → value("pi.op.tool_memo", ...)
|
||||
pendingToolOutput(operationId, invocationId) → value("pi.pending.tool_output", ...)
|
||||
pendingEntry(resultEntryId) → value("pi.pending.entry", ...)
|
||||
```
|
||||
|
||||
Implement `outcome_ready` and invocation facts before progress checkpoints. The state solves incorrect parallel replay by itself; checkpoints improve reconnect observation and unsafe interruption diagnostics without becoming completion authority.
|
||||
Implement `outcome_ready` and invocation memos before progress checkpoints. The state solves incorrect parallel replay by itself; checkpoints improve reconnect observation and unsafe interruption diagnostics without becoming completion authority.
|
||||
|
|
|
|||
75
packages/agent/docs/work-packages/00-runtime1-removal.md
Normal file
75
packages/agent/docs/work-packages/00-runtime1-removal.md
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
# WP00 — Runtime1 removal
|
||||
|
||||
## Status
|
||||
|
||||
Approved; implementation not started. No tag. Do not release while the public runtime2 execution path is incomplete.
|
||||
|
||||
## Goal
|
||||
|
||||
Make runtime2 the sole public harness implementation, delete runtime1 and its obsolete tests, then stop before adding runtime2 behavior.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The approved acceptance/hook redesign and durability handoffs are present in `runtime2.md`, `registers.md`, `assistant-durability.md`, and `tool-durability.md`.
|
||||
- Existing tests are evidence, not authority.
|
||||
|
||||
## Work, in order
|
||||
|
||||
1. **Reconcile the contract.** Fold the approved acceptance/hook redesign into `harness.md`, including durable `starting`, atomic hook-free acceptance, driver-owned `before_run`, `before_drive`, request-local system-prompt transformation, trusted restore, and removal of process-origin activation semantics. Audit §§0.4, 1.2, 3.1–3.6, 4.1–4.2, 4.5, 5.1–5.2, 5.5–5.6, and Parts 8–9; remove every stale `BeforeResumePrepared`, `before_resume`, `resumeData`, `systemPromptOverride`, stable-ID routing, reservation, and `fresh | continue | resume` activation reference. Then remove the temporary supersession notice and duplicated redesign from `runtime2.md`.
|
||||
2. **Harvest before deletion.** Inspect `agent-harness-runtime.test.ts`, `agent-harness-r2/r3/r4.test.ts`, and old `restore.test.ts`. Preserve unique scenarios in the detailed future rows or a temporary categorized inventory; explicitly discard old reservation, `before_resume`, `resumeData`, persisted hook prompt override, semantic restore audit, and pre-`outcome_ready` tool-crash behavior.
|
||||
3. **Remove runtime1-only public members.** Delete `before_resume`, `BeforeResumePrepared`, `resumeData`, `systemPromptOverride`, and stable hook-ID routing. Add the approved `before_drive` and `transform_context` shapes. Update telemetry schema source and regenerate its document. Do not implement `starting` or acceptance behavior here.
|
||||
4. **Switch the factory.** Add `packages/agent/src/harness/runtime2/index.ts`, point `agent-harness.ts` at it, and add a constructor-selection regression. Verify the experimental coding-agent worker still creates the harness, subscribes to events, and closes it.
|
||||
5. **Delete runtime1 source and tests** listed below.
|
||||
6. **Update `[Unreleased]`** for the public breaking removals and temporarily incomplete factory.
|
||||
7. Run the retained tests and checks. Fix every failure; do not restore compatibility shims.
|
||||
|
||||
## Delete
|
||||
|
||||
```text
|
||||
packages/agent/src/harness/runtime/**
|
||||
packages/agent/src/harness/restore.ts
|
||||
packages/agent/test/harness/agent-harness-runtime.test.ts
|
||||
packages/agent/test/harness/agent-harness-r2.test.ts
|
||||
packages/agent/test/harness/agent-harness-r3.test.ts
|
||||
packages/agent/test/harness/agent-harness-r4.test.ts
|
||||
packages/agent/test/harness/restore.test.ts
|
||||
packages/agent/test/harness/scratch/r1.ts
|
||||
packages/agent/test/harness/scratch/r2.ts
|
||||
packages/agent/test/harness/scratch/r3.ts
|
||||
packages/agent/test/harness/scratch/r4.ts
|
||||
```
|
||||
|
||||
## Retain
|
||||
|
||||
- all `test/harness/runtime2/**` tests;
|
||||
- Session, SessionTree, storage, repository, backend-conformance, and instrumentation tests;
|
||||
- execution assistant/tool/primitives tests;
|
||||
- config, hooks, events, telemetry, compaction, and branch-summary code/tests;
|
||||
- `types.test.ts`, updated for the reduced public contract;
|
||||
- `packages/agent/src/agent-loop.ts` unchanged.
|
||||
|
||||
Do not parameterize obsolete runtime1 suites against runtime2 and do not keep a runtime1 smoke suite.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- No source import references `harness/runtime/*`.
|
||||
- Public `AgentHarness.create()` selects runtime2.
|
||||
- Runtime2 creation, events, inspection, close, and fault tests pass.
|
||||
- These coding-agent tests pass:
|
||||
- `experimental-remote-runtime.test.ts`
|
||||
- `experimental-session-worker.test.ts`
|
||||
- `experimental-session-worker-lifecycle.test.ts`
|
||||
- Every modified test passes individually.
|
||||
- Agent and root TypeScript pass.
|
||||
- `git diff --check` and `npm run check` pass.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No bound-value/list implementation.
|
||||
- No implementation of `starting` or atomic acceptance.
|
||||
- No drive owner, provider, retry, deferred, or tool execution.
|
||||
- No runtime1 parity work, compatibility layer, archaeology tag, or release.
|
||||
|
||||
## Stop condition
|
||||
|
||||
Stop once runtime1 is absent, runtime2 is the public factory, retained coverage is green, and all checks pass. Report the deletion and harvested scenarios; do not begin another work package.
|
||||
57
packages/agent/docs/work-packages/01-bound-values-lists.md
Normal file
57
packages/agent/docs/work-packages/01-bound-values-lists.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
# WP01 — Bound values and lists
|
||||
|
||||
## Status
|
||||
|
||||
Ready after WP00. The normative design is [`registers.md`](../registers.md).
|
||||
|
||||
## Goal
|
||||
|
||||
Replace the retained register/custom-state API with bound `Value<T>` and `ValueList<T>` addresses across every existing backend and application surface, then stop before runtime execution work.
|
||||
|
||||
## Scope
|
||||
|
||||
1. Add `session/values.ts` with universal `value()` / `list()`, typed write helpers, all `pi.*` built-in address/prefix constructors, and no registry, catalog, privilege path, or global type map.
|
||||
2. Replace `RegisterValues`, namespace unions, `Register`, `getRegister`, `listRegisters`, raw register writes, custom-state APIs, and old persisted semantic names.
|
||||
3. Expose the same bound-address reads through Storage, SessionReader, SessionMutator, Session, and SessionTree; add direct Session scalar/list writes and retain one-commit `Session.mutate()` composition.
|
||||
4. Implement scalar replacement and list append/page/whole-list-delete in Memory, JSONL, and SQLite, including replay, compaction, snapshots, forks/rewrites, instrumentation, and shared conformance.
|
||||
5. Use `pi.session.name`, `pi.entry.label`, and the complete built-in namespace/key grammar from `registers.md`. Applications define their own addresses directly.
|
||||
6. Update telemetry schema source for value/list write kinds and regenerate `telemetry-schema.md`.
|
||||
7. Update all retained source/tests and public exports; no old API shim remains.
|
||||
|
||||
## Preserve
|
||||
|
||||
- transaction atomicity and global write `seq` ordering;
|
||||
- JSONL torn-tail and compaction behavior, preserving list-element sequences;
|
||||
- SQLite `BEGIN IMMEDIATE`, writer fencing, and indexed list paging without temporary sorting;
|
||||
- repository/fork semantics, branch indexes, stats, UUIDv7/follower IDs, v3 normalization, and existing entry/usage behavior;
|
||||
- projection-only restore: no list read until bounded consumption-time hydration;
|
||||
- trusted in-process values with no codecs, cloning, or runtime shape validation.
|
||||
|
||||
## Required coverage
|
||||
|
||||
- type inference and incompatible scalar/list write rejection;
|
||||
- exact `pi.*` built-in addresses and five scan-prefix constructors;
|
||||
- application-wide and dynamically keyed addresses with no second operation-time key;
|
||||
- scalar set/get/delete/recreate and namespace-scoped prefix scans;
|
||||
- list append, global element sequences, asc/desc cursors, limits, absent reads, whole delete, and delete-then-append;
|
||||
- atomic entry + usage + scalar + list transactions and rollback;
|
||||
- identical Memory/JSONL/SQLite behavior;
|
||||
- JSONL replay, torn transactions, and cursor-preserving compaction;
|
||||
- SQLite query plans;
|
||||
- forks excluding operation-owned values/lists while preserving semantic session values;
|
||||
- close rejecting later reads while admitted commits drain;
|
||||
- zero old register or custom-state API references outside immutable release history.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No runtime acceptance, drive, provider, assistant-frame consumer, tool memo/checkpoint consumer, or operation-state redesign.
|
||||
- No generic event log, list truncation, per-element deletion, registry, catalog, or compatibility facade.
|
||||
- Do not rewrite applied backend migrations solely for terminology; follow the compatibility policy in `registers.md`.
|
||||
|
||||
## Validation
|
||||
|
||||
Run every modified test individually, the shared backend conformance suites, SQLite package tests, agent/root TypeScript, `git diff --check`, and `npm run check`.
|
||||
|
||||
## Stop condition
|
||||
|
||||
Stop when every existing backend and retained Session surface uses bound values/lists, old APIs are absent, and all checks pass. Report schema/compatibility choices; do not begin runtime execution work.
|
||||
Loading…
Add table
Add a link
Reference in a new issue