From 7ccb2a088252bad7cc863bb6a84f9f5fbbe51b81 Mon Sep 17 00:00:00 2001 From: Mario Zechner Date: Mon, 24 Aug 2026 19:49:15 +0200 Subject: [PATCH] docs(agent): simplify durable drive ownership --- packages/agent/docs/harness.md | 89 ++- .../work-packages/05-direct-durable-drive.md | 741 ++++++++---------- 2 files changed, 379 insertions(+), 451 deletions(-) diff --git a/packages/agent/docs/harness.md b/packages/agent/docs/harness.md index 72883d46f..8258e59c9 100644 --- a/packages/agent/docs/harness.md +++ b/packages/agent/docs/harness.md @@ -1463,17 +1463,17 @@ interface LaneState { } ``` -Attachment restores only the small durable projection needed to construct each lane and inventory open operations. It reads the lane's `laneLeaf`, `laneConfig`, `laneState`, optional `laneLastResult`, and, when `currentOperationId` names O, `operationMeta(O)` plus `operationState(O)`. These values become the lane's owned process-local projection. `laneLastResult` supports inspection but is never a recovery input (§3.13). +Attachment restores only the small durable projection needed to construct each lane and inventory open operations. It reads the lane's `laneLeaf`, `laneConfig`, `laneState`, optional `laneLastResult`, and, when `currentOperationId` names O, `operationMeta(O)` plus `operationState(O)`. These values become the lane's owned process-local projection. While that harness remains alive and owns the session, this projection is the authoritative current lane and operation control state. Every supported control-state mutation commits through the owning `Lane` and publishes its exact next projection before the lane mutation line is released. `laneLastResult` supports inspection but is never a recovery input (§3.13). -Projection restore validates the relationships required to construct that projection: required value existence, lane ownership, operation-id agreement, and intent/state kind compatibility. Missing or contradictory projection values fault attachment and no harness is returned. +Projection restore validates the relationships required to construct that projection: required value existence, lane ownership, operation-id agreement, and intent/state kind compatibility. Missing or contradictory projection values fault attachment and no harness is returned. Process loss discards the projection; a replacement process rebuilds it from these durable values during attachment. -Attachment does not dereference transcript, queue payloads, deferred sources, assistant frames, tool arguments, tool checkpoints, preparations, memos, or staged outcomes. Those references are validated by the operation that consumes them: `watch(context)` validates presentation references while capturing its snapshot, and drive procedures validate transition inputs. A missing or mismatched referenced payload is terminal storage corruption and faults that consumer. Optional frame lists and tool checkpoints may be absent. This consumption-time validation keeps attachment bounded without weakening durable transition fences (§3.4, §4.3). +Attachment does not dereference transcript, queue payloads, deferred sources, assistant frames, tool arguments, tool checkpoints, preparations, memos, or staged outcomes. Those references are validated by the operation that consumes them: `watch(context)` validates presentation references while capturing its snapshot, and drive procedures validate transition inputs. A missing or mismatched referenced payload is terminal storage corruption and faults that consumer. Optional frame lists and tool checkpoints may be absent. This consumption-time validation keeps attachment bounded: live procedures use the owned projection for control flow and the callback reader only to dereference content named by that projection (§3.4, §4.3). ## 3.4 The atomic transition rule > Compute the next total state in memory, then atomically commit every entry insert, usage insert, and value/list write that makes that state true. -A transition rereads the latest operation and lane values inside the lane mutation line, verifies the semantic state it extends, and changes only the fields it owns. In particular, settlement preserves newer inbox/control fields, and the terminal transaction clears `currentOperationId` while preserving concurrently accepted `pendingNextRun`. An unexpected non-terminal phase or effect identity is an invariant defect, not ordinary stale-state replanning; absent operation values mean a terminal transaction already won (§4.3, §4.9). Every edge below is exactly one `commit()`. +A transition plans from the authoritative owned projection supplied by the lane mutation line, verifies the semantic state it extends, and changes only the fields it owns. In particular, settlement preserves newer inbox/control fields, and the terminal transaction clears `currentOperationId` while preserving concurrently accepted `pendingNextRun`. A procedure that awaited external work rechecks its expected phase and effect identity against that current projection before committing. Cancellation routes to reconciliation; a missing or different operation means ownership was lost; any other unexpected phase or identity is an invariant defect (§4.3, §4.9). The callback reader is used only for referenced payloads and operation-owned cleanup scans. Every edge below is exactly one `commit()`. ## 3.5 The graph @@ -1592,7 +1592,7 @@ The provider loop never awaits storage per frame; existing assistant event deliv At stream settlement, the procedure stops frame admission and awaits the latest frame-write promise before `after_response`; lane FIFO means its completion implies completion of every earlier append. There is **no** timer, batcher, coalescer, active/waiting state, or flush API — public or internal. A failed frame append faults the harness before `after_response` starts; the complete final response stays process-local and never commits after a storage fault. -Each append mutation verifies that the same operation, attempt, and reserved response id still own the lane when it executes; a terminal transaction that won first fences a stale append without recreating operation state (§4.9). Frames are auxiliary: a missing list is valid, frames never prove request admission or completion, never select a restart point, and a crash after the final frame append but before settlement still restores `effect_pending` — a complete-looking draft is not a settled response. Structural summary streams persist no frames (§3.9). There is no frame-persistence event, and reconnect replays no historical updates (§5.4, §5.5). +Each append mutation verifies against the current owned projection that the same operation, attempt, and reserved response id still own the lane when it executes, and exact `Drive` identity is checked again at commit admission; a terminal transaction that won first fences a stale append without recreating operation state (§4.9). Frames are auxiliary: a missing list is valid, frames never prove request admission or completion, never select a restart point, and a crash after the final frame append but before settlement still restores `effect_pending` — a complete-looking draft is not a settled response. Structural summary streams persist no frames (§3.9). There is no frame-persistence event, and reconnect replays no historical updates (§5.4, §5.5). ### Classification order @@ -1856,7 +1856,7 @@ TX[ { +async function executeDrivePass( + lane: Lane, drive: Drive, options: DriveOptions, +): Promise { try { - let current = await loadCurrent(operationId); - if (!current) return settledFromLastResult(operationId); + let current = lane.state.operation; + if (current === null || current.meta.operationId !== drive.operationId) + return hydrateMatchingLastResult(lane, drive.operationId); if (current.state.control.status === "cancel_requested") return reconcileCancellation(current); await runBeforeDrive(current); while (true) { - current = await loadCurrent(operationId); - if (!current) return settledFromLastResult(operationId); + current = lane.state.operation; + if (current === null || current.meta.operationId !== drive.operationId) + return hydrateMatchingLastResult(lane, drive.operationId); if (current.state.control.status === "cancel_requested") return reconcileCancellation(current); if (mustWaitForRetry(current, options)) return retryWait(current); if (isDeferredSuspended(current) && !options.pollDeferred) return deferredWait(current); - if (current.state.phase.kind === "starting") + if (current.state.kind === "run" && current.state.phase.kind === "starting") await settleBeforeRun(current); else if (hasOrphanedEffect(current)) await recoverOrphanedEffect(current, options); @@ -1965,12 +1969,13 @@ async function executeDrivePass(operationId: string, } catch (error) { if (error instanceof AbortRequested) { await error.cancellation; - const current = await loadCurrent(operationId); - if (!current) return settledFromLastResult(operationId); + const current = lane.state.operation; + if (current === null || current.meta.operationId !== drive.operationId) + return hydrateMatchingLastResult(lane, drive.operationId); return reconcileCancellation(current); } if (error instanceof OperationEnded) - return settledFromLastResult(operationId); + return hydrateMatchingLastResult(lane, drive.operationId); throw error; } } @@ -1993,7 +1998,7 @@ Normal execution handles checkpoints, ready/retry states, planned or locally own ### Task installation, joining, and removal -`accept` commits the first durable state and returns without installing a `Drive`. A durably accepted but locally unowned operation is ordinary. `drive({ operationId }, context)` owns one pass. On the lane mutation line it reads current ownership and `pi.lane.lastResult`, then applies exactly one rule: +`accept` commits the first durable state and returns without installing a `Drive`. A durably accepted but locally unowned operation is ordinary. `drive({ operationId }, context)` owns one pass. On the lane mutation line it inspects the current owned projection, including its latest result, then applies exactly one rule: - current operation absent and `pi.lane.lastResult.operationId === operationId`: return its already-settled outcome; - current operation absent with another/no last id, or current operation has another id: return `OperationMismatch` without starting work; @@ -2004,7 +2009,7 @@ The first caller to install a pass supplies that pass's retry-wait and deferred- A newly installed real pass checks cancellation, then invokes `before_drive` once before recovery or ordinary work. Joiners do not rerun it. A later pass after retry wait, deferred suspension, close/reopen, or process restart invokes it again. Durable phase and local ownership alone determine dispatch: `starting` requires `before_run`; an unowned `effect_pending` phase is orphaned; every other phase follows its ordinary procedure. No fresh/continue/restore marker exists. -Every task has one outer `finally` path: after its last durable commit, wait decision, or terminal observation and final local events, it synchronously removes itself only when `lane.activeDrive === drive`. Normal completion settles only after that exact cleanup. Invocation abandonment and close/fault fail completion and remove the exact Drive immediately so non-cooperative work cannot block recovery or shutdown; every late planner and progress write also checks exact Drive identity. External finalization instead settles completion immediately but deliberately retains the Drive until the task's eventual `finally`, preserving `LaneBusy` during detached cleanup (§4.9). +Every task has one outer `finally` path: after its last durable commit, wait decision, or terminal observation and final local events, it synchronously removes itself only when `lane.activeDrive === drive`. Normal completion settles only after that exact cleanup. Invocation abandonment and close/fault fail completion and remove the exact Drive immediately so non-cooperative work cannot block recovery or shutdown; every late planner and progress write also checks exact Drive identity. External finalization instead closes the gate in committed materialization and settles completion immediately after terminal event delivery without waiting for detached work. The exact Drive remains installed until both that settlement and pass ending have occurred; whichever side arrives second removes it. This preserves `LaneBusy` and same-id joining throughout event delivery and detached cleanup (§4.9). Once `finalizedOutcome` is set, the pass continuation may neither settle nor fail the shared completion; only the finalizer settles it after event delivery. Convenience methods add policy, not another execution path. `prompt`/`skill`/`compact`/`navigateTree` call `accept` and then drive the returned id; they wait locally through retry waits and return on terminal or deferred suspension. `resume(context)` inspects the current id and drives it with one deferred-poll permit. `abort(context)` calls `requestAbort`, then ensures cancellation reconciliation has a drive pass; it still returns once cancellation is durable. A scheduler calls the expected-id primitives directly and may persist the admission or set an alarm from `waiting.notBefore`. Normal applications do not manage operation ids when resuming. @@ -2099,13 +2104,15 @@ Every catalog item has abort-first/admission-first tests. Preparation must prece Every state-dependent mutation on a lane is linearized by `Session.mutate`: read latest state, decide, commit at most once, and complete the process-local update before the next mutation starts. Provider, tool, hook, and timer work never occupies the line. The callback-scoped `SessionMutator` enforces the one-commit limit; a high-level `SessionTree` write uses the same mechanism internally. -An operation transition rereads the latest `pi.lane.state`, `pi.op.state`, leaf, and any dependent values inside that mutation. It verifies the expected semantic restart point or pending effect identity, preserves fields owned by concurrent public calls, and commits the complete next state. A mismatch is not an ordinary boolean "replan" result: +An operation transition receives the latest owned projection inside that mutation. It verifies the expected semantic restart point or pending effect identity, preserves fields owned by concurrent public calls, and commits the complete next state. A mismatch is classified directly: - `cancel_requested` routes through the task's cancellation control flow; -- absent operation values mean its terminal transaction already committed (§4.9); +- a missing or different operation means the task lost ownership and commits nothing (§4.9); - any other unexpected phase/identity is an invariant defect. -Settlement therefore cannot overwrite steer/write acceptance or cancellation with a stale snapshot. It rereads the latest state, verifies that the same request or tool call remains pending, and merges the output while preserving newer inbox/control fields. +Settlement therefore cannot overwrite steer/write acceptance or cancellation with a stale snapshot. It verifies that the same request or tool call remains pending in the current projection and merges the output while preserving newer inbox/control fields. Referenced payloads and operation-owned cleanup prefixes are still read from storage when consumed; lane/operation control values are not reread during live execution. + +A drive-owned commit checks `lane.activeDrive === drive` synchronously at commit admission, immediately before invoking `mutator.commit()`, with no `await` between the check and invocation. A planner-side check alone is insufficient because abandonment can replace the exact owner after an asynchronous planner returns. If ownership was lost before admission, the command commits nothing; if commit admission won first, that transition may finish before owner removal. What serializes on the lane line: operation acceptance, drive installation/join/removal, execution inspection, detailed watch capture, queue enqueue/cancel/consumption, deferred writes, cancellation requests, assistant frame appends, tool progress checkpoints, invocation memo access, outcome staging, source-ordered outcome materialization, lane-configuration setters, finish, and lane creation. All durable races consequently have the orders listed in Part 9. The effect gate is needed only because synchronous external-call admission is not a durable mutation. @@ -2165,7 +2172,7 @@ The lane's configured model identity comes from `laneConfig`. A current operatio Recovery begins only when an open operation has no `Drive` and a matching `drive({ operationId }, context)` installs a real pass owner. `AgentHarness.create(options, context)` never drives. `resume(context)` inspects and drives the current operation without exposing its id and grants the same pass one deferred-poll permit. `requestAbort(operationId, context)` with no task commits cancellation but installs nothing; the next drive enters cancellation reconciliation directly. -The pass first reloads durable control. If cancellation is requested, it invokes neither `before_drive` nor `before_run` and enters §4.6. Otherwise it gates and invokes `before_drive`; failure rejects this pass without faulting the harness or writing durable progress. Model/tool implementations are resolved only at the actual operation boundary that needs them. An unavailable provider/model or configured request tool becomes a non-retryable configuration failure before request intent; an unavailable requested tool becomes a synthetic error result. Neither condition suspends the operation. Durable phase then decides the work: `starting` runs and settles `before_run` as §3.6 specifies; an unowned pending effect is orphaned by construction and follows the table below; all other phases continue ordinarily. +The pass first inspects the authoritative owned control projection. If cancellation is requested, it invokes neither `before_drive` nor `before_run` and enters §4.6. Otherwise it gates and invokes `before_drive`; failure rejects this pass without faulting the harness or writing durable progress. Model/tool implementations are resolved only at the actual operation boundary that needs them. An unavailable provider/model or configured request tool becomes a non-retryable configuration failure before request intent; an unavailable requested tool becomes a synthetic error result. Neither condition suspends the operation. Durable phase then decides the work: `starting` runs and settles `before_run` as §3.6 specifies; an unowned pending effect is orphaned by construction and follows the table below; all other phases continue ordinarily. | Orphaned restart point | Activation recovery | |---|---| @@ -2196,12 +2203,12 @@ retry_wait retry_wait → drive({ waitForRetry: true }) admits and immediately starts the retry timer through `drive.gate` -→ timer reaches notBefore: Session.mutate verifies the same wait and commits ready +→ timer reaches notBefore: Lane.command verifies the same current wait and commits ready → requestAbort: timer wakes after durable cancellation and reconciliation runs → close: local task rejects; no durable write ``` -At or after `notBefore`, either policy verifies the same durable wait and commits `ready` without an unnecessary timer. +At or after `notBefore`, either policy verifies the same current durable wait state in the owned projection and commits `ready` without an unnecessary timer. ## 4.6 Abort and cancellation reconciliation @@ -2213,7 +2220,7 @@ For a live operation, the first matching `requestAbort()` performs this order: ```text 1. synchronously call `beginAbort` on the matching Drive and install the abort-mutation promise; -2. `Session.mutate` rereads the latest operation and either commits +2. `Lane.command` inspects the current owned operation and either commits `cancel_requested` or observes that the expected id no longer owns the lane; 3. for runs, atomically move current steer/follow-up ids to `control.drained*` and empty those inbox lists; their `pendingEntry(id)` values remain; @@ -2226,7 +2233,7 @@ The cancellation mutation leaves the current phase untouched. A later `requestAb The effect gate orders only whether external work started. The lane mutation line separately orders cancellation against settlement: -- abort commit first → settlement rereads cancelled control and applies cancellation classification; +- abort commit first → settlement observes cancelled control in the current projection and applies cancellation classification; - settlement first → its normal next state commits, then abort marks that state cancelled; - terminal first → abort returns `NoActiveOperation`. @@ -2285,11 +2292,11 @@ Close rejects active drive and convenience-operation promises with `HarnessClose ## 4.9 External finalization -An authorized administrative operation may commit the terminal transaction while a task still exists. It uses the same lane mutation line. This specification adds no public `forceFinalize()` method: authorization, SLA policy, and initiation belong to repository or hosting administration, while the terminal-transaction helper and live-task notification remain package-internal. Live admitted work receives `OperationEnded` when the finalizer closes `drive.gate`. Any queued or later planner rereads ownership, returns `lost_ownership`, and commits nothing. +An authorized administrative operation may commit the terminal transaction while a task still exists. In-process finalization must use the live owning `Lane`'s command path, on the same lane mutation line, so its terminal commit publishes the idle projection before notifying the owner. Bypassing that path and mutating reserved control values behind the live `Lane` is a programming defect. This specification adds no public `forceFinalize()` method: authorization, SLA policy, and initiation belong to repository or hosting administration, while terminal cleanup and live-task notification remain package-internal. Live admitted work receives `OperationEnded` when the finalizer closes `drive.gate`. Any queued or later planner observes ownership loss in the current projection, returns `lost_ownership`, and commits nothing. -With a live in-process Drive, the finalizer itself publishes the end observation, records `drive.finalizedOutcome`, closes the gate, and settles shared completion immediately; the detached task stops without publishing or writing. With no local Drive—for example, finalization by a replacement process—a later matching `drive()` hydrates from `pi.lane.lastResult`. Neither path recreates operation values/lists or commits a competing terminal result. Scalar and list invocation writes are fenced the same way: invocation memo writes, tool checkpoint replacements, and assistant frame appends each verify durable ownership when their lane job executes, so a write enqueued before finalization is deleted by the terminal cleanup (which removes all operation-owned values/lists) and a write after it declines without committing or recreating state. An unowned operation needs no task to stop; a later matching `drive()` returns the settled result from `pi.lane.lastResult`, while `resume(context)` sees an idle lane and returns `NothingToResume`. +With a live in-process Drive, the finalizer itself records `drive.finalizedOutcome` and closes the gate in committed materialization, publishes and awaits the end observation, then settles shared completion without waiting for detached work; the detached task stops without publishing or writing. With no local Drive—for example, finalization by a replacement process—a later matching `drive()` hydrates from `pi.lane.lastResult`. Neither path recreates operation values/lists or commits a competing terminal result. Scalar and list invocation writes are fenced the same way: invocation memo writes, tool checkpoint replacements, and assistant frame appends each verify the current owned projection when their lane job executes and exact Drive identity at commit admission, so a write enqueued before finalization is deleted by the terminal cleanup (which removes all operation-owned values/lists) and a write after it declines without committing or recreating state. An unowned operation needs no task to stop; a later matching `drive()` returns the settled result from `pi.lane.lastResult`, while `resume(context)` sees an idle lane and returns `NothingToResume`. -At most one terminal transaction commits because every candidate rereads ownership on the lane line before writing. An in-process finalizer publishes terminal events, records the matching result in `drive.finalizedOutcome`, closes its gate with `OperationEnded`, and settles the shared completion immediately so non-cooperative provider/tool work cannot delay callers. It deliberately retains the Drive; acceptance returns `LaneBusy` until the detached task reaches its exact-object `finally`. The task publishes no duplicate end event and cannot write after ownership loss. A replacement-process finalizer first acquires ownership after close/crash, so no live Drive exists to notify. +At most one terminal transaction commits because every candidate verifies the current owned projection on the lane line before writing. An in-process finalizer records the matching result in `drive.finalizedOutcome` and closes its gate with `OperationEnded` in committed materialization, publishes and awaits the terminal event, then settles shared completion without waiting for non-cooperative provider/tool work. It retains the Drive until both terminal event delivery/settlement and the detached task's exact-object `finally` have occurred. Acceptance returns `LaneBusy`, and same-id drive callers join that retained completion, until the later boundary removes it. The task publishes no duplicate end event and cannot write after ownership loss. A replacement-process finalizer first acquires ownership after close/crash, so no live Drive exists to notify. # Part 5 — Public surface @@ -2588,7 +2595,7 @@ interface LaneInfo { // QueueMode, RetryPolicy, and CompactionSettings use the source types named in §0.7. ``` -Passing an open `Session` to `create(options, context)` transfers orchestration ownership to the attachment attempt and then the returned harness until `close(context)` resolves. During that interval callers must not invoke raw `Session.mutate`, `Session.createLane`, or write reserved `pi.*` addresses through another facade; doing so can race lane inventory and violates the single-owner contract. If create rejects, ownership returns to the caller. +Passing an open `Session` to `create(options, context)` transfers orchestration ownership to the attachment attempt and then the returned harness until `close(context)` resolves. During that interval every retained `Session` and pre-attachment `SessionTree` facade is read-only. Callers must not invoke `Session.mutate`, `Session.createLane`, `Session.appendMessage`, `Session.appendCustomEntry`, any mutating `session.view(lane)` method, or write reserved `pi.*` addresses through another facade. Application values and lists must be mutated through the harness-owned lane facade. Bypassing the owning `Lane` can stale its authoritative projection and is a programming defect, not a supported concurrency path. If create rejects, ownership returns to the caller. `create(options, context)` initializes an unconfigured main when needed, then restores the small durable projection for every lane under that Context before returning. `open` contains exactly one item per lane with a durable current operation and omits idle lanes; `aborting:true` is copied only from durable cancellation control. The array is an inventory that may become stale after return, not a reservation, identity prediction, or drive claim. Detailed snapshot payloads are read only by `watch(context)`. @@ -3502,7 +3509,7 @@ Tree: Operations: -12. `laneState(lane)` confers lane ownership, and `operationState(operationId)` confers operation-state ownership. An open lane names operation O, `operationMeta(O)` holds that lane's compatible `OperationMeta`, and `operationState(O)` holds an `OperationState` compatible with O's intent kind; state values carry no duplicate owner metadata. +12. `laneState(lane)` confers lane ownership, and `operationState(operationId)` confers operation-state ownership. An open lane names operation O, `operationMeta(O)` holds that lane's compatible `OperationMeta`, and `operationState(O)` holds an `OperationState` compatible with O's intent kind; state values carry no duplicate owner metadata. While a harness owns the session, exactly one live `Lane` owns each lane's authoritative projection and every supported write to that lane's control addresses commits through it. 13. Operation-owned values and lists may exist only while their operation is open: the terminal transaction deletes them atomically with clearing `currentOperationId` (§3.13). Lane-owned `pendingNextRun` values are never deleted by it. 14. Acceptance must observe `currentOperationId === null`, commits no `Drive`, and returns before any hook/provider/tool/timer work begins. Run acceptance commits payload-free `starting`; only its consuming command may apply `before_run` output and replace it with `checkpoint`. A supplied operation id obeys §1.2 and is the exact id written to `pi.op.meta`, events, and `pi.lane.lastResult`. 15. A reserved id may exist only with the content its intent named. Queued-content ids begin in `pi.pending.entry`; settlement-family ids begin as strings in `pi.op.state`. A tool-result id may then move through `string only → outcome-ready pi.pending.entry → immutable entry`; no two representations coexist at a commit boundary (§2.2). An effect-pending response id may additionally key its auxiliary frame list (§3.7); frames are observation, not a content representation, and die with settlement. @@ -3510,9 +3517,9 @@ Operations: 17. At most one operation is open per lane. Two is corruption. 18. `overflowRecoveryUsed` is `true` only after overflow compaction. A transition that adds projecting conversational input or tool results and requires an assistant writes `false`; an unprojected custom write preserves it. 19. **The settlement transaction that commits a response with `stopReason: "aborted"` must, in that same transaction, write an operation state with `control.status === "cancel_requested"`.** The invariant is scoped to the committing transaction — later terminal cleanup or forks may remove the state without violating it. Providers must comply with the harness-owned signal contract; violation is corruption. -20. Attachment restores and validates only the small lane/operation projection (§3.3, §4.4). Detailed presentation references are validated by `watch(context)` under the lane line; drive references are validated by their consuming procedure. Missing or contradictory required data faults that consumer, while optional frame/checkpoint absence is legal. Live phase/effect-identity checks remain concurrency fences. `pi.lane.lastResult` never determines an open operation's next procedure. -21. At most one terminal transaction ever commits per operation. In-process finalization closes the live Drive gate with `OperationEnded`, records `drive.finalizedOutcome`, and settles immediately; queued/later planners return `lost_ownership` and commit nothing. With no local Drive, later matching callers hydrate from `pi.lane.lastResult` (§4.9). -22. At most one `Drive` exists per lane. Acceptance and taskless `requestAbort` never install one. A matching `drive` installs it before releasing the lane mutation line; another matching drive joins that pass, and a stale id starts nothing. Every late transition checks exact Drive identity. Normal `finally` removes only itself; abandonment and close/fault remove immediately; external finalization settles immediately but retains ownership until `finally`. Each newly installed pass invokes `before_drive` once after the cancellation check; joiners do not. `starting + cancel_requested` invokes neither `before_drive` nor `before_run`. +20. Attachment restores and validates only the small lane/operation projection (§3.3, §4.4). That owned projection is authoritative until close, fault, or process loss. Detailed presentation references are validated by `watch(context)` under the lane line; drive payload references are validated by their consuming procedure. Missing or contradictory required data faults that consumer, while optional frame/checkpoint absence is legal. Live phase/effect-identity and exact-`Drive` checks remain concurrency fences. `pi.lane.lastResult` never determines an open operation's next procedure. +21. At most one terminal transaction ever commits per operation. In-process finalization closes the live Drive gate with `OperationEnded` and records `drive.finalizedOutcome` in committed materialization, then settles after terminal event delivery without waiting for detached work; once that outcome is recorded, the pass continuation may neither settle nor fail shared completion. The exact Drive is removed only after both finalizer-owned settlement and pass ending, so no caller hydrates before terminal event delivery. Queued/later planners return `lost_ownership` and commit nothing. With no local Drive, later matching callers hydrate from `pi.lane.lastResult` (§4.9). +22. At most one `Drive` exists per lane. Acceptance and taskless `requestAbort` never install one. A matching `drive` installs it before releasing the lane mutation line; another matching drive joins that pass, and a stale id starts nothing. Every drive-owned write rechecks exact Drive identity synchronously at commit admission. Normal `finally` removes only itself; abandonment and close/fault remove immediately; external finalization settles after terminal event delivery and retains ownership until both settlement and `finally` have occurred. Each newly installed pass invokes `before_drive` once after the cancellation check; joiners do not. `starting + cancel_requested` invokes neither `before_drive` nor `before_run`. 23. The §4.2 `Gate.admit()` catalog is complete. Every listed hook/provider/tool/timer integration calls `admit(() => operation())` after preparation; no unlisted code calls it. Admitted asynchronous provider setup/delegation owns `drive.gate.signal`. 24. `drive` and `requestAbort` are fenced by expected operation id. They may affect only that current operation; `drive` may also return its matching latest terminal result. A stale wake for A cannot drive or cancel B. 25. No public drive option encodes a wall-clock budget or partial-progress return. An admitted effect settles normally or is recovered from durable state after task loss; host scheduling and process termination remain outside the harness contract. @@ -3524,7 +3531,7 @@ Operations: 31. Scalar assistant/deferred state is the sole restart authority for streamed partials. One effect-pending response id constructs exactly one `pendingAssistantFrames(operationId, responseEntryId)` address; every element is an exported pi-ai `AssistantMessageFrame`; frame order equals provider event order; terminal `done`/`error` events are never stored; frames never establish provider completion or suppress unknown-outcome recovery. 32. Every final or synthetic response settlement — normal, recovery, cancellation, or external finalization — atomically deletes its exact frame list. Idle forks contain no frame lists. A restored partial may appear in `streamingMessage` but never in `transcript` before settlement. 33. The provider loop never awaits storage per frame; frame appends are enqueued synchronously in provider-event order, and awaiting the latest frame-write promise at stream settlement implies every accepted append completed. -34. Successful attachment publishes only complete lane projections and an open-operation inventory. It resolves no model/tool identity and starts no work. A later drive rereads and fences durable state. +34. Successful attachment publishes only complete lane projections and an open-operation inventory. It resolves no model/tool identity and starts no work. A later drive fences the authoritative owned projection and exact `Drive` identity; storage reads only dereference payloads named by that projection. 35. Every event-producing committing harness lane job publishes its owned projection and calls `emitBatch` with its complete event batch in the exact continuation that observes commit, as the callback's final action; this includes direct appends, lane and metadata setters, acceptance, and harness lane creation. The mutation never awaits delivery, but the public operation does. A lane watch registers buffering and clones live presentation synchronously, then performs bounded durable reads while holding the line. Snapshot plus buffered events has no gap or duplicate and replays no pre-registration lifecycle. `emitBatch` binds recipients and the emitting Context immediately; a delayed watcher receives the object-identical source Context, never its start Context. 36. Shared harness/lane/Session/SessionTree receivers retain no invocation Context and expose no receiver-level telemetry default. Concurrent calls preserve independent telemetry and cancellation lineage. Context and its values are neither durable operation data nor serialized business arguments. RPC cancel/disconnect reaches only the matching invocation through `context.abortSignal` and never becomes durable cancellation. 37. Process-local model/tool registry absence never becomes durable waiting state or an acceptance error. Pre-intent request-configuration absence fails in-band without fabricating a response/usage; missing requested tools stage `isError` tool-result messages with no invented details; uncertain effects settle under their existing recovery rules first. @@ -3545,7 +3552,7 @@ Each durable mutation race has exactly two durable histories. Process-local owne | assistant frame append vs response settlement | settlement awaits the latest frame write, then its transaction deletes the list; a crash between leaves the committed frame prefix under `effect_pending` | | assistant frame append vs external finalization | append first → terminal cleanup deletes the list; finalization first → the fenced append declines without committing | | live update event vs its queued frame/checkpoint commit | either finishes first; events are observation, and reconnect uses only committed frames/checkpoints | -| external finalization vs memo/checkpoint mutation | mutation first → terminal cleanup deletes it; finalization first → durable ownership check declines without committing | +| external finalization vs memo/checkpoint mutation | mutation first → terminal cleanup deletes it; finalization first → owned-projection/exact-Drive check declines without committing | | later tool B settles vs earlier tool A | B stages outcome-ready immediately; tree placement waits for A | | `abort` vs `before_run_end` follow-up | follow-up dropped; or committed and the run continues | | `cancelQueued` vs checkpoint consumption | `cancelled`; or `already_consumed` | @@ -3556,7 +3563,7 @@ Each durable mutation race has exactly two durable histories. Process-local owne | deferred write vs abort | write survives abort either way | | `requestAbort` vs `before_drive`/`before_run` admission | admission first → the complete hook pipeline runs and its consuming command observes cancellation; cancellation first → reconciliation runs and neither hook starts | | `requestAbort` vs ordinary operation admission | admission first → operation is invoked with the signal; cancellation first → gate refuses invocation | -| attachment vs concurrent resume | attachment owns the session before publication; after return, resume rereads durable state and stale `open` remains harmless | +| attachment vs concurrent resume | attachment owns the session before publication; after return, resume uses the authoritative owned projection and stale `open` remains harmless | | watcher registration vs state publication | watcher first → old snapshot plus the complete buffered event batch; publication/`emitBatch` first → new snapshot without that old batch | | close vs attachment | create completes and publishes a fully open harness; or close/fault rejects attachment without a partial harness | | snapshot capture vs resume | capture first yields pre-resume snapshot plus events; resume publication first yields post-transition snapshot | @@ -3568,7 +3575,7 @@ Each durable mutation race has exactly two durable histories. Process-local owne **Tier A — state and drive.** For every state in Part 3, construct it durably, close, reopen, drive its expected operation id, and assert the next durable transition, wait, or terminal result. Coverage must include: accepted and restored `starting` with hook output applied once by its consuming transaction; minimal projection restore; ad-hoc watch capture with one compaction-bounded transcript scan and state-directed payload reads, with required presentation references faulting watch and optional frame/checkpoint absence remaining legal; drive references are detected at consumption and captured step configuration is not re-resolved; assistant intent with no settlement — with no frames, partial frames, and authoritative end-frame content — recovered into synthetic partial settlements below and at the retry cap; settlement followed by each classification branch; every settled stop reason surviving except the two deliberate normalizations; a self-contained deferred step with copied configuration, consecutive polls, repeated equal-handle pending responses, ready and terminal responses, and handle-mismatch normalization into durable failure; every tool state including planned, effect-pending safe/unsafe with and without a checkpoint, outcome-ready, and completed; parallel B/C outcomes staged before A and never replayed after reopen; source-ordered materialization; invocation memos surviving safe replay and dying at outcome staging; a batch where every call terminates; genuine-length synthetic outcomes; every overflow crash position; every navigation state; abort at every position; unavailable model/configured tools as non-retryable configuration failure; missing requested tools as detail-free error results; every terminal transaction proving deletion of tool args, memos, checkpoints, frame lists, staged outcomes, and pending payloads; `pi.lane.lastResult`; preserved `pendingNextRun`; representation exclusivity for every reserved id; and every half-completed recovery prefix. -For each recovery prefix: close, reopen, drive, and compare against uninterrupted recovery. Invoking recovery twice from the initial prefix is **not** sufficient. Every operation kind also covers accept → close before first drive → reopen → drive. +For each recovery prefix: close, reopen, drive, and compare against uninterrupted recovery. Invoking recovery twice from the initial prefix is **not** sufficient. Every operation kind also covers accept → close before first drive → reopen → drive. At every test-controlled committed lane boundary, compare the published `Lane.state` with a fresh `restoreLaneState` result; divergence is an implementation defect, never silently healed by the next transition. One corruption assertion constructs an `aborted` response with running control directly and requires the consuming transition to reject it as an invariant defect. Provider conformance separately proves implementations emit `aborted` only for the supplied signal. @@ -3627,7 +3634,7 @@ One corruption assertion constructs an `aborted` response with running control d | **Invocation memo** | Tool-invocation-scoped durable current value used for replay-safe memoization; valid only while the call is effect-pending. | | **Terminal transaction** | The commit that deletes an operation's values/lists, writes `pi.lane.lastResult`, and clears `currentOperationId`. | | **Segment** | A branch-index range that references an older branch instead of copying it. | -| **External finalization** | A terminal transaction committed outside the live task. An in-process finalizer publishes once and settles the live Drive immediately from `finalizedOutcome`; with no local Drive, later callers hydrate from `pi.lane.lastResult` (§4.9). | +| **External finalization** | A terminal transaction committed outside the live task. An in-process finalizer closes the gate in committed materialization, publishes once, then settles the live Drive from `finalizedOutcome` after event delivery without waiting for detached work; the exact owner remains installed until both settlement and pass ending, preventing early hydration; with no local Drive, later callers hydrate from `pi.lane.lastResult` (§4.9). | | **Precise rewrite** | The administrative copy-retained-and-swap rebuild of a session store — the sole sanctioned path that removes entries or usage rows (§2.9). | # Appendix B — Coding-agent v3-format compatibility diff --git a/packages/agent/docs/work-packages/05-direct-durable-drive.md b/packages/agent/docs/work-packages/05-direct-durable-drive.md index e3b823137..128ca8e93 100644 --- a/packages/agent/docs/work-packages/05-direct-durable-drive.md +++ b/packages/agent/docs/work-packages/05-direct-durable-drive.md @@ -1,6 +1,6 @@ # WP05 — Direct durable drive -**Status: implementation-ready.** Independent Opus and GPT-5.6-Sol reviews report no blockers. +**Status: implementation-ready.** M1 is complete. Independent Opus and GPT-5.6-Sol reviews approved the authoritative-`Lane.state`, straight-line procedure correction below with no blockers. Removes breakpoints/manual drive, then implements the complete durable execution graph. Public drive stays disabled until every reachable phase and reconciliation path exists. @@ -22,7 +22,7 @@ Read completely, in this order, before editing anything. 3. `packages/agent/src/harness/session/types.ts` — durable types (`OperationState`, `RunState`, `RunPhase`, `Generation`, `Deferred`, `ToolBatch`, `ToolCall`, `StructuralDecision`, `NavigationState`, `LaneLastResult`, `SessionReader`, `Write`, `CommitResult`). 4. `packages/agent/src/harness/session/values.ts` — every typed address and write helper. 5. `packages/agent/src/harness/runtime2/lane.ts` — `Lane`, `Lane.command`, `LaneCommand`, `accept`. -6. `packages/agent/src/harness/runtime2/types.ts` — `Config`, `LaneState`, `LaneCommand`, `Drive`, `StepResult`, `SliceNotImplemented`. +6. `packages/agent/src/harness/runtime2/types.ts` — `Config`, `LaneState`, `LaneCommand`, `Drive`, the current `StepResult` that M2 simplifies to `ProcedureResult`, and `SliceNotImplemented`. 7. `packages/agent/src/harness/runtime2/harness.ts` — `Harness`, config store, `fault`, `close`. 8. `packages/agent/src/harness/runtime2/restore.ts` — projection restore. 9. `packages/agent/src/harness/execution/effect-gate.ts`, `execution/assistant.ts`, `execution/tools.ts`. @@ -41,7 +41,7 @@ session/** durable storage + session imports no harness run execution/effect-gate Gate / GateControl / AbortRequested execution/assistant provider streaming execution/tools tool phase helpers -runtime2/types.ts Config / LaneState / LaneCommand / Drive / StepResult +runtime2/types.ts Config / LaneState / LaneCommand / Drive / ProcedureResult runtime2/progress.ts progress channels -> runtime2/types, type-only runtime2/lane runtime2/drive/*.ts procedures -> runtime2/types, progress, type-only runtime2/lane runtime2/drive.ts the switch -> runtime2/drive/*, type-only runtime2/lane (M7 only) @@ -79,6 +79,7 @@ export class Drive { readonly waitForRetry: boolean; deferredPermits: number; finalizedOutcome?: DriveOutcome; + passEnded = false; private readonly control: GateControl; private settled = false; @@ -95,6 +96,7 @@ export class Drive { settle(outcome: DriveOutcome): void { /* first call wins */ } fail(error: unknown): void { /* first call wins */ } + hasSettled(): boolean { return this.settled; } beginAbort(cancellation: Promise): void { this.control.beginAbort(cancellation); } signalAbort(): void { this.control.signalAbort(); } closeGate(error: Error): void { this.control.close(error); } @@ -105,8 +107,8 @@ Procedures receive the concrete lane and drive: ```ts async function runGeneration( - lane: Lane, drive: Drive, run: DurableRun, generation: Generation, -): Promise { /* straight-line procedure */ } + lane: Lane, drive: Drive, generation: Generation, +): Promise { /* straight-line procedure */ } ``` `drive/**` imports `Lane` with `import type`, so no runtime cycle exists. `Lane` is package-internal and is not exported from the package root. @@ -127,15 +129,9 @@ private removeDrive(drive: Drive): void { } ``` -Every late transition and progress write fences both durable identity and exact process identity: +Every late transition and progress write checks the current operation in the authoritative `Lane.state` supplied by the lane mutation line and exact process identity. Exact identity is checked again synchronously at commit admission, immediately before `mutator.commit()`, with no `await` between the check and invocation. A planner-side check alone is insufficient because abandonment may replace the owner after an asynchronous planner returns. -```ts -const read = await readDurableRun(reader, lane.name, drive.operationId, drive.context); -if (read.kind === "lost") return { kind: "return", result: { kind: "lost_ownership" } }; -if (!lane.isDriveActive(drive)) return { kind: "return", result: { kind: "lost_ownership" } }; -``` - -This prevents ABA writes: after invocation abandonment, a replacement `Drive` may own the same durable operation id; the old object still fails exact identity and cannot write or remove the replacement. +This prevents ABA writes: after invocation abandonment, a replacement `Drive` may own the same durable operation id; the old object still fails exact identity and cannot write or remove the replacement. The small package-internal `Lane.commandOwned(drive, plan, context)` variant performs only this admission check; it otherwise has the same visible `LaneCommand` decision as `command` and introduces no planner or transition framework. `startDrive` is ordinary Lane code: @@ -155,10 +151,22 @@ startDrive(drive: Drive): void { return await runDrive(this, drive); } finally { drive.installerSignal?.removeEventListener("abort", abandon); - this.removeDrive(drive); + drive.passEnded = true; + // Normal passes remove before settlement. A finalized pass remains joinable until + // both its terminal event was delivered/settled and detached work ended. + if (drive.finalizedOutcome === undefined || drive.hasSettled()) this.removeDrive(drive); } })(); - pass.then((outcome) => drive.settle(outcome), (error) => drive.fail(error)); + pass.then( + (outcome) => { + // Once finalization records its outcome, the finalizer owns completion ordering: + // terminal event delivery first, then settle. The detached pass must not win. + if (drive.finalizedOutcome === undefined) drive.settle(outcome); + }, + (error) => { + if (drive.finalizedOutcome === undefined) drive.fail(error); + }, + ); void pass.catch(() => {}); } ``` @@ -168,7 +176,7 @@ Completion and owner removal are independent actions, but they need no superviso | Path | Completion | Owner | |---|---|---| | normal return/rejection | pass calls `drive.settle` / `drive.fail` | pass `finally` removes exact drive first | -| external finalization | terminal continuation records `finalizedOutcome`, closes gate, and calls `drive.settle(outcome)` immediately | deliberately **does not** remove; the pass `finally` removes it after detached work returns, preserving `LaneBusy` | +| external finalization | committed materialization records `finalizedOutcome` and closes the gate; after terminal event delivery, the finalizer settles without waiting for detached work | exact owner remains installed until **both** event-delivered settlement and pass ending have occurred; whichever happens second removes it | | installer abandonment | close gate and call `drive.fail(new DriveAbandoned())` immediately | remove exact drive immediately so a replacement pass can recover `effect_pending` | | close/fault | close gate and fail completion immediately | remove exact drive; admission is already sealed | @@ -176,7 +184,7 @@ Omitting removal is the retention operation; there is no no-op `retainOwner()` m Install and join share `drive.completion`. Therefore installer abandonment abandons that shared pass and every current joiner receives `DriveAbandoned`; durable state remains at its last commit and any caller may drive it again. A joiner's own invocation cancellation ends only that joiner's observation and does not abandon the installed pass. The retained pass context is cancellation-free; `installerSignal` is observed separately for installer abandonment. -An in-process finalizer records `drive.finalizedOutcome` before closing the gate. `runDrive` catches `OperationEnded` and returns that outcome without entering another lane job or publishing another end event. A replacement-process finalizer has no local drive and hydration uses `laneLastResult`. +An in-process finalizer records `drive.finalizedOutcome` before closing the gate in committed materialization, delivers its terminal event, then settles the Drive without waiting for detached work. `runDrive` catches `OperationEnded` and returns that outcome without entering another lane job or publishing another end event. A replacement-process finalizer has no local drive and hydration uses `laneLastResult`. ### 2.2 Gate / GateControl — approved @@ -275,38 +283,31 @@ async drive(options: DriveOptions, context: Context): Promise { } this.assertOpen(); - const claim = await this.command(async (_projection, reader) => { - // 1. A matching local owner wins over durable idle: after the terminal - // commit the lane is durably idle while the old owner is still - // delivering final events and removing itself. No durable read needed. + const claim = await this.command(async (state, reader) => { + // 1. A matching local owner wins over owned-projection idle: after the terminal + // commit the lane is idle while the old owner is still delivering final events + // and removing itself. const owner = this.activeDrive; if (owner !== undefined && owner.operationId === options.operationId) { return { kind: "return", result: { kind: "join", drive: owner } }; } - // A configured lane always has laneState (Session.createLane and seedMain write - // it). Absence is corruption, not idleness — treating it as idle would silently - // accept a drive against a lane that does not durably exist. - const durableLane = await reader.getValue(laneStateValue(this.name), context); - if (durableLane === undefined) { - throw new SessionInvariantError(`Lane ${JSON.stringify(this.name)} has no lane state`); - } - const currentId = durableLane.value.currentOperationId; - const last = (await reader.getValue(laneLastResult(this.name), context))?.value; + const currentId = state.operation?.meta.operationId ?? null; + const last = state.lastResult; // 2. Another operation is still owned locally. if (owner !== undefined) { return { kind: "return", result: { kind: "mismatch", error: this.mismatch(options.operationId, currentId, last) } }; } - // 3. Durable operation matches and has no owner: install. + // 3. Current operation matches and has no owner: install. if (currentId === options.operationId) { const drive = new Drive(options, context); this.activeDrive = drive; return { kind: "return", result: { kind: "install", drive } }; } - // 4. Idle and the latest result is ours: hydrate. + // 4. Idle and the latest result is ours: dereference only entries named by it. if (currentId === null && last?.operationId === options.operationId) { const outcome = await hydrateTerminalOutcome(reader, last, context); return { kind: "return", result: { kind: "settled", outcome: { kind: "settled", operationId: options.operationId, outcome } } }; @@ -375,27 +376,27 @@ private mismatch(expected: string, currentOperationId: string | null, last: Lane ### 2.4 Drive switch contract — approved -**Compile ordering.** `Drive` and `StepResult` live in `runtime2/types.ts` from M1. `runtime2/drive.ts` is created in M7, once every procedure module it imports exists. M2–M6 call procedures directly from tests with a concrete `Lane` and `Drive`; public drive stays guarded until M8. +**Compile ordering.** `Drive` lives in `runtime2/types.ts` from M1; M2 replaces the initial `StepResult` with `ProcedureResult`. `runtime2/drive.ts` is created in M7, once every procedure module it imports exists. M2–M6 call procedures directly from tests with a concrete `Lane` and `Drive`; public drive stays guarded until M8. ```ts // packages/agent/src/harness/runtime2/types.ts -export type StepResult = - | { kind: "advance" } - | { kind: "reload" } +export type LostOwnership = { kind: "lost_ownership" }; + +export type ProcedureResult = + | { kind: "continue" } | { kind: "waiting"; outcome: DriveOutcome } | { kind: "settled"; outcome: TerminalOperationOutcome } - | { kind: "lost_ownership" }; + | LostOwnership; ``` | Result | Obligation | |---|---| -| `advance` | exactly one commit landed during this step | -| `reload` | no commit landed; state moved under this planner | -| `waiting` | the last commit parked the operation in retry or deferred wait | -| `settled` | the terminal transaction committed | -| `lost_ownership` | no commit landed; this task terminates and never writes again | +| `continue` | inspect the current `Lane.state` and continue the ordinary drive loop; this procedure either committed or observed a supported concurrent change since its pre-work snapshot | +| `waiting` | the operation is durably parked in retry or deferred wait | +| `settled` | this procedure's terminal transaction committed | +| `lost_ownership` | commit nothing; this task terminates and never writes again | -`advance` is created only by `LaneCommand.materialize`, except `foldSteps` may re-emit an existing child `advance`. `runDrive` bounds consecutive `reload` results. +There is no global `advance`/`reload` distinction. A procedure must not return `continue` from an unchanged state it does not understand; that is an invariant defect, not a spin-and-hope path. Whether a particular intent commit landed matters only to the straight-line procedure that is about to start its effect. Such a helper returns a small local result such as `"committed" | "state_changed" | LostOwnership`; it does not become drive-loop vocabulary. Tool children likewise use a tool-local result when the batch must know whether a child staged or materialized anything. ```ts // packages/agent/src/harness/runtime2/drive.ts — M7 @@ -403,206 +404,124 @@ export async function runDrive( lane: Lane, drive: Drive, ): Promise { try { - return await drivePass(lane, drive); + await runBeforeDriveUnlessCancelled(lane, drive); + for (;;) { + if (lane.closedError !== undefined) throw lane.closedError; + if (!lane.isDriveActive(drive)) throw new DriveAbandoned(); + const before = lane.state; + const operation = before.operation; + if (operation === null || operation.meta.operationId !== drive.operationId) { + return hydrateSettled(lane, drive); + } + if (operation.state.control.status === "cancel_requested") { + return { kind: "settled", operationId: drive.operationId, + outcome: await reconcile(lane, drive, operation) }; + } + const result = await dispatch(lane, drive, operation); + if (result.kind === "continue") { + if (lane.state === before) { + throw new SessionInvariantError("drive procedure returned continue without progress"); + } + continue; + } + if (result.kind === "lost_ownership") { + if (drive.finalizedOutcome !== undefined) return drive.finalizedOutcome; + if (lane.closedError !== undefined) throw lane.closedError; + throw new DriveAbandoned(); + } + if (result.kind === "settled") { + return { kind: "settled", operationId: drive.operationId, outcome: result.outcome }; + } + return result.outcome; + } } catch (error) { if (error instanceof OperationEnded) return hydrateSettled(lane, drive); if (error instanceof AbortRequested) { await error.cancellation; - const current = await loadOperation(lane, drive); - if (current.kind === "lost") return hydrateSettled(lane, drive); + const operation = lane.state.operation; + if (operation === null || operation.meta.operationId !== drive.operationId) { + return hydrateSettled(lane, drive); + } return { kind: "settled", operationId: drive.operationId, - outcome: await reconcile(lane, drive, current.value) }; + outcome: await reconcile(lane, drive, operation) }; } - throw error; // DriveAbandoned, HarnessClosed, HarnessFault, OperationMismatch + throw error; } } ``` -`drivePass` loads durable control before `before_drive`, skips ordinary hooks under `cancel_requested`, then runs one total phase switch. Procedures receive `(lane, drive, durableState)`. `drive.waitForRetry` and mutable `drive.deferredPermits` are pass-local and never mutate caller options. +The body is illustrative, not a required helper inventory. The unchanged-`continue` guard rejects the pass with `SessionInvariantError`; it is a liveness assertion outside `Lane.command`, not another transition or a storage reread. `drivePass`/`dispatch` inspect the authoritative `Lane.state`; they do not load lane or operation control values from storage. `hydrateSettled` first returns `drive.finalizedOutcome` when present; otherwise it uses matching `lane.state.lastResult` and dereferences only entries named by that result. A foreign current/latest operation produces `OperationMismatch` and no write. -`hydrateSettled(lane, drive)` first returns `drive.finalizedOutcome` when present; otherwise it reads matching `laneLastResult`. A foreign current/latest operation produces `OperationMismatch` and no write. +### 2.5 Authoritative lane state and owned commands — approved -### 2.5 Transition planner / decision shape — approved +`Lane.command` passes the authoritative current lane/operation projection. Every supported control-state mutation commits through that `Lane`, and a successful commit publishes `decision.next` before releasing the mutation line. A failed commit publishes nothing and faults the harness. A process crash destroys the projection; attachment restores and validates it from durable control values. -`Lane.command` passes `this.state` (the **process-local projection**), not a durable read. Drive transitions must not trust it. +Procedures therefore do **not** reread `laneState`, `operationMeta`, `operationState`, `laneLeaf`, or `laneConfig` on each transition. They use the callback reader only to dereference content named by the projection: pending entries, assistant frames, tool arguments/checkpoints, preparations, memos, staged outcomes, tree entries, and cleanup-prefix scans. -One authoritative read shape serves every transition **and every operation kind**. It is deliberately the simplest thing that is correct: the lane fence plus five indexed point reads. It must never conflate these three outcomes: - -| Outcome | Meaning | Caller | -|---|---|---| -| `{ kind: "ok", value }` | the fence names this operation and every required value validated | proceed | -| `{ kind: "lost" }` | the fence no longer names this operation | return `lost_ownership`; terminate the task | -| throws `SessionInvariantError` | the fence names us, but durable state is missing or self-inconsistent | corruption; faults the harness | - -A missing `operationMeta` behind a matching fence is not a lost race — it is a broken session, and silently treating it as "someone else took the lane" would hide it forever. +Drive-owned commands use one small `Lane` variant so exact-object ownership is checked at commit admission rather than only in an asynchronous planner: ```ts -// packages/agent/src/harness/runtime2/drive/transition.ts -// Address constructors are aliased exactly as runtime2/lane.ts already aliases them, -// because session/types.ts and runtime2/types.ts both export a type named LaneState. -import { - laneConfig, laneLastResult, laneLeaf, - laneState as laneStateValue, - operationMeta as operationMetaValue, - operationState as operationStateValue, -} from "../../session/values.ts"; - -export type DurableRead = { kind: "ok"; value: T } | { kind: "lost" }; - -/** Authoritative durable read of the current operation, whatever its kind. */ -export interface DurableOperation { - readonly meta: OperationMeta; - readonly state: OperationState; - readonly leafId: string | null; - readonly configuration: LaneConfiguration; - readonly pendingNextRun: string[]; - readonly lastResult: LaneLastResult | undefined; -} - -/** The same read, narrowed to a run. */ -export interface DurableRun extends DurableOperation { - readonly state: RunState; -} - -export async function readDurableOperation( - reader: SessionReader, lane: string, operationId: string, context: Context, -): Promise> { - // A configured lane has all three of laneState, laneLeaf, laneConfig — Session.createLane - // writes them in one transaction and harness attachment requires them (harness.md §4.4). - // Absence of the lane record is corruption, and must not be read as "someone else owns it". - const laneStateStored = await reader.getValue(laneStateValue(lane), context); - if (laneStateStored === undefined) { - throw new SessionInvariantError(`Lane ${JSON.stringify(lane)} has no lane state`); - } - if (laneStateStored.value.currentOperationId !== operationId) return { kind: "lost" }; - - const [meta, state, leaf, configuration, lastResult] = await Promise.all([ - reader.getValue(operationMetaValue(operationId), context), - reader.getValue(operationStateValue(operationId), context), - reader.getValue(laneLeaf(lane), context), - reader.getValue(laneConfig(lane), context), - reader.getValue(laneLastResult(lane), context), - ]); - - // Past this point the fence names us, so every absence is corruption — including laneLeaf. - // laneLeaf is `Value`: a lane with no entries stores the VALUE null, which is - // valid. An absent leaf ADDRESS is a different fact and means the lane record is incomplete. - if (meta === undefined || state === undefined || leaf === undefined || configuration === undefined) { - throw new SessionInvariantError( - `Lane ${JSON.stringify(lane)} owns ${operationId} with incomplete durable state ` + - `(meta=${meta !== undefined}, state=${state !== undefined}, ` + - `leaf=${leaf !== undefined}, configuration=${configuration !== undefined})`, - ); - } - if (meta.value.operationId !== operationId || meta.value.lane !== lane) { - throw new SessionInvariantError( - `Operation meta for ${operationId} names ${JSON.stringify(meta.value.lane)}/${meta.value.operationId}`, - ); - } - // OperationMeta.intent.kind and OperationState.kind are the same three-member set. - if (meta.value.intent.kind !== state.value.kind) { - throw new SessionInvariantError( - `Operation ${operationId} has ${meta.value.intent.kind} intent but ${state.value.kind} state`, - ); - } - - return { - kind: "ok", - value: { - meta: meta.value, - state: state.value, - leafId: leaf.value, // may legitimately be null - configuration: configuration.value, - pendingNextRun: laneStateStored.value.pendingNextRun, - lastResult: lastResult?.value, - }, - }; -} - -/** A non-run state behind a matching fence is corruption, not a lost race. */ -export function narrowRun(operation: DurableOperation): DurableRun { - if (operation.state.kind !== "run") { - throw new SessionInvariantError( - `Operation ${operation.meta.operationId} is ${operation.state.kind}, expected run`, - ); - } - return { ...operation, state: operation.state }; -} - -export async function readDurableRun( - reader: SessionReader, lane: string, operationId: string, context: Context, -): Promise> { - const read = await readDurableOperation(reader, lane, operationId, context); - return read.kind === "lost" ? read : { kind: "ok", value: narrowRun(read.value) }; -} - -/** The exact next LaneState published with a committed operation state, for any kind. */ -export function projectOperation( - operation: DurableOperation, state: OperationState, leafId?: string | null, -): LaneState { - return { - leafId: leafId === undefined ? operation.leafId : leafId, - configuration: operation.configuration, - pendingNextRun: operation.pendingNextRun, - ...(operation.lastResult === undefined ? {} : { lastResult: operation.lastResult }), - operation: { meta: operation.meta, state }, - }; -} - -/** Run-typed convenience over projectOperation. */ -export function projectRun(run: DurableRun, state: RunState, leafId?: string | null): LaneState { - return projectOperation(run, state, leafId); -} - -/** Terminal projection for any operation kind: the operation ceases to exist. */ -export function projectIdle( - operation: DurableOperation, lastResult: LaneLastResult, leafId?: string | null, -): LaneState { - return { - leafId: leafId === undefined ? operation.leafId : leafId, - configuration: operation.configuration, - pendingNextRun: operation.pendingNextRun, - lastResult, - operation: null, - }; +// packages/agent/src/harness/runtime2/lane.ts +async commandOwned( + drive: Drive, + plan: (state: LaneState, reader: SessionReader) => + LaneCommand | Promise>, + context: Context, +): Promise { + // Same implementation and visible LaneCommand decision as command(), with two additions: + // 1. before planning, state.operation must name drive.operationId and activeDrive must be drive; + // 2. for a commit decision, assertOpen() and activeDrive === drive are checked immediately + // before mutator.commit(), with no await between the checks and invocation. } ``` -Usage inside any drive transition (approved shape; body illustrative). The planner returns a `StepResult` directly, which makes "no `advance` without a commit" structural rather than a rule to remember: +`commandOwned` is not a scheduler, planner, transaction wrapper, or capability facade. It is `Lane.command` plus the exact-Drive admission fence. If its initial operation/id check or final exact-object check fails, it returns `lost_ownership` and commits nothing. Close/fault is checked first at final admission and propagates its lifecycle error rather than being mislabeled as ownership loss. Ordinary public lane commands continue to use `command`. + +A representative transition stays direct: ```ts -const step = await lane.command(async (_projection, reader) => { - const read = await readDurableRun(reader, lane.name, drive.operationId, drive.context); - // Declines are EXPECTED. Lane.command turns a planner throw into a harness fault, so - // return an explicit no-commit outcome instead of throwing. - if (read.kind === "lost") return { kind: "return", result: { kind: "lost_ownership" } }; - const run = read.value; - if (run.state.phase.kind !== "assistant") return { kind: "return", result: { kind: "reload" } }; - if (run.state.phase.generation.status !== "ready") return { kind: "return", result: { kind: "reload" } }; - if (run.state.phase.generation.context.stepId !== expectedStepId) return { kind: "return", result: { kind: "reload" } }; - if (!lane.isDriveActive(drive)) return { kind: "return", result: { kind: "lost_ownership" } }; - - const nextState: RunState = { ...run.state, phase: { kind: "assistant", generation } }; +const intent = await lane.commandOwned(drive, (state) => { + const operation = state.operation; + if (operation === null || operation.state.kind !== "run") { + throw new SessionInvariantError("generation intent reached a non-run operation"); + } + const run = operation.state; + if (run.control.status === "cancel_requested") { + return { kind: "return", result: { kind: "state_changed" } as const }; + } + if (run.phase.kind !== "assistant" || + run.phase.generation.status !== "ready" || + run.phase.generation.context.stepId !== expectedStepId) { + throw new SessionInvariantError("generation intent lost its ready boundary"); + } + const nextState: RunState = { ...run, phase: { kind: "assistant", generation } }; return { kind: "commit", writes: [setValue(operationStateValue(drive.operationId), nextState)], - next: projectRun(run, nextState), - materialize: (): StepResult => ({ kind: "advance" }), - events: () => [{ type: "turn_start", runId: drive.operationId, turnId: stepId, lane: lane.name }], + next: { ...state, operation: { meta: operation.meta, state: nextState } }, + materialize: () => ({ kind: "committed" } as const), + events: () => [{ type: "turn_start", runId: drive.operationId, + turnId: expectedStepId, lane: lane.name }], }; }, drive.context); -if (step.kind !== "advance") return step; // reload / lost_ownership propagate unchanged +if (intent.kind !== "committed") return intent.kind === "lost_ownership" + ? intent + : { kind: "continue" }; +// The effect may start only on this branch. ``` -**Do not optimize this now.** No focused/partial readers, no read-budget accounting, no caching or memoized read layer, no batching `getValues` API. These are indexed point lookups and SQLite is expected to serve them cheaply. Narrowing them is a later, profiling-driven change with a measurement to justify it. +The transition derives `next` from the state supplied to that planner, never from a snapshot captured before the lane job. Inbox and other concurrent fields that do not invalidate this transition are preserved in the committed next state. Cancellation returns to reconciliation without committing; a newly non-empty inbox invalidates only a terminal-finish boundary; relevant tool-sibling movement uses the tool-local `state_changed` result. Ownership loss is handled in band. A matching operation with any other impossible kind/phase relationship is a `SessionInvariantError`. + +The single-owner contract is explicit: while a harness owns a `Session`, retained `Session` and pre-attachment `SessionTree` mutators are forbidden. In-process administrative finalization also commits through the owning `Lane`. External storage mutation is not a supported second authority, and the runtime does not silently heal it. Conformance tests compare each committed projection with a fresh restore at controlled boundaries. ### 2.6 Generation procedure — approved shape, illustrative bodies ```ts // packages/agent/src/harness/runtime2/drive/generation.ts async function runReadyGeneration( - lane: Lane, drive: Drive, run: DurableRun, gen: Extract, -): Promise { + lane: Lane, drive: Drive, run: RunState, + gen: Extract, +): Promise { const config = lane.readConfig(); const identity = gen.context.configuration.model; const model = lane.models.getModel(identity.provider, identity.modelId); @@ -613,7 +532,8 @@ async function runReadyGeneration( const systemPrompt = await resolveSystemPrompt(config, drive.context); const patch = await lane.hooks.runWithGate( "before_request", - { lane: lane.name, runId: drive.operationId, model, step: "assistant", attempt: gen.nextAttempt, streamOptions: gen.context.streamOptions }, + { lane: lane.name, runId: drive.operationId, model, step: "assistant", + attempt: gen.nextAttempt, streamOptions: gen.context.streamOptions }, drive.gate, drive.context, ); const streamOptions = applyStreamOptionsPatch(gen.context.streamOptions, patch?.streamOptions); @@ -627,10 +547,9 @@ async function runReadyGeneration( responseEntryId, usageId, intendedOutputLimit: model.maxTokens, contextWindow: model.contextWindow, }; - // commitGenerationIntent returns a StepResult, never a boolean. A declined planner - // committed nothing, so the only honest answers are `reload` and `lost_ownership`. const intent = await commitGenerationIntent(lane, drive, gen.context.stepId, generation); - if (intent.kind !== "advance") return intent; + if (intent.kind === "lost_ownership") return intent; + if (intent.kind === "state_changed") return { kind: "continue" }; // ---- effect: admitted, off the lane line ---- const progress = openFrameProgress(lane, drive, responseEntryId); @@ -646,20 +565,15 @@ async function runReadyGeneration( } finally { progress.seal(); } - await progress.drain(); // rejects if any frame write failed + await progress.drain(); // ---- outcome: one visible commit ---- - return settleAssistant(lane, drive, { responseEntryId, usageId, stepId: gen.context.stepId }, response); + return settleAssistant(lane, drive, + { responseEntryId, usageId, stepId: gen.context.stepId }, response); } ``` -Constraints: preparation precedes the intent commit; settlement writes under the reserved ids; the frame observer enqueues synchronously and never awaits storage; `progress.drain()` precedes settlement. - -**`{ kind: "advance" }` is constructed in exactly one kind of place: a `materialize(commit)` callback.** `Lane.command` calls `materialize` only after its commit landed, so "advance proves a commit" becomes structural rather than a rule to remember. A procedure that returns `advance` from anywhere else is guessing. - -The single exception is `foldSteps` (§2.8), which re-emits `advance` only when a child step already carried one; it never manufactures the first. `enterConfigurationFailure`, `settleAssistant`, `commitGenerationIntent`, and every other transition follow the rule and propagate a declined planner's `reload`/`lost_ownership` unchanged. - -Grep guard for the final review: every `kind: "advance"` outside `runtime2/types.ts` must sit inside a `materialize` callback or inside `foldSteps`. +Preparation precedes the intent commit; settlement writes under the reserved ids; the frame observer enqueues synchronously and never awaits storage; `progress.drain()` precedes settlement. `commitGenerationIntent` has a small local result because this procedure must know whether it may start the effect. Successful settlement returns `continue`, `waiting`, `settled`, or `lost_ownership` according to the state it committed; there is no global commit-count result. ### 2.7 Progress helper — approved @@ -676,20 +590,20 @@ function openProgress( lane: Lane, drive: Drive, commitWrite: (item: T) => Write, clear: () => Write, - stillOwns: (reader: SessionReader) => Promise, + stillOwns: (state: LaneState) => boolean, ): ProgressChannel { let sealed = false; let latest: Promise = Promise.resolve(); return { write(item) { if (sealed) return; - const write = lane.command(async (projection, reader) => { - if (!(await stillOwns(reader))) return { kind: "return", result: undefined }; - return { kind: "commit", writes: [commitWrite(item)], next: projection, + const write = lane.commandOwned(drive, (state) => { + if (!stillOwns(state)) return { kind: "return", result: undefined }; + return { kind: "commit", writes: [commitWrite(item)], next: state, materialize: () => undefined }; - }, drive.context); - latest = write; // raw rejecting promise retained for drain - void write.catch(() => {}); // separate harness-fault observer + }, drive.context).then(() => undefined); + latest = write; + void write.catch(() => {}); }, seal() { sealed = true; }, async drain() { await latest; }, @@ -698,83 +612,32 @@ function openProgress( } ``` -`openFrameProgress(lane, drive, responseEntryId)` and `openToolProgress(lane, drive, invocationId)` fence both the durable phase/id and `lane.isDriveActive(drive)`. The provider/tool callback enqueues synchronously, settlement seals then drains, and the settling transaction includes `clearWrite()`. +`openFrameProgress(lane, drive, responseEntryId)` and `openToolProgress(lane, drive, invocationId)` inspect the current projection's phase/id and use `commandOwned` for the final exact-Drive admission fence. They perform no durable control reads. The provider/tool callback enqueues synchronously, settlement seals then drains, and the settling transaction includes `clearWrite()`. ### 2.8 Tool batch — approved shape, illustrative bodies +The tool procedure is one ordinary async routine. It keeps its process-local started-promise map, but every status patch is computed from the current `Lane.state` supplied to that specific `commandOwned` job. It never patches a batch snapshot captured before the job. + +Each child uses a module-local result only: + ```ts -// packages/agent/src/harness/runtime2/drive/tools.ts -// Every child procedure returns a StepResult. None of them may invent `advance`. -async function runToolBatch( - lane: Lane, drive: Drive, run: DurableRun, batch: ToolBatch, -): Promise { - const steps: StepResult[] = []; - - // A crash can leave staged outcome_ready calls with no live task. Nothing else - // places them, so materialize any existing ready prefix before starting work. - const prefix = await materializeReadyPrefix(lane, drive, batch.turnId); - steps.push(prefix); - // Short-circuit BEFORE resolving toolContext or starting any call. If we already lost the - // fence there is nothing to run; if the state moved, re-dispatch reads it fresh. Starting - // effects here would run tools for a batch that is no longer current. - if (prefix.kind === "lost_ownership" || prefix.kind === "reload") return prefix; - - const sequential = run.state.settings.toolExecution === "sequential"; - const started = new Map>(); // process-local, never durable - const toolContext = await resolveToolContext(lane, drive); // once per batch, not per call - - try { - for (const call of batch.calls) { - if (call.status === "completed" || call.status === "outcome_ready") continue; - const work = call.status === "effect_pending" - ? recoverCall(lane, drive, batch, call, toolContext) // orphan by construction - : runCall(lane, drive, batch, call, toolContext); - started.set(call.sourceIndex, work); - if (sequential) { - const step = await work; - steps.push(step); - // Sequential mode starts the next call only if this one still owned the lane. - // A later call would be planned against a batch that already moved. - if (step.kind === "lost_ownership" || step.kind === "reload") return foldSteps(steps); - } - } - // Parallel mode cannot unstart what it already admitted, so it observes and settles - // every started call. The `started` loop above is the only place calls begin, and it - // has finished, so no LATER call can start after a loss is observed here. - if (!sequential) steps.push(...(await Promise.all(started.values()))); - } catch (error) { - if (!(error instanceof AbortRequested)) throw error; - await error.cancellation; - for (const settled of await Promise.allSettled(started.values())) { - if (settled.status === "fulfilled") steps.push(settled.value); - } - steps.push(await stageUnstartedCalls(lane, drive, batch)); - } - // Skip the closing materialization once ownership is gone: it would only decline anyway, - // and `lost_ownership` must reach runDrive without another lane job. - if (!steps.some((step) => step.kind === "lost_ownership")) { - steps.push(await materializeReadyPrefix(lane, drive, batch.turnId)); - } - return foldSteps(steps); -} - -/** - * Aggregate child outcomes. Losing the fence dominates: the task must stop. Otherwise - * the batch advanced only if at least one child actually committed; if every child - * declined, nothing changed durably and the honest answer is `reload`. - */ -function foldSteps(steps: StepResult[]): StepResult { - if (steps.some((step) => step.kind === "lost_ownership")) return { kind: "lost_ownership" }; - const waiting = steps.find((step) => step.kind === "waiting"); - if (waiting !== undefined) return waiting; // tools never produce it; kept total - if (steps.some((step) => step.kind === "settled")) { - throw new SessionInvariantError("a tool child procedure may not settle the operation"); - } - return steps.some((step) => step.kind === "advance") ? { kind: "advance" } : { kind: "reload" }; -} +type ToolChildResult = + | { kind: "committed" } + | { kind: "state_changed" } + | LostOwnership; ``` -Every status patch is derived from the state reread inside its own `Lane.command`: +This local result answers whether the batch may continue starting effects and whether a source-ready prefix may be materialized. It is not returned to the global drive switch. The batch procedure: + +1. materializes any already-staged source-ready prefix; +2. resolves `toolContext` once; +3. makes one source-ordered start pass, sequentially or in parallel; +4. retains every started promise and observes every started result; +5. stages finalized outcomes immediately in completion order; +6. materializes the contiguous ready prefix in source order; +7. returns `lost_ownership` if the exact owner was replaced, otherwise `continue` so the outer loop inspects the now-current phase. + +Under cancellation, no later call starts. Already-started calls are observed and staged, unstarted calls receive their specified synthetic cancellation outcomes, and source-order materialization finishes before reconciliation continues. ```ts function patchCall(state: RunState, sourceIndex: number, next: ToolCall): RunState { @@ -783,58 +646,108 @@ function patchCall(state: RunState, sourceIndex: number, next: ToolCall): RunSta ...state, phase: { kind: "tools", batch: { ...state.phase.batch, - calls: state.phase.batch.calls.map((c) => (c.sourceIndex === sourceIndex ? next : c)), + calls: state.phase.batch.calls.map((call) => + call.sourceIndex === sourceIndex ? next : call), } }, }; } -// call site: const read = await readDurableRun(reader, lane, operationId, ctx); -// patchCall(read.value.state, i, next) -// NEVER patchCall(batchCapturedBeforeTheJob, …) ``` Staging is completion-ordered (`outcome_ready`); `materializeReadyPrefix` places the contiguous ready prefix in source order in one commit, and `addedToolNames` takes effect only at placement. -### 2.9 Terminal transaction — approved +### 2.9 Terminal transactions stay in their procedures — approved -The terminal transaction is shared by runs, standalone compactions, and navigations, so it uses the **general** reader and projector. Nothing here narrows to a run. +The procedure that owns a terminal boundary also owns its terminal `Lane.commandOwned`. An authorized in-process external finalizer is the one exception: it owns an ordinary `Lane.command` because it may finalize an unowned operation. Its `materialize` continuation records `finalizedOutcome` and closes any matching live Drive gate synchronously after projection publication. The finalizer settles that Drive only after `Lane.command` returns, so terminal event delivery still precedes public completion. The finalizer must still verify that the current projection names the exact operation it intends to end; a terminal winner observed first commits nothing. Each procedure already knows its exact semantic state, local result, publication writes, public outcome, and terminal event. There is no generic `commitTerminal`, `TerminalRequest`, terminal validator callback, or terminal planner. + +Inside that one command, the procedure: + +1. verifies its exact terminal boundary against the current authoritative projection; +2. computes its bounded `LaneLastResult` directly from known state and reserved ids; +3. prepends any procedure-specific publication writes (summary entry, navigation leaf/label movement, usage); +4. appends the shared mechanical operation-cleanup writes; +5. writes `laneLastResult` and idle `laneState`; +6. publishes the exact idle process-local projection, including any leaf movement in its publication writes; +7. constructs the complete `{ kind: "settled", outcome }` and its one terminal event from `CommitResult`, which supplies storage-assigned entry sequence and timestamp when needed. + +A normal completed-run finish is representative. Its terminal check is visibly part of the checkpoint procedure: ```ts -// packages/agent/src/harness/runtime2/drive/terminal.ts -async function commitTerminal( - lane: Lane, drive: Drive, request: TerminalRequest, -): Promise { - return lane.command(async (_projection, reader) => { - const read = await readDurableOperation(reader, lane.name, drive.operationId, drive.context); - // The owner reached its own terminal transaction; losing the fence here is corruption. - if (read.kind === "lost") throw new SessionInvariantError("terminal transaction lost its operation"); - const operation = read.value; +return lane.commandOwned(drive, async (state, reader) => { + const operation = state.operation; + if (operation === null || operation.state.kind !== "run") { + throw new SessionInvariantError("run finish reached a non-run operation"); + } + const run = operation.state; + if (run.control.status === "cancel_requested") { + return { kind: "return", result: { kind: "continue" } as const }; + } + if (run.phase.kind !== "checkpoint" || run.phase.continuation.kind !== "may_finish") { + throw new SessionInvariantError("run finish lost its checkpoint boundary"); + } + if (run.inbox.steer.length !== 0 || run.inbox.followUp.length !== 0 || run.inbox.writes.length !== 0) { + return { kind: "return", result: { kind: "continue" } as const }; + } + if (state.leafId === null) throw new SessionInvariantError("completed run has no leaf"); - const [toolArgs, toolMemos, preparations, toolOutputs] = await Promise.all([ - reader.scanValues(operationToolArgsPrefix(drive.operationId), drive.context), - reader.scanValues(operationToolMemoPrefix(drive.operationId), drive.context), - reader.scanValues(operationPreparationPrefix(drive.operationId), drive.context), - reader.scanValues(pendingToolOutputPrefix(drive.operationId), drive.context), - ]); - - const writes: Write[] = [ - deleteValue(operationMeta(drive.operationId)), - deleteValue(operationState(drive.operationId)), - ...toolArgs.map((v) => deleteValue(v.address)), - ...toolMemos.map((v) => deleteValue(v.address)), - ...preparations.map((v) => deleteValue(v.address)), - ...toolOutputs.map((v) => deleteValue(v.address)), - ...liveFrameListWrites(operation.state), // run only: exact deleteList for an unsettled response - ...stagedPendingEntryWrites(operation.state), // run only: staged results + queued/drained ids + const finalId = run.phase.continuation.includeFinalAssistant + ? run.latestAssistantEntryId + : null; + const finalEntry = finalId === null + ? undefined + : (await reader.getEntries([finalId], drive.context)).get(finalId); + if (finalId !== null && + (finalEntry?.type !== "message" || finalEntry.message.role !== "assistant" || + finalEntry.message.stopReason === "pending")) { + throw new SessionInvariantError(`Run finish references invalid assistant ${finalId}`); + } + const final = finalEntry?.type === "message" && finalEntry.message.role === "assistant" + ? { finalEntryId: finalEntry.id, finalMessage: finalEntry.message } + : {}; + const outcome: TerminalOperationOutcome = { + operation: "run", runId: drive.operationId, kind: "completed", + leafId: state.leafId, ...final, + }; + const lastResult: LaneLastResult = { + operationId: drive.operationId, + kind: "run", + outcome: "completed", + leafId: state.leafId, + runCompletion: finalId === null ? "terminated_tools" : "assistant", + ...(finalId === null ? {} : { finalAssistantEntryId: finalId }), + }; + const cleanup = await operationCleanupWrites( + reader, drive.operationId, run, drive.context, + ); + return { + kind: "commit", + writes: [ + ...cleanup, setValue(laneLastResult(lane.name), lastResult), - setValue(laneStateValue(lane.name), { currentOperationId: null, pendingNextRun: operation.pendingNextRun }), - ]; - return { kind: "commit", writes, next: projectIdle(operation, lastResult), materialize: () => outcome, - events: () => [endEvent] }; - }, drive.context); -} + setValue(laneStateValue(lane.name), { + currentOperationId: null, + pendingNextRun: state.pendingNextRun, + }), + ], + next: { ...state, lastResult, operation: null }, + materialize: () => ({ kind: "settled", outcome } as const), + events: () => [{ + type: "run_end", lane: lane.name, runId: drive.operationId, + leafId: state.leafId, outcome: "completed", ...final, + }], + }; +}, drive.context); ``` -`pendingNextRun` is lane-owned and survives. Defensive deletes of already-removed addresses are no-ops. **Never** add `delete_prefix`. +Standalone structural and navigation procedures use the same visible suffix after their own result-publication writes. They do not call a generic terminal procedure. + +`runtime2/drive/terminal.ts` contains only mechanical helpers: + +- `operationCleanupWrites(reader, operationId, operationState, context)` scans typed operation prefixes and returns concrete deletes for meta/state, tool arguments, memos, preparations, tool-output checkpoints, the exact live assistant-frame list, and operation-owned pending entries; +- `hydrateTerminalOutcome(reader, lastResult, context)` dereferences only entries directly named by the bounded value. + +Each procedure constructs its own `LaneLastResult` visibly. Entry-producing terminal procedures construct public entries and events in `materialize(commit)` / `events(commit)`, after storage assigned sequence and timestamp. Operation-id matching remains visible at the drive claim/hydration call site rather than hiding in another helper. + +`pendingNextRun` is lane-owned and survives. Defensive deletes of already-removed addresses are no-ops. **Never** add `delete_prefix`. The outer drive loop performs process-local completion once after receiving `settled`; it does not construct or commit the terminal transaction. ### 2.10 GatingStorage + representative test — approved @@ -967,15 +880,15 @@ Drive procedures are package-internal and receive the concrete `Lane` ```ts // every drive procedure import type { Lane } from "../lane.ts"; -import type { Drive, StepResult } from "../types.ts"; +import type { Drive, ProcedureResult } from "../types.ts"; async function runCheckpoint( - lane: Lane, drive: Drive, run: DurableRun, + lane: Lane, drive: Drive, run: RunState, phase: Extract, -): Promise { /* ... */ } +): Promise { /* ... */ } ``` -`Lane` already exposes the package-internal members procedures need: `session`, `models`, `hooks`, `command`, `emitBatch`, `readConfig`, `mismatch`, and `isDriveActive`. The package root exports only `AgentLane` / `AgentHarness`, so these members do not widen public API. Procedure imports of `Lane` are type-only and erase; `lane.ts` may value-import `drive.ts` in M8 without a runtime cycle. +`Lane` already exposes the package-internal members procedures need: `state`, `session`, `models`, `hooks`, `command`, `emitBatch`, `readConfig`, `mismatch`, and `isDriveActive`; M2 adds only `commandOwned` for exact-Drive commit admission. The package root exports only `AgentLane` / `AgentHarness`, so these members do not widen public API. Procedure imports of `Lane` are type-only and erase; `lane.ts` may value-import `drive.ts` in M8 without a runtime cycle. Rules: - `TContext extends object | undefined` everywhere; no other bound. @@ -1067,8 +980,9 @@ const ENTRY_WRITE = 0; const USAGE_WRITE = 1; return { - kind: "commit", writes, next: projectRun(run, nextState), - materialize: (): StepResult => ({ kind: "advance" }), + kind: "commit", writes, + next: { ...state, operation: { meta: operation.meta, state: nextState } }, + materialize: (): ProcedureResult => ({ kind: "continue" }), events: (commit) => [ { type: "entry_added", lane: lane.name, entry: { ...responseEntry, seq: commit.seqs[ENTRY_WRITE], timestamp: commit.timestamp } }, @@ -1188,7 +1102,7 @@ Only `packages/coding-agent/src/core/cache-stats.ts` (comment) and vendored `hig | `src/harness/execution/effect-gate.ts` | add `Gate`, `GateControl`, `createGate()`, `DriveAbandoned` (§2.2); keep `AbortRequested`; retire `EffectGate.assertOpen` from the procedure-facing surface | | `src/harness/hooks.ts` | `runWithGate`/`runToolWithGate` accept `Gate` and use `admit` | | `src/harness/result.ts`, `src/harness/agent-harness.ts` | keep tagged/runtime errors in the dependency-neutral result module and re-export them from `agent-harness.ts`; this lets `agent-harness.ts` directly wire `createAgentHarness` while concrete Lane imports remain order-independent | -| `src/harness/runtime2/types.ts` | add the `Drive` class (completion, gate control, stripped pass Context, installer signal, wait policy, mutable `deferredPermits`, mutable `finalizedOutcome`) and `StepResult`; move `LaneCommand` here from `lane.ts` | +| `src/harness/runtime2/types.ts` | add the `Drive` class (completion, gate control, stripped pass Context, installer signal, wait policy, mutable `deferredPermits`, mutable `finalizedOutcome`) and the initial procedure-result union, simplified in M2; move `LaneCommand` here from `lane.ts` | | `src/harness/runtime2/lane.ts` | make the class `Lane`; expose package-internal `session`, `models`, `hooks`, `emitBatch`, `command`, `readConfig`, and `mismatch` for direct procedure use; add package-internal `activeDrive: Drive | undefined` and `isDriveActive(drive)`; import `LaneCommand` from `types.ts`; remove the obsolete `AcceptanceConfig` type | | `src/harness/runtime2/harness.ts` | `Harness extends Lane`; `buildLane` returns `Lane`; pass `HookRegistry` and full `Config` into `Lane` | | `src/harness/runtime2/restore.ts` | name `Lane` wherever it names `Lane` | @@ -1223,40 +1137,43 @@ Only `packages/coding-agent/src/core/cache-stats.ts` (comment) and vendored `hig --- -### M2 — Terminal, hydration, synthetic settlement, orphan recovery +### M2 — Terminal mechanics and bounded hydration -**Goal.** The exits of the graph, testable from constructed state. +**Goal.** Land only the mechanical pieces used by terminal commands. No procedure finishes an operation in M2; each later phase-owning procedure writes its terminal transaction inline when that phase lands. **Read completely before editing** -- `src/harness/session/types.ts` (`LaneLastResult`, `TerminalOperationOutcome` in `agent-harness.ts`) -- `src/harness/session/values.ts` (all `*Prefix` constructors) -- `src/harness/runtime2/restore.ts` -- `test/harness/runtime2/{harness,watch}.test.ts` (durable-state construction patterns) +- `src/harness/session/types.ts` (`OperationState`, `LaneLastResult`; `TerminalOperationOutcome` in `agent-harness.ts`) +- `src/harness/session/values.ts` (all operation cleanup prefix constructors) +- `src/harness/runtime2/{lane,types,restore,progress}.ts` +- `test/harness/runtime2/{harness,watch,restore}.test.ts` **Create** -- `src/harness/runtime2/drive/transition.ts` — `DurableRead`, `DurableOperation`, `DurableRun`, `readDurableOperation`, `narrowRun`, `readDurableRun`, `projectOperation`, `projectRun`, `projectIdle` (§2.5) -- `src/harness/runtime2/drive/terminal.ts` — `commitTerminal`, `hydrateTerminalOutcome`, `settledFromLastResult` (§2.9) -- `src/harness/runtime2/drive/recovery.ts` — synthetic settlement from committed frames -- `test/harness/runtime2/drive-transition.test.ts` +- `src/harness/runtime2/drive/terminal.ts` — `operationCleanupWrites`, `hydrateTerminalOutcome` (§2.9) - `test/harness/runtime2/drive-terminal.test.ts` -**Modify** — none beyond exports. +**Modify** +| Path | Change | +|---|---| +| `src/harness/runtime2/types.ts` | replace global `StepResult.advance/reload` with `ProcedureResult.continue`; retain waiting, settled, and lost ownership (§2.4) | +| `src/harness/runtime2/lane.ts` | add `commandOwned`, with exact-Drive identity checked immediately before commit admission (§2.5); document `state` as the live control authority | +| `src/harness/runtime2/progress.ts` | use the planner's current `LaneState` and `commandOwned`; remove durable lane/operation control reads (§2.7) | +| `src/harness/session/session.ts` | document retained mutators as forbidden while a harness owns the Session; no runtime compatibility layer | -**Behavior/invariants landed.** The reader is total over operation kinds and separates the three outcomes: fence mismatch is `lost`, missing/self-inconsistent durable state behind a matching fence throws `SessionInvariantError`, and a non-run state behind a matching fence is corruption rather than a lost race. Terminal uses the general reader and `projectIdle`, so it serves runs, standalone compactions, and navigations; it deletes every operation-owned address via typed prefix scans and concrete `deleteValue`; `pendingNextRun` survives; hydration returns the same outcome the live caller receives; synthetic settlement writes under reserved ids and deletes the exact frame list. +**Behavior/invariants landed.** A live `Lane.state` is the sole control-flow authority. Process loss restores it from durable values. Runtime reads dereference payloads only. Exact-Drive replacement between planner return and commit admission declines the commit. Cleanup enumerates every operation-owned address through typed prefix scans and concrete deletes; `pendingNextRun` is not cleanup-owned. Hydration reads only the bounded latest-result value and entries it directly names. **Focused tests** | Path | Theme | |---|---| -| `test/harness/runtime2/drive-transition.test.ts` | absent `laneState` → throws (**not** `lost`); fence names another operation → `lost`; fence names us with missing meta / missing state / missing configuration → throws; **absent `laneLeaf` address → throws**, while a **stored `laneLeaf` value of `null` → `leafId: null` and is valid** — these two are separate cases and must be asserted separately; meta naming another lane or operation → throws; intent/state-kind mismatch → throws; `readDurableRun` on a compaction state → throws, not `lost`; `projectOperation`/`projectIdle` output equals a fresh restore for each of run, compaction, navigation | -| `test/harness/runtime2/drive-terminal.test.ts` | cleanup completeness incl. leftovers, for **each** of run, compaction, navigation; `pendingNextRun` preserved; hydration equality per kind; synthetic settlement content and zero usage | +| `test/harness/runtime2/lane.test.ts` | exact owner replaced after an async planner returns but before commit admission → no write; commit admitted first → write and projection publish complete before replacement | +| `test/harness/runtime2/progress.test.ts` | current projection permits matching frame/tool progress; terminal or phase movement first → queued write declines; no durable control reads | +| `test/harness/runtime2/drive-terminal.test.ts` | cleanup completeness for run, compaction, navigation, including leftovers and exact live frame list; `pendingNextRun` excluded; bounded hydration equality for directly constructed latest-result values per kind | +| `test/harness/runtime2/restore.test.ts` | required durable projection absence/contradiction remains attachment corruption; a stored null leaf remains valid | -**Commands.** `npm run check` · `node …/vitest --run test/harness/runtime2/drive-transition.test.ts test/harness/runtime2/drive-terminal.test.ts` +**Commands.** `npm run check` · targeted vitest for every modified test file -**Do not implement yet.** No switch (`drive.ts` does not exist until M7), no generation, no public wiring. +**Do not implement yet.** No terminal commit procedure, synthetic recovery, switch, generation, or public wiring. -**Exit condition / review questions.** Is any operation-owned address missing from cleanup, for any operation kind? Can any durable inconsistency be silently reported as `lost`? Does hydration read only bounded state? - ---- +**Exit condition / review questions.** Does any supported control write bypass the owning Lane? Is exact Drive identity adjacent to commit admission? Does terminal.ts contain anything beyond mechanical cleanup and bounded hydration? ### M3 — Generation (review checkpoint) @@ -1271,28 +1188,28 @@ Only `packages/coding-agent/src/core/cache-stats.ts` (comment) and vendored `hig **Create** - `src/harness/runtime2/drive/checkpoint.ts` — `startRun`, `runCheckpoint` - `src/harness/runtime2/drive/generation.ts` — `runGeneration`, `settleAssistant`, `enterConfigurationFailure` (§2.6) +- `src/harness/runtime2/drive/recovery.ts` — synthetic interrupted-assistant settlement from committed frames, owned beside generation classification - `test/harness/runtime2/drive-generation.test.ts` -**Do not create `drive.ts` here.** A switch written now would have to import `drive/deferred.ts` (M4), `drive/tools.ts` (M5), `drive/structural.ts` (M6), and `drive/reconcile.ts` (M7), none of which exist — the package would not compile, and stubbing those modules to make it compile would create exactly the half-built graph this plan forbids. `StepResult` and `Drive` already live in `runtime2/types.ts` (M1), so procedures compile and are tested standalone. `drive.ts` is created once, in M7. +**Do not create `drive.ts` here.** A switch written now would have to import `drive/deferred.ts` (M4), `drive/tools.ts` (M5), `drive/structural.ts` (M6), and `drive/reconcile.ts` (M7), none of which exist — the package would not compile, and stubbing those modules to make it compile would create exactly the half-built graph this plan forbids. `Drive` lives in `runtime2/types.ts` from M1 and `ProcedureResult` from M2, so procedures compile and are tested standalone. `drive.ts` is created once, in M7. **Modify** | Path | Change | |---|---| | `src/harness/execution/assistant.ts` | `request` admitted through `Gate`; observer receives Context | -| `src/harness/runtime2/drive/transition.ts` | add a generation-specific verification helper only if it enforces one invariant; do not split or narrow `readDurableRun` | -**Behavior/invariants landed.** Preparation precedes intent; ids reserved at intent and honoured at settlement; frames enqueued synchronously without per-frame awaits; settlement deletes the frame list; `advance` only after a commit; every transition rereads durable state and publishes the exact projection; checkpoint separates tree parent/leaf from `triggerEntryId` for unprojected custom writes. +**Behavior/invariants landed.** Preparation precedes intent; ids reserved at intent and honoured at settlement; frames enqueued synchronously without per-frame awaits; settlement deletes the frame list; every transition plans from the current owned projection and publishes its exact next projection; checkpoint separates tree parent/leaf from `triggerEntryId` for unprojected custom writes. **Focused tests** | Path | Theme | |---|---| -| `test/harness/runtime2/drive-generation.test.ts` | each procedure invoked directly with a concrete `Lane` and `Drive`, no switch; intent before provider; reserved-id settlement; frame order without storage awaits; orphan recovery without provider call; mixed projecting/unprojected drains (leaf vs trigger); queued public mutation observed by the next planner → procedure returns `reload` and commits nothing; fence taken by another operation → returns `lost_ownership` and commits nothing; published projection equals fresh restore | +| `test/harness/runtime2/drive-generation.test.ts` | each procedure invoked directly with a concrete `Lane` and `Drive`, no switch; intent before provider; reserved-id settlement; frame order without storage awaits; orphan recovery without provider call; mixed projecting/unprojected drains (leaf vs trigger); queued inbox mutation before intent → intent commits while preserving it; cancellation before intent → procedure returns `continue` and commits nothing; fence taken by another operation → returns `lost_ownership` and commits nothing; published projection equals fresh restore | **Commands.** `npm run check` · targeted vitest **Do not implement yet.** `drive.ts`, retry, deferred, tools, structural, reconciliation, public wiring. -**Exit condition / review questions.** Does any planner throw for an expected verification failure? Does any procedure return `advance` on a path that did not commit? Does any procedure derive state from a pre-job snapshot? Does `packages/agent/src/harness/runtime2/` contain any module importing a file that does not exist yet? +**Exit condition / review questions.** Does any planner throw for an expected verification failure? Does any procedure start an effect on a path whose intent did not commit? Does any procedure derive state from a pre-job snapshot? Does `packages/agent/src/harness/runtime2/` contain any module importing a file that does not exist yet? --- @@ -1428,8 +1345,9 @@ const request: NestedRequest = async (label, aiContext, options, context) => { const index = nextRequestIndex(label); // 0 for history, 1 for turn_prefix const usageId = lane.session.idGenerator.next(); const intent = await commitRequestIntent(lane, drive, taskId, index, usageId); - // Carries the planner's own StepResult; the attempt loop converts it straight back. - if (intent.kind !== "advance") throw new StructuralAttemptDeclined(intent); + // NestedRequest has no return channel for a declined intent, so this module-local + // signal unwinds to the structural procedure without starting the provider. + if (intent.kind !== "committed") throw new StructuralAttemptDeclined(intent); // Preparation finishes here, BEFORE the gate check, so admit() and the Models call stay // adjacent (harness.md §4.2). The admitted context composes the invocation signal with @@ -1462,44 +1380,42 @@ The default `retryingNestedRequest` keeps calling `completeSimpleWithRetries` un ```ts // packages/agent/src/harness/runtime2/drive/structural.ts — module-private, never exported. -/** - * A NestedRequest must return Promise, so a declined intent commit — the - * planner returned `reload` or `lost_ownership` and wrote nothing — has no return channel - * and can only unwind by throwing. This never escapes the module: the attempt loop converts - * it straight back into the StepResult the planner produced. - */ +/** A NestedRequest cannot return without an AssistantMessage, so a declined local intent + * unwinds to the owning structural procedure. This signal never escapes the module. */ +type StructuralIntentDecline = { kind: "state_changed" } | LostOwnership; class StructuralAttemptDeclined extends Error { - readonly step: Extract; - constructor(step: Extract) { - super(`Structural request intent declined: ${step.kind}`); + readonly result: StructuralIntentDecline; + constructor(result: StructuralIntentDecline) { + super(`Structural request intent declined: ${result.kind}`); this.name = "StructuralAttemptDeclined"; - this.step = step; + this.result = result; } } -/** Narrower than StepResult: an intent commit can only advance, reload, or lose the fence. */ -type IntentResult = Extract; +type StructuralIntentResult = { kind: "committed" } | StructuralIntentDecline; declare function commitRequestIntent( lane: Lane, drive: Drive, taskId: string, index: number, usageId: string, -): Promise; +): Promise; async function runStructuralAttempt( - lane: Lane, drive: Drive, operation: DurableOperation, decision: StructuralDecision, -): Promise { + lane: Lane, drive: Drive, operation: Operation, decision: StructuralDecision, +): Promise { try { const result = await compact(preparation, lane.models, model, customInstructions, thinkingLevel, /* retry */ undefined, /* callbacks */ undefined, drive.context, request); return await publishStructuralResult(lane, drive, operation, decision, result); } catch (error) { - // The one signal this module throws for a no-commit decline. Hand the planner's own - // StepResult back to runDrive, which re-reads and re-dispatches. - if (error instanceof StructuralAttemptDeclined) return error.step; - throw error; // AbortRequested, OperationEnded, provider errors + if (error instanceof StructuralAttemptDeclined) { + return error.result.kind === "lost_ownership" + ? error.result + : { kind: "continue" }; + } + throw error; // AbortRequested, OperationEnded, provider errors } } ``` -The throw site is the only one: `if (intent.kind !== "advance") throw new StructuralAttemptDeclined(intent);` +The throw site is the only one: `if (intent.kind !== "committed") throw new StructuralAttemptDeclined(intent);` A test must pin the option composition: assert the options reaching a faux `Models.completeSimple` from the drive path carry `cacheRetention: "none"`, a **distinct** `sessionId` per request within one split-turn attempt, the gate-composed signal, and the invocation telemetry parent — and that they match what `retryingNestedRequest` produces for the same input apart from the signal source. Assert also that no `message_start`/`message_update`/`message_end` event is emitted and no `pendingAssistantFrames` address is written during a structural attempt. @@ -1545,7 +1461,7 @@ The parameter is optional and defaulted precisely so existing callers are untouc **Create** - `src/harness/runtime2/drive/reconcile.ts` — reconciliation per phase **and** the package-private cancellation primitive `requestOperationAbort(lane, operationId, context)` -- `src/harness/runtime2/drive.ts` — the total switch: `runDrive`, `loadOperation`, `hydrateSettled`, `dispatch`, `dispatchRun` (§2.4). Created **here**, not earlier, because this is the first milestone at which every module it imports (`checkpoint`, `generation`, `deferred`, `tools`, `structural`, `reconcile`, `terminal`, `recovery`) exists +- `src/harness/runtime2/drive.ts` — the total switch: `runDrive`, `hydrateSettled`, `dispatch`, `dispatchRun` (§2.4). Created **here**, not earlier, because this is the first milestone at which every module it imports (`checkpoint`, `generation`, `deferred`, `tools`, `structural`, `reconcile`, `terminal`, `recovery`) exists - `test/harness/runtime2/drive-reconcile.test.ts` - `test/harness/runtime2/drive-lifecycle.test.ts` - `test/harness/runtime2/drive-switch.test.ts` @@ -1561,15 +1477,23 @@ The parameter is optional and defaulted precisely so existing callers are untouc ```ts // packages/agent/src/harness/runtime2/lane.ts /** Package-internal M7 lifecycle seam; not part of AgentLane. */ -notifyOwnerOfFinalization(operationId: string, outcome: TerminalOperationOutcome): void { +interruptOwnerForFinalization( + operationId: string, outcome: TerminalOperationOutcome, +): Drive | undefined { const drive = this.activeDrive; - if (drive === undefined || drive.operationId !== operationId) return; + if (drive === undefined || drive.operationId !== operationId) return undefined; const settled: DriveOutcome = { kind: "settled", operationId, outcome }; drive.finalizedOutcome = settled; // before OperationEnded can reach runDrive - drive.closeGate(new OperationEnded()); - drive.settle(settled); // public completion does not wait on stuck work - // Deliberately do not removeDrive(drive): acceptance remains LaneBusy until pass finally. + drive.closeGate(new OperationEnded()); // stop/detach work immediately after commit + // Do not remove here: ownership spans event delivery and any remaining pass work. + return drive; // finalizer settles it after event delivery +} + +settleOwnerAfterFinalization(drive: Drive, outcome: DriveOutcome): void { + // Called only after the finalizer's Lane.command delivered its terminal event. + if (drive.passEnded) this.removeDrive(drive); + drive.settle(outcome); } /** Package-internal M7 lifecycle seam; exercised through startDrive cancellation tests. */ @@ -1581,19 +1505,19 @@ abandonDrive(drive: Drive, error: DriveAbandoned): void { } ``` -Both methods run on `Lane`; procedures do not remove ownership. +Both methods run on `Lane`; procedures do not remove ownership. The external finalizer calls `interruptOwnerForFinalization` from its committed `materialize`, returns that owner in its internal command result, and calls `settleOwnerAfterFinalization(owner, { kind: "settled", operationId, outcome })` only after `Lane.command` has delivered the terminal event. If the pass ended first, this removes before settling; if event delivery/settlement happened first, pass `finally` removes later. Until both happen, same-id callers join the retained Drive rather than hydrating early. -`OperationEnded` is added to `execution/effect-gate.ts` alongside `AbortRequested` and `DriveAbandoned`. A non-cooperative effect that ignores the pulled signal is **observed and detached** under the same policy as close (§4.7): the pass does not block on it, and its later rejection is observed so it produces no unhandled rejection. It can no longer commit — every drive write is fenced by `readDurableOperation`, which now returns `lost` for it. +`OperationEnded` is added to `execution/effect-gate.ts` alongside `AbortRequested` and `DriveAbandoned`. A non-cooperative effect that ignores the pulled signal is **observed and detached** under the same policy as close (§4.7): the pass does not block on it, and its later rejection is observed so it produces no unhandled rejection. It can no longer commit — every drive-owned commit is fenced by the current projection and exact Drive identity at commit admission. -**Behavior/invariants landed.** `requestOperationAbort` order: `beginAbort` → interrupt → commit marker → `signalAbort`. `runDrive` loads the operation and its control status **before** `before_drive`, and skips `before_drive` entirely when the operation is already `cancel_requested` — that pass only reconciles. Reconciliation walks `assistant.effect_pending`, `deferred`, `tools` (mixed statuses), `compaction`, `checkpoint`, `failure_drain`. Invocation cancellation (`DriveAbandoned`) never writes durable cancellation. A `lost_ownership` step terminates the pass immediately and writes nothing more. Close writes no synthetic settlement, does not block on a non-cooperative effect, and produces no unhandled rejection. External finalization notifies the owner synchronously after the terminal commit; the old task stops without writing, resolves from `finalizedOutcome` (falling back to `laneLastResult` when it was finalized by a replacement process), and emits no duplicate terminal event. +**Behavior/invariants landed.** `requestOperationAbort` order: `beginAbort` → interrupt → commit marker → `signalAbort`. `runDrive` inspects the current owned operation and its control status **before** `before_drive`, and skips `before_drive` entirely when the operation is already `cancel_requested` — that pass only reconciles. Reconciliation walks `assistant.effect_pending`, `deferred`, `tools` (mixed statuses), `compaction`, `checkpoint`, `failure_drain`. Invocation cancellation (`DriveAbandoned`) never writes durable cancellation. A `lost_ownership` step terminates the pass immediately and writes nothing more. Close writes no synthetic settlement, does not block on a non-cooperative effect, and produces no unhandled rejection. External finalization interrupts the owner synchronously after the terminal commit, delivers the terminal event, then settles it; the old task stops without writing and resolves from `finalizedOutcome` (falling back to `laneLastResult` when it was finalized by a replacement process), and emits no duplicate terminal event. **Focused tests** | Path | Theme | |---|---| | `test/harness/runtime2/drive-reconcile.test.ts` | `requestOperationAbort` exercised directly with a concrete `Lane` and `Drive`, not through `Lane.requestAbort`; abort before/after admission at each admit site; mid-batch abort; reconciliation per phase; `aborted` response always under `cancel_requested` | -| `test/harness/runtime2/drive-switch.test.ts` | `before_drive` runs once, after the operation is loaded; `before_drive` is **not** invoked for an already `cancel_requested` operation; `reload` re-dispatches without a commit; `MAX_CONSECUTIVE_RELOADS` consecutive declines fault; `lost_ownership` ends the pass with zero further commits; external finalization → `hydrateSettled` returns the same outcome; foreign operation on the fence → `OperationMismatch` | +| `test/harness/runtime2/drive-switch.test.ts` | `before_drive` runs once, after the current owned operation is inspected; `before_drive` is **not** invoked for an already `cancel_requested` operation; `continue` requires a replaced projection and re-dispatches; unchanged `continue` faults instead of spinning; `lost_ownership` ends the pass with zero further commits; external finalization → `hydrateSettled` returns the same outcome; foreign operation on the fence → `OperationMismatch` | | `test/harness/runtime2/drive-lifecycle.test.ts` | close **before** admission returns `Result.err(Closed)`; close **during** admitted work rejects `HarnessClosed` (these two are asserted separately, never with a shared loose matcher); non-cooperative effect; fault sealing; invocation cancel abandons at `effect_pending` with no durable write | -| `test/harness/runtime2/drive-lifecycle.test.ts` (finalization) | finalize while a **provider stream** is active, and again while a **tool** is active: the owner's gate closes and its signal fires within the finalizing continuation; the admitted effect rejects with `OperationEnded`, `runDrive`'s catch resolves from `drive.finalizedOutcome` **without entering a lane job**, and the task writes nothing afterwards (asserted through `InstrumentedStorage`: zero commit attempts after the terminal one); exactly **one** `run_end` is delivered; `accept` returns `LaneBusy` until the owner's `finally` removes it, then succeeds; a non-cooperative effect that ignores the signal delays nothing and produces no unhandled rejection | +| `test/harness/runtime2/drive-lifecycle.test.ts` (finalization) | finalize while a **provider stream** is active, and again while a **tool** is active; also let an effect return successfully after committed finalization but before terminal event delivery, and attempt a same-id `drive` during that delivery: the owner's gate closes and its signal fires within committed materialization; the terminal event is delivered before shared completion settles; a cooperative effect rejects with `OperationEnded` and resolves through `drive.finalizedOutcome` **without entering a lane job**, while a successful return racing event delivery reaches `lost_ownership` but still returns `finalizedOutcome`; neither path settles/fails the Drive from the pass continuation or writes after the terminal commit (asserted through `InstrumentedStorage`: zero commit attempts after the terminal one); exactly **one** `run_end` is delivered; `accept` returns `LaneBusy` until both event-delivered settlement and the owner's `finally` have occurred, then succeeds; a non-cooperative effect that ignores the signal delays nothing and produces no unhandled rejection | | `test/harness/runtime2/drive-lifecycle.test.ts` (catch boundary) | each signal from admitted work reaches its own arm and no other: `OperationEnded` → settled outcome, no second end event; `AbortRequested` → marker awaited, reconciliation runs, `aborted` under `cancel_requested`; `DriveAbandoned` → pass rejects, state stays `effect_pending`, **zero** durable writes, next drive recovers it as an orphan; `HarnessClosed` and `HarnessFault` → propagate unchanged, no synthetic settlement; `OperationMismatch` from `hydrateSettled` → propagates to `Result.err`. Every arm ends with the owner removed exactly once (§2.1 ordering) | **Commands.** `npm run check` · targeted vitest @@ -1634,10 +1558,10 @@ Both methods run on `Lane`; procedures do not remove ownership. | Path | Theme | |---|---| | `test/harness/runtime2/drive-ownership.test.ts` | join before hydration; same-id join observes delivered events; mismatch ids. Both paths use the same result/error adapter: normal outcome → `Result.ok`; `OperationMismatch` and `Closed` → `Result.err`; `HarnessClosed`, `HarnessFault`, `DriveAbandoned` → rejection. Installer cancellation rejects installer and existing joiners with `DriveAbandoned`; joiner cancellation rejects only that joiner and leaves the Drive installed | -| `test/harness/runtime2/drive-ownership.test.ts` (independent completion/removal) | run each case with a **non-cooperative provider** and again with a **non-cooperative tool** that ignores its signal and never settles. **External finalization**: the public completion resolves immediately with the finalizer's outcome while the effect is still stuck; `accept` returns `LaneBusy` throughout; exactly **one** `run_end` is delivered; after the effect finally returns the owner clears and `accept` succeeds. **Invocation abandonment**: the public promise rejects `DriveAbandoned`, a new `drive` claims the lane **before** the old effect returns, durable state is still `effect_pending`, and the new pass runs orphan recovery. **Close/fault**: resolves without waiting; public promises reject `HarnessClosed`/`HarnessFault`. In every case the detached pass produces no unhandled rejection, and `Drive.settle`/`Drive.fail` and exact `removeDrive` remain idempotent | -| `test/harness/runtime2/drive-ownership.test.ts` (ABA fencing) | abandon an invocation whose effect is still running, let a **replacement owner claim the same `operationId`**, then let the old effect return: its transition planner and its progress writes both decline via `isDriveActive` even though `readDurableOperation` returns `ok`; zero commits attributed to the old task (asserted through `InstrumentedStorage`); the replacement owner is **not** cleared by the old task's cleanup; the replacement's own commits succeed | +| `test/harness/runtime2/drive-ownership.test.ts` (independent completion/removal) | run each case with a **non-cooperative provider** and again with a **non-cooperative tool** that ignores its signal and never settles. **External finalization**: after the terminal event is delivered, public completion resolves with the finalizer's outcome while the effect is still stuck; `accept` returns `LaneBusy` throughout; exactly **one** `run_end` is delivered; after the effect finally returns the owner clears and `accept` succeeds. **Invocation abandonment**: the public promise rejects `DriveAbandoned`, a new `drive` claims the lane **before** the old effect returns, durable state is still `effect_pending`, and the new pass runs orphan recovery. **Close/fault**: resolves without waiting; public promises reject `HarnessClosed`/`HarnessFault`. In every case the detached pass produces no unhandled rejection, and `Drive.settle`/`Drive.fail` and exact `removeDrive` remain idempotent | +| `test/harness/runtime2/drive-ownership.test.ts` (ABA fencing) | abandon an invocation whose effect is still running, let a **replacement owner claim the same `operationId`**, then let the old effect return: its transition planner and its progress writes both decline via `isDriveActive` even though the operation id is unchanged; zero commits attributed to the old task (asserted through `InstrumentedStorage`); the replacement owner is **not** cleared by the old task's cleanup; the replacement's own commits succeed | | `test/harness/runtime2/drive-surfaces.test.ts` | `accept({kind:"compaction"})` and `accept({kind:"navigation"})` admit and drive before any convenience method is used; convenience = accept + drive; `resume`; `requestAbort` through the public method reaches the M7 primitive; `Closed` returned not thrown; listeners settle before the next procedure hook; usage totals per settlement equal `getStats()`; **reopen a session with historical usage** → the first new usage event's totals include the history; **two lanes committing usage concurrently** → each event's totals match its own commit position and the final totals equal `getStats()`; Context lineage | -| `test/harness/runtime2/drive-crash-matrix.test.ts` | release N commits → discard → reopen → drive, for every N in every phase path; exactly one commit per `advance` and **zero** per `reload`; gated vs ungated byte-identical writes | +| `test/harness/runtime2/drive-crash-matrix.test.ts` | release N commits → discard → reopen → drive, for every N in every phase path; every gated boundary exposes its exact committed writes; gated vs ungated byte-identical writes | **Commands.** `npm run check` · `./test.sh` @@ -1656,19 +1580,17 @@ Milestone in parentheses where creation order matters. **Create** - `packages/agent/src/harness/runtime2/drive.ts` (**M7** — not before; it imports every procedure module) -- `packages/agent/src/harness/runtime2/drive/transition.ts` - `packages/agent/src/harness/runtime2/drive/checkpoint.ts` - `packages/agent/src/harness/runtime2/drive/generation.ts` +- `packages/agent/src/harness/runtime2/drive/recovery.ts` - `packages/agent/src/harness/runtime2/drive/deferred.ts` - `packages/agent/src/harness/runtime2/drive/tools.ts` - `packages/agent/src/harness/runtime2/drive/structural.ts` - `packages/agent/src/harness/runtime2/drive/reconcile.ts` - `packages/agent/src/harness/runtime2/drive/terminal.ts` -- `packages/agent/src/harness/runtime2/drive/recovery.ts` - `packages/agent/src/harness/runtime2/progress.ts` - `packages/agent/src/harness/session/testing/gating-storage.ts` - `packages/agent/test/harness/gating-storage.test.ts` -- `packages/agent/test/harness/runtime2/drive-transition.test.ts` - `packages/agent/test/harness/runtime2/drive-terminal.test.ts` - `packages/agent/test/harness/runtime2/drive-generation.test.ts` - `packages/agent/test/harness/runtime2/drive-retry-deferred.test.ts` @@ -1732,7 +1654,6 @@ Milestone in parentheses where creation order matters. | Path | Decide by | |---|---| | `packages/agent/src/harness/session/values.ts` | create a new address only if a required durable fact has none; default is no change | -| `packages/agent/src/harness/session/session.ts` | modify only if `idGenerator` injection is chosen over test-side id normalization (M1) | `session/types.ts` is listed under **Modify** rather than Conditional because §2.12 requires the `CommitResult.stats` field. That is an in-memory result shape, not a durable record, so it needs no migration. Any change to a *durable* shape (`RunPhase`, `Generation`, `OperationMeta`, `LaneLastResult`, …) remains forbidden without a stated migration. @@ -1758,7 +1679,7 @@ grep -rn "Breakpoint\|action_required\|ActionInfo\|ActionRequired\|peekAction\|e grep -rn "delete_prefix\|interface Deps\b" packages/agent/src grep -rn "SliceNotImplemented" packages/agent/src/harness/runtime2 # watchSession only grep -rn ": any\b\|\|as unknown as\|@ts-expect-error" packages/agent/src/harness/runtime2 -grep -rn 'kind: "advance"' packages/agent/src/harness/runtime2 # materialize() or foldSteps only +grep -rn 'kind: "advance"\|kind: "reload"' packages/agent/src/harness/runtime2 # must stay empty grep -n "runtime2" packages/agent/src/index.ts # must stay empty (§2.11) # CommitResult.stats reaches all THREE backends, not just the two in packages/agent: grep -rn "firstSeq" packages/agent/src packages/agent/test packages/session-backends