Commit graph

185 commits

Author SHA1 Message Date
Mario Zechner
92ab536024 feat: Durable storage adopt callback inside the commit, watch types in Chord, explicit watch retirement
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
  synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
  ended (was session_closed) and failed (was session_failed); retirement is an
  explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
  and DocumentState; use watchDoc(), watch(), watchTaskGraph()
2026-10-22 10:52:59 +00:00
Armin Ronacher
2685de7a17 fix: update deps for security advisories
Bump vitest to 4.1.12 (vite 8.3.4) and the gondolin example extension to 0.14.0, which drops node-forge. Gondolin's undici moves to 6.29.1.
2026-10-22 11:24:50 +02:00
Armin Ronacher
58c0b02ce5 Add [Unreleased] section for next cycle 2026-10-22 00:29:24 +02:00
Armin Ronacher
4a080252f0 Release v1.4.0 2026-10-22 00:29:13 +02:00
Armin Ronacher
f76b248cae docs(ai,tui,coding-agent): audit changelogs since v1.3.0
Move two post-release entries out of the 1.3.0 sections, add missing entries, duplicate user-facing ai and tui changes into coding-agent, and add New Features.
2026-10-22 00:16:31 +02:00
Armin Ronacher
cae9fd28a2 Revert "feat(durable): replicate storages as a commit log"
This reverts commit 4c25119f54. The work continues on the replication branch.
2026-10-21 14:39:26 +02:00
Armin Ronacher
4c25119f54 feat(durable): replicate storages as a commit log
Every built-in storage implements ReplayableStorage: head(), replay() at an explicit sequence, and export() of the current state as a log that replays to the same reads. SQLite schema version 3 adds sequence indexes so export streams. @earendil-works/pi-durable/sync adds ReplicatingStorage, which appends each committed batch to an outbox with fork copies resolved, compactRecords(), and versioned JSON log lines.
2026-10-21 14:26:19 +02:00
Mario Zechner
04a93bad8f fix(ai,agent,coding-agent,durable): client identity follow-ups
- ClientAttribution is { name, url, categories? } with required name and url, so
  OpenRouter always gets an HTTP-Referer. identity.ts maps the fields to each
  provider's headers; NVIDIA gets the name with its first letter upper-cased.
  attribution: null turns it off. PI_ATTRIBUTION is pi's, including the
  cli-agent category; agent-core, durable, and the coding agent use it.
- Categories are a string array, joined for OpenRouter.
- withClientIdentity() is a typed wrapper instead of a Proxy and keeps a
  MutableModels mutable. Durable no longer caches wrappers in a WeakMap.
- The Codex originator stays pi; login agentName no longer follows attribution.
- Models and the coding agent's ModelRuntime add x-opencode-session for custom
  providers pointing at OpenCode, replacing the coding agent's transformHeaders.
2026-10-21 11:18:16 +00:00
Armin Ronacher
adc5fa9f80 feat(ai,agent,coding-agent): move client identity and provider attribution into pi-ai
pi-ai gets a ClientIdentity request option and createModels() default that choose the User-Agent (detailed with version, short, or custom) and provider attribution headers for OpenRouter, NVIDIA NIM, and OpenCode. Attribution is off by default in pi-ai and on by default in pi-agent-core and pi-durable (HarnessOptions.identity). pi-coding-agent sets its identity from the telemetry setting instead of adding headers itself.
2026-10-21 11:17:02 +00:00
Mario Zechner
16f5f0af4d feat(durable): IndexedDbStorage, a browser storage backed by IndexedDB (@earendil-works/pi-durable/storage/indexeddb) 2026-10-21 03:02:55 +02:00
Petr Houška
61060ddf8c fix(coding-agent): resolve Windows shells from installation directories
Use installation environment variables for PowerShell discovery and legacy WSL launcher detection.
2026-10-19 21:55:12 +02:00
Mario Zechner
5c1736f82d feat: checkpointWhen is required on Durable document definitions
Breaking change: defineDoc and defineDocFamily now require an explicit checkpointWhen predicate; no default is supplied. Set policies on built-in documents, examples, tests, and triage state, and document the required API.
2026-10-19 18:17:42 +00:00
Mario Zechner
2ec55d808d feat: Durable and pi-ai telemetry; Context.telemetry is always a span
- pi-ai: AI_TELEMETRY_SCHEMA; pi.ai.request and pi.ai.poll spans around Models requests via a lazyStream observer
  (no provider changes), first_chunk and block events (never per delta), usage/cost end attributes, provider_error
  status with a bounded message, aborted stays ok; duration, first-chunk, token and cost metrics; option renamed
  telemetryContext -> telemetry (a parent span)
- pi-telemetry: TelemetryContext carries readonly metrics and propagation capabilities on every span
- chord/context: Context.telemetry is always a TelemetrySpan (NOOP default); withTelemetry takes spans
- durable: DURABLE_TELEMETRY_SCHEMA; pi.durable.submit (inputs start their own trace linked to the caller) and
  pi.durable.phase spans for every task phase; stored carriers on submissions and tasks (TaskOptions.trace, links);
  entry, submission, checkpoint, replay and report events after commit; resumed tasks stay in their trace; phase
  duration, active task and report metrics; failed phases carry fixed categories with bounded messages
2026-10-19 06:26:02 +00:00
Mario Zechner
48909d987a feat: telemetry propagation, span links and metrics; telemetry member in chord/context
- pi-telemetry: TelemetryPropagation (inject/extract, opaque carriers), span links (SpanOptions.links, addLink),
  TelemetryMetrics (counter, up-down counter, histogram), typed metric schemas (createTypedMetrics), in-memory
  measurement log, conformance tests for propagation and metrics
- chord/context: fixed Context.telemetry member (NOOP default), withTelemetry, traceIn
- durable: HarnessOptions.telemetry option type (no instrumentation yet)
2026-10-18 20:36:08 +00:00
Mario Zechner
91b33cab3b feat(durable): ReportInfo says what happened, with typed reasons
onReport info is tagged by type (session_failed, task_faulted, task_parked,
migration_failed, task_outdated, task_error, extension_error, host_error,
listener_error). Task reports carry a ReportTask; restart-limit and outdated
definition reasons are typed fields instead of error.cause strings; host_error
names the failing HarnessOptions callback.
2026-10-18 01:58:07 +00:00
Mario Zechner
5ba1cbaf3d fix(durable): close keeps admitted aborts, reports name the extension that ran, review nits
- Close reads the records it writes back on the line, after admitted commits, so a queued abort mark survives.
- A task's report names the extension that installed the definition it ran, captured at reservation.
- Terminal and completing records drop the restart count; retryTask() ignores them.
- wake() counts a reconcile pass only when it has work, so a run's end sends no extra active wake.
- A failed agent resolution no longer breaks the tool result path that names a tool's extension.
- Spec §10.1 list, changelog sections, test cleanup of killed storages.
2026-10-18 01:00:42 +00:00
Mario Zechner
440a5fcdb8 fix(durable): review fixes for the crash-loop guard, reports, and blocked tasks
- A close writes the tasks it stopped back as pending, so only a crash counts a restart.
- An abort mark starts the count over; an abort handler cut off restartLimit times ends the task faulted.
- Only a changed state resets the count; raised and lowered limits reach the scheduler and the graph.
- wake() stays active while a reconcile pass is scheduled or underway; failed migrations recompute it.
- Reports name the extension whose code failed (hooks, wrappers, sections, tools, a task's installer),
  runtime.report() takes a source, and the parked report carries blocked: restart_limit.
- The task graph keeps only what blocked reasons read, and shows abandoned tasks waiting for a definition.
2026-10-18 00:53:14 +00:00
Mario Zechner
a30c2a81df feat(durable): crash-loop guard, report sources, inbox after failed runs, blocked tasks in the task graph
- Open counts restarts per task; at settings.restartLimit (default 5) a task is parked as blocked
  (restart_limit) until Harness.retryTask().
- onReport(error, info) with ReportInfo: session, task (faulted), extension, listener, host, timer, wake.
  Task faults are reported.
- settings.afterFailedRun: keep, continue, or withdraw the inputs queued behind a failed run.
- TaskGraphNode.blocked follows registry changes and failed migrations.
- README: fallback alarm before Harness.open in the Durable Object example.
2026-10-18 00:22:10 +00:00
Mario Zechner
e17e2e1055 fix(durable): release the second slow call when the first finished, not after 200 polls
The code mode ordering test released call 1 from an eventually() poll with a turn limit; on a loaded CI
runner the sandbox worker took more turns and the test hung until its timeout.
2026-10-17 23:54:43 +00:00
Mario Zechner
00bf7193d6 fix(durable): arm before resume in the Durable Object example; timer wording
The README example and the test worker arm the recovery alarm before resume() in the constructor and in
alarm(). Timers should not fire early, and callers recheck; nativeTimer no longer needs a non-null assertion.
2026-10-17 23:41:24 +00:00
Mario Zechner
ec1be23eb9 fix(durable): review fixes for sleeping tasks and host timers
Due sleeps are picked up at every scheduling pass, so a pass that drops the timer cannot strand them.
Code mode drops a global's result after the script finished instead of reading an ended invocation's clock.
A foreground timer the host declines falls back to setTimeout; the default timer never fires early; Progress
rechecks its clock; an undelivered onWake is offered again. Spec, README, and tests follow.
2026-10-17 23:38:18 +00:00
Mario Zechner
26b5535a51 docs(durable): clarify the SQLite schema migration in the changelog 2026-10-17 23:25:11 +00:00
Mario Zechner
de47e488f2 docs(durable): changelog for sleeping tasks, timers, and wakeups 2026-10-17 23:08:06 +00:00
Mario Zechner
ea7c648430 feat(durable): workerd tests for Durable Object hosting
Runs the storage conformance suite, the schema migration, an alarm-woken sleep across an eviction, and a
turn recovered through the recovery alarm in workerd via @cloudflare/vitest-plugin (npm run test:workers).

fix(durable): decline background timers in Cloudflare Workers. With nodejs_compat, timers have an unref(),
so the idle-memory timers were armed and a pending one kept a Durable Object from hibernating or being evicted.
2026-10-17 23:02:48 +00:00
Mario Zechner
f75b3240ab feat(durable): durable sleeping state replaces runtime.sleep
Tasks end a phase asleep with runtime.sleepUntil() or { status: "sleeping", until, checkpoint }:
no invocation is held, the deadline survives restarts, and the task resumes at its checkpoint once due.
Generation retry/poll and compaction retry sleep this way. Harness.wake() and HarnessOptions.onWake let
a host such as a Durable Object wake the process itself. HarnessOptions.setTimer arms every Harness
timer, and chord Progress paces on an injectable clock. SQLite migrates to schema version 2 to admit
the sleeping status.
2026-10-17 22:29:26 +00:00
Mario Zechner
c268154ca1 feat(chord): move Durable's SQLite drivers into chord/sqlite 2026-10-17 16:48:29 +00:00
Mario Zechner
2331785c33 fix(durable): run the Codex prompt-cache e2e test on gpt-6.1-sol
Codex no longer serves gpt-5.5 to ChatGPT accounts ("The 'gpt-5.5' model is not
supported when using Codex with a ChatGPT account"), so the test failed in its
first turn as unanswered.
2026-10-16 19:42:02 +00:00
Mario Zechner
7ec1f648e9 feat(durable): accept functions for HarnessOptions.models and settings
Like env, models and settings may be a function returning the current value.
Generation, compaction, hooks and tools read it at each use (api.models at each
access), so a host can replace its model configuration without reopening the
Harness. A plain object still works.
2026-10-16 19:30:35 +00:00
Mario Zechner
790271cd02 fix(durable): restore ToolControl.terminate changes dropped by c4cf7957b
c4cf7957b was built from a tree that predates 6085986e4 and reverted its
terminate type change, regression test, and changelog entries.
2026-10-16 08:20:58 +00:00
Mario Zechner
c4cf7957b4 feat(durable): per-model compaction settings via compaction.modelOverrides
Overrides keyed by provider/modelId apply at the threshold check, overflow
recovery, and compaction range selection.

closes #10942
2026-10-16 08:20:12 +00:00
Mario Zechner
6085986e4b fix(durable): type ToolControl.terminate as boolean
The type change and changelog entry landed in 1de1b98e7; this adds the
regression test for terminate: false.

closes #10858
2026-10-16 08:19:56 +00:00
Armin Ronacher
ae7b16d43b Add [Unreleased] section for next cycle 2026-10-15 20:57:35 +02:00
Armin Ronacher
71023aec15 Release v1.3.0 2026-10-15 20:57:27 +02:00
Christian Klotz
f7e3cb5433
feat(durable): support namespaced SQLite storage (#10937)
* feat(durable): namespace SQLite storage tables

Render migrations and runtime queries from one schema map, add optional prefixes, and simplify the Cloudflare Durable Object adapter.

* feat(durable): expose SQLite prefixes through storage helpers

* docs: explain shared Durable Object storage

* Update changelog
2026-10-15 14:02:27 +01:00
Mario Zechner
4a5d372915 feat(chord,durable): move OutputBuffer and Progress to @earendil-works/chord/output
OutputBuffer, Progress, boundOutput, sanitizeOutput, characterEnd and the
output types move from Durable's internal harness/output.ts to the new
Chord subpath, with OutputSkip (the shape of Durable's ShellOutputSkip) and
utf8ByteLength. Durable imports them from Chord; its truncate.ts re-exports
Chord's utf8ByteLength instead of keeping a second copy.
2026-10-15 09:59:46 +00:00
Mario Zechner
5713c9507c fix(durable): lead with the system baseline after a compaction that keeps assistant messages
Context derivation moved a system message to the front only when user messages
alone preceded it. After a compaction that keeps an assistant message, the
baseline written for the next run followed the kept history and the new input,
so Anthropic and OpenAI Codex requests had no initial prompt or tool set.

Derivation now identifies the harness baseline: the first pi.system entry after
the head marker, or of the transcript when no earlier entry carries a system
message, even one omitted later. Its message leads even past assistant
messages. Only the first system message ever moves, so replay is unchanged;
other system messages after an assistant message stay in place.

closes #10904
2026-10-14 09:47:25 +00:00
Mario Zechner
b042997178 fix(durable): encode base64 with btoa, without retaining concatenation trees
toBase64 grew each 8 KiB chunk a character at a time; V8 kept every
concatenation node until the final join, about 138 MB of heap for a
3 MiB image, beyond a Durable Object's 128 MB limit.

closes #10923
2026-10-14 09:41:45 +00:00
Armin Ronacher
f13c00fb3a fix: pin internal dependencies to exact versions
Update release synchronization and dependency checks to enforce exact versions. Refresh both lockfiles and add regression coverage.

closes #10865
2026-10-13 14:28:15 +02:00
Armin Ronacher
d547949184 Add [Unreleased] section for next cycle 2026-10-13 00:47:08 +02:00
Armin Ronacher
06a4669355 Release v1.2.0 2026-10-13 00:46:55 +02:00
Mario Zechner
55df12bf62 docs(durable,env): changelog entries for listFiles 2026-10-12 15:22:48 +00:00
Mario Zechner
abfe62928b feat(durable): optional ExecutionEnv.listFiles for tree listings
An environment may list the files, directories and symbolic links below a
directory: relative / paths, depth first in name order, without .git and
what git ignores in work trees, bounded with a truncated flag. Callers fall
back to their own walk when it is missing or fails with not_supported.
The env conformance suite checks it with the new listFiles option.
2026-10-12 15:22:14 +00:00
Mario Zechner
6db4070f17 docs: describe only the current Chord, Durable, and coding-agent code
Chord: README rewritten for the new runtime; notes.md and requests.md removed, their open limits and
decisions folded into spec.md section 12; guide and spec status lines updated. Durable: drops the
pico-v5 handoff and delta findings, renames the Pico spec, and uses current Chord terms. Delta README,
issue categories, and PI_RADIUS_GATEWAY updated.
2026-10-12 10:47:09 +00:00
Mario Zechner
0104fef1e9 feat(chord): replace the facet and service runtime with the chord2 host runtime
Moves chord2 (contracts, objects, cells, roots, plugins, hot reload, memory/IPC/WebSocket links)
from p2 into packages/chord, with its tests (now on vitest) and docs. The root export is the new
runtime plus the shared base (Context, JsonValue, JsonRepresentation, Draft, copyJson, isJsonValue,
replicatedState and its types). ./node now holds the host manager, child control, process locks,
and nodeLoader. ./context and ./delta are unchanged; ./bundler is removed.

replicatedState stays a standalone module with the same exports. It no longer exposes the exact-op
subscription the removed service provider used, so listener failures of mutable states are always
reported asynchronously. The package depends on typebox instead of esbuild. The wire format stays JSON.
2026-10-12 10:41:59 +00:00
Mario Zechner
2ee5c1e56d docs(durable): drop the Chord facet usage guide and its test
The guide and chord-guide.test.ts used Chord's facet and service runtime, which is being replaced.
2026-10-12 10:41:59 +00:00
Mario Zechner
eb2fce43d2 fix(durable): reject waits that close a wait cycle
A wait is rejected when it closes a cycle in the wait-for graph: waits of
tasks that are not abort-marked, and holds of live ordinary owned work. The
rejection names the cycle. Replaces the owner-chain rule, so a background task
may wait on its conversation's owner. Approach and tests from #10533 by
@odysseus0.

fixes #10411
2026-10-12 10:15:26 +00:00
Mario Zechner
77332f7a34 feat(durable): deferred tools and the tool_search extension
- ToolRegistration.deferred: not offered to the model until modelTools names it
- modelTools gains { remove?, add? }; control.addTools writes names to add, so
  loaded tools follow the others in load order and a load only appends
- Agent.deferred lists deferred tools not loaded yet
- @earendil-works/pi-durable/tool-search: the coding agent's tool_search (BM25)
- the codemode section leaves deferred tools out
2026-10-12 08:05:26 +00:00
Armin Ronacher
f11e50f72e fix(coding-agent): make codemode output joining linear
joinAdjacentText checked endsWith on the growing joined string, which
makes V8 flatten the concatenation on every item. Collect each text run
in an array, check the separator against the last non-empty part, and
join once. Same fix in the durable code mode copy.

closes #10678
2026-10-12 10:03:56 +02:00
Armin Ronacher
c661bc574f fix(durable): drop shared MCP types from the codemode section
pi-codemode no longer exports MCP_TYPESCRIPT_PREAMBLE or mcpStructuredContentSchema(); output schemas render as their own types, as in the coding agent.
2026-10-12 09:21:04 +02:00
Mario Zechner
8bb180bec1 feat(durable): code mode listing, tool search, output, models.*, grammar; tool error rule
- structuredOutput is always the tool's own value; a schema tool's error text
  without one becomes a tool_error diagnostic on nested results
- code mode: a schema tool's error result with its value resolves (bash nonzero
  exit), schema-less tools resolve with their text, as in the coding agent
- codemode prompt section with tool declarations by namespace within an inline
  budget; searchTools/describeTool/describeNamespace from the start data
- result layout of the coding agent: header, text markers, console block,
  truncation within max_output_tokens and the tool's output limits, spill and
  image files in a recorded output directory (a rerun overwrites them)
- models.*: catalog globals; classify/generateImages as hidden internal tools
  (nested calls with usage), at most four at once
- Lark grammar of the script source as constrainedSampling
2026-10-12 00:06:09 +00:00