Commit graph

12 commits

Author SHA1 Message Date
Mario Zechner
e5c6182bab feat(durable): group commit and batch Storage.commit
A Session no longer holds its line while a commit is stored. Commits that
arrive while one storage call is in flight go out together in the next call,
each with its own seq and outcome; a job runs until it touches a key an
unpublished commit wrote, then waits for that commit's publication. Outcomes
are delivered in line order; publication is in line order, one commit per
macrotask. The scheduler's decisions run as regions that record the keys they
read and rerun after a wait, with side effects only after a clean check.

Storage.commit(batches, receive, context) replaces commit(writes, adopt,
context): prefix semantics (Durable's own checks refuse a batch and keep the
ones before it; any other failure rejects the call), receipts per stored
batch, optional publishing() scope. Memory, SQLite, JSONL, IndexedDB and
StoreStorage implement it natively; StoreStorage through the new
ChordStore.commitSeries (one store transaction per call), replacing
commitInBatch.

A normal run makes no storage reads: bounded record cache, conversation
tails, task-document index, kept context ranges. A tool slot's running status
and a generation's attempt are stored with the next commit (12 instead of 14
commits per run).

One Harness with 128 conversations over storage with a 4 ms round trip: about
5 -> 440 agent runs per second. Design: packages/durable/docs/group-commit.md.
2026-10-30 10:41:11 +00:00
Mario Zechner
8025c414d1 feat(chord): SqliteExecutor.runCounted resolves with the rows a statement changed
The Node driver takes the count from node:sqlite; the Durable Object driver reads
SELECT changes() after the statement, since the cursor's rowsWritten also counts
index writes (verified in workerd). run is unchanged and costs nothing more.
2026-10-27 08:54:53 +00:00
Mario Zechner
92ab536024 feat: Durable storage adopt callback inside the commit, watch types in Chord, explicit watch retirement
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
  synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
  ended (was session_closed) and failed (was session_failed); retirement is an
  explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
  and DocumentState; use watchDoc(), watch(), watchTaskGraph()
2026-10-22 10:52:59 +00:00
Mario Zechner
c268154ca1 feat(chord): move Durable's SQLite drivers into chord/sqlite 2026-10-17 16:48:29 +00:00
Mario Zechner
eba8497395 feat(durable): nested tool calls, structured output, callers; storage errors end the Session
Nested tool calls: ToolExecutionApi.executeTool() runs another tool as its own pi.tool task owned by the caller,
keyed for replay-safe reattachment. Results for programs live in the caller's NestedResultDoc until it settles;
pi.live.nestedTools shows each call with the arguments it runs with and a summary. TaskOptions.abandonOnRestart
aborts work its creator awaited only in memory before it runs again after a restart.

Structured output: structuredOutputSchema and structuredOutput, validated; a tool without a schema gives programs its
bounded output (one text as a string, one image as itself, else the list). ToolExecutionResult.content is now output.
callers and modelTools decide who may call a tool.

Storage errors: any error a Storage method throws fails the Session, which closes itself; later calls and waits reject
with SessionFailed, watches end with session_failed, and Session.closed settles. Nothing retries in the Harness.
StorageRejected is removed; StorageRequestError marks invalid reads that fail only their call. Listener and onReport
errors are contained and reported.

Also updates scripts/issues/triage.ts and the vacation experiment for the content -> output rename.
2026-10-09 17:13:34 +00:00
Mario Zechner
ed391c4f0a fix(durable): harden async SQLite adapter queue and close
- drop AsyncLocalStorage misuse guards from the Node adapter
- run calls immediately when the queue is idle; transactions publish their barrier before the callback starts
- SqliteStorage.close() waits for admitted multi-query reads and shares one close promise
2026-09-30 22:32:34 +02:00
Christian Klotz
cd0ba2fabc
feat(durable): execute SQLite queries by SQL text and cache statements per connection
Replace prepare/SqliteStatement with run/get/all(sql, ...params) on the database and transaction handles. The Node adapter caches prepared statements per connection, so transactions reuse them instead of preparing again on every commit.

Read documents in one transaction so a concurrent commit cannot replace the base between the record and revision queries.
2026-09-30 19:49:55 +01:00
Christian Klotz
fd3af5ee2b
feat(durable): pass a transaction handle to SQLite transaction callbacks 2026-09-30 11:39:02 +01:00
Christian Klotz
f3e68e8eaf
feat(durable): make SQLite storage asynchronous 2026-09-30 11:10:50 +01:00
Mario Zechner
04b5bad666 feat(durable): add typed IDs and explicit ownership 2026-09-25 17:58:29 +02:00
Mario Zechner
507d7649e9 feat(durable): add conversation document forks 2026-09-25 16:17:53 +02:00
Mario Zechner
5901c9b9e5 feat: add durable SQLite storage backend
Add the portable SQLite storage core, Node adapter, versioned migrations, conformance coverage, and storage benchmarks.
2026-09-22 17:46:42 +02:00