Commit graph

10 commits

Author SHA1 Message Date
Mario Zechner
e5c6182bab feat(durable): group commit and batch Storage.commit
A Session no longer holds its line while a commit is stored. Commits that
arrive while one storage call is in flight go out together in the next call,
each with its own seq and outcome; a job runs until it touches a key an
unpublished commit wrote, then waits for that commit's publication. Outcomes
are delivered in line order; publication is in line order, one commit per
macrotask. The scheduler's decisions run as regions that record the keys they
read and rerun after a wait, with side effects only after a clean check.

Storage.commit(batches, receive, context) replaces commit(writes, adopt,
context): prefix semantics (Durable's own checks refuse a batch and keep the
ones before it; any other failure rejects the call), receipts per stored
batch, optional publishing() scope. Memory, SQLite, JSONL, IndexedDB and
StoreStorage implement it natively; StoreStorage through the new
ChordStore.commitSeries (one store transaction per call), replacing
commitInBatch.

A normal run makes no storage reads: bounded record cache, conversation
tails, task-document index, kept context ranges. A tool slot's running status
and a generation's attempt are stored with the next commit (12 instead of 14
commits per run).

One Harness with 128 conversations over storage with a 4 ms round trip: about
5 -> 440 agent runs per second. Design: packages/durable/docs/group-commit.md.
2026-10-30 10:41:11 +00:00
Mario Zechner
92ab536024 feat: Durable storage adopt callback inside the commit, watch types in Chord, explicit watch retirement
Breaking changes:
- Durable Storage.commit(writes, adopt, context): storages call adopt(seq) once,
  synchronously, when the batch is durable and before their next commit
- Chord traceIn(options, fn, context): context last
- WatchHandle and WatchEnd move to Chord (Durable re-exports them); end reasons
  ended (was session_closed) and failed (was session_failed); retirement is an
  explicit source event, and an ended watch drops its references into the Session
- Removed Session.documentState(), Conversation.viewState(), Harness.taskGraph()
  and DocumentState; use watchDoc(), watch(), watchTaskGraph()
2026-10-22 10:52:59 +00:00
Armin Ronacher
cae9fd28a2 Revert "feat(durable): replicate storages as a commit log"
This reverts commit 4c25119f54. The work continues on the replication branch.
2026-10-21 14:39:26 +02:00
Armin Ronacher
4c25119f54 feat(durable): replicate storages as a commit log
Every built-in storage implements ReplayableStorage: head(), replay() at an explicit sequence, and export() of the current state as a log that replays to the same reads. SQLite schema version 3 adds sequence indexes so export streams. @earendil-works/pi-durable/sync adds ReplicatingStorage, which appends each committed batch to an outbox with fork copies resolved, compactRecords(), and versioned JSON log lines.
2026-10-21 14:26:19 +02:00
Mario Zechner
f75b3240ab feat(durable): durable sleeping state replaces runtime.sleep
Tasks end a phase asleep with runtime.sleepUntil() or { status: "sleeping", until, checkpoint }:
no invocation is held, the deadline survives restarts, and the task resumes at its checkpoint once due.
Generation retry/poll and compaction retry sleep this way. Harness.wake() and HarnessOptions.onWake let
a host such as a Durable Object wake the process itself. HarnessOptions.setTimer arms every Harness
timer, and chord Progress paces on an injectable clock. SQLite migrates to schema version 2 to admit
the sleeping status.
2026-10-17 22:29:26 +00:00
Mario Zechner
c268154ca1 feat(chord): move Durable's SQLite drivers into chord/sqlite 2026-10-17 16:48:29 +00:00
Christian Klotz
cd0ba2fabc
feat(durable): execute SQLite queries by SQL text and cache statements per connection
Replace prepare/SqliteStatement with run/get/all(sql, ...params) on the database and transaction handles. The Node adapter caches prepared statements per connection, so transactions reuse them instead of preparing again on every commit.

Read documents in one transaction so a concurrent commit cannot replace the base between the record and revision queries.
2026-09-30 19:49:55 +01:00
Christian Klotz
f3e68e8eaf
feat(durable): make SQLite storage asynchronous 2026-09-30 11:10:50 +01:00
Mario Zechner
04b5bad666 feat(durable): add typed IDs and explicit ownership 2026-09-25 17:58:29 +02:00
Mario Zechner
5901c9b9e5 feat: add durable SQLite storage backend
Add the portable SQLite storage core, Node adapter, versioned migrations, conformance coverage, and storage benchmarks.
2026-09-22 17:46:42 +02:00