sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

docs: refine Pico5 JSONL publication design

Browse source
This commit is contained in:
Mario Zechner 2026-09-23 00:26:05 +02:00
parent a32782520f
commit 9672462143
2 changed files with 38 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

34
packages/durable/docs/pico-v5-handoff.md
View file

@ -55,12 +55,38 @@ deleted-page reuse, and representative storage sizes.
Implement table writes in `main.jsonl`, one document sidecar per incarnation,
one sidecar per live task, and one main marker per commit. Do not add a
standalone-sidecar protocol. Serialization must also provide the storage ownership
boundary: retained indexes/materializations are detached from write arguments,
and reads never expose backend-owned cached objects.
standalone-sidecar protocol.
Copy, rather than import, the current `ExecutionEnv`, `FileSystem`, `Shell`, Node
implementation, and their required utility files from `packages/agent/src/harness`
into `packages/durable/src/env`. Copy only the environment-related slice, not
agent skills, prompts, telemetry, or tool definitions. Extend the copied
filesystem contract with exact-byte file truncation and file flushing. Keep the
portable environment and JSONL entry points free of Node built-ins; expose Node
implementations only from `/env/node` and `/storage/jsonl/node`. Do not use the
Pico3 implementation as source material.
Refactor the current `MemoryStorage` state machinery into a two-phase prepared
mutation: validation and detachment produce a candidate that can later be
applied without failure. Build `MemoryStorage.commit()` on that pair, and reuse
the same machinery for JSONL. JSONL must append every prepared sidecar record,
append the main marker, and only then apply the prepared in-memory mutation.
Serialization or preparation failure occurs before file I/O and does not poison
the backend. Retained indexes/materializations remain detached from write
arguments, and reads never expose backend-owned cached objects.
JSONL creation has `fsync?: boolean`, defaulting to `false`. With `false`, append
sidecars and then the marker without an explicit flush. With `true`, append all
affected sidecars, flush each affected sidecar, and then append the main marker.
Do not explicitly flush `main.jsonl`. A main-only commit has no sidecars to
flush. Any uncertain append or flush failure poisons the open backend and
publishes no prepared in-memory mutation.
Fault-test torn/short sidecar writes, failures between sidecars, every marker
boundary, unconfirmed tails, missing confirmed data, and poisoned writes.
boundary, unconfirmed tails, missing confirmed data, poisoned writes, exact-byte
tail truncation, both fsync settings and their call ordering, detached retained
state and reads, and browser-safe portable entry points. Run the complete storage
conformance suite directly and after reopen.
## 5. JSONL reclamation

11
packages/durable/docs/pico-v5.md
View file

@ -2115,9 +2115,14 @@ Publication protocol:
3. Publish in memory only after the marker write succeeds.
Every commit uses this protocol; there is no standalone-sidecar fast path.
Without `fsync`, it guarantees ordinary process-crash consistency, not survival
of power, host, kernel, or filesystem failure. Durable mode flushes sidecars
before the marker.
JSONL creation accepts an `fsync` option that defaults to `false`. Without
`fsync`, it guarantees ordinary process-crash consistency, not survival of
power, host, kernel, or filesystem failure. With `fsync: true`, the backend
appends all affected sidecar records, flushes each affected sidecar, and only
then appends the main marker. It does not explicitly flush `main.jsonl`; an
acknowledged tail commit may therefore still disappear, but a marker that
survives should not overtake its sidecar data. A main-only commit has no
sidecars to flush.
Recovery: