mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
docs(agent): finalize Pico5 architecture and presentation
This commit is contained in:
Mario Zechner
parent
ea7f84678c
commit
c8e4a5a552
4 changed files with 2490 additions and 1343 deletions
364
packages/agent/docs/pico-v5-chord-usage.md
Normal file
364
packages/agent/docs/pico-v5-chord-usage.md
Normal file
|
|
@ -0,0 +1,364 @@
|
|||
# Pico5 documents through Chord
|
||||
|
||||
This guide uses the contracts in the [Pico5 specification](pico-v5.md).
|
||||
|
||||
- **Session:** a durable container for conversations, entries, tasks, and documents;
|
||||
it serializes mutations on one commit line.
|
||||
- **Facet:** a feature's setup unit in a Chord host. Setup declares services and
|
||||
dependencies synchronously; `onActivate` runs after dependencies are ready.
|
||||
- **Context:** explicitly passed cancellation and invocation values. Forward the caller's context.
|
||||
- **Service:** a typed token and implementation acquired with `env.use`. Remote
|
||||
services contain replicated state and async methods with JSON arguments plus a trailing `Context`.
|
||||
- **Durable document:** JSON persisted as a complete base, Chord operations, and
|
||||
checkpoints. Its definition token declares schema, version, scope, and initialization;
|
||||
conversation scope also declares history and fork behavior.
|
||||
- **Transaction draft:** the mutable tracked object from `await tx.doc`, valid
|
||||
only inside that commit callback, including all nested objects.
|
||||
- **DocumentSource:** an opaque handle to one document incarnation's committed changes,
|
||||
not a mutable value or a public subscription API.
|
||||
- **ReplicatedState:** Chord's immutable complete values through `value` and
|
||||
`subscribe(listener)`. `value` is `undefined` before hydration or while disconnected.
|
||||
The listener receives `(value, context, delivery)`; delivery contains `kind`
|
||||
(`hydrate` or `update`) and a stream `sequence`, not a Session commit number.
|
||||
|
||||
Documents are scoped directly to a Session, conversation, or task. A task is
|
||||
durable work attached to a conversation; its documents retire when it becomes
|
||||
terminal and never participate in conversation forks.
|
||||
|
||||
## Imports and the adapter boundary
|
||||
|
||||
Examples build on one another. Pico5 names (`defineDoc`, `defineDocFamily`,
|
||||
`Session`, `DocumentSource`, `Id`, `TaskRuntime`, `DocumentObserver`)
|
||||
refer to normative contracts, without a specified import path or runnable Pico5
|
||||
package. These Chord imports exist today:
|
||||
|
||||
```ts
|
||||
import {
|
||||
createFacetHost, createRemoteServiceBinding, defineFacet, defineService,
|
||||
type Context, type Facet, type JsonValue, type RemoteServiceTransport,
|
||||
type ReplicatedState,
|
||||
} from "@earendil-works/chord";
|
||||
import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context";
|
||||
|
||||
// The spec calls this JsonObject; current Chord exports JsonValue, not JsonObject.
|
||||
type JsonObject = { [key: string]: JsonValue };
|
||||
|
||||
// Required by Pico5, but not a current Chord export yet.
|
||||
declare function documentReplicatedState<T extends JsonObject>(
|
||||
source: DocumentSource<T>, context: Context,
|
||||
): Promise<{ readonly state: ReplicatedState<T | null>; dispose(): void }>;
|
||||
```
|
||||
|
||||
The adapter must register Chord-recognized state, atomically hydrate matching
|
||||
committed value/revision and subscribe, and forward committed ops without another
|
||||
tracker or re-diff. Disposal releases observation, not the document. Even Chord's
|
||||
hydration requests must never flush an uncommitted draft.
|
||||
|
||||
Current `env.replicatedState(initial)` creates a **mutable, in-memory** tracker
|
||||
with `.state` and `.publish(context)`, not a durable source adapter. There is no
|
||||
`replicatedState.own(source)`. Do not mirror documents into another tracker.
|
||||
|
||||
## 1. A Session-wide canvas
|
||||
|
||||
All conversations in this Session share one canvas. Session documents are
|
||||
current-only and have no history or fork settings.
|
||||
|
||||
```ts
|
||||
type Stroke = { color: string; points: { x: number; y: number }[] };
|
||||
type CanvasState = { strokes: Stroke[] };
|
||||
|
||||
const CanvasDoc = defineDoc<CanvasState>({
|
||||
id: "app.canvas", version: 1, scope: "session",
|
||||
initial: () => ({ strokes: [] }),
|
||||
// This append-only example adds one stroke per commit.
|
||||
checkpointWhen: value => value.strokes.length % 100 === 0,
|
||||
});
|
||||
|
||||
interface CanvasService {
|
||||
readonly state: ReplicatedState<CanvasState | null>;
|
||||
addStroke(stroke: Stroke, context: Context): Promise<void>;
|
||||
}
|
||||
const Canvas = defineService<CanvasService>("app.canvas");
|
||||
|
||||
async function createCanvasFacet(session: Session, context: Context): Promise<Facet> {
|
||||
const target = { scope: "session" } as const;
|
||||
// Get-or-create: an absent canvas is initialized in a serialized commit.
|
||||
const source = await session.documentSource(CanvasDoc, target, context);
|
||||
const replica = await documentReplicatedState(source, context);
|
||||
return defineFacet({
|
||||
id: "app.canvas/session",
|
||||
setup(env) {
|
||||
env.own(() => replica.dispose());
|
||||
env.provide(Canvas, {
|
||||
state: replica.state,
|
||||
async addStroke(stroke, context) {
|
||||
await session.commit(async tx => {
|
||||
const draft = await tx.doc(CanvasDoc, target);
|
||||
draft.strokes.push(stroke); // The draft now owns stroke; caller must not mutate it.
|
||||
}, context);
|
||||
},
|
||||
});
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const CanvasConsumer = defineFacet({
|
||||
id: "app.canvas/consumer",
|
||||
setup(env) {
|
||||
const canvas = env.use(Canvas); // Declare now; access only after activation.
|
||||
env.onActivate(() => {
|
||||
env.own(canvas.state.subscribe((value, _context, delivery) => {
|
||||
console.log(delivery.kind, delivery.sequence, value?.strokes.length ?? "retired");
|
||||
})); // subscribe delivers the current hydrated value, then updates.
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
async function runCanvasExample(session: Session): Promise<void> {
|
||||
const context = BACKGROUND_CONTEXT;
|
||||
const provider = await createCanvasFacet(session, context);
|
||||
const host = await createFacetHost({ facets: [provider, CanvasConsumer] });
|
||||
try {
|
||||
await host.services.use(Canvas).addStroke({
|
||||
color: "black", points: [{ x: 10, y: 20 }, { x: 30, y: 40 }],
|
||||
}, context);
|
||||
} finally {
|
||||
await host.dispose(); // Unsubscribes; does not delete the canvas or close Session.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The application owns the open Session. Acquire state **before** synchronous
|
||||
`setup`; `env.provide` cannot run in `onActivate`. Install one provider per
|
||||
Session host. Chord may reload presentation facets independently, but v1 does
|
||||
not use facet reload to replace Session-side task or hook implementations. A
|
||||
host plugin code change closes and reopens the Harness with the new definition
|
||||
set.
|
||||
|
||||
Remote clients supply a `RemoteServiceTransport` connected to the host's `services`
|
||||
provider; Chord prescribes no socket protocol. `CanvasConsumer` also works unchanged
|
||||
in a UI host configured with a remote service source.
|
||||
|
||||
```ts
|
||||
async function connectCanvas(transport: RemoteServiceTransport, context: Context) {
|
||||
const services = createRemoteServiceBinding({ services: [Canvas], transport });
|
||||
const canvas = services.use(Canvas);
|
||||
const stop = canvas.state.subscribe((value, _context, delivery) => {
|
||||
console.log(delivery.kind, delivery.sequence, value?.strokes ?? "retired");
|
||||
});
|
||||
try {
|
||||
await services.ready(context); // Initial snapshot installed; not all future updates.
|
||||
} catch (error) {
|
||||
stop();
|
||||
await services.dispose(BACKGROUND_CONTEXT);
|
||||
throw error;
|
||||
}
|
||||
return async () => {
|
||||
stop();
|
||||
await services.dispose(BACKGROUND_CONTEXT);
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
```text
|
||||
first acquisition -> commit base { strokes: [] }
|
||||
addStroke(A) -> commit A -> local and remote subscribers see A
|
||||
worker restarts -> open same durable storage; install canvas facet again
|
||||
acquire source -> load base + committed deltas, without rerunning initial()
|
||||
late client -> hydrate complete canvas including A, then ordered updates
|
||||
conversation fork -> still uses this same Session canvas
|
||||
```
|
||||
|
||||
Use persistent storage for restart survival. The memory backend is not persistent.
|
||||
A method's successful commit does not promise every remote callback has run yet.
|
||||
|
||||
## 2. Conversation-scoped diff reviews
|
||||
|
||||
A **document family** uses one definition for many instances. The logical key is
|
||||
`(definition, conversationId, instanceId)`; the persisted numeric document ID
|
||||
identifies an incarnation and is never reused. Initializer input is not identity.
|
||||
|
||||
```ts
|
||||
type ReviewInput = { path: string; patch: string };
|
||||
type ReviewComment = { id: string; line: number; text: string };
|
||||
type ReviewState = ReviewInput & { comments: ReviewComment[] };
|
||||
const ReviewDoc = defineDocFamily<ReviewState, ReviewInput>({
|
||||
id: "app.diff-review", version: 1, family: true, scope: "conversation",
|
||||
history: "latest", fork: "current",
|
||||
initial: input => ({ path: input.path, patch: input.patch, comments: [] }),
|
||||
checkpointWhen: value => value.comments.length % 50 === 0,
|
||||
});
|
||||
interface DiffReviewService {
|
||||
readonly state: ReplicatedState<ReviewState | null>;
|
||||
identity(context: Context): Promise<{ conversationId: Id; instanceId: string }>;
|
||||
addComment(comment: ReviewComment, context: Context): Promise<void>;
|
||||
}
|
||||
const DiffReviews = defineService<DiffReviewService>("app.diff-reviews");
|
||||
|
||||
function reviewFacet(
|
||||
session: Session, conversationId: Id,
|
||||
reviews: readonly { instanceId: string; initial: ReviewInput }[], context: Context,
|
||||
): Facet {
|
||||
return defineFacet({
|
||||
id: "app.diff-reviews/session",
|
||||
setup(env) {
|
||||
const instances = env.provideMany(DiffReviews);
|
||||
env.onActivate(async () => {
|
||||
for (const review of reviews) {
|
||||
const target = { scope: "conversation" as const, conversationId, ...review };
|
||||
const source = await session.documentSource(ReviewDoc, target, context);
|
||||
const replica = await documentReplicatedState(source, context);
|
||||
env.own(() => replica.dispose());
|
||||
// Chord instance keys route services; they are not persisted document IDs.
|
||||
instances.spawn(JSON.stringify([conversationId, review.instanceId]), {
|
||||
state: replica.state,
|
||||
async identity() { return { conversationId, instanceId: review.instanceId }; },
|
||||
async addComment(comment, context) {
|
||||
await session.commit(async tx => {
|
||||
const draft = await tx.doc(ReviewDoc, target);
|
||||
draft.comments.push(comment); // Do not mutate comment after this assignment.
|
||||
}, context);
|
||||
},
|
||||
}); // The facet owns spawned service lifetimes automatically.
|
||||
}
|
||||
});
|
||||
},
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Example input: `[{ instanceId: "review-7", initial: { path: "a.ts", patch: "-old\n+new" } }]`.
|
||||
Pass distinct instance IDs and a real conversation ID; install the facet as above.
|
||||
Keyed consumers use `env.observe(DiffReviews, handler)`, not `env.use`. The handler
|
||||
receives `(review, context)`; call `review.identity(context)` to identify it and
|
||||
`review.state.subscribe` to observe comments.
|
||||
|
||||
These comments are conversation-scoped: ending the generating task or unloading
|
||||
the facet does not retire them. `current` copies every logically
|
||||
present review at fork-commit time into independent child instances with new
|
||||
numeric IDs, retaining family instance IDs. Parent and child then diverge.
|
||||
|
||||
```text
|
||||
parent review-7: comment A -> transcript entry E -> comment B
|
||||
fork at E with current: child review-7 contains A and B
|
||||
child adds C: parent still contains only A and B
|
||||
```
|
||||
|
||||
With `fork: "initial"`, no instance copies; first child access uses supplied input.
|
||||
`latest` cannot read history or fork `asOf`; use `history: "rewindable"` with `asOf`
|
||||
to reflect E's commit. Reaccess ignores `initial`; updating the patch requires
|
||||
an explicit mutation or a distinct review instance ID.
|
||||
|
||||
## 3. Task-scoped output and a tool/task watch
|
||||
|
||||
This is a bounded progress window, not the permanent result. Scope itself gives
|
||||
it task lifetime; no history, fork, owner, or conversation setting is needed.
|
||||
|
||||
```ts
|
||||
type JobInput = { command: string };
|
||||
type JobOutput = { stdout: string; chunks: number };
|
||||
const JobOutputDoc = defineDoc<JobOutput>({
|
||||
id: "app.job-output", version: 1, scope: "task",
|
||||
initial: () => ({ stdout: "", chunks: 0 }),
|
||||
checkpointWhen: value => value.chunks % 100 === 0,
|
||||
});
|
||||
function jobOutputTarget(taskId: Id) {
|
||||
return { scope: "task" as const, taskId };
|
||||
}
|
||||
async function appendJobOutput(
|
||||
runtime: TaskRuntime<JobInput, { phase: "running" }, null, {}>,
|
||||
chunk: string, context: Context,
|
||||
): Promise<void> {
|
||||
// Read process output outside this callback. The runtime gates the live task.
|
||||
await runtime.commit(async tx => {
|
||||
const draft = await tx.doc(JobOutputDoc, jobOutputTarget(runtime.taskId));
|
||||
draft.stdout = (draft.stdout + chunk).slice(-50_000);
|
||||
draft.chunks += 1;
|
||||
}, context);
|
||||
}
|
||||
async function observeJob(
|
||||
api: DocumentObserver, producerTaskId: Id,
|
||||
finished: Promise<void>, context: Context,
|
||||
): Promise<void> {
|
||||
const target = jobOutputTarget(producerTaskId);
|
||||
const watch = await api.watchDoc(JobOutputDoc, target, context);
|
||||
try {
|
||||
console.log(watch.revision, watch.value.stdout); // Consume captured initial value.
|
||||
watch.start((value, ops, revision, _context) => {
|
||||
console.log(revision, value === null ? "retired" : value.stdout, ops);
|
||||
}); // Drain buffered changes in order, then continue live delivery.
|
||||
await finished; // Caller-supplied observation lifetime; outside any commit.
|
||||
} finally {
|
||||
watch.stop(); // Idempotent; discards any remaining buffer.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The running phase calls `appendJobOutput` and must also commit its next checkpoint
|
||||
or terminal outcome. `api` is the invocation's `DocumentObserver`; `TaskRuntime`
|
||||
includes that interface. Acquisition validates the target task is live and
|
||||
derives its conversation from the task record.
|
||||
|
||||
```text
|
||||
watchDoc: capture value V at revision 8 + register listener atomically on Session line
|
||||
producer commits revision 9 before start: buffer 9
|
||||
consumer reads watch.value (V), calls start: deliver 9, then subsequent revisions
|
||||
producer becomes terminal: same commit retires output; watch receives null + ["r", null]
|
||||
```
|
||||
|
||||
Watches automatically stop when their invocation ends. A Session-acquired watch
|
||||
is caller-owned and stops on Session close. Retirement ends the incarnation's
|
||||
stream; recreation requires a new watch/source and a new numeric
|
||||
document ID. A terminal task cannot create more output. Task-scoped documents
|
||||
never copy into forks. Preserve required output in result entries or Session- or
|
||||
conversation-scoped documents in the terminal commit before retirement. Use a
|
||||
task-scoped family only when one task needs several independently keyed documents.
|
||||
|
||||
## 4. Built-in conversation views and commit delivery
|
||||
|
||||
`ConversationView` contains `conversation`, raw transcript `entries`, and
|
||||
`docs: Readonly<Record<string, JsonObject>>`. Selected built-in singletons mount
|
||||
under `docs` by stable definition ID, not numeric incarnation ID. The spec's
|
||||
illustrative path mapping is:
|
||||
|
||||
```text
|
||||
document ["s", ["message"], value]
|
||||
-> view ["s", ["docs", "pi.live", "message"], value]
|
||||
```
|
||||
|
||||
Built-in IDs/fields await approval; `pi.live` is illustrative, not an available API.
|
||||
The mount publishes one batch per complete Session commit: entry/head changes and
|
||||
changed mounted documents together, without a tracker or semantic projection.
|
||||
Third-party documents are **not automatically mounted**; use their own Chord
|
||||
services or trusted invocation watches. A retired source publishes `null`; a
|
||||
dynamic service should then withdraw its instance rather than expose stale data.
|
||||
|
||||
```text
|
||||
addStroke -> hold Session mutation line -> await tx.doc -> mutate tracked draft
|
||||
callback succeeds -> tracker flush: incremental ops + candidate value
|
||||
atomic storage commit: all document and record writes; checkpoint predicate selects ops or base
|
||||
storage succeeds -> adopt committed baseline + enqueue source publication, still on line
|
||||
release line -> deliver committed source ops -> adapter -> local/remote Chord consumers
|
||||
late subscriber -> atomically capture committed value/revision + subscription
|
||||
```
|
||||
|
||||
No visible-undurable path exists. Callback failure restores the unchanged
|
||||
baseline. Failure after flush, including checkpoint/storage failure, publishes
|
||||
nothing and poisons the open Session; close and reopen it instead of continuing.
|
||||
|
||||
## Footguns
|
||||
|
||||
- Never retain drafts, nested proxies, or bound array methods across commits.
|
||||
Never mutate inserted aliases or insert one mutable object at multiple paths.
|
||||
- Async commit holds the line through storage settlement and baseline adoption.
|
||||
Await document access there, not models, processes, network calls, or humans.
|
||||
- Normal `snapshot`, `documentSource`, and `watchDoc` reads are get-or-create.
|
||||
Family `initial` is first-creation input, not an update.
|
||||
- An unstarted watch buffers without a fixed bound. Start or stop promptly.
|
||||
- Definitions own checkpoints, not storage heuristics. Revise the counting predicates
|
||||
above if mutations change. Keep schema IDs, versions, fork policies, and public
|
||||
paths stable; schema changes require migration, not a source-only rename.
|
||||
|
||||
Chord sources: [types](../../chord/src/types.ts), [facet examples](../../chord/test/facets.test.ts),
|
||||
[state](../../chord/src/services/state.ts), [provider](../../chord/src/services/provider.ts),
|
||||
[Context](../../chord/src/context/index.ts), [Delta](../../chord/src/delta/README.md).
|
||||
|
|
@ -1,555 +1,266 @@
|
|||
# Pico5 handoff and decision register
|
||||
|
||||
> Working coordination document. `pico-v5.md` is the compact design. This file
|
||||
> records accepted choices, unresolved decisions, examples, and layer sign-offs so
|
||||
> work survives conversation compaction. No unresolved item may be decided silently
|
||||
> during implementation.
|
||||
|
||||
## Current status
|
||||
|
||||
- `packages/agent/src/harness/pico/` and its tests were removed.
|
||||
- The untracked `pico4` source/tests were removed.
|
||||
- `pico3` remains as a reference, not an implementation base.
|
||||
- Storage benchmark artifacts are in `/tmp/pi-pico5-storage-bench/`.
|
||||
- Astra approved starting the memory + SQLite storage/conformance slice after the
|
||||
storage decisions below are signed off.
|
||||
|
||||
## Accepted direction
|
||||
|
||||
1. Entries are immutable and paged; the Session is not one materialized DOM.
|
||||
2. Authors mutate ordinary JSON documents; Chord produces nested/list/string ops.
|
||||
3. History policy is per document: `latest` or `rewindable`.
|
||||
4. Built-ins use three conversation documents:
|
||||
- `rewindable`: historical configuration;
|
||||
- `sticky`: present state such as queues and retry policy;
|
||||
- `live`: high-churn turn, generation, tool, and compaction presentation state.
|
||||
5. Typed document definitions and dynamic families are required. Family instances
|
||||
may be task-owned and retire automatically.
|
||||
6. Task recovery state is a complete replacement, separate from presentation and
|
||||
application documents.
|
||||
7. There is no visible-but-undurable publication. Every Chord-visible update follows
|
||||
a successful durable commit.
|
||||
8. Interrupted assistant/tool partials are converted into durable aborted/interrupted
|
||||
entries by their recovering task kind, then removed from `live`.
|
||||
9. A built-in needs cross-task document observation, so `watchDoc` is initial scope.
|
||||
10. UI status is state, not an event protocol. There is no public event channel
|
||||
initially; clients observe document paths. A bounded `live` notice field may be
|
||||
designed with the built-in view if concrete transient facts require it.
|
||||
11. Third-party documents initially use their own Chord services. Automatic mounting
|
||||
into the agent view is deferred.
|
||||
12. SQLite is the long-running/cloud backend. JSONL remains local and readable.
|
||||
13. Implementation proceeds layer by layer and stops for user sign-off after each.
|
||||
|
||||
## Decisions required before storage implementation
|
||||
|
||||
### S1. Document identity and families — accepted
|
||||
|
||||
A definition carries type, logical ID, scope, history, initializer, and optional
|
||||
family ownership rules:
|
||||
|
||||
```ts
|
||||
const live = defineDoc<LiveState>({
|
||||
id: "pi.live",
|
||||
scope: "conversation",
|
||||
history: "latest",
|
||||
initial: () => ({ tools: [] }),
|
||||
});
|
||||
|
||||
const jobOutput = defineDocFamily<JobState, JobInput>({
|
||||
id: "pi.job",
|
||||
scope: "conversation",
|
||||
history: "latest",
|
||||
owner: "task",
|
||||
initial: input => ({ command: input.command, stdout: "" }),
|
||||
});
|
||||
```
|
||||
|
||||
A persisted instance has a unique, never-reused storage identity plus definition ID,
|
||||
scope owner, family instance ID, creation sequence, history policy, and retirement
|
||||
sequence. Reusing one logical family key creates a new incarnation.
|
||||
|
||||
### S2. Historical document membership — accepted
|
||||
|
||||
A rewindable document retired today still exists in an as-of query from yesterday.
|
||||
A latest document can be deleted after retirement.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
10 create rewindable review A
|
||||
20 append entry E
|
||||
30 retire A
|
||||
40 fork at E
|
||||
```
|
||||
|
||||
The fork enumerates A at commit 20 and copies its as-of value into a new child-owned
|
||||
instance. Current enumeration at commit 40 omits A.
|
||||
|
||||
### S3. Storage ownership and copying — proposed recommendation
|
||||
|
||||
Simplest safe contract:
|
||||
|
||||
- Commit inputs are borrowed until `Storage.commit()` settles.
|
||||
- A backend retaining anything after settlement detaches it first.
|
||||
- Memory uses a small recursive `copyJson`; JSONL/SQLite detach through serialization.
|
||||
- Every storage read returns a fresh owned JSON copy. Mutating it cannot corrupt the
|
||||
backend or another caller. Public types remain readonly, but correctness does not
|
||||
rely on TypeScript at runtime.
|
||||
- Large strings are immutable and may share their string value during an in-memory
|
||||
copy; object/array containers never share mutably.
|
||||
|
||||
Example failure this prevents:
|
||||
# Pico5 implementation handoff
|
||||
|
||||
```ts
|
||||
const task = { phase: "running" };
|
||||
await storage.commit([{ type: "task", task }]);
|
||||
task.phase = "done"; // must not mutate MemoryStorage behind its back
|
||||
```
|
||||
|
||||
Decision: approve this contract, or replace it with explicit ownership transfer and
|
||||
state which caller may still use document ops for Chord publication.
|
||||
|
||||
### S4. Table reads and writes inside `Tx` — proposed recommendation
|
||||
|
||||
Adopt the other design's simple rule:
|
||||
|
||||
```text
|
||||
table reads first -> first table write -> later table reads throw ReadAfterWrite
|
||||
document reads/mutations always read their transaction draft
|
||||
```
|
||||
`packages/agent/docs/pico-v5.md` is normative. Implement this list in order.
|
||||
After every package: run its tests, run `npm run check`, and stop for user review.
|
||||
Do not redesign later packages while implementing the current one.
|
||||
|
||||
Example:
|
||||
Pico3 is reference material only. Preserve useful behavior, not its capability
|
||||
facades, membranes, document routing, view projection, events, or clone chains.
|
||||
|
||||
```ts
|
||||
const inputs = await tx.inputs(ids); // read
|
||||
const head = await tx.newestEntry(c); // read
|
||||
const entry = tx.appendEntry(c, assistant); // first table write
|
||||
tx.doc(live, c).message = undefined; // document draft is allowed
|
||||
// await tx.entry(other) now throws ReadAfterWrite
|
||||
```
|
||||
## Status
|
||||
|
||||
Benefit: no shadow indexes for newly created entries/tasks/inputs. Cost: helpers must
|
||||
load everything before their first table write. Validate all built-in settlement
|
||||
traces before accepting.
|
||||
- Obsolete `pico` and `pico4` prototypes were removed.
|
||||
- `pico3` remains.
|
||||
- No Pico5 implementation exists.
|
||||
|
||||
Decision: adopt this rule, or support read-your-writes for tables and pay for overlays.
|
||||
## 1. Records, cursors, and memory tables
|
||||
|
||||
### S5. Escaped document draft — open
|
||||
Implement IDs, sequences, conversations, entries, inputs, live/terminal tasks,
|
||||
document metadata, storage writes, cursors, and detached `MemoryStorage` tables.
|
||||
|
||||
A callback can retain a document proxy:
|
||||
Test mixed atomic commits, rollback, detached reads/writes, cursor boundaries,
|
||||
fork-aware entry scans through deep ancestor caps, head lookup,
|
||||
entry-to-commit lookup, and full task replacement.
|
||||
|
||||
```ts
|
||||
let escaped: LiveState;
|
||||
await session.commit(tx => { escaped = tx.doc(live, c); });
|
||||
escaped.message = partial;
|
||||
```
|
||||
## 2. Memory document records
|
||||
|
||||
Options:
|
||||
Add document creation bases, deltas, required/checkpoint bases, retirement,
|
||||
reincarnation, current/as-of membership, and base-plus-tail reconstruction.
|
||||
|
||||
- **Detached stale draft:** mutation succeeds locally but has no effect. Simplest,
|
||||
but silently hides a bug.
|
||||
- **Revoked draft:** every later read/write throws. Clearest, but requires tracker
|
||||
proxies to share an active guard or another membrane.
|
||||
- **Debug assertion:** stale mutation is detected at the next transaction. Cheaper,
|
||||
but delayed.
|
||||
Test Session-, conversation-, and task-scoped documents, retired historical
|
||||
membership, family queries, and no scans of unrelated document records.
|
||||
|
||||
Recommendation: first determine whether Chord tracker proxies can cheaply share one
|
||||
revocation guard. Do not let an escaped proxy mutate committed state or contaminate a
|
||||
later commit.
|
||||
## 3. SQLite backend
|
||||
|
||||
### S6. Base policy — partly accepted
|
||||
Implement the complete storage contract with ordinary rows and indexed document
|
||||
records. Do not translate Chord operations into SQL JSON patches.
|
||||
|
||||
Every document may receive bases. Storage gets both incremental ops and a borrowed
|
||||
candidate value, serializing only one:
|
||||
Run the memory conformance suite after reopen. Test SQL transaction rollback,
|
||||
recent/ancient as-of reads, query plans, latest reclamation, WAL checkpointing,
|
||||
deleted-page reuse, and representative storage sizes.
|
||||
|
||||
```ts
|
||||
persist(shouldBase(id, ops) ? [["r", value]] : ops);
|
||||
```
|
||||
## 4. JSONL publication
|
||||
|
||||
- `latest`: a committed base permits reclaiming older records.
|
||||
- `rewindable`: bases bound as-of replay; historical segments remain.
|
||||
- Trigger on both encoded bytes and record count.
|
||||
Implement table writes in `main.jsonl`, one document sidecar per incarnation,
|
||||
one sidecar per live task, and one main marker per commit. Do not add a
|
||||
standalone-sidecar protocol.
|
||||
|
||||
Open values:
|
||||
Fault-test torn/short sidecar writes, failures between sidecars, every marker
|
||||
boundary, unconfirmed tails, missing confirmed data, and poisoned writes.
|
||||
|
||||
- byte threshold;
|
||||
- record-count threshold;
|
||||
- whether document definitions may request `rebaseWhenInitial`.
|
||||
## 5. JSONL reclamation
|
||||
|
||||
`pi.live` benefits from `rebaseWhenInitial`: when a turn ends at `{ tools: [] }`, a
|
||||
tiny base allows immediate reclamation of streamed output. The definition/document
|
||||
manager can recognize its initial state; storage still owns physical reclamation.
|
||||
Implement task-document retirement and current-only base reclamation using
|
||||
committed markers, temporary replacement, rename, and descriptor invalidation.
|
||||
|
||||
### S7. Task rows — accepted
|
||||
Crash-test every rewrite/rename boundary. Verify that rewindable history is
|
||||
never reclaimed and default no-fsync behavior matches the specification.
|
||||
|
||||
Task records are full replacements. SQLite uses one live row and `DELETE` on terminal.
|
||||
JSONL uses one task sidecar and unlinks it only after terminal publication.
|
||||
## 6. Tracker transaction core
|
||||
|
||||
The runtime envelope owns kind/input/dependencies/memos/effect IDs. Kind-defined
|
||||
recovery state cannot overwrite memos. A terminal transaction transfers any needed
|
||||
outcome and retires every task-owned document family instance atomically.
|
||||
Keep one tracker per loaded document. Add transaction begin, callback rollback
|
||||
from the unchanged baseline, flush, fatal post-flush failure, committed baseline
|
||||
adoption, and eviction.
|
||||
|
||||
This atomic retirement removes the need for an application-level orphan sweep:
|
||||
Do not add membranes, transaction proxy graphs, capability facades, or defensive
|
||||
full-document clones. Test callback failure, no-op transactions, storage failure
|
||||
poisoning, and unload/reload.
|
||||
|
||||
- crash before terminal: task and owned docs remain live and recover;
|
||||
- crash after terminal marker: logical retirement is known, and storage may clean
|
||||
leftover files during open.
|
||||
## 7. Document definitions and access
|
||||
|
||||
### S8. SQLite representation — accepted direction
|
||||
Implement `defineDoc`, `defineDocFamily`, identity validation, the three direct
|
||||
scopes, and get-or-create `tx.doc`, `snapshot`, and `documentSource` acquisition.
|
||||
|
||||
Use one SQL transaction per Session commit:
|
||||
Test concurrent initialization once, initial bases, detached snapshots, family
|
||||
initializer use only on first creation, scope/target mismatch, terminal-task
|
||||
rejection, task-derived conversation identity, retirement, and
|
||||
reincarnation-bound sources. Include create-task-then-document,
|
||||
document-after-terminal rejection, and create-document-then-terminal settlement
|
||||
in one transaction; internal candidate validation must not trigger
|
||||
`ReadAfterWrite`.
|
||||
|
||||
```text
|
||||
conversations / entries / inputs
|
||||
document metadata
|
||||
document base+delta records indexed by document and commit
|
||||
a current live-task row
|
||||
```
|
||||
## 8. Checkpoints and migration
|
||||
|
||||
Do not translate Chord ops into SQLite JSON functions initially. The benchmark found
|
||||
full JSONB replacement slower than TEXT and generic SQL patching slower with more WAL
|
||||
writes. Current/as-of reads select a base and indexed tail; they do not scan unrelated
|
||||
documents.
|
||||
Inside `Storage.commit()`, evaluate `checkpointWhen(value, ops)` exactly once
|
||||
for ordinary mutations. Implement required creation/version bases and
|
||||
all-older-version migration.
|
||||
|
||||
Still choose schema details, WAL checkpoint cadence, and `synchronous` default during
|
||||
the SQLite layer.
|
||||
Test latest migration persistence, rewindable migration on current/historical
|
||||
read, first post-migration mutation base, newer-version rejection, absent plugin
|
||||
preservation, and checkpoint starvation without backend heuristics.
|
||||
|
||||
### S9. JSONL physical layout — current lean: uniform sidecars
|
||||
## 9. Conversation document forks
|
||||
|
||||
Simplest layout from the other design:
|
||||
Using fixture conversations and entry-to-commit mappings, implement the `asOf`,
|
||||
`current`, and `initial` settings for singleton and family documents.
|
||||
|
||||
```text
|
||||
main.jsonl tables + one publication marker per commit
|
||||
doc-<storage-id>.jsonl one file per document instance
|
||||
task-<storage-id>.jsonl one file per live task
|
||||
```
|
||||
Test unknown definitions, retired membership, new child incarnations, lazy
|
||||
`initial` creation, and exclusion of task- and Session-scoped documents.
|
||||
|
||||
Every sidecar record is prepared. Sidecars append first; one main marker is always
|
||||
last. This deliberately permits records such as:
|
||||
## 10. Chord structural array operations
|
||||
|
||||
```json
|
||||
{"seq":14,"refs":["task-9"]}
|
||||
{"seq":15,"refs":["task-9"]}
|
||||
{"seq":16,"refs":["pi.live-1"]}
|
||||
```
|
||||
Make ordinary positional mutations encode scattered removals without carrying
|
||||
retained payloads. Callers must not write operations manually.
|
||||
|
||||
The markers are cosmetic overhead but make every commit use one protocol. They can be
|
||||
compact; fields such as `writes:[]` and repeated `maxId` are unnecessary if high-water
|
||||
is represented elsewhere. Without `fsync`, the second append is unlikely to be the
|
||||
performance bottleneck; the benchmark's no-fsync sidecar backend still handled about
|
||||
9.5k–30k commits/s depending on workload while opening/closing each sidecar.
|
||||
Test front/tail/middle/scattered/all/no removal, retained 256 KiB and 1 MiB
|
||||
payloads, append plus removal, later nested/index writes, exact replay, and
|
||||
unchanged previous immutable snapshots. Internal flushes must remain one commit.
|
||||
|
||||
Alternative: standalone one-file commits avoid markers but introduce two record modes,
|
||||
distributed commit discovery, and more recovery cases.
|
||||
## 11. Chord document source
|
||||
|
||||
Decision: approve uniform marker publication or select the standalone optimization.
|
||||
Add the thin opaque-source adapter to `ReplicatedState`. It must use committed
|
||||
value/ops directly, with no tracker or re-diff.
|
||||
|
||||
### S10. JSONL process-crash contract — accepted direction
|
||||
Test contiguous revisions, atomic hydrate/subscribe, attachment races,
|
||||
retirement ending one incarnation, recreation requiring reacquisition, and
|
||||
listener isolation.
|
||||
|
||||
No `fsync` by default. Awaited/full sidecar writes precede the awaited/full main
|
||||
marker. For an ordinary process crash with the OS/filesystem alive:
|
||||
## 12. Document watches
|
||||
|
||||
- torn final lines are removed;
|
||||
- sidecar records without a marker are uncommitted and removed;
|
||||
- a complete marker publishes the transaction.
|
||||
Implement get-or-create `watchDoc`: capture value/revision and register its
|
||||
listener atomically, buffer without a cap until `start()` or `stop()`, and always
|
||||
deliver value plus ops.
|
||||
|
||||
Power loss, host/kernel failure, or weak filesystem ordering may lose acknowledged
|
||||
commits. Optional durable mode may fsync sidecars before the marker.
|
||||
Test ordered draining, live delivery, retirement as absence, idempotent stop,
|
||||
listener errors, and explicit cleanup. Invocation-owned cleanup is integrated in
|
||||
package 15.
|
||||
|
||||
Any uncertain write error poisons the open backend until reopen. Reclamation occurs
|
||||
only after its base/retirement commits and uses temp + rename; cached descriptors are
|
||||
then reopened.
|
||||
## 13. Conversations and entries
|
||||
|
||||
### S11. Missing confirmed sidecar — open
|
||||
Implement conversation history/ownership records, entry creation, cursor-based
|
||||
fork-aware scans, head lookup, and entry edits.
|
||||
|
||||
Example:
|
||||
Test conversation creation and actual forks, deep ancestor caps, same-commit
|
||||
entry prefixes, newest-edit wins, self-head resolution, raw head-to-tail
|
||||
transcript, and ownership traversal.
|
||||
|
||||
```text
|
||||
sidecar record 20 was reported written
|
||||
main marker 20 survives
|
||||
sidecar record 20 is absent after machine failure
|
||||
```
|
||||
## 14. Context derivation and system messages
|
||||
|
||||
This should not arise from the stated ordinary process-crash contract. Options:
|
||||
Implement model-context reduction, PR #9548 positional `SystemMessage` replay,
|
||||
tool-result ordering, and missing post-fork tool results. Replay `content`,
|
||||
ordered named `sections` with `null` removal, then tool removals/additions.
|
||||
|
||||
- **Fail open:** report corruption and preserve all bytes for diagnosis/repair.
|
||||
Simplest and never invents state.
|
||||
- **Roll back globally:** truncate every file to the latest complete global prefix.
|
||||
Recovers availability but may discard acknowledged later commits and must interact
|
||||
correctly with reclaimed prefixes and standalone records if those exist.
|
||||
- **Best-effort skip:** invalid because it exposes a partial atomic commit.
|
||||
Test model-less and excluded-stop-reason entries, replacements/omissions,
|
||||
multiple heads, section replacement/removal/re-addition order, rejection of
|
||||
integer-like section keys, tool addition/removal/replacement order, and raw-view
|
||||
versus model context.
|
||||
|
||||
Recommendation: fail open initially. Add a separate repair tool only after physical
|
||||
fault tests establish a safe prefix algorithm.
|
||||
## 15. Task definitions and invocations
|
||||
|
||||
### S12. Main JSONL growth — accepted limitation
|
||||
Implement `defineTask`, exhaustive phase maps, full checkpoint replacement,
|
||||
kind migration, runtime commits, memos, and invocation close gates.
|
||||
|
||||
Document/task sidecars reclaim high-churn data; `main.jsonl` still grows by entries,
|
||||
lifecycle records, and commit markers. JSONL may scan it on open. It is the local
|
||||
backend. Years-long products use SQLite. Do not add global JSONL compaction initially.
|
||||
Use a fake two-phase effect. Test intent/effect/outcome recovery,
|
||||
unchanged-checkpoint faulting, same-phase checkpoint progress, cancellation
|
||||
precedence, thrown-handler faulting, close/reopen without abort marks, outcomes,
|
||||
or task-document retirement, no fresh phase/abort dispatch while closing,
|
||||
first-writer-wins memos, and automatic watch cleanup.
|
||||
|
||||
### S13. Storage query surface — needs exact signatures
|
||||
## 16. Scheduler and terminal tasks
|
||||
|
||||
The first storage contract must support:
|
||||
Implement reservation, running-task reopen reconciliation, dependencies,
|
||||
terminal outcomes, waits, holds, joins, and orphaning.
|
||||
|
||||
- create/current/retired conversation records;
|
||||
- paged immutable entries and entry -> commit mapping;
|
||||
- current task lookup/scan;
|
||||
- current and as-of document enumeration;
|
||||
- current and as-of document value;
|
||||
- family-instance enumeration by definition/scope/owner;
|
||||
- one atomic mixed commit;
|
||||
- close/reopen.
|
||||
Test result values and entry IDs, terminal records after reopen, dependency
|
||||
eligibility, unknown kinds, and terminal removal of checkpoints/memos.
|
||||
|
||||
As-of results distinguish not-yet-created, retired after target, and unavailable
|
||||
history. Concrete TypeScript signatures are the first layer's first sign-off.
|
||||
## 17. Abort and owned conversations
|
||||
|
||||
## Decisions required before document/task implementation
|
||||
Implement durable abort marks, signal/join/fresh-abort invocation, owned
|
||||
conversation creation, subtree traversal, background behavior, and idle waits.
|
||||
|
||||
### D1. Three built-in documents — accepted
|
||||
Test commit rejection after a run task is marked, crashes at every abort stage,
|
||||
close precedence over a previously marked task, deep ownership trees, and atomic
|
||||
retirement of task-scoped documents.
|
||||
|
||||
```ts
|
||||
rewindable // model/tool/config choices inherited by historical fork
|
||||
sticky // inbox, queue modes, retry policy
|
||||
live // current message/tools/generation/collapse and maybe notices
|
||||
```
|
||||
## 18. Inputs and positional inbox
|
||||
|
||||
`live` is durable because recovery converts its partial output into transcript entries.
|
||||
Its high churn is independently reclaimable when it returns to initial state.
|
||||
Define the initial inbox and turn-control documents, then implement request-ID
|
||||
deduplication, awaitable input handles, busy admission, withdrawal, queue modes,
|
||||
and `postTools`/`final` boundaries. Use a fake successor task.
|
||||
|
||||
### D2. Document adapter to Chord — accepted direction
|
||||
Table-test every input transition, interleaved steer/follow-up/write selection,
|
||||
self-head cuts, stale targets, successor triggers, reopen waits, compact
|
||||
large-payload removals, and orphan/fault cleanup of active turn control.
|
||||
|
||||
A thin adapter such as `replicatedState.own(documentSource)` exposes committed value
|
||||
and ops. It owns no second tracker. Each document and mounted view has a contiguous
|
||||
publication revision separate from sparse global Session commit sequence.
|
||||
## 19. Remaining built-in documents and view
|
||||
|
||||
Hydration captures value/revision and subscribes atomically. Listener errors after
|
||||
persistence are isolated. Current Chord needs a small public source adapter; it does
|
||||
not need harness knowledge.
|
||||
Define the concrete configuration, preference, and live presentation documents;
|
||||
reuse the approved inbox/turn definitions. Record all IDs, fields, history,
|
||||
fork settings, migration, and checkpoint predicates in the normative
|
||||
specification.
|
||||
|
||||
### D3. Cross-task `watchDoc` — accepted, exact scope open
|
||||
Implement `{ conversation, entries, docs }`. Test direct task writes, one
|
||||
publication per Session commit, atomic
|
||||
entry/preview settlement, head changes, contiguous revisions, stable public
|
||||
paths, retry/collapse late-join status, bounded-output truncation metadata, and
|
||||
absence of semantic projection. Specify which diagnostics become entries,
|
||||
terminal details, or bounded document state.
|
||||
|
||||
Required semantics:
|
||||
## 20. Registries, hooks, and sections
|
||||
|
||||
```ts
|
||||
const watch = await api.watchDoc(jobOutput, job.id, context);
|
||||
consume(watch.value); // snapshot first
|
||||
watch.start((ops, value) => consume(value));
|
||||
```
|
||||
Implement task/tool/entry/section registries, Session and owned-subtree hooks,
|
||||
positional PR #9548 section/tool updates, complete baselines after a head cut,
|
||||
and preparation revision checks. Do not add a public event stream or
|
||||
plugin-state router.
|
||||
|
||||
- capture + subscription are one Session-line operation;
|
||||
- callbacks run off the line and cannot re-enter a commit synchronously;
|
||||
- watch is volatile and invocation-owned;
|
||||
- recovery re-enters the handler, captures a fresh snapshot, and subscribes again;
|
||||
- retirement appears as document absence;
|
||||
- `stop()` is idempotent.
|
||||
Test registration lifetimes, hook replay with memos, exact persisted rendered
|
||||
section strings, minimal section patches and `null` removals, complete baseline
|
||||
tool declarations, and a head cut that retains earlier system messages. Verify
|
||||
the new baseline entry omits those messages through `ContextEdit` before replay,
|
||||
including retained `content`, section order, and tool changes. Include a retained
|
||||
delta whose ID precedes the head-carrying entry: select omissions by retained
|
||||
context membership, not an ID comparison with the head entry. Test tool loadout
|
||||
additions/removals and preparation retry after registry movement. Do
|
||||
not implement in-process replacement of Session-side plugin code; a host plugin
|
||||
change uses the Harness close/reopen boundary.
|
||||
|
||||
Open scope: same conversation only, or any conversation owned by the observing task's
|
||||
subtree. Initial built-in use must provide the concrete required edge.
|
||||
## 21. Tool and post-tools tasks
|
||||
|
||||
### D4. Reopen conversion — accepted
|
||||
Implement offered-set checks, argument validation, tool hooks, durable bounded
|
||||
progress, owned APIs, interrupted/replay-safe recovery, result entries,
|
||||
post-tools joining, controls, and boundaries using fake tools.
|
||||
|
||||
Generation recovery reads `live.message`, appends an assistant entry with
|
||||
`stopReason: "aborted"`, clears the preview, and retries/fails atomically. Tool
|
||||
recovery turns persisted output into its interrupted result before clearing the slot.
|
||||
The entry uses the normal model/render shape while model projection excludes aborted
|
||||
content.
|
||||
Test recovery from every phase, both stored/current replay-policy directions,
|
||||
and atomic assistant/tool/post-tools settlement. Use a fake generation
|
||||
successor; package 22 replaces it and reruns integration.
|
||||
|
||||
Conversion runs before retiring any task-owned family document it needs.
|
||||
## 22. Generation task
|
||||
|
||||
### D5. Document versioning/migration — open
|
||||
Implement preparation, request intent, durable throttled partials, attempts,
|
||||
retry policy, response classification, continuation, and deferred polling/
|
||||
cancellation with the faux provider.
|
||||
|
||||
Document paths and persisted values are public/durable schema. A plugin upgrade may
|
||||
change `{ comments: [...] }` to `{ threads: [...] }`.
|
||||
Test every phase before and after reopen, aborted partial conversion, overflow
|
||||
through a fake collapse kind, input settlement, and no visible-undurable update.
|
||||
Replace the fake tool successor and rerun the package 21 integration tests.
|
||||
|
||||
Options:
|
||||
## 23. Collapse task
|
||||
|
||||
- definition has an integer version and migration from complete current value;
|
||||
- task/document kind owns migration during open;
|
||||
- reject mismatched versions initially.
|
||||
Implement manual, threshold, and overflow collapse; exchange-boundary range
|
||||
selection; summarization; retries; staleness; and headed summary entries.
|
||||
|
||||
Recommendation: persist definition version now even if initial implementation only
|
||||
rejects mismatches. Retrofitting identity/version after stored products ship is harder.
|
||||
Test context before/after collapse, provider failure, declined/stale work, and
|
||||
reopen from every phase. Replace generation's fake overflow target and rerun its
|
||||
overflow integration test.
|
||||
|
||||
## Decisions required before view implementation
|
||||
## 24. Job and plugin tasks
|
||||
|
||||
### V1. Public view shape — open
|
||||
Implement process jobs with bounded durable output, interruption recovery,
|
||||
rescheduling, and registered plugin handlers.
|
||||
|
||||
Fixed top-level fields:
|
||||
Test process exit/error/abort/reopen, background idle behavior, notifications as
|
||||
state or entries, and plugin handler recovery.
|
||||
|
||||
```ts
|
||||
{ conversation, entries, rewindable, sticky, live }
|
||||
```
|
||||
## 25. Harness integration
|
||||
|
||||
Typed and direct, but every added built-in document changes the protocol.
|
||||
Expose conversation/input/task handles, watches, dynamic registries, close,
|
||||
suspend, service withdrawal/client detach, and product wiring. Implement the v1
|
||||
host-plugin reload path as stop admission, close/join, dispose, rebuild with all
|
||||
new definitions, reopen/migrate, and resume. Test that closing seals commit and
|
||||
get-or-create admission, lets storage settlement for already-flushed admitted
|
||||
commits finish despite caller cancellation, writes no abort or terminal outcome, starts no fresh abort
|
||||
invocation, and does not run old and new generations concurrently.
|
||||
|
||||
Document map:
|
||||
|
||||
```ts
|
||||
{
|
||||
conversation,
|
||||
entries,
|
||||
docs: {
|
||||
"pi.rewindable": RewindableState,
|
||||
"pi.sticky": StickyState,
|
||||
"pi.live": LiveState,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Makes document identity/lifetime explicit and can add selected documents later, but
|
||||
paths and IDs become public protocol. It can remain typed for the three built-ins.
|
||||
|
||||
Recommendation: decide before writing any UI adapter. In either shape, mount only the
|
||||
three built-ins initially and treat every visible path as versioned public API.
|
||||
|
||||
### V2. Public events — accepted: none initially
|
||||
|
||||
A compaction-start event cannot describe current state to a late joiner. Instead:
|
||||
|
||||
```ts
|
||||
live.collapse = { through, stage: "summarizing" }; // running
|
||||
live.collapse = undefined; // finished
|
||||
```
|
||||
|
||||
Clients react to state ops by path and always hydrate the current truth. Hooks remain
|
||||
pre-decision interception. Internal commit subscribers may exist mechanically, but
|
||||
there is no separate public semantic event protocol initially.
|
||||
|
||||
Whether failed attempts need a bounded `live.notices` list is deferred until the UI
|
||||
is implemented. It remains state with explicit retention, not fire-and-forget events.
|
||||
|
||||
### V3. Atomic publication — accepted
|
||||
|
||||
The built-in mount consumes the complete Session commit, prefixes every changed
|
||||
built-in document path, combines entry changes, and publishes one Chord batch. It has
|
||||
no tracker or semantic projection. Independent third-party Chord services may publish
|
||||
separately.
|
||||
|
||||
### V4. Active transcript bound — open
|
||||
|
||||
A Slack conversation cannot hydrate every entry ever written. Options include:
|
||||
|
||||
- durable `head` entry: active transcript is the range beginning at its target;
|
||||
- fixed recent entry/count/byte window;
|
||||
- product-selected window with a stable cursor.
|
||||
|
||||
Example requirement:
|
||||
|
||||
```text
|
||||
10 years / 500,000 entries
|
||||
late UI attach -> bounded snapshot
|
||||
older history -> explicit paged query
|
||||
model context -> independently derived bounded projection
|
||||
```
|
||||
|
||||
Choose before view implementation, not before storage. Entry paging and indexes are
|
||||
required now regardless.
|
||||
|
||||
### V5. Multi-entry fork boundary — open
|
||||
|
||||
One commit may append entries E and F and leave one final rewindable state. Fork at E:
|
||||
|
||||
- **entry prefix:** child transcript includes E but not F, while document state is the
|
||||
final state of the containing commit;
|
||||
- **commit boundary:** selecting E means inherit all entries from its indivisible
|
||||
commit, including F;
|
||||
- **restriction:** only designated forkable entries/commits may be targets.
|
||||
|
||||
Choose before fork/view API. Storage records entry -> commit either way.
|
||||
|
||||
### V6. Sticky fork initialization — current lean: defaults
|
||||
|
||||
A historical fork inherits rewindable documents as-of its target and no live tasks.
|
||||
Copying current sticky/live state can copy queued inputs or running previews from the
|
||||
future. Simplest initial rule:
|
||||
|
||||
```text
|
||||
rewindable -> historical copy
|
||||
sticky -> fresh defaults
|
||||
live -> fresh defaults
|
||||
```
|
||||
|
||||
Explicit sticky-copy options remain deferred.
|
||||
|
||||
### V7. Third-party mounting — accepted defer
|
||||
|
||||
Third-party documents initially expose application-specific Chord services. Do not add
|
||||
`inView`, privacy flags, dynamic mount registries, or client-selected arbitrary docs.
|
||||
A document is not a security boundary; services deliberately choose what crosses a
|
||||
process boundary.
|
||||
|
||||
## Layer plan and sign-off gates
|
||||
|
||||
### Layer 0 — cleanup
|
||||
|
||||
- Remove `pico` source/tests: done.
|
||||
- Remove `pico4` source/tests: done.
|
||||
- Keep `pico3`: done.
|
||||
- Run repository check after deletions and document additions: pending.
|
||||
|
||||
### Layer 1 — logical storage + memory reference
|
||||
|
||||
Before code: sign off S3 and S6, retain accepted S1/S2/S7, and approve the exact S13 interfaces.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- atomic mixed commits;
|
||||
- document creation, bases, current/as-of membership and values;
|
||||
- family instances and task ownership;
|
||||
- full task replacement/retirement;
|
||||
- entry paging and entry -> commit;
|
||||
- ownership isolation and failure rollback.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 2 — SQLite
|
||||
|
||||
Before code: approve concrete schema and durability defaults.
|
||||
|
||||
Acceptance: Layer 1 conformance plus reopen, indexed query plans, latest reclamation,
|
||||
retired rewindable history, WAL checkpoint behavior, and realistic storage benchmark.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 3 — JSONL
|
||||
|
||||
Before code: decide S9 and S11.
|
||||
|
||||
Acceptance: semantic conformance plus physical fault injection at every sidecar,
|
||||
marker, rewrite, rename, retirement, and reopen boundary.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 4 — document manager + Chord source
|
||||
|
||||
Before code: decide S4, S5, D3 scope, and D5.
|
||||
|
||||
Acceptance: ordinary mutations, no duplicate tracker, failed-commit rollback,
|
||||
contiguous publication, hydration fencing, listener isolation, eviction, and
|
||||
`watchDoc` snapshot/update/retirement behavior.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 5 — tasks
|
||||
|
||||
Acceptance: phase-narrowed full replacements, effect sandwich, runtime-owned memos,
|
||||
abortable ownership tree, atomic family retirement/outcome handoff, generation/tool
|
||||
reopen conversion, and no visible-undurable state.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 6 — built-in documents and conversation view
|
||||
|
||||
Before code: decide V1, V4, V5, and V6.
|
||||
|
||||
Acceptance: direct built-in writes, fixed structural mount, one publication per commit,
|
||||
bounded late hydration, crash-visible progress, and no public event dependency.
|
||||
|
||||
Stop for user review.
|
||||
|
||||
### Layer 7 — remaining harness
|
||||
|
||||
Inputs, hooks, tools, system sections, dynamic tools, deferred providers, and product
|
||||
wiring. Preserve accepted APIs from earlier design work rather than redesigning them
|
||||
inside the state implementation.
|
||||
Run all package-specific tests and the repository check. Verify a local
|
||||
coding-agent turn and a reopened interrupted turn, then stop for final review.
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue