mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
docs(agent): persist pico entry projections
This commit is contained in:
Mario Zechner
parent
2188891bf3
commit
ce5ec9ca35
3 changed files with 309 additions and 223 deletions
|
|
@ -69,10 +69,10 @@ conversation has three things:
|
|||
- **state**: keyed **values** and **lists**, for the model in use, plan mode, a game board, whatever
|
||||
a plugin needs to remember.
|
||||
|
||||
What the model sees, the **context**, is not stored. The newest **head** supplies a replacement
|
||||
(summary, handoff or nothing) and the first retained transcript entry. The harness prepends that
|
||||
head, reads forward from the retained entry, applies immutable **edit** entries that omit or replace
|
||||
individual projections, then projects messages. Compaction appends a head; tool-result pruning
|
||||
What the model sees, the **context**, is not stored as a list. Each entry may store `model`
|
||||
messages, a **head** boundary, and **edits** that omit or replace earlier model messages. The harness
|
||||
prepends the newest head, reads forward from its stored boundary, folds retained edits, then performs
|
||||
request-local tool and provider normalization. Compaction appends a head; tool-result pruning
|
||||
appends an edit. Forks, compaction and reset never mutate old entries.
|
||||
|
||||
Every id is a session sequence number, minted when the write is built and never changed:
|
||||
|
|
@ -147,7 +147,7 @@ w.start(event => {
|
|||
|
||||
// accept the input, drive the conversation until it is idle, return the answer to that input
|
||||
const answer = await c.prompt('Inspect the parser and list the public API');
|
||||
console.log(answer?.content);
|
||||
console.log(answer?.model[0]?.content);
|
||||
|
||||
w.unsubscribe();
|
||||
await h.close(); // cancels nothing durable; reopening the file continues exactly here
|
||||
|
|
@ -193,8 +193,10 @@ const h = await Harness.open(storage, { models, tools, replace: {
|
|||
}});
|
||||
```
|
||||
|
||||
Open checks that every kind found in storage is registered (without scanning the transcript: the
|
||||
set of kinds ever written is kept) and reports live work, but starts nothing:
|
||||
Open checks the recorded kind strings without scanning the transcript and reports live work, but
|
||||
starts nothing. An unregistered historical entry kind is reported, not rejected: its stored
|
||||
`model`, `head` and `edits` still build context, while its typed data and custom renderer are
|
||||
unavailable. A live task kind is still required.
|
||||
|
||||
```typescript
|
||||
const { start, inflight } = await h.inspect();
|
||||
|
|
@ -284,13 +286,14 @@ definitions). pico is built on that, and keeps two things apart:
|
|||
- **What you want right now.** Config (model, thinking, selected tools, a profile) plus whatever
|
||||
the host has: cwd, skills, context files, the tool catalogue. The host answers a hook with it
|
||||
each turn. None of it is stored.
|
||||
- **What the model has been told.** `system` entries in the transcript: the baseline that opened the
|
||||
current context and one entry per change delivered since. Immutable facts about requests that
|
||||
happened, so they fork and compact like every other entry.
|
||||
- **What the model has been told.** `system` entries in the transcript: structured `data` for the
|
||||
baseline or delta and the exact materialized `SystemMessage` in `model`. These are immutable facts
|
||||
about requests that happened, so they fork and compact like every other entry.
|
||||
|
||||
At the top of every turn the generation renders the first, folds the second out of the context,
|
||||
and appends the difference, if any, before it projects the request. The host keeps rendering "the
|
||||
whole prompt" the way it always did; the harness turns that into append-only deltas.
|
||||
At the top of every turn the generation renders the first, folds the second out of the context, and
|
||||
appends the difference, if any, with both its structured data and model message before it projects
|
||||
the request. The host keeps rendering "the whole prompt" the way it always did; the harness turns
|
||||
that into append-only deltas.
|
||||
|
||||
### Answering the Hook
|
||||
|
||||
|
|
@ -318,9 +321,9 @@ After the first turn the transcript holds what was sent:
|
|||
12 assistant ...
|
||||
```
|
||||
|
||||
The tool definitions are stored in full, not as names. Rendering a persisted message may depend
|
||||
only on what is stored on it, so a later change to a tool's description is a change the model gets
|
||||
told about, and an old transcript renders the way it was sent.
|
||||
The tool definitions are stored in full, not as names, and the model message is materialized in the
|
||||
same append. A later change to a tool's description is therefore a change the model gets told about,
|
||||
and an old transcript is sent the way it was originally recorded without running entry-kind code.
|
||||
|
||||
### Changing the Loadout
|
||||
|
||||
|
|
@ -436,7 +439,7 @@ const answer = await c.prompt('Inspect the parser');
|
|||
// AssistantEntry | undefined (the run ended without an answer: aborted, failed)
|
||||
```
|
||||
|
||||
The pieces are available separately, and they compose across retries. `accept` is idempotent by
|
||||
The pieces are available separately, and they work across retries. `accept` is idempotent by
|
||||
`requestId`, and `answerTo` finds the answer to an input however far the conversation has moved
|
||||
since, so a client that lost the reply can ask again:
|
||||
|
||||
|
|
@ -622,7 +625,7 @@ await alt.prompt('Try a different implementation');
|
|||
const back = await c.fork(earlier.id, { abort: true }); // "go back": aborts the source's foreground first
|
||||
```
|
||||
|
||||
Any content entry is a valid fork point, including an assistant with unanswered tool calls or one
|
||||
Any transcript entry is a valid fork point, including an assistant with unanswered tool calls or one
|
||||
of several results. The request projection supplies missing results for a successful incomplete
|
||||
exchange without inheriting or executing the source tasks. Which conversation a UI treats as
|
||||
"current" is the UI's business; the harness only has conversations.
|
||||
|
|
@ -635,10 +638,11 @@ const children = await h.conversations({ parent: c.id }); // forks and
|
|||
|
||||
## Compaction and Reset
|
||||
|
||||
Compaction appends a summary head whose content names the first retained entry. The context becomes
|
||||
the summary followed by the transcript from that entry. It runs as a background task and may run
|
||||
while the model keeps working: ordinary entries landing meanwhile remain after the prepared retained
|
||||
boundary. Only a competing head makes the summary stale; edit entries do not.
|
||||
Compaction appends a summary entry with its model message and first retained entry id stored on the
|
||||
entry. The context becomes the summary followed by the transcript from that boundary. It runs as a
|
||||
background task and may run while the model keeps working: ordinary entries landing meanwhile
|
||||
remain after the prepared boundary. Only a competing head makes the summary stale; edit entries do
|
||||
not.
|
||||
|
||||
```typescript
|
||||
const collapseId = await c.collapse();
|
||||
|
|
@ -752,7 +756,7 @@ inside it see committed state; ids are final when returned; a throw discards eve
|
|||
```typescript
|
||||
const entry = await c.commit(tx => {
|
||||
tx.value(planMode).set(false); // rewindable state first
|
||||
const id = tx.entry(myPlugin.noteKind, { text: 'plan accepted' });
|
||||
const id = tx.entry(myPlugin.noteKind, { data: { text: 'plan accepted' } });
|
||||
tx.value(expanded).set(true); // sticky state may follow
|
||||
tx.task(myPlugin.reminderKind, { background: true, // tasks anywhere
|
||||
state: { about: id, at: Date.now() + 3600_000 } });
|
||||
|
|
@ -825,8 +829,9 @@ its own conversation through `ctx.conversation.commit(...)`: a job, a child conv
|
|||
Anything that is *about* the call rather than its output goes through `diag`: truncation, a spilled
|
||||
file, a corrected path, "the file changed on disk since you read it". The harness emits the ones it
|
||||
owns (the sink itself reports truncation and spill); a tool adds only what it alone knows.
|
||||
Projection puts the data first and the commentary after it, delimited, so the model can treat tool
|
||||
output as data, and a UI renders diagnostics as callouts by severity:
|
||||
Tool settlement puts the output first and the commentary after it and stores that exact message in
|
||||
entry `model`. Non-message details, usage, control flags, diagnostics and truncation metadata become
|
||||
entry `data`. A UI combines the two and renders diagnostics as callouts by severity:
|
||||
|
||||
```text
|
||||
...last matching line
|
||||
|
|
@ -914,56 +919,78 @@ effects need their own idempotence.
|
|||
|
||||
## Writing Kinds
|
||||
|
||||
Kinds are how you extend the harness with new behaviour rather than new state. An entry kind says
|
||||
what a kind of entry means to the model; a task kind says how a kind of work runs.
|
||||
Kinds are how you extend the harness with new behaviour rather than new state. An entry kind is a
|
||||
typed name for an immutable transcript shape; a task kind says how a kind of work runs.
|
||||
|
||||
### Entry Kinds
|
||||
|
||||
Entry facets combine. `data` is optional kind-specific JSON for logic and custom UI rendering;
|
||||
`model` is an optional stored `Message[]`; `head` and `edits` are optional stored context controls.
|
||||
No facet is derived while reading.
|
||||
|
||||
```typescript
|
||||
interface NoteContent { text: string; pinned?: boolean }
|
||||
interface NoteData { text: string; pinned?: boolean }
|
||||
type NoteEntry = EntryBase & EntryData<NoteData>;
|
||||
|
||||
// the model never sees this kind: no `project`
|
||||
export const noteKind = defineEntryKind<NoteContent>({ kind: 'myplugin.note' });
|
||||
export const noteKind = defineEntryKind<NoteEntry>('myplugin.note');
|
||||
|
||||
// the model sees this one
|
||||
export const pinnedKind = defineEntryKind<NoteContent>({
|
||||
kind: 'myplugin.pinned',
|
||||
project: e => [{ role: 'user', content: `<pinned>${e.content.text}</pinned>`, timestamp: 0 }],
|
||||
await c.commit(tx => tx.entry(noteKind, {
|
||||
data: { text: 'Parser plan accepted', pinned: false },
|
||||
})); // transcript/UI only; the model sees nothing
|
||||
```
|
||||
|
||||
A plugin that wants both typed data and a model message keeps the append-time conversion in an
|
||||
ordinary helper:
|
||||
|
||||
```typescript
|
||||
type PinnedEntry = EntryBase & EntryData<NoteData> & ModelProjection<UserMessage>;
|
||||
export const pinnedKind = defineEntryKind<PinnedEntry>('myplugin.pinned');
|
||||
|
||||
function appendPinned(tx: ConversationTx, data: NoteData, timestamp: number): Id {
|
||||
return tx.entry(pinnedKind, {
|
||||
data,
|
||||
model: [{ role: 'user', content: `<pinned>${data.text}</pinned>`, timestamp }],
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Heads and edits are supplied the same way:
|
||||
|
||||
```typescript
|
||||
type WindowEntry = EntryBase & EntryData<{ retainFrom: Id }> & ContextHead;
|
||||
export const windowKind = defineEntryKind<WindowEntry>('myplugin.window');
|
||||
|
||||
tx.entry(windowKind, {
|
||||
data: { retainFrom },
|
||||
head: retainFrom, // first retained entry, inclusive
|
||||
});
|
||||
|
||||
// a head kind: prepend this entry, then retain the transcript from retainFrom
|
||||
export const windowKind = defineEntryKind<{ retainFrom: Id }>({
|
||||
kind: 'myplugin.window',
|
||||
head: entry => entry.content.retainFrom,
|
||||
});
|
||||
type ToolResultEditEntry = EntryBase & ContextEdits;
|
||||
export const toolResultEditKind = defineEntryKind<ToolResultEditEntry>('myplugin.tool_result_edit');
|
||||
|
||||
type ToolResultEdit = { target: Id; messages?: Message[] };
|
||||
export const toolResultEditKind = defineEntryKind<ToolResultEdit>({
|
||||
kind: 'myplugin.tool_result_edit',
|
||||
edit: entry => entry.content.messages === undefined
|
||||
? [{ target: entry.content.target, action: 'omit' }]
|
||||
: [{ target: entry.content.target, action: 'replace', messages: entry.content.messages }],
|
||||
tx.entry(toolResultEditKind, {
|
||||
edits: replacement === undefined
|
||||
? [{ target, action: 'omit' }]
|
||||
: [{ target, action: 'replace', messages: replacement }],
|
||||
});
|
||||
```
|
||||
|
||||
The presence of `head` makes writes of that kind persist `Entry.head = true`, which supports an
|
||||
indexed newest-head lookup. The function itself returns the first retained entry id. Commit
|
||||
validation requires that value not move before the previous visible head's returned value. The
|
||||
latest head is prepended and older heads are excluded from its retained range.
|
||||
|
||||
`edit` interprets immutable entry content as projection operations. Edits are applied in transcript
|
||||
order; the newest edit for a target wins. They omit or replace individual retained projections
|
||||
without changing the target entry. There is no dynamic `compose`: append another head or edit when
|
||||
context should change.
|
||||
`head: 'self'` in an append draft stores the new entry id, which is how reset and handoff discard
|
||||
everything earlier. Commit validation requires a stored head boundary not to move before the
|
||||
previous visible boundary. Edits apply in transcript order; the newest edit per target wins. Append
|
||||
another head or edit when context should change; no entry-kind callback runs during context reads.
|
||||
|
||||
Reads are typed by the kind, or untyped and narrowed:
|
||||
|
||||
```typescript
|
||||
const note = await h.getEntry(noteKind, id); // Entry<NoteContent> | undefined (also undefined for another kind)
|
||||
const any = await h.getEntry(id);
|
||||
if (noteKind.is(any)) any.content.pinned;
|
||||
const note = await h.getEntry(noteKind, id); // NoteEntry | undefined (also undefined for another kind)
|
||||
const any = await h.getEntry(id); // Entry | undefined
|
||||
if (noteKind.is(any)) any.data.pinned;
|
||||
```
|
||||
|
||||
A missing plugin removes that narrowing and its custom renderer, but not context behavior: the
|
||||
entry's model messages, head and edits are stored independently of the kind.
|
||||
|
||||
### Task Kinds
|
||||
|
||||
A task kind declares its statuses and their roles, its config, its hooks, and three functions.
|
||||
|
|
@ -985,7 +1012,7 @@ export const reminderKind = defineTaskKind({
|
|||
const { skip } = await ctx.hooks(reminderKind).run('before_fire', { about: task.state.about });
|
||||
await ctx.commit(tx => {
|
||||
if (!skip && !tx.getTask(task.id)!.abort) // a marked task creates no successor
|
||||
tx.entry(noticeKind, { text: `Reminder: see entry ${task.state.about}` });
|
||||
tx.entry(noticeKind, { model: [noticeMessage(`Reminder: see entry ${task.state.about}`)] });
|
||||
tx.settle(task.id, 'done', { ...task.state, fired: !skip });
|
||||
});
|
||||
},
|
||||
|
|
|
|||
|
|
@ -52,134 +52,157 @@ are never model input by themselves.
|
|||
```ts
|
||||
type Id = number; // a session sequence number; one alias so the representation can change
|
||||
|
||||
interface Entry<Content = JsonValue> {
|
||||
readonly id: Id; // its sequence number
|
||||
readonly conversationId: Id;
|
||||
readonly kind: string;
|
||||
readonly content: Content;
|
||||
readonly key?: string; // optional indexed key, e.g. the tool call a result answers
|
||||
readonly head?: true; // this kind is a context head; indexed
|
||||
readonly byTaskId?: Id; // which task wrote it
|
||||
}
|
||||
```
|
||||
|
||||
Entries append. They are never patched, reordered or renumbered. Ids increase in transcript order
|
||||
but need not be consecutive.
|
||||
|
||||
Kinds are code, not data: the built-ins are registered by `Harness.open`, plugins add theirs
|
||||
next to the task kinds (§5.2) and tools (§9.5). Storage sees only the `kind` string on each entry and never runs kind code; the scan
|
||||
and projection look the string up in the registry.
|
||||
|
||||
```ts
|
||||
type ContextEdit =
|
||||
| { readonly target: Id; readonly action: "omit" }
|
||||
| { readonly target: Id; readonly action: "replace"; readonly messages: readonly Message[] };
|
||||
|
||||
interface EntryKind<Content> {
|
||||
interface EntryIdentity {
|
||||
readonly id: Id; // its sequence number
|
||||
readonly conversationId: Id;
|
||||
readonly kind: string;
|
||||
project?(entry: Entry<Content>): readonly Message[]; // absent: the model never sees this kind
|
||||
head?(entry: Entry<Content>): Id; // first retained entry, inclusive
|
||||
edit?(entry: Entry<Content>): readonly ContextEdit[];
|
||||
is(entry: Entry): entry is Entry<Content>;
|
||||
readonly byTaskId?: Id; // which task wrote it
|
||||
}
|
||||
|
||||
interface EntryBase extends EntryIdentity {
|
||||
readonly key?: string; // optional indexed key, e.g. the tool call a result answers
|
||||
}
|
||||
|
||||
interface EntryData<Data extends JsonValue = JsonValue> {
|
||||
readonly data: Data; // kind-specific durable data; never model input by itself
|
||||
}
|
||||
|
||||
interface ModelProjection<Model extends Message = Message> {
|
||||
readonly model: readonly Model[]; // provider-neutral messages materialized at append
|
||||
}
|
||||
|
||||
interface ContextHead {
|
||||
readonly head: Id; // first retained transcript entry, inclusive
|
||||
}
|
||||
|
||||
interface ContextEdits {
|
||||
readonly edits: readonly ContextEdit[];
|
||||
}
|
||||
|
||||
type Entry = EntryBase & Partial<
|
||||
EntryData & ModelProjection & ContextHead & ContextEdits
|
||||
>;
|
||||
|
||||
type UserEntry = EntryBase & ModelProjection<UserMessage>;
|
||||
type AssistantEntry = EntryBase & ModelProjection<AssistantMessage>;
|
||||
|
||||
interface SummaryData { readonly summarizedThrough: Id }
|
||||
type SummaryEntry = EntryBase & EntryData<SummaryData> & ModelProjection<UserMessage> & ContextHead;
|
||||
```
|
||||
|
||||
The facets combine. A user entry may be only `EntryBase & ModelProjection<UserMessage>`; a tool
|
||||
result may also carry non-message `EntryData<ToolResultData>` for its renderer; a summary has data,
|
||||
a model projection and a head; a reset needs only a head. Entries append and are never patched,
|
||||
reordered or renumbered. Ids increase in transcript order but need not be consecutive.
|
||||
|
||||
`model` is the exact provider-neutral message projection chosen by the writer. It is not the final
|
||||
provider request: context selection, edit folding, tool-exchange normalization and provider
|
||||
conversion still happen per request. `data` is optional arbitrary kind-specific JSON for typed
|
||||
logic and rendering. Nothing implicitly derives one from the other, so storing both never requires
|
||||
duplicating a message.
|
||||
|
||||
An entry kind is only a name and type witness:
|
||||
|
||||
```ts
|
||||
interface EntryKind<E extends Entry = Entry> {
|
||||
readonly kind: string;
|
||||
is(entry: Entry): entry is E;
|
||||
}
|
||||
```
|
||||
|
||||
`project` turns an entry into the messages the model receives. A kind without it is invisible to the
|
||||
model: it is in the transcript and never in the context.
|
||||
A plugin normally keeps append-time construction in a typed helper. It may compute `model`, `head`
|
||||
and `edits` from its data before calling `Tx.entry`, but no such computation is registered on the
|
||||
kind or run while reading. Asynchronous inputs are prepared before entering the commit line.
|
||||
|
||||
A kind with `head` defines a context head. `Tx.entry` stores only `head: true`, so storage can index
|
||||
heads without understanding the kind. When deriving context, the harness calls `head(entry)` to get
|
||||
the first retained entry id, inclusive. A summary returns the first entry it did not summarize;
|
||||
`handoff` and `reset` return their own id, so nothing earlier is retained. A plugin can define its
|
||||
own head kind.
|
||||
|
||||
For every new head:
|
||||
A stored `head` is the inclusive first retained entry id. Presence of the column marks and indexes
|
||||
the entry as a head. `Tx.entry` accepts `"self"` for reset and handoff drafts and stores the minted
|
||||
entry id. Every new head must satisfy:
|
||||
|
||||
```text
|
||||
newKind.head(newHead) >= oldKind.head(previousVisibleHead)
|
||||
newHead.head >= previousVisibleHead.head
|
||||
```
|
||||
|
||||
The commit line loads and interprets both entries and enforces this. Plugin code supplies the
|
||||
candidate but cannot move context backwards. Older heads are controls, not retained context entries:
|
||||
the newest head replaces them and carries any predecessor summary information it still needs.
|
||||
The commit line validates this directly from stored entries. A head cannot move context backwards.
|
||||
Older heads are controls, not retained context entries: the newest head replaces them and carries
|
||||
any predecessor summary information it still needs.
|
||||
|
||||
A kind with `edit` defines a context edit. Its immutable entry content is interpreted as standard
|
||||
operations that omit or replace the projection of earlier visible entries. Edits are folded in
|
||||
transcript order; the newest applicable edit for a target wins. The target entry itself is never
|
||||
changed, and an edit whose target is outside the selected range is a no-op. Retained edit entries
|
||||
are applied again on later turns. Tool-result pruning and shortening are edits. Dynamic policies
|
||||
such as "always keep only the last N exchanges" are unsupported; code appends a new head or edit
|
||||
when it wants context to change. There is no `compose` callback.
|
||||
Stored `edits` omit or replace the model projection of earlier visible entries. Edits are folded in
|
||||
transcript order; the newest applicable edit for a target wins. The target entry is never changed,
|
||||
and an edit whose target is outside the selected range is a no-op. Retained edit entries apply again
|
||||
on later turns. Tool-result pruning and shortening are edits. Dynamic policies such as "always keep
|
||||
only the last N exchanges" are unsupported; code appends a new head or edit when context should
|
||||
change. There is no `compose` callback or read-time entry-kind behavior.
|
||||
|
||||
The built-ins are `user`, `assistant`, `tool_result`, `system`, `notice`, `summary`, `handoff` and
|
||||
`reset`.
|
||||
All have `project` except `reset`, which starts an empty context; `handoff` starts a context with
|
||||
its own message; `system` projects to pi-ai's `SystemMessage`, each at its position in the
|
||||
transcript, so the prompt in force at any entry is on record and a provider's cached prefix is
|
||||
never rewritten:
|
||||
`reset`. Their writers materialize model messages when they append. `reset` stores no model;
|
||||
`handoff` stores its message and starts a context; `system` stores the exact pi-ai `SystemMessage`
|
||||
at its transcript position, so the prompt in force at any entry is on record and a provider's
|
||||
cached prefix is never rewritten:
|
||||
|
||||
```ts
|
||||
interface SystemContent {
|
||||
interface SystemData {
|
||||
readonly baseline?: true; // opens an epoch (§8.2)
|
||||
readonly sections?: readonly { key: string; text: string }[]; // in render order; baseline: all; delta: the changed ones
|
||||
readonly toolsAdded?: readonly Tool[]; // complete definitions as sent, never names resolved later
|
||||
readonly toolsRemoved?: readonly Tool[];
|
||||
}
|
||||
|
||||
type SystemEntry = EntryBase & EntryData<SystemData> & ModelProjection<SystemMessage>;
|
||||
```
|
||||
|
||||
Definitions are stored, not looked up, and sections are stored in the order they were rendered,
|
||||
because rendering a persisted message must depend only on what is stored on it; a changed
|
||||
description, schema or section order is therefore a delta like any other. `project` turns the
|
||||
entry into pi-ai's `SystemMessage` at request time: a baseline's sections into the top-level prompt
|
||||
(or one system message where there is no baseline slot), a delta's into "the `plan_mode` guidance
|
||||
now applies: …" with `toolsAdded`/`toolsRemoved` attached. Plugin-defined kinds usually have no `project`. The
|
||||
kind is the type witness for reads and writes, so nothing is cast and nothing is validated:
|
||||
stored objects are trusted (§7.3), and a kind that changes its shape ships a migration.
|
||||
Definitions and sections are stored in `data` for the next generation's structured comparison and
|
||||
in the materialized message in `model`. A changed description, schema or section order is therefore
|
||||
a delta like any other. Plugin-defined entries omit `model` when the model should not see them.
|
||||
The kind types reads and writes; stored objects are trusted after wire validation (§7.3), and a kind
|
||||
that changes its data shape ships a migration.
|
||||
|
||||
```ts
|
||||
tx.entry(toolResultKind, conversationId, content); // content: ToolResultContent
|
||||
const e = await harness.getEntry(summaryKind, id); // Entry<SummaryContent>
|
||||
const any = await harness.getEntry(id); // Entry<JsonValue>
|
||||
if (summaryKind.is(any)) any.content.through; // narrowed
|
||||
type ToolResultEntry = EntryBase & EntryData<ToolResultData> & ModelProjection<ToolResultMessage>;
|
||||
const toolResultKind = defineEntryKind<ToolResultEntry>("tool_result");
|
||||
|
||||
tx.entry(toolResultKind, conversationId, { data: resultData, model: [message], key: callId });
|
||||
const e = await harness.getEntry(summaryKind, id); // SummaryEntry | undefined
|
||||
const any = await harness.getEntry(id); // Entry | undefined
|
||||
if (summaryKind.is(any)) any.data.summarizedThrough; // narrowed
|
||||
```
|
||||
|
||||
The kind is optional on reads: without it you get the untyped entry, and `kind.is(entry)` narrows
|
||||
it. A typed read of an entry of another kind returns `undefined`, the same as a missing entry;
|
||||
batched reads omit it.
|
||||
|
||||
Open checks the set of entry kinds ever written against the registry: on SQLite that is a `kinds`
|
||||
table with one row per distinct kind, filled by `INSERT OR IGNORE` in the commit that first uses a
|
||||
kind; on JSONL and memory it falls out of the replay open does anyway. No scan of entries; an
|
||||
unknown kind rejects before anything runs.
|
||||
Without a kind, a read returns the untyped entry. A typed read of another kind returns `undefined`,
|
||||
the same as a missing entry; batched reads omit it. The session records the distinct kind strings
|
||||
without scanning entries. Open reports an unregistered entry kind but does not reject: stored
|
||||
`model`, `head` and `edits` preserve context, while only kind-specific typing and rendering are
|
||||
unavailable.
|
||||
|
||||
### 2.2 Context is derived, not stored
|
||||
|
||||
The generation derives context from the transcript:
|
||||
|
||||
```text
|
||||
H = newest head visible at the request/fork target (one indexed query)
|
||||
from = no H → transcript start; otherwise kind(H).head(H)
|
||||
H = newest entry with stored `head` visible at the request/fork target
|
||||
from = no H → transcript start; otherwise H.head
|
||||
range = fork-aware transcript entries from `from` through target, inclusive
|
||||
controls= fold edit entries in transcript order
|
||||
controls= fold stored `edits` in transcript order
|
||||
context = no H → eligible range entries
|
||||
H exists → H, then eligible range entries excluding every head
|
||||
omitted targets disappear; replaced targets keep their id and use replacement messages
|
||||
messages= project through each entry's kind; normalize tool exchanges for the selected model
|
||||
messages= concatenate stored `model` arrays; normalize tool exchanges for the selected model
|
||||
```
|
||||
|
||||
Edit entries are controls and do not project unless their kind separately defines `project`.
|
||||
Entries whose kind has neither control behavior nor `project` contribute nothing. Existing tool
|
||||
results are placed with their calls in call order. For a successful assistant cut before all its
|
||||
results, request-local pi-ai transformation supplies missing results; results beyond the fork cutoff
|
||||
and source tasks are never inherited. Error/aborted assistant outputs are excluded from subsequent
|
||||
requests and never create tool tasks/results.
|
||||
An entry without `model` contributes no messages. An edit entry may independently have a model
|
||||
projection. Existing tool results are placed with their calls in call order. For a successful
|
||||
assistant cut before all its results, request-local pi-ai transformation supplies missing results;
|
||||
results beyond the fork cutoff and source tasks are never inherited. Error/aborted assistant
|
||||
outputs are excluded from subsequent requests and never create tool tasks/results.
|
||||
|
||||
No context list is stored, so nothing can drift from the transcript. A conversation handle keeps an
|
||||
optional process-local cache of the current derived entries, candidate membership and winning edits.
|
||||
`contextEntries(through)` loads it once, catches it up with entries after its cursor, slices it when
|
||||
a new head arrives and folds new edits. Generation tasks receive that handle; `ConversationView`
|
||||
exposes the resulting context ids. A historical target older than the cache is derived separately
|
||||
and does not rewind it. Each request receives an immutable snapshot.
|
||||
`contextEntries(through)` loads it once, catches it up with entries after its cursor, slices it from
|
||||
the stored boundary when a new head arrives and folds stored edits. Generation tasks receive that
|
||||
handle; `ConversationView` exposes the resulting context ids. A historical target older than the
|
||||
cache is derived separately and does not rewind it. Each request receives an immutable snapshot.
|
||||
|
||||
On a cold handle, cost is the fork-aware candidate range plus edit folding and fork depth, not
|
||||
necessarily the final projected-message count. Memory and JSONL answer from their in-memory indexes;
|
||||
|
|
@ -188,12 +211,11 @@ convenience owned by the handle and may be garbage-collected with it; the transc
|
|||
|
||||
### 2.3 Compaction and reset
|
||||
|
||||
**Compaction** appends one summary head. Its typed content names the first retained entry, and its
|
||||
kind returns that id:
|
||||
**Compaction** appends one summary with a stored model projection and head boundary:
|
||||
|
||||
```text
|
||||
transcript [10 user, 20 asst, 30 user, 40 asst] context [10, 20, 30, 40]
|
||||
append 50 summary { retainFrom: 30 }, head:true
|
||||
append 50 summary { model:[summary], head:30 }
|
||||
scan [30 user, 40 asst, 50 summary]
|
||||
context [50 summary, 30 user, 40 asst]
|
||||
```
|
||||
|
|
@ -205,9 +227,9 @@ may still land later: ordinary entries appended while it runs are at or after it
|
|||
boundary and remain in the range. Only a competing head invalidates it; intervening context edits
|
||||
do not (§8.4).
|
||||
|
||||
**Reset** appends a `handoff` head (with a message the model sees) or a `reset` head (nothing).
|
||||
Their `head(entry)` returns `entry.id`. The transcript keeps everything. Reset by itself neither
|
||||
cancels tasks nor requests a response.
|
||||
**Reset** appends a `handoff` head with `model` or a `reset` head without it. Both pass
|
||||
`head: "self"` to `Tx.entry`, which stores the new entry id as the boundary. The transcript keeps
|
||||
everything. Reset by itself neither cancels tasks nor requests a response.
|
||||
|
||||
### 2.4 Forks share history, not future changes
|
||||
|
||||
|
|
@ -254,7 +276,7 @@ One session-wide sequence orders every mutation. A creation's sequence number is
|
|||
// last committed: 99
|
||||
const entryId = await conv.commit(tx => {
|
||||
tx.value(planMode).set(true); // 100
|
||||
const id = tx.entry(userKind, message); // 101
|
||||
const id = tx.entry(userKind, { model: [message] }); // 101
|
||||
tx.task(generationKind, { state: { input: id } }); // 102
|
||||
return id;
|
||||
});
|
||||
|
|
@ -280,7 +302,7 @@ history through X:
|
|||
// tool settlement, one commit
|
||||
await ctx.commit(tx => {
|
||||
tx.value(planMode).set(true); // 300
|
||||
const result = tx.entry(toolResultKind, content); // 301
|
||||
const result = tx.entry(toolResultKind, { data: resultData, model: [message] }); // 301
|
||||
tx.settle(task, { result }); // 302
|
||||
});
|
||||
|
||||
|
|
@ -294,7 +316,7 @@ before persistence and consumes no ids. Session state and sticky conversation st
|
|||
anywhere because forks do not reconstruct their history, so they may reference ids minted earlier
|
||||
in the same commit. Tasks also go anywhere; they are not inherited by forks and usually come after
|
||||
the entries they name. Historical reads (§4) take an entry as their position; "before the first
|
||||
entry" is the empty position. Any content entry is a valid fork point, including an assistant call
|
||||
entry" is the empty position. Any transcript entry is a valid fork point, including an assistant call
|
||||
or one of several tool results. Projection repairs the resulting successful incomplete exchange for
|
||||
the request only; it never inherits or executes the source tasks.
|
||||
|
||||
|
|
@ -423,7 +445,7 @@ const partial = await ctx.scratch(sc => sc.list(frames).read());
|
|||
|
||||
// settlement: the result becomes an entry; settle retires the scope in the same commit
|
||||
await ctx.commit(tx => {
|
||||
const id = tx.entry(assistantKind, assemble(partial));
|
||||
const id = tx.entry(assistantKind, { model: [assemble(partial)] });
|
||||
tx.settle(task, { result: id });
|
||||
});
|
||||
```
|
||||
|
|
@ -587,7 +609,7 @@ The generation's settlement publishes the exchange atomically:
|
|||
|
||||
```ts
|
||||
await ctx.commit(tx => {
|
||||
const assistant = tx.entry(assistantKind, message); // calls [A, B]
|
||||
const assistant = tx.entry(assistantKind, { model: [message] }); // calls [A, B]
|
||||
const tools = message.calls.map(call =>
|
||||
tx.task(toolKind, { state: { call, assistant } }));
|
||||
tx.task(postToolsKind, { after: tools, state: { assistant, tools } });
|
||||
|
|
@ -610,7 +632,9 @@ async execute(task, ctx) {
|
|||
const selectedTools = added.length ? [...current, ...added] : current;
|
||||
if (added.length) tx.value(generationKind.config.selectedTools).set(selectedTools);
|
||||
const handoff = outcomes.find(o => o.handoff);
|
||||
if (handoff) tx.entry(handoffKind, handoff.handoff);
|
||||
if (handoff) tx.entry(handoffKind, {
|
||||
data: { text: handoff.handoff }, model: [handoffMessage(handoff.handoff)], head: "self",
|
||||
});
|
||||
landSteering(tx, task.conversationId); // §8.1
|
||||
tx.task(generationKind, { state: nextGeneration(task, { selectedTools }) });
|
||||
tx.settle(task, {});
|
||||
|
|
@ -824,9 +848,10 @@ successor. Background work: `abortTask(job)`, or `abort(childConversation)` for
|
|||
|
||||
### 6.4 Open, close, shutdown, delete
|
||||
|
||||
**Open** opens storage, checks the set of entry kinds written (§2.1) and the live tasks' kinds
|
||||
against the registry, exposes `inspect()` (live tasks by role) and queries,
|
||||
starts nothing. **Close** stops admitting commits, cancels owned executions, waits for them and for
|
||||
**Open** opens storage, reports unregistered entry kind strings (§2.1), checks live task kinds
|
||||
against the registry, exposes `inspect()` (live tasks by role) and queries, and starts nothing.
|
||||
Unknown entries remain fully usable for context from their stored facets; an unknown live task kind
|
||||
still rejects. **Close** stops admitting commits, cancels owned executions, waits for them and for
|
||||
admitted persistence, releases storage; it writes no outcomes, so unfinished work resumes on the
|
||||
next open. **Shutdown** marks all live work, drains queued input, drives cancellation to settlement,
|
||||
then closes. **Delete** of a conversation rejects while its ownership subtree has live work, then
|
||||
|
|
@ -889,7 +914,7 @@ interface TaskQuery extends Cursor { readonly conversationIds: readonly
|
|||
interface ListQuery extends Cursor { readonly at?: Id } // at = an entry id (§3.2)
|
||||
interface ValueQuery { readonly scope: Scope; readonly namespace: string; readonly after?: string; readonly limit: number } // keys, ordered
|
||||
|
||||
type EntryHeader = Omit<Entry, "content">;
|
||||
type EntryHeader = Omit<Entry, "data">; // context fields without arbitrary plugin data
|
||||
interface Version<T> { readonly seq: Id; readonly value: T }
|
||||
interface Element<T> { readonly id: Id; readonly value: T }
|
||||
|
||||
|
|
@ -924,7 +949,7 @@ Pages carry a cursor and the sequence they were read at. Transcript reads are lo
|
|||
reads: they combine each fork's capped source prefix with local entries, carrying every ancestor
|
||||
cutoff recursively. `from` and `through` are inclusive entry positions in that logical transcript;
|
||||
source entries after a fork point never appear. `newestHead(conversationId, at)` applies the same
|
||||
cutoffs and returns the newest visible `head:true` entry at or before `at`. Rewindable conversation
|
||||
cutoffs and returns the newest visible entry with `head` at or before `at`. Rewindable conversation
|
||||
state follows the same ancestor cutoffs. Session state stays current; sticky conversation state is
|
||||
copied from current state only when the fork explicitly selects it. Tasks are never inherited.
|
||||
Historical reads on sticky or scratch addresses reject. Coherent multi-read operations run on the
|
||||
|
|
@ -963,11 +988,11 @@ from there; the line guarantees nothing landed in between, so storage numbering
|
|||
storage check that for free. The harness validates against
|
||||
committed state plus earlier writes in the batch (kinds validate status transitions; the harness
|
||||
enforces ownership, dependencies, exchange rules and admission; nobody validates payload shapes:
|
||||
stored objects are trusted, and schema validation belongs at wire boundaries). For a `head:true`
|
||||
entry, validation calls its kind's `head(entry)`, requires the returned id to be visible, and requires
|
||||
it to be at or after the previous fork-visible head kind's returned id. Edit targets must be earlier
|
||||
visible entries. Storage checks scope, sequences and structure, prepares only touched data, persists
|
||||
atomically, publishes. A commit with no writes writes nothing.
|
||||
stored objects are trusted, and schema validation belongs at wire boundaries). For an entry with
|
||||
`head`, validation requires the stored id to be visible and at or after the previous fork-visible
|
||||
head's stored boundary. Stored edit targets must be earlier visible entries. No entry-kind code runs.
|
||||
Storage checks scope, sequences and structure, prepares only touched data, persists atomically,
|
||||
publishes. A commit with no writes writes nothing.
|
||||
|
||||
### 7.4 Memory and JSONL
|
||||
|
||||
|
|
@ -979,7 +1004,7 @@ one line per batch:
|
|||
```json
|
||||
{"first":100,"writes":[
|
||||
{"type":"value.set","conversationId":1,"namespace":"plugin.plan","value":true},
|
||||
{"type":"entry","conversationId":1,"kind":"user","content":"Inspect"},
|
||||
{"type":"entry","conversationId":1,"kind":"user","model":[{"role":"user","content":"Inspect","timestamp":0}]},
|
||||
{"type":"task","conversationId":1,"kind":"generation","status":"pending"}]}
|
||||
```
|
||||
|
||||
|
|
@ -1005,12 +1030,15 @@ must never remove a deliberately reused scope.
|
|||
|
||||
Tables: conversations, entries, tasks (current records including terminal), values and
|
||||
value_versions, list_elements and clear_markers, scratch, commit_boundaries, kinds (one row per
|
||||
distinct entry kind), session_metadata (storage version, §6.5). Keys start with the session id. Indexes follow the
|
||||
query table: entries by (conversation, id), (conversation, kind, key, id) and a partial index
|
||||
(conversation, id) where head is set; tasks by (role, id) and (conversation, role, id) with a
|
||||
distinct entry kind), session_metadata (storage version, §6.5). An entry row has nullable JSON
|
||||
columns for `data`, `model` and `edits`, plus a nullable integer `head`; storage understands those
|
||||
core fields but never a plugin's data shape. Keys start with the session id. Indexes follow the query
|
||||
table: entries by (conversation, id), (conversation, kind, key, id) and a partial index
|
||||
(conversation, id) where head is not null; tasks by (role, id) and (conversation, role, id) with a
|
||||
partial index on non-terminal roles; value_versions by (conversation, namespace, key, seq);
|
||||
conversations by parent and by owner. One commit is one transaction: number the writes from `lastSeq + 1`,
|
||||
insert rows, record the boundary, advance `lastSeq`. No full rewrite of anything per append.
|
||||
conversations by parent and by owner. One commit is one transaction: number the writes from
|
||||
`lastSeq + 1`, insert rows, record the boundary, advance `lastSeq`. No full rewrite of anything per
|
||||
append.
|
||||
|
||||
Values are stored whole, current and historical: a point read must not become a replay chain, and
|
||||
disk is cheaper than that. Scratch is rows in the `scratch` table keyed by (task, address), written
|
||||
|
|
@ -1064,7 +1092,7 @@ execute: if a collapse is needed by threshold: { create collapse C; create g
|
|||
none of the inputs is stored
|
||||
sent = the newest baseline `system` entry in the context plus the deltas after it;
|
||||
deltas before it (kept by a compaction) are subsumed and ignored, by the fold
|
||||
and by projection alike
|
||||
and by request projection alike
|
||||
{ if no sent: system entry with the baseline (text + full tool definitions);
|
||||
else if sent ≠ desired: system entry with the diff, per section ("the plan_mode guidance
|
||||
now applies: …", "the cwd section changed: …") and toolsAdded / toolsRemoved as full
|
||||
|
|
@ -1091,10 +1119,10 @@ The generation diffs the two at the top of every turn and writes only the delta.
|
|||
handoff, reset) starts a new epoch: the context has no baseline after it, so the next generation
|
||||
writes a fresh one, and any older delta a compaction kept is subsumed by it and dropped at projection; a fork carries
|
||||
the sent-state in its prefix and diffs against its own config; a restart with a changed host
|
||||
emits "these sections now apply" and nothing else. Projection puts the epoch's baseline in the
|
||||
provider's baseline slot, later entries as system messages at their positions, and the folded tool
|
||||
set as the callable list, or folds everything into one top-level prompt on providers without
|
||||
native system messages.
|
||||
emits "these sections now apply" and nothing else. Request projection puts the epoch's stored
|
||||
baseline message in the provider's baseline slot, later stored system messages at their positions,
|
||||
and the folded tool set as the callable list, or folds
|
||||
everything into one top-level prompt on providers without native system messages.
|
||||
|
||||
One task lives through retries and deferral; retries share a budget carried in state. Overflow
|
||||
does not keep the generation alive: it settles and hands the retry to the collapse's settlement,
|
||||
|
|
@ -1177,7 +1205,7 @@ create: capture the prefix to replace (ending on a complete exchange), its fir
|
|||
execute: { status summarizing }
|
||||
before_collapse (may decline or supply the summary); call the summarizer; candidate to scratch
|
||||
{ if no head newer than the captured one:
|
||||
append summary { retainFrom:firstRetained }, whose kind marks it head:true; settle done
|
||||
append summary { model:[summary], head:firstRetained }; settle done
|
||||
else: settle failed(stale) }
|
||||
```
|
||||
|
||||
|
|
@ -1187,8 +1215,8 @@ but do not invalidate or recompute the summary. A summary may land while a gener
|
|||
since the stream already projected its context and the replaced prefix is old. Automatic collapses
|
||||
(threshold, overflow) are foreground; manual ones are background. A collapse never creates a
|
||||
generation except through the settlement chain in §8.2. Abort records usage and settles without
|
||||
publishing. Prefix pruning/windowing uses another head kind; pruning or replacing individual
|
||||
retained entries uses edit entries (§2.1). Dynamic `compose` policies are unsupported.
|
||||
publishing. Prefix pruning/windowing appends another entry with a stored head; pruning or replacing
|
||||
individual retained entries uses edit entries (§2.1). Dynamic `compose` policies are unsupported.
|
||||
|
||||
### 8.5 Subagents
|
||||
|
||||
|
|
@ -1280,7 +1308,9 @@ const jobKind: TaskKind<JobState> = {
|
|||
if (ctx.signal.aborted) return tx.settle(task.id, "killed", { ...task.state, output });
|
||||
const r = getOrThrow(result);
|
||||
if (task.state.notify) // the call that started it returned early (§9.3): tell the model
|
||||
tx.entry(noticeKind, { text: `job ${task.id} (${task.state.origin?.tool}: ${task.state.cmd.join(" ")}) finished, exit ${r.exitCode}` });
|
||||
tx.entry(noticeKind, {
|
||||
model: [noticeMessage(`job ${task.id} (${task.state.origin?.tool}: ${task.state.cmd.join(" ")}) finished, exit ${r.exitCode}`)],
|
||||
});
|
||||
if (task.state.every) // recurring: same task, next time
|
||||
tx.patch(task.id, { status: "planned", state: { ...task.state, notBefore: Date.now() + task.state.every } });
|
||||
else
|
||||
|
|
@ -1297,8 +1327,8 @@ shell doesn't hand out an adoptable process, recover reruns if the job declared
|
|||
otherwise settles `lost`. A recurring job is the same task looping `planned → running → planned`;
|
||||
`abortTask(id)` ends it wherever it is. Output stays in scratch while it runs and in the terminal state
|
||||
after, where `jobOutput`, the `job` tool and UIs read it; there is no result entry, since a call that
|
||||
waited already carries the output. A job whose call returned early appends a `notice` entry (a kind
|
||||
that projects to a `SystemMessage`) when it finishes, so the next generation learns of it without
|
||||
waited already carries the output. A job whose call returned early appends a `notice` entry with a stored `SystemMessage` when it
|
||||
finishes, so the next generation learns of it without
|
||||
polling. Interval and catch-up policy are in state,
|
||||
so a restart never launches a backlog.
|
||||
|
||||
|
|
@ -1419,20 +1449,35 @@ the same answer however far the conversation has moved since. Nothing else maps
|
|||
```ts
|
||||
const entryId = await harness.commit(tx => {
|
||||
tx.value(planMode).set(true); // state first (§3.2)
|
||||
const id = tx.entry(noteKind, conversationId, content); // final id, inside the closure
|
||||
const id = tx.entry(noteKind, conversationId, {
|
||||
data: { text: "plan accepted" },
|
||||
model: [{ role: "user", content: "<note>plan accepted</note>", timestamp: 0 }],
|
||||
}); // final id, inside the closure
|
||||
tx.task(reminderKind, { background: true, state: { about: id } });
|
||||
return id; // any value; resolved after commit
|
||||
return id; // any value; resolved after commit
|
||||
});
|
||||
```
|
||||
|
||||
```ts
|
||||
type EntryInput<E extends Entry> =
|
||||
Omit<E, keyof EntryIdentity | "head"> &
|
||||
(E extends ContextHead ? { readonly head: Id | "self" } : { readonly head?: never });
|
||||
|
||||
interface EntryDraft {
|
||||
readonly key?: string;
|
||||
readonly data?: JsonValue;
|
||||
readonly model?: readonly Message[];
|
||||
readonly head?: Id | "self";
|
||||
readonly edits?: readonly ContextEdit[];
|
||||
}
|
||||
|
||||
interface Tx {
|
||||
// reads (committed state)
|
||||
getEntry(id) / getEntry(kind, id) / getEntries(...) / getTask(id) / getTask(kind, id) / getTasks(...)
|
||||
value<T>(addr: Value<T>): { get(at?): T | undefined; set(v: T): void; delete(): void };
|
||||
list<T>(addr: List<T>): { append(v: T): Id; remove(id: Id): void; clear(): void; read(q): Page<T> };
|
||||
// writes; rewindable conversation state before entries (§3.2)
|
||||
entry<C>(kind: EntryKind<C>, conversationId: Id, content: C): Id;
|
||||
entry<E extends Entry>(kind: EntryKind<E>, conversationId: Id, input: EntryInput<E>): Id;
|
||||
task<S>(kind: TaskKind<S>, spec: { state: S; after?: Id[]; background?: true; owns?: Id[] }): Id;
|
||||
patch<S>(task: Task<S> | Id, changes: { status?: string; state?: Partial<S> }): void;
|
||||
settle<S>(task: Task<S> | Id, outcome: Partial<S> & { status?: string }): void; // terminal; retires scratch
|
||||
|
|
@ -1473,12 +1518,13 @@ class Tx {
|
|||
};
|
||||
}
|
||||
|
||||
entry<C>(kind: EntryKind<C>, conversationId: number, content: C): number {
|
||||
entry(kind: EntryKind, conversationId: number, draft: EntryDraft): number {
|
||||
this.sawEntry = true;
|
||||
const id = this.seq + 1;
|
||||
const entry: Entry<C> = {
|
||||
id, conversationId, kind: kind.kind, content,
|
||||
...(kind.head === undefined ? {} : { head: true }),
|
||||
const { head, ...fields } = draft;
|
||||
const entry: Entry = {
|
||||
id, conversationId, kind: kind.kind, ...fields,
|
||||
...(head === "self" ? { head: id } : head === undefined ? {} : { head }),
|
||||
};
|
||||
this.push({ type: "entry", entry });
|
||||
return id;
|
||||
|
|
@ -1556,7 +1602,7 @@ interface ToolContext {
|
|||
jobOutput(id: Id): Promise<ToolOutputState & { exitCode?: number }>; // preview or terminal state
|
||||
}
|
||||
|
||||
// what the sink folds to; recorded on the tool task and the tool_result entry at settlement
|
||||
// what the live sink folds to; recorded on the tool task and used to build the result entry
|
||||
interface ToolOutputState {
|
||||
content: (TextContent | ImageContent)[];
|
||||
details: JsonValue;
|
||||
|
|
@ -1568,6 +1614,8 @@ interface ToolOutputState {
|
|||
diags: readonly { severity: "info" | "warn" | "error"; message: string; code?: string }[];
|
||||
truncation: ShellOutputTruncation; // totals over everything ever written
|
||||
}
|
||||
|
||||
type ToolResultData = Omit<ToolOutputState, "content">;
|
||||
```
|
||||
|
||||
`execute` returns nothing and reports failure by throwing; the harness sets `isError` when it
|
||||
|
|
@ -1582,8 +1630,8 @@ spilled to a file, the path was resolved differently, the file changed on disk s
|
|||
the search stopped at 500 matches) goes through `out.diag`, never into the text the model reads as
|
||||
the tool's data. The harness emits the ones it owns: the sink calls `diag` itself when it truncates
|
||||
or spills, the path resolver when it corrects a path, the blocking budget when a call continues as
|
||||
a job. A tool adds only what it alone knows. Projection renders the data first and the diagnostics
|
||||
after it, delimited:
|
||||
a job. A tool adds only what it alone knows. Tool settlement renders the data first and the
|
||||
diagnostics after it, delimited, and stores that message on the result entry:
|
||||
|
||||
```text
|
||||
…last line of the file
|
||||
|
|
@ -1594,8 +1642,10 @@ after it, delimited:
|
|||
|
||||
so the model can parse tool output as data, a plugin can post-process it without stripping notices
|
||||
it doesn't know, and a UI renders diagnostics as callouts by severity. `isError` says whether the
|
||||
call failed; a `warn` diag does not change it. The transcript records exactly the commentary the
|
||||
model saw, because it is in the entry's `ToolOutputState`.
|
||||
call failed; a `warn` diag does not change it. Tool settlement stores the exact rendered result as
|
||||
entry `model`; the non-message fields of `ToolOutputState` become `ToolResultData` for typed logic
|
||||
and rendering. The transcript therefore records exactly the commentary the model saw without
|
||||
copying the message into `data`.
|
||||
|
||||
### 9.4 Watch
|
||||
|
||||
|
|
@ -1754,12 +1804,12 @@ h.kinds.generation; h.kinds.tool; h.kinds.postTools; h.kinds.collapse; h.kin
|
|||
|
||||
The built-in entry kinds (`user`, `assistant`, `tool_result`, `system`, `notice`, `summary`,
|
||||
`handoff`, `reset`) and task kinds (`generation`, `tool`, `post_tools`, `collapse`, `job`) are
|
||||
registered by `open` itself, because `accept`, `prompt`, `steer` and `collapse` cannot work without
|
||||
them. A replacement is registered under the built-in's name and must understand its persisted
|
||||
statuses and keep its hook names, so existing handlers keep working; a wrapper that delegates to
|
||||
the original is the usual shape. `h.kinds.<name>` is how handles and plugins refer to whatever is
|
||||
currently registered, so `c.settings` is the config of the generation kind in use, not of a
|
||||
specific import.
|
||||
registered by `open` itself, because `accept`, `prompt`, `steer` and `collapse` need them to write
|
||||
and type new records. Reading context does not need entry kinds. A task-kind replacement is
|
||||
registered under the built-in's name and must understand its persisted statuses and keep its hook
|
||||
names, so existing handlers keep working; a wrapper that delegates to the original is the usual
|
||||
shape. `h.kinds.<name>` is how handles and plugins refer to whatever is currently registered, so
|
||||
`c.settings` is the config of the generation kind in use, not of a specific import.
|
||||
|
||||
`Harness.open` takes storage, models, tools, additional kinds, replacements and initial root values; it creates
|
||||
the root only for empty storage and never creates work on reopen.
|
||||
|
|
|
|||
|
|
@ -11,19 +11,23 @@ shows the surface each package has to end up with.
|
|||
|
||||
## 1. Types and ids
|
||||
|
||||
`Id`, `Entry`, `EntryKind`, `ContextEdit`, `Task`, `TaskRole`, `Conversation`,
|
||||
`Address`/`Value`/`List`/`Scope`, `Write`/`CommitBatch`, `Page`/`Cursor` and the query shapes
|
||||
(§2, §4.1, §5.1, §7.2–7.3). No code.
|
||||
`Id`, `EntryIdentity`, `EntryBase`, the composable `EntryData` / `ModelProjection` /
|
||||
`ContextHead` / `ContextEdits` facets, `Entry`, `EntryKind`, `EntryInput`, `ContextEdit`, `Task`,
|
||||
`TaskRole`, `Conversation`, `Address`/`Value`/`List`/`Scope`, `Write`/`CommitBatch`, `Page`/`Cursor`
|
||||
and the query shapes (§2, §4.1, §5.1, §7.2–7.3). No code.
|
||||
|
||||
Test: it compiles; a fixture file with one instance of every record shape.
|
||||
Test: it compiles; fixtures cover entries with no data, no model, every individual facet and the
|
||||
built-in facet combinations.
|
||||
|
||||
## 2. Memory storage
|
||||
|
||||
`Storage` and `MemoryStorage`: `commit` numbering from `lastSeq`, point reads, fork-aware
|
||||
`scanEntries` / `scanTasks` / `scanConversations` with cursors, target-capped `newestHead`, value
|
||||
versions and list elements read at a position, `ownedFrom`, the set of entry kinds written.
|
||||
`scanEntries` / `scanTasks` / `scanConversations` with cursors, target-capped `newestHead` over stored
|
||||
numeric boundaries, stored entry data/model/edits, value versions and list elements read at a
|
||||
position, `ownedFrom`, the set of entry kind strings written.
|
||||
|
||||
Tests: every row of the §7.2 query table against hand-built batches; ids are `lastSeq + 1 + i`; a
|
||||
head is found without a kind; entry headers omit arbitrary data but retain model/head/edits; a
|
||||
live-task scan decodes no terminal rows; `remove` and `clear` hide by position.
|
||||
|
||||
## 3. The line and `Tx`
|
||||
|
|
@ -41,21 +45,23 @@ append/remove/clear after an entry each reject.
|
|||
|
||||
## 4. Entry kinds and context
|
||||
|
||||
`EntryKind`, the registry, the built-in kinds (`user`, `assistant`, `tool_result`, `system`,
|
||||
`notice`, `summary`, `handoff`, `reset`) with `project`, `head` and `edit`; context = newest head
|
||||
prepended to the fork-aware range from its returned id, older heads excluded, edits folded in
|
||||
transcript order; projection to pi-ai messages with tool results ordered by call index; the `system`
|
||||
kind's epoch fold (newest baseline plus the deltas after it, older deltas dropped) and its projection
|
||||
into the baseline slot / `SystemMessage`s. No `compose`.
|
||||
`EntryKind` as `kind` plus `is`, the registry and typed append helpers for the built-in kinds
|
||||
(`user`, `assistant`, `tool_result`, `system`, `notice`, `summary`, `handoff`, `reset`). Writers
|
||||
materialize optional model messages and stored controls; context = newest stored head prepended to
|
||||
the fork-aware range from its numeric boundary, older heads excluded, stored edits folded in
|
||||
transcript order, stored model arrays concatenated, then pi-ai tool results ordered by call index.
|
||||
The `system` data fold selects the newest baseline plus later deltas and places their stored messages
|
||||
in the baseline slot / `SystemMessage`s. There is no read-time entry-kind behavior.
|
||||
|
||||
Tests: summary keeps the tail; handoff and reset; repeated compaction subsumes; a returned head id
|
||||
below the previous visible head kind's returned id is rejected; edits omit/replace targets and persist
|
||||
across turns; the fold on a context that kept an old delta; tool-result order.
|
||||
Tests: data-only and model-only entries; summary keeps the tail; handoff/reset normalize `"self"`;
|
||||
repeated compaction subsumes; a stored head below the previous visible boundary is rejected; edits
|
||||
omit/replace targets and persist across turns; context is identical with its plugin kind unregistered;
|
||||
the fold on a context that kept an old delta; tool-result order.
|
||||
|
||||
## 5. Forks and historical reads
|
||||
|
||||
`createConversation` with `parent`, the shared prefix in fork-aware `scanEntries`, capped-source
|
||||
lookup for values and lists, arbitrary content-entry fork points.
|
||||
lookup for values and lists, arbitrary transcript-entry fork points.
|
||||
|
||||
Tests: a fork sees the head, edits and values in force at its entry; heads/results the source adds
|
||||
later are invisible; deep fork chains; successful incomplete tool exchanges project with missing
|
||||
|
|
@ -87,7 +93,8 @@ new attempt clears its list.
|
|||
`settings`, `fork` with `abort`, `abort`, `hooks.on` scoped with `subtree`), `abortTask`,
|
||||
`conversations` with `parent` / independent filtering. No agent behaviour yet.
|
||||
|
||||
Tests: open on empty vs existing storage; an unknown kind rejects without scanning entries;
|
||||
Tests: open on empty vs existing storage; an unregistered entry kind is reported without scanning
|
||||
entries and its stored model/head/edits still derive context; an unknown live task kind rejects;
|
||||
replace by name keeps `h.kinds.<name>` consistent; `settings` round-trips; scoped hooks run after
|
||||
harness-wide ones, innermost last.
|
||||
|
||||
|
|
@ -95,7 +102,8 @@ harness-wide ones, innermost last.
|
|||
|
||||
Statuses pending → streaming → done / failed / retry_wait / deferred / aborted on a faux provider;
|
||||
config capture (model, thinking, selected tools, profile, budget); `system_instructions` with
|
||||
sections merged across handlers and the diff writing `system` entries; `before_request`,
|
||||
sections merged across handlers and the diff writing `system` data plus its materialized model
|
||||
message; `before_request`,
|
||||
`after_response`, `on_yield`; retry sleeps in execute; recover from frames; usage recorded per
|
||||
attempt.
|
||||
|
||||
|
|
@ -107,7 +115,8 @@ crash while streaming publishes the partial; retry budget exhausted → failed w
|
|||
## 10. Tools, post_tools, exchanges
|
||||
|
||||
The tool kind with the sink (`ToolOutput`, `ToolOutputState`, limits enforced by the sink, `diag`,
|
||||
`delegate`, `handoff`, `addTools`, `terminate`), `before_tool` (fail-closed) and `after_tool`,
|
||||
`delegate`, `handoff`, `addTools`, `terminate`), tool-result entries with structured data plus their
|
||||
materialized model message, `before_tool` (fail-closed) and `after_tool`,
|
||||
replay policy on recover; post_tools with `after`, terminate / handoff / steer / next generation;
|
||||
`accept` idle vs busy; `prompt` and `answerTo`.
|
||||
|
||||
|
|
@ -177,17 +186,17 @@ fold, failed and aborted attempts included.
|
|||
|
||||
## 17. JSONL storage
|
||||
|
||||
Append-only batches, replay on open, torn tail discarded, malformed line fails open, the kinds set
|
||||
from replay, values as Chord deltas (full when first or small, ops otherwise), scratch sidecars
|
||||
retired after the main-file settle.
|
||||
Append-only batches, replay on open, torn tail discarded, malformed line fails open, entry
|
||||
`data`/`model`/`head`/`edits` and the kind-string set from replay, values as Chord deltas (full when
|
||||
first or small, ops otherwise), scratch sidecars retired after the main-file settle.
|
||||
|
||||
Tests: conformance against memory (one mutation stream, identical query results); linear file
|
||||
growth under repeated sets of a large value; crash between the settle write and the sidecar unlink.
|
||||
|
||||
## 18. SQLite storage
|
||||
|
||||
Tables and indexes of §7.5, scratch rows deleted in the settle transaction, storage version and
|
||||
`migrate`.
|
||||
Tables and indexes of §7.5, nullable entry JSON columns for data/model/edits and an indexed integer
|
||||
head boundary, scratch rows deleted in the settle transaction, storage version and `migrate`.
|
||||
|
||||
Tests: conformance three ways; a cold reopen decodes only live rows (count them); a version
|
||||
mismatch rejects.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue