This adds experimental support for virtual models. A virtual model is a catalog entry that an extension registers with pi.registerVirtualModel(). It does not talk to a provider itself; for every request it picks a physical model and thinking level. The motivation is to make routing policies pluggable: plan on a strong model and implement on a cheap one, pick a model with a classifier, or move to a larger window when the context grows. Today this needs model switching by hand or hacks in prepareRequest.
The selection stays virtual: model_change records it, resume restores it, and each response records the physical model and thinking level that produced it. Router state can be stored on the session branch, so it follows /tree and forks and survives compaction. Context limits, compaction and image sizing use the physical model a request was routed to. examples/extensions/jev-router.ts shows a full router that uses the Jev classifier to pick Sol or Terra for planning and hands off to Luna after the first edit. docs/virtual-models.md describes the API.
Separate conversation content from instructions and replace prefix/suffix wording with continuation-oriented summarization guidance.
Hopefully this avoids the refusals.
Fixes#9652
Fall back to the latest safe assistant boundary when trailing tool results exceed the retained-token budget, allowing older history to compact before the next provider request.
Closes#9740
This change makes system prompt text and tool changes part of the transcript rather than silently rewriting its starting conditions. This lets Pi record when instructions changed or tools became available, restore that state after resuming or navigating branches, and preserve cached prompt prefixes where the upstream supports it.
Run next-turn preparation only when the agent loop continues, allowing threshold compaction before the next provider request. Restore interactive progress when the same run resumes.
Closes#6879
Important to notice:
this changes when prepareNextTurn & prepareNextTurnWithContext are invoked:
the now run only when a next turn actually happens, and after shouldStopAfterTurn.
* fix(coding-agent): resume after context-limited length stops
Treat length stops as context overflow when prompt usage is within 1% of
Pi's configured context window. Include cache-write tokens in that usage
and allow non-zero output because OpenAI may emit reasoning tokens before
returning an incomplete response.
After compaction, remove any retryable error or truncated-length assistant
message restored from persisted session history so the interrupted agent
turn can continue.
This is specifically for handling this case for openai models:
https://developers.openai.com/api/docs/guides/reasoning
> If the generated tokens reach the context window limit or the max_output_tokens value you’ve set, you’ll receive a response with a status of incomplete and incomplete_details with reason set to max_output_tokens. This might occur before any visible output tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response.
This detection currently assumes the reported prompt usage is within 1% of
the context window configured in Pi. Its reliability therefore depends on
Pi's model contextWindow metadata matching the provider's effective limit.
Maybe we should consider treating OpenAI incomplete responses as requiring compaction
regardless of the local context-window estimate?
fixes#7020
* fix(coding-agent): recover truncated responses after compaction
Classify length stops by comparing provider-reported output usage with the model's intended output limit. Compact and retry once when generation ends below that limit, avoiding unreliable context-window estimates while preserving genuine max-output stops.
Normalize OpenAI Responses incomplete reasons so only max_output_tokens is treated as a length stop. Preserve qualified provider reasons for diagnostics, surface other reasons as errors, and use neutral truncation wording in the TUI.
* add usage info to branch summary entries
* add usage to compaction entries
* allow custom tools to report llm usage in tool results
* allow observing and patching usage in tool_result hooks
* agent-harness: save usage in entries
for compaction, branch summaries and tool results
