The `openai-responses` client sent the session-affinity header as
`session_id`. Proxies such as nginx with `underscores_in_headers off`
silently drop headers with underscores, and Envoy can be configured to
reject them. Codex now sends `session-id`, and the
`openai-codex-responses` path already followed it (#4967).
With the default `openai` session-affinity format, `openai-responses`
now sends `session-id`. `x-client-request-id` and `prompt_cache_key` are
unchanged. The `openai-nosession` and `openrouter` formats behave as
before.
The compat schema description for `sessionAffinityFormat`
(`openai-responses`) and `docs/models-configuration.md` are updated to
match.
Not changed: `openai-completions` still sends `session_id`. I did not
check whether xAI or other backends on the Responses API depend on the
underscored name. Their `prompt_cache_key` is still sent, so prompt
caching should not depend on the header.
Testing:
- `test/openai-responses-compat.test.ts` now captures `session-id`. It
also checks that `session_id` is no longer sent, and the
explicit-override test uses the new name. `test/xai-responses.test.ts`
expects `session-id` and no `session_id`.
- Without the source change, these tests fail (3 failures); with it,
both files pass (50 tests).
- `npm run check` passes.
- The full `packages/ai` suite passes, except
`generate-models-strict.test.ts`. That failure is unrelated: the test
depends on live model catalog data.
Fixes#11639
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Fixes#8409
## Summary
- Pass the request abort signal through lazy stream setup wrappers.
- Report setup failures as aborted when the request signal is already
aborted.
- Added a regression test for aborting during a tool execution before
next auth setup.
`qwen-token-plan-individual` filters the models.dev `alibaba-token-plan`
catalog through a fixed list of model IDs. That list still stopped at
`glm-5.2`. models.dev now has `glm-5.3`, and `qwen-token-plan` and
`qwen-token-plan-cn` already list it. Individual did not, because
`glm-5.3` was not on its list.
This change adds `glm-5.3` to the Individual list and updates the
"verified" date in the comment above it.
Tests in `packages/ai/test/qwen-token-plan-models.test.ts`:
- `glm-5.3` is added to the expected list for the exact-list test.
- A new test checks that Individual's `glm-5.3` uses the international
Token Plan base URL. It also checks the thinking levels models.dev gives
for it: low, high and max.
How I tested it:
- `npm run generate-models`, then `test/qwen-token-plan-models.test.ts`:
all 90 tests pass. Without the change to the list, 3 tests fail.
- `npm run check` passes.
- Before: `pi --list-models qwen-token-plan-individual` listed `glm-5.2`
but not `glm-5.3`. After: it lists both.
Fixes#11658
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
When a request fails before the provider answers (connection refused or
reset, DNS, TLS or certificate, proxy), the `openai` and Anthropic SDKs
throw an error with the generic message `"Connection error."`. The
actual reason is in the `cause` chain, usually two levels down: undici's
`TypeError: fetch failed` wraps `connect ECONNREFUSED 127.0.0.1:8080`.
pi built `errorMessage` from `error.message` only, so users, print mode,
and session files saw only "Connection error." and had nothing to go on.
This change adds `withCauseDetail()` in `utils/error-body.ts`:
- It walks the `cause` chain, up to 5 levels, and appends the deepest
message, plus its `code` if the message doesn't already contain it. For
example: `Connection error.: connect ECONNREFUSED 127.0.0.1:8080`.
- `normalizeProviderError` applies it only when the error has no HTTP
status and no body. That covers every adapter that goes through
`formatProviderError`.
- The Anthropic adapter's catch block also uses it, since it read
`error.message` directly.
Errors without a cause come out unchanged. Retry classification still
matches, because the message still contains "Connection error".
**Testing**
- Added tests to `packages/ai/test/error-body.test.ts`:
- a synthetic nested cause chain;
- adding the code when the message lacks it;
- no cause leaves the message unchanged;
- a real `openai` SDK call against a closed local port, which now gives
`Connection error.: connect ECONNREFUSED 127.0.0.1:<port>`.
- Without the source change, the nested-chain test and the real SDK test
fail, because the message is just `Connection error.`.
- The full `packages/ai` test suite passes, and `npm run check` passes.
Fixes#11695
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Azure's OpenAI models are copied from the direct `openai` provider. That
provider caps the GPT-6 family at 272K so requests stay in OpenAI's
short-context pricing tier.
The Azure provider overrides this cap through
`CONTEXT_WINDOW_OVERRIDES`, but that list only covered gpt-5.4, gpt-5.5
and gpt-5.6. As a result, `azure/gpt-6-astra`, `gpt-6-luna`,
`gpt-6-sol`, `gpt-6.1-sol` and `gpt-6.1-terra` showed a 272K context
window, and pi compacted sessions far earlier than needed. Azure Foundry
deploys these models with a 1,050,000-token context window.
This change adds the five models to the Azure override list.
OpenAI-direct is unchanged.
Testing:
- Added a test in `packages/ai/test/supports-xhigh.test.ts` that checks
each of these Azure models has a 1,050,000 context window. It fails on
all five models without the override (5 failed) and passes with it.
- After regenerating the model data, the generated Azure catalog lists
all five models at 1050000. Before the change they were at 272000.
- `npm run check` passes.
Fixes#11742
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
Anthropic can keep a conversation on the server (message threads beta).
A request can then continue from the previous reply and send only the
new messages, instead of the whole history every turn.
pi uses this when a session id is set and caching is on. It still builds
the full request, compares it with the previous one, and only sends the
new part if nothing earlier changed.
- Default: on only for the anthropic provider on api.anthropic.com.
Gateways that drop unknown fields would lose the conversation.
Override with compat.supportsMessageThreads.
- If the server rejects a continuation, pi resends the full request
as a new thread. Other errors turn threads off for the session.
- New option StreamOptions.disableContinuation. The cache warmer sets
it so warm requests don't replace the session's thread.
- cleanupSessionResources(sessionId) frees a session's thread state.
The Codex Responses provider's local `sleep()` helper attached an
anonymous `abort` listener to the caller's signal and never removed it
when the timer fired. The SSE retry loop calls this helper before each
retry, so every completed backoff left one listener on the signal. Three
429 responses followed by a success left three listeners behind.
The helper now uses a named `onAbort` handler, removes it when the timer
fires, and registers it with `{ once: true }`. It still rejects with
`Error("Request was aborted")`, which the retry loop's catch block
depends on.
### Testing
- New regression test in `packages/ai/test/openai-codex-stream.test.ts`.
A stub fetch returns 429 with `retry-after: 0` three times and then a
completed SSE stream, using `maxRetries: 3` and the caller's own
`AbortController`. The test asserts that `getEventListeners(signal,
"abort")` is empty afterwards. It fails without the change ("got 3") and
passes with it.
- `node ../../node_modules/vitest/dist/cli.js --run
test/openai-codex-stream.test.ts`: 41/41 pass.
- A standalone script against the built package printed `listeners 3`
before the change and `listeners 0` after.
- `npm run check`: biome and the repo scripts pass. `tsc` fails, but
only in files this change does not touch: generated model ids in
`packages/ai` tests, `github-copilot.ts`, and missing bench dependencies
in `packages/schema`. I did not verify whether those failures also
happen on a clean checkout.
Fixes#11708
Requested by @badlogic
Co-authored-by: triagent <triagent@localhost>
OpenRouter checks input + max_tokens against the context window before
routing, using its own tokenizer. For prose it counted up to 12% more
input tokens than the provider. pi left only 4096 tokens spare, so
models whose output limit is close to the window (e.g. DeepSeek flash)
were rejected with a 400, which pi then treated as a context overflow
and compacted.
The default output limit now leaves 15% of the estimated input unused
(at least 4096). When that would leave less than 32768 tokens of output,
only the fixed 4096 margin is kept and output is capped at 32768.
fixes#10497
Closes#11678
Changes:
- `catalog/providers/github-copilot.ts`: add `microsoft-decision-1` as a
`typesafe-system-one` classifier (text input, 32768 context, $0.042/M
input, Copilot's static headers).
- `src/providers/github-copilot.ts`: register the classifier and wrap
`typesafeSystemOneApi()` to add `/v1` to the base URL, since Copilot
serves `/v1/systemone` and the OAuth login replaces the base URL with
the account endpoint.
- README and `docs/models.md`: add the model to the classifier tables.
Tests:
- `test/classifier-models.test.ts`: with an OAuth credential, the
request goes to `https://api.enterprise.githubcopilot.com/v1/systemone`
with the Copilot headers, and the model stays available although
`availableModelIds` lists only chat models. With an API key, it goes to
`https://api.individual.githubcopilot.com/v1/systemone`. Both tests fail
without the wrapper.
- Ran against `api.enterprise.githubcopilot.com` with a Copilot login:
bool, choice, and score questions return answers and usage.
Co-authored-by: Hongwei Ma <11957602+Marvae@users.noreply.github.com>
The non-strict input_schema kept only type/properties/required, so
property $refs into root $defs or definitions pointed at nothing. Claude
then produced wrong arguments (a JSON string instead of an object), and
some Anthropic-compatible gateways rejected the request.
Root anyOf/oneOf/allOf are still left out: Anthropic rejects them with a
400. That part of #9557 and #9134 stays open.
Enable per-message effort for OpenCode Claude models that support it, which
also sends thinking.block_binding drop_block. OpenCode Go rejected replayed
thinking with a 400 after the tool list or system prompt changed.
closes#10954
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
Replace build-time shell utilities with Node filesystem operations and retire the native sandbox example in favor of existing isolation options. Upgrade glob matching and worker-test dependencies and refresh the lockfiles.
When a Zhipu/Z.ai (`open.bigmodel.cn`) plan hits its usage cap, the API
returns HTTP 429 with error code `1308`. The cap resets after hours, but
`isRetryableAssistantError()` still marked the error as retryable,
because the bare `429` in the message matched the transient-error
pattern. The agent-level retry then re-sent a request that could not
succeed and slowed down the final error. With default settings that is 3
attempts, about 14 s.
This change adds the Zhipu `"code":"1308"` error code to the
non-retryable provider-limit patterns in
`packages/ai/src/utils/retry.ts`, next to the existing quota and
subscription-limit entries. The pattern matches the JSON code field, not
a bare number, so request IDs or byte counts that contain 1308 do not
trigger it.
Not changed: the provider-level HTTP retry (`retry.provider.maxRetries`,
off by default). It still retries every 429 based on the status code
alone.
Testing:
- `npm run check` passes.
- `test/retry.test.ts` passes.
- Calling `isRetryableAssistantError` directly on the reported message,
`429 {"code":"1308","message":"已达到 5 小时的使用上限。…"}`, returned `true`
before the change and returns `false` after.
Fixes#11537
Requested by @davidbrai
Co-authored-by: triagent <triagent@localhost>
This builds the model catalog from per-provider definitions and
models.dev v2 data.
It eplacer `generate-models.ts` with one `defineProvider()` file per
built-in provider, shared model-family rules (Claude, GPT, DeepSeek,
Kimi, Google), and source adapters. models.dev is read through its v2
API. Outputs and CLI flags are unchanged.
The test copied the generator's model-ID rule and snapshotted upstream catalog IDs, so it broke on catalog changes without describing behavior beyond the code.
The ~anthropic/claude-*-latest aliases used Chat Completions, where pi
dropped the thinking signature and OpenRouter ignored the unsigned
thinking on replay. Route them through Anthropic Messages with adaptive
thinking, like the versioned Claude models.
refs #9534
LiteLLM's pre-call context check rejects requests that are too large
with HTTP 400 and a message like:
```
litellm.ContextWindowExceededError: litellm.BadRequestError: litellm._pre_call_checks: Context Window exceeded for given call. ... Max Input Tokens=262144, Got=265262 ...
```
None of the `OVERFLOW_PATTERNS` matched this wording, so
`isContextOverflow()` returned false. As a result, the agent session
never ran overflow compaction and retry, and the turn failed with the
400 error.
This change adds `/context window exceeded/i` to `OVERFLOW_PATTERNS` and
documents the error in the doc comment that lists providers.
Testing:
- Added a regression test in `packages/ai/test/overflow.test.ts` with
the exact error message from the issue. It fails without the pattern and
passes with it.
- `node ../../node_modules/vitest/dist/cli.js --run
test/overflow.test.ts` in `packages/ai`: 32 passed.
- `npm run check` passes.
- Running `isContextOverflow()` directly on the reported message
returned `false` before this change and returns `true` after it.
Fixes#11492
Requested by @rwachtler
Co-authored-by: triagent <triagent@localhost>
Retry Bedrock strict tool limit rejections once with strict: "prefer" tools non-strict, as the Anthropic provider already does. Share the limit error messages and the prefer check between both providers.
closes#11416
GPT-5.5 and older namespace tools loaded through additional_tools or tool
search and reject replayed calls without the namespace, so switching to them
after another model's tool calls failed with 400. Drop dynamic tool loading
for these models on OpenAI and OpenCode; keep mid-conversation system messages.
closes#11415
closes#11132
pi sends every opencode and opencode-go request to the shared base
(`https://opencode.ai/inference/v1`, `/inference/go/v1`), Jev included.
this switches them to the documented protocol-specific endpoints, which
is also what models.dev lists per model:
| api | before | after |
|---|---|---|
| openai-completions / openai-responses | `/inference/v1` |
`/inference/openai/v1` |
| anthropic-messages | `/inference` | `/inference/anthropic` (SDK
appends `/v1/messages`) |
| google-generative-ai | `/inference/v1` | `/inference/google/v1beta` |
| jev-1.13 / jev-1.13-free | `/inference/v1` | `/inference/systemone/v1`
|
go uses the same paths under `/inference/go/...`.
changes: base URLs in `generate-models.ts`, the jev URL expectations in
`test/classifier-models.test.ts`, and a stale comment in
`providers/opencode.ts`. `models.generated.ts` has no diff since the
catalog data is gitignored.
verified with the regenerated catalog:
- all 111 opencode/opencode-go models hit their new URL, none 404
- real requests with an OpenCode key returned replies on
anthropic-messages, google-generative-ai, openai-responses,
openai-completions and jev
- go returned 403 "subscription required" from the new paths (my key has
no Go sub), so routing works there too
`npm run check` passes. `./test.sh` passes for ai and coding-agent; the
only remaining failures were timeouts that pass in isolation and pi-env
tests that need cargo locally.
---
_Drafted with AI assistance, reviewed by me._
Add gpt-6.1-terra to the OpenAI, Azure OpenAI Responses, and OpenAI
Codex providers with official pricing ($1 input, $0.05 cached input,
$1.25 cache writes, $5 output, long-context tier above 272k), tool
search, additional tools, and mid-conversation effort changes, as for
GPT-6.1 Sol. Unlike Sol it accepts reasoning.effort "none", so off is
supported; both reject "minimal" (verified against Codex). Codex serves
it with a 500K context window.
- ai: OAuth logins only accept http(s) URLs from servers (shared
trustedHttpUrl, now also checked by Radius); pi-messages rejects
contentIndex values that are not the next or an existing block; the
CLI auth file is written with mode 0600
- ai: OpenCode DeepSeek V4.1 Flash (renamed from V4 Flash upstream) no
longer sends prompt_cache_retention
- tui: setTitle() replaces control characters; export
replaceControlCharacters()
- coding-agent: the login dialog only opens web URLs and never prints
control characters from sign-in, device-code, or info URLs
- durable: partial commits no longer merge a new __proto__ member into
Object.prototype
Kimi Coding, Moonshot, and OpenCode Go generated K3.1 Pro and K3.1 Pro Highspeed with Kimi K2 settings because K3 detection matched exact IDs. Detect the K3 family by ID pattern, and Kimi Coding K3-style models by their effort options. Add implied subscription costs and clamp gateway output limits like K3.
- ai: StreamingJsonParser / ToolCallArgumentsStream read each argument delta once (tolerant,
immutable snapshots, linear cost); every provider, pi-messages and the faux provider use it
and keep stream state (partialJson, partialArgs, index, customInput, streamIndex, Bedrock
reasoning bytes) off the content blocks; grammar tool input appends per delta
- agent: streamProxy uses the same parser
- chord: Delta append ops own their text (ownedSlice), so ops and replicas no longer keep the
whole growing string alive
- durable, ai, mcp: bounded error excerpts and summaries own their characters
- coding-agent: tool calls render streamed arguments once per frame, not once per delta
With a sessionId, the faux provider reported newly cached prompt tokens
both as input and as cacheWrite, so input + cacheRead + cacheWrite
exceeded the estimated prompt. Every prompt token now goes either to
cacheRead or to cacheWrite, and input is 0 while caching is simulated.
Fix docs and `--help` drift: update the provider checklist to the
`src/api` layout, fix broken MCP anchors, and document `/mcp`, `pi mcp
add|remove`, `PI_CLEAR_ON_SHRINK`, and `"disabled"` timeouts.
closes#11243
This commit makes it so we store the responseModel (i.e. the "phyiscal"
model used) on each PiMessage. This allows e.g. Radius to know which
physical model was used by virtual models (like `cheap`). It's
especially important after we merged
https://github.com/earendil-works/pi/pull/11000
(9113094291), where Pi will drop thinking
traces when the model name doesn't match - with this change we can check
the physical model instead.
Knowing how annoying re-logins in corporate environments can be, I think
we can open the browser and copy the code to the clipboard (what the user
asks for) without forcing it upon users (which I assume was the reason
it was removed in 8f0e9251b9).
This change adds optional browser opening and code copying during
device-code login after an additional confirmation (usually Enter).
It should work for all providers, with an optional opt-out controlled
by the provider author, not user settings.
Fixes#9282.
A mid-conversation system message was placed before checking whether the
next assistant message converts to any blocks. When it did not (empty text,
or another model's reasoning-only reply), the update landed before a user
message and Claude rejected the request with a 400.
closes#11091
Use authoritative OpenRouter billing for Chat Completions and Anthropic Messages while retaining catalog component estimates and fallback behavior.
Fixes#9980
Add Usage.cost.totalFromProvider to record whether cost.total is the
billed cost reported by the provider or an estimate from catalog rates.
Catalog-derived totals set it to false; a missing value also means
false. combineUsage marks a sum as provider-reported only when both
parts are.
