Add createGatewayBindingFetch: a FetchFunction factory that intercepts
requests bound for an AI Gateway HTTPS endpoint and re-issues them
through the Workers AI binding's universal endpoint
(env.AI.gateway(id).run()). Binding calls are pre-authenticated
in-account, so Workers running inside the gateway's account need no
cf-aig-authorization API token.
The shim is transport-only: it composes with the existing gateway
passthrough base URLs and every API implementation that honors
StreamOptions.fetch, preserving each provider's native wire format
(verified: workers-ai v1/chat/completions OpenAI-compat SSE streams
incrementally; anthropic v1/messages and openai responses route
end-to-end; cf-aig-log-id is present and readable via gateway getLog).
Binding types are structural, so no @cloudflare/workers-types
dependency and no impact on browser bundling.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
vLLM caps the reasoning phase with a top-level thinking_token_budget. On
OpenAI-compatible endpoints reasoning and the answer share max_tokens, so a
reasoning-heavy turn can consume the entire response ceiling and return no
answer and no tool call — the agent loop then sees a message without tool
calls and ends the run as if it had finished.
thinkingBudgets from #529 already carries per-level budgets, but only the
Anthropic and Google paths read it. This wires it into openai-completions
behind a new opt-in compat flag, and clamps the budget so at least
MIN_ANSWER_TOKENS remain for the answer.
The flag is independent of thinkingFormat because one vLLM server can serve
zai, qwen and chat-template models.
Co-authored-by: Ben Shaharizad <ben.shaharizad@taboola.com>
GA qwen3.8-max is on models.dev for alibaba-token-plan[-cn]. Point the
Qwen3.8 effort map at the GA id and exclude the retired preview so
regenerate does not revive it.
* fix(coding-agent): resume after context-limited length stops
Treat length stops as context overflow when prompt usage is within 1% of
Pi's configured context window. Include cache-write tokens in that usage
and allow non-zero output because OpenAI may emit reasoning tokens before
returning an incomplete response.
After compaction, remove any retryable error or truncated-length assistant
message restored from persisted session history so the interrupted agent
turn can continue.
This is specifically for handling this case for openai models:
https://developers.openai.com/api/docs/guides/reasoning
> If the generated tokens reach the context window limit or the max_output_tokens value you’ve set, you’ll receive a response with a status of incomplete and incomplete_details with reason set to max_output_tokens. This might occur before any visible output tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response.
This detection currently assumes the reported prompt usage is within 1% of
the context window configured in Pi. Its reliability therefore depends on
Pi's model contextWindow metadata matching the provider's effective limit.
Maybe we should consider treating OpenAI incomplete responses as requiring compaction
regardless of the local context-window estimate?
fixes#7020
* fix(coding-agent): recover truncated responses after compaction
Classify length stops by comparing provider-reported output usage with the model's intended output limit. Compact and retry once when generation ends below that limit, avoiding unreliable context-window estimates while preserving genuine max-output stops.
Normalize OpenAI Responses incomplete reasons so only max_output_tokens is treated as a length stop. Preserve qualified provider reasons for diagnostics, surface other reasons as errors, and use neutral truncation wording in the TUI.
The google-vertex and google-generative-ai adapters called
client.models.generateContentStream bare, so a transient 429/5xx before
the first token became a terminal errored assistant message instead of
being retried like in the Anthropic, OpenAI, Azure, and OpenRouter
adapters. Wrap the initial request in the shared retryProviderRequest
via a new retryGoogleRequest helper that also normalizes @google/genai's
ApiError, which has a status property but no headers property, so
retryProviderRequest's provider-error guard recognizes it.
Opt-in via the existing maxRetries stream option; default behavior is
unchanged.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Gemini can attach thoughtSignature to a part whose visible text is
empty and requires it echoed back; convertMessages dropped such blocks
together with the signature, breaking the reasoning chain. With the
chain broken, gemini flash models intermittently end a mid-task turn
with a thought-only STOP and no tool call. Empty blocks are now dropped
only when they are unsigned, mirroring the anthropic converter.
Co-authored-by: Jingtao Xu <jingtaozf@gmail.com>
