diff --git a/.pi/skills/add-llm-provider.md b/.pi/skills/add-llm-provider.md index 884d3da7f..d137e548e 100644 --- a/.pi/skills/add-llm-provider.md +++ b/.pi/skills/add-llm-provider.md @@ -23,11 +23,11 @@ Most providers reuse an existing API such as `openai-completions`; skip this ste - Register the lazy wrapper in the builtin API list in `src/compat.ts`. - Add `export type` re-exports in `src/index.ts` for option types that should be available from the root entry. The `./api/*` subpath export in `package.json` already covers the new module. -## 3. Model Generation (`packages/ai/scripts/generate-models.ts`) +## 3. Model Catalog (`packages/ai/catalog/providers/.ts`) -- Add logic to fetch/parse models from the provider source. -- Map to the standardized `Model` interface. -- Regenerate. This writes the stable catalog wrapper `src/providers/.models.ts` (exporting `_MODELS`) that the provider factory imports, and updates `src/models.generated.ts`. +- Add a provider definition with `defineProvider()` and register it in `catalog/providers/index.ts`. Read `packages/ai/catalog/README.md` first. +- For a models.dev provider, use `modelsDevModels()`; set compat defaults per API in `compat` and provider-specific behavior in `finalize`. +- Regenerate with `npm run generate-models` in `packages/ai`. This writes the stable catalog wrapper `src/providers/.models.ts` (exporting `_MODELS`) that the provider factory imports, and updates `src/models.generated.ts`. ## 4. Provider Factory and Auth (`packages/ai/src/providers/`) diff --git a/AGENTS.md b/AGENTS.md index 9876d8031..04c9c32be 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ - Always ask before removing functionality or code that appears intentional. - Do not preserve backward compatibility unless the user asks for it. - Never hardcode key checks (e.g. `matchesKey(keyData, "ctrl+x")`). Add defaults to `DEFAULT_EDITOR_KEYBINDINGS` or `DEFAULT_APP_KEYBINDINGS` so they stay configurable. -- Never modify `packages/ai/src/models.generated.ts` directly; update `packages/ai/scripts/generate-models.ts` instead, then regenerate. Including the resulting `models.generated.ts` diff is always OK, even if regeneration includes unrelated upstream model metadata changes. The generated model data in `packages/ai/src/providers/data/` is gitignored: never stage it, and don't look for it in git history. +- Never modify `packages/ai/src/models.generated.ts` directly; update the catalog definitions in `packages/ai/catalog/` instead (see its README), then regenerate. Including the resulting `models.generated.ts` diff is always OK, even if regeneration includes unrelated upstream model metadata changes. The generated model data in `packages/ai/src/providers/data/` is gitignored: never stage it, and don't look for it in git history. ## Commands diff --git a/biome.json b/biome.json index a27fb2366..2c5b8a2fa 100644 --- a/biome.json +++ b/biome.json @@ -29,6 +29,7 @@ "scripts/model-catalog-protocol.test.ts", "packages/*/src/**/*.ts", "packages/*/test/**/*.ts", + "packages/ai/catalog/**/*.ts", "packages/coding-agent/examples/**/*.ts", "packages/agent/examples/**/*.ts", "!**/node_modules/**/*", diff --git a/packages/ai/CHANGELOG.md b/packages/ai/CHANGELOG.md index 444c3be23..996207fbd 100644 --- a/packages/ai/CHANGELOG.md +++ b/packages/ai/CHANGELOG.md @@ -15,6 +15,7 @@ ### Changed - The faux provider streams partial tool-call arguments, so `toolcall_delta` partials show growing arguments as real providers do ([#9265](https://github.com/earendil-works/pi/issues/9265)) +- The built-in model catalog is generated from models.dev's v2 catalog, and its per-provider rules moved from `scripts/generate-models.ts` into one TypeScript definition per provider under `catalog/`. Models that models.dev types as realtime or image models (`gpt-realtime-2.1`, `gemini-3.1-flash-live-preview`, `gemini-3.1-flash-lite-image`) are no longer offered as chat models, and models.dev's newer OpenAI and Google chat models (`chat-latest`, `gpt-5.6-cyber`, `gemini-robotics-er-2-preview`) are now included ### Fixed diff --git a/packages/ai/README.md b/packages/ai/README.md index 37fbcb7e2..583d55bc3 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -1928,12 +1928,12 @@ Create a new API implementation file (for example `bedrock-converse-stream.ts`) Add a lazy wrapper `src/api/.lazy.ts` (`Api()` via `lazyApi()`) so providers can reference the implementation without importing its SDK. Add any root-level `export type` re-exports in `src/index.ts` that should remain available from `@earendil-works/pi-ai`. -#### 3. Model Generation (`scripts/generate-models.ts`) +#### 3. Model Catalog (`catalog/providers/.ts`) -- Add logic to fetch and parse models from the provider's source (e.g., models.dev API) +- Add a provider definition and register it in `catalog/providers/index.ts`; see `catalog/README.md` for the build order and helpers - Map chat/tool-capable provider data to `Model`, image-generation data to `ImageModel`, and models.dev `type: "decision"` entries to `ClassifierModel`; hydration groups the ignored `src/providers/data/.json` values by API while stable `src/providers/.models.ts` wrappers derive exact model/API types directly from those JSON keys - Keep model ids unique within each provider and model type; emit separate entries when an upstream model supports multiple operations -- Handle provider-specific quirks (pricing format, capability flags, model ID transformations) +- Handle provider-specific quirks (pricing format, capability flags, model ID transformations) in the provider definition #### 4. Provider Factory (`src/providers/.ts`) diff --git a/packages/ai/catalog/README.md b/packages/ai/catalog/README.md new file mode 100644 index 000000000..24f3641a4 --- /dev/null +++ b/packages/ai/catalog/README.md @@ -0,0 +1,47 @@ +# Model catalog + +`node catalog/generate.ts` (`npm run generate-models`) fetches upstream catalogs and writes the built-in model catalog: + +- `src/providers/data/.json`: model values, grouped by API (gitignored) +- `src/providers/.models.ts` and `src/models.generated.ts`: typed wrappers (committed) +- with `--json-output `: the public catalog JSON served by pi.dev + +## Layout + +| Path | Contents | +| --- | --- | +| `sources/` | Upstream fetchers. `models-dev.ts` normalizes the [models.dev v2](https://models.dev/experimental/v2.0/api.json) catalog; OpenRouter, Vercel AI Gateway, NVIDIA NIM, and Radius have their own modules. All sources are fetched before anything is built. | +| `providers/.ts` | One file per built-in provider: where its models come from, its API and base URL, compat defaults, corrections, and models upstream does not list. Register new files in `providers/index.ts`. | +| `families/` | Facts about a model family that hold on every host, such as Claude effort levels or GPT-6 thinking levels. They apply to every provider's models. | +| `define.ts` | `defineProvider()` and the helpers provider files use. | +| `build.ts` | Runs the provider definitions in a fixed order. | + +## Build order + +For each provider: + +1. `models(ctx)` returns its chat models: upstream entries mapped to pi models, data corrections (limits, prices), and authored models. `ctx.providerModels(id)` returns another provider's models from this step, e.g. Azure derives from OpenAI. +2. `compat` defaults for each API are merged under each model's own compat. Then the shared rules run: models.dev effort levels for APIs that send them natively, and the rules in `families/`. +3. `finalize(model, ctx)` applies provider-specific behavior. It runs after the shared rules, so its values win. +4. Image input limits get the default resize profile, then `finalizeAll(models, ctx)` runs cross-model logic such as Anthropic's allowed fallback models. + +## Adding a provider + +Most providers are a models.dev provider served on one API: + +```ts +import { defineProvider, modelsDevModels } from "../define.ts"; + +export default defineProvider({ + id: "example", + models: (ctx) => + modelsDevModels(ctx, "example", { api: "openai-completions", baseUrl: "https://api.example.com/v1" }), + compat: { "openai-completions": { supportsStore: false, supportsDeveloperRole: false } }, +}); +``` + +`modelsDevModels()` keeps tool-capable chat models and maps names, input modalities, costs (including context tiers), and limits. Use its `include` and `map` options for provider-specific filtering and fields. Prefer fixing wrong upstream data in models.dev; keep local corrections small and say why they exist. + +## Checking changes + +`npm run diff:model-catalog -- [provider ...]` from the repository root generates the catalog at `HEAD` and in the worktree and prints the JSON differences. diff --git a/packages/ai/catalog/build.ts b/packages/ai/catalog/build.ts new file mode 100644 index 000000000..1a47dce05 --- /dev/null +++ b/packages/ai/catalog/build.ts @@ -0,0 +1,164 @@ +import type { + AnyModel, + Api, + ClassifierApi, + ClassifierModel, + ImageApi, + ImageModel, + KnownApi, + Model, + OpenAICompletionsCompat, +} from "../src/types.ts"; +import type { ApiCompatDefaults, ChatModel, ProviderContext, ProviderDefinition } from "./define.ts"; +import { mergeThinking } from "./define.ts"; +import { applyClaudeFamilyRules } from "./families/anthropic.ts"; +import { applyDeepSeekCompatRules, applyDeepSeekFamilyRules } from "./families/deepseek.ts"; +import { applyOpenAiFamilyRules } from "./families/openai.ts"; +import { getEffortThinkingLevelMap } from "./reasoning.ts"; +import type { CatalogSources } from "./sources/index.ts"; +import type { ModelsDevModel } from "./sources/models-dev.ts"; + +export interface ProviderCatalog { + chat: (Model & { type: "chat" })[]; + image: ImageModel[]; + classifier: ClassifierModel[]; +} + +// Keep the generated default no less restrictive than coding-agent's historical +// image preprocessing. Provider limits can narrow this profile, but unknown +// providers retain the cache-safe 2000px / 4.5 MiB behavior. +const DEFAULT_IMAGE_RESIZE = { + maxWidth: 2000, + maxHeight: 2000, + maxBytes: 4.5 * 1024 * 1024, + jpegQuality: 80, +} as const; + +function modelKey(model: Pick): string { + return `${model.provider}:${model.id}`; +} + +function applyCompatDefaults(model: ChatModel, defaults: ApiCompatDefaults | undefined): void { + const entry = defaults?.[model.api as KnownApi] as + | Model["compat"] + | ((model: Model) => Model["compat"]) + | undefined; + if (!entry) return; + const compat = typeof entry === "function" ? entry(model) : entry; + model.compat = { ...compat, ...model.compat } as Model["compat"]; +} + +/** Whether the model's API sends models.dev effort values to the host as-is. */ +function sendsNativeEffort(model: ChatModel): boolean { + if (model.api === "anthropic-messages") { + return (model as Model<"anthropic-messages">).compat?.forceAdaptiveThinking === true; + } + if ( + model.api === "openai-responses" || + model.api === "azure-openai-responses" || + model.api === "openai-codex-responses" + ) { + return true; + } + if (model.api !== "openai-completions") return false; + const compat: OpenAICompletionsCompat = (model as Model<"openai-completions">).compat ?? {}; + return (compat.thinkingFormat ?? "openai") === "openai" && (compat.supportsReasoningEffort ?? true); +} + +function applyModelsDevEffort(model: ChatModel, source: ModelsDevModel | undefined): void { + if (!source?.reasoningOptions || !sendsNativeEffort(model)) return; + const map = getEffortThinkingLevelMap(source.reasoningOptions); + if (map) mergeThinking(model, map); +} + +function applyImageResizeDefaults(model: AnyModel): void { + if (!model.input.includes("image")) return; + const images = model.inputLimits?.images; + model.inputLimits = { + ...model.inputLimits, + images: { ...images, resize: { ...DEFAULT_IMAGE_RESIZE, ...images?.resize } }, + }; +} + +/** Build every provider's catalog from the fetched sources. */ +export function buildCatalog( + definitions: readonly ProviderDefinition[], + sources: CatalogSources, + strict: boolean, +): Record { + const definitionsById = new Map(definitions.map((definition) => [definition.id, definition])); + const modelsDevSources = new Map(); + const stageOne = new Map(); + const building = new Set(); + + function context(providerId: string): ProviderContext { + return { + providerId, + sources, + strict, + providerModels: (id) => structuredClone(buildModels(id)), + modelsDevSource: (model) => modelsDevSources.get(modelKey(model)), + recordModelsDevSource: (model, source) => { + modelsDevSources.set(modelKey(model), source); + }, + }; + } + + function buildModels(providerId: string): ChatModel[] { + const built = stageOne.get(providerId); + if (built) return built; + const definition = definitionsById.get(providerId); + if (!definition) throw new Error(`Unknown provider: ${providerId}`); + if (building.has(providerId)) throw new Error(`Provider ${providerId} depends on itself`); + building.add(providerId); + const models = definition.models?.(context(providerId)) ?? []; + building.delete(providerId); + for (const model of models) { + if (model.provider !== providerId) { + throw new Error(`${providerId} produced ${modelKey(model)} for another provider`); + } + } + stageOne.set(providerId, models); + return models; + } + + for (const definition of definitions) buildModels(definition.id); + + const catalogs: Record = {}; + for (const definition of definitions) { + const ctx = context(definition.id); + const chat: ChatModel[] = []; + const seen = new Set(); + for (const model of stageOne.get(definition.id) ?? []) { + // The first source listing a model wins. + if (seen.has(model.id)) continue; + seen.add(model.id); + chat.push(model); + } + + for (const model of chat) { + applyCompatDefaults(model, definition.compat); + applyDeepSeekCompatRules(model); + applyModelsDevEffort(model, ctx.modelsDevSource(model)); + applyOpenAiFamilyRules(model); + applyClaudeFamilyRules(model); + applyDeepSeekFamilyRules(model); + } + for (const model of chat) definition.finalize?.(model, ctx); + for (const model of chat) { + if (model.compat && Object.keys(model.compat).length === 0) delete model.compat; + applyImageResizeDefaults(model); + } + definition.finalizeAll?.(chat, ctx); + + const image = definition.images?.(ctx) ?? []; + for (const model of image) applyImageResizeDefaults(model); + const classifier = definition.classifiers?.(ctx) ?? []; + catalogs[definition.id] = { + chat: chat.map((model) => ({ ...model, type: "chat" as const })), + image, + classifier, + }; + } + return catalogs; +} diff --git a/packages/ai/catalog/define.ts b/packages/ai/catalog/define.ts new file mode 100644 index 000000000..71517c6e1 --- /dev/null +++ b/packages/ai/catalog/define.ts @@ -0,0 +1,142 @@ +import type { + Api, + ClassifierApi, + ClassifierModel, + ImageApi, + ImageModel, + KnownApi, + Model, + ModelInputLimits, + ThinkingLevelMap, +} from "../src/types.ts"; +import type { CatalogSources } from "./sources/index.ts"; +import { type ModelsDevModel, modelsDevChatModels, modelsDevCost } from "./sources/models-dev.ts"; + +export type ChatModel = Model; +export type ModelCompat = NonNullable["compat"]>; + +/** + * Compat defaults per API. They sit under each model's own compat, so model + * values win. A function receives the model to vary defaults by model. + */ +export type ApiCompatDefaults = { + [TApi in KnownApi]?: ModelCompat | ((model: Model) => ModelCompat); +}; + +/** Passed to every provider hook. */ +export interface ProviderContext { + /** The provider being built. */ + readonly providerId: string; + readonly sources: CatalogSources; + /** Fail instead of degrading when upstream data is unexpected. */ + readonly strict: boolean; + /** Copies of another provider's chat models before shared rules run. */ + providerModels(providerId: string): ChatModel[]; + /** The models.dev entry a model was built from, if it was recorded. */ + modelsDevSource(model: ChatModel): ModelsDevModel | undefined; + /** Record the models.dev entry so shared rules can use its reasoning metadata. */ + recordModelsDevSource(model: ChatModel, source: ModelsDevModel): void; +} + +/** + * One built-in provider's catalog. The build runs, in order: + * 1. `models()`: upstream models plus authored ones, with data corrections. + * 2. `compat` defaults and the shared model-family rules in `families/`. + * 3. `finalize()`: provider-specific behavior that must win over shared rules. + * 4. Shared normalization (image input limits), then `finalizeAll()`. + */ +export interface ProviderDefinition { + id: string; + models?(ctx: ProviderContext): ChatModel[]; + images?(ctx: ProviderContext): ImageModel[]; + classifiers?(ctx: ProviderContext): ClassifierModel[]; + compat?: ApiCompatDefaults; + finalize?(model: ChatModel, ctx: ProviderContext): void; + finalizeAll?(models: ChatModel[], ctx: ProviderContext): void; +} + +export function defineProvider(definition: ProviderDefinition): ProviderDefinition { + return definition; +} + +/** Shallow-merge compat flags into a model. */ +export function mergeCompat(model: Model, compat: ModelCompat): void { + model.compat = { ...model.compat, ...compat } as ModelCompat; +} + +/** Shallow-merge thinking level mappings into a model. Later values win. */ +export function mergeThinking(model: ChatModel, map: ThinkingLevelMap): void { + model.thinkingLevelMap = { ...model.thinkingLevelMap, ...map }; +} + +/** Provider input limits for image-capable models. Limits the model already carries take precedence. */ +export function withInputLimits(model: ChatModel, limits: ModelInputLimits): void { + if (!model.input.includes("image")) return; + model.inputLimits = { + ...limits, + ...model.inputLimits, + images: { ...limits.images, ...model.inputLimits?.images }, + }; +} + +/** Add models only when upstream has not listed them (yet). */ +export function addMissing(models: ChatModel[], additions: readonly ChatModel[]): ChatModel[] { + for (const addition of additions) { + if (!models.some((model) => model.id === addition.id)) models.push(addition); + } + return models; +} + +export interface ModelsDevOptions { + api: TApi; + baseUrl: string; + compat?: ModelCompat; + headers?: Record; + /** Only models models.dev lists on this API protocol (e.g. "responses"). */ + protocol?: string; + /** Skip models models.dev marks deprecated. */ + skipDeprecated?: boolean; + /** Expose models.dev effort levels through the shared rules. Defaults to true. */ + recordReasoning?: boolean; + /** Fallback limits when models.dev has none. Defaults to 4096 each. */ + defaultContextWindow?: number; + defaultMaxTokens?: number; + /** Extra filter. Models without tool support are always skipped. */ + include?(source: ModelsDevModel): boolean; + /** Adjust or replace the mapped model. Return undefined to skip it. */ + map?(model: Model, source: ModelsDevModel): ChatModel | undefined; +} + +/** Map one models.dev provider's tool-capable chat models onto pi models. */ +export function modelsDevModels( + ctx: ProviderContext, + sourceProvider: string, + options: ModelsDevOptions, +): ChatModel[] { + const models: ChatModel[] = []; + for (const source of modelsDevChatModels(ctx.sources.modelsDev, sourceProvider)) { + if (!source.toolCall) continue; + if (options.protocol && !source.apis.some((api) => api.protocol === options.protocol)) continue; + if (options.skipDeprecated && source.status === "deprecated") continue; + if (options.include && !options.include(source)) continue; + const base: Model = { + id: source.id, + name: source.name || source.id, + api: options.api, + provider: ctx.providerId, + baseUrl: options.baseUrl, + reasoning: source.reasoning, + input: source.imageInput ? ["text", "image"] : ["text"], + cost: modelsDevCost(source.cost), + contextWindow: source.contextWindow || options.defaultContextWindow || 4096, + maxTokens: source.maxTokens || options.defaultMaxTokens || 4096, + ...(options.headers ? { headers: { ...options.headers } } : {}), + ...(options.compat ? { compat: { ...options.compat } } : {}), + }; + const model = options.map ? options.map(base, source) : base; + if (!model) continue; + if (options.recordReasoning !== false) ctx.recordModelsDevSource(model, source); + models.push(model); + } + return models; +} diff --git a/packages/ai/catalog/families/anthropic.ts b/packages/ai/catalog/families/anthropic.ts new file mode 100644 index 000000000..f9d6d8a64 --- /dev/null +++ b/packages/ai/catalog/families/anthropic.ts @@ -0,0 +1,134 @@ +// Anthropic Claude models: facts that hold on every host that serves them. +// Ids appear dashed (claude-opus-4-8), dotted (claude-opus-4.8), prefixed +// (anthropic/claude-opus-4.8), and dated (claude-opus-5-20260723). + +import type { Model, ThinkingLevelMap } from "../../src/types.ts"; +import { type ChatModel, mergeCompat, mergeThinking } from "../define.ts"; + +function stripPrefix(modelId: string): string { + return modelId.toLowerCase().replace(/^~?anthropic\//, ""); +} + +/** Models that accept per-message `output_config` effort changes. */ +export function supportsClaudeMidConvoEffort(modelId: string): boolean { + const id = stripPrefix(modelId); + return ( + /^claude-opus-(?:5|5[.-]5)(?:-\d{8})?$/.test(id) || + /^claude-(?:sonnet|haiku)-5[.-]5(?:-\d{8})?$/.test(id) || + /^claude-(?:fable|mythos)-5(?:[.-]1)(?:-\d{8})?$/.test(id) + ); +} + +// Claude models that accept `thinking: {type: "disabled"}` alongside per-message effort. +// Opus 5.5, Sonnet 5.5, Fable 5.1, and Mythos 5.1 reject it. +// https://platform.claude.com/docs/en/build-with-claude/thinking#configuring-thinking +const DISABLED_THINKING_MODELS = new Set(["claude-opus-5", "claude-haiku-5-5", "claude-haiku-5.5"]); + +/** Models that accept system messages and tool changes after the conversation starts. */ +export function supportsClaudeMidConvoSystemMessages(modelId: string): boolean { + return ( + /^claude-opus-(?:4[.-]8|5(?:[.-]5)?)(?:-\d{8})?$/.test(modelId) || + /^claude-(?:sonnet|haiku)-5[.-]5(?:-\d{8})?$/.test(modelId) || + /^claude-(?:fable|mythos)-5(?:[.-]1)?(?:-\d{8})?$/.test(modelId) + ); +} + +/** Enable per-message effort changes. Models that cannot disable thinking with it lose "off". */ +export function enableClaudeMidConvoEffort(model: Model<"anthropic-messages">): void { + mergeCompat(model, { supportsMidConvoEffort: true }); + if (!DISABLED_THINKING_MODELS.has(stripPrefix(model.id).replace(/-\d{8}$/, ""))) { + mergeThinking(model, { off: null }); + } +} + +function idIncludesAny(modelId: string, fragments: readonly string[]): boolean { + return fragments.some((fragment) => modelId.includes(fragment)); +} + +const ADAPTIVE_THINKING = [ + "opus-4-6", + "opus-4.6", + "opus-4-7", + "opus-4.7", + "opus-4-8", + "opus-4.8", + "opus-5", + "opus.5", + "sonnet-4-6", + "sonnet-4.6", + "sonnet-5", + "sonnet.5", + "haiku-5", + "haiku.5", + "fable-5", + "mythos-5", +]; +/** + * OpenRouter's `~anthropic/claude-*-latest` aliases follow the newest model of each family, + * all of which use adaptive thinking. + */ +function isClaudeLatestAlias(modelId: string): boolean { + return /^~anthropic\/claude-[a-z]+-latest$/.test(modelId); +} + +const TEMPERATURE_UNSUPPORTED = [ + "opus-4-7", + "opus-4.7", + "opus-4-8", + "opus-4.8", + "opus-5", + "opus.5", + "sonnet-5-5", + "sonnet-5.5", + "haiku-5-5", + "haiku-5.5", +]; +const MAX_EFFORT_ONLY = ["opus-4-6", "opus-4.6", "sonnet-4-6", "sonnet-4.6"]; +const XHIGH_AND_MAX_EFFORT = [ + "opus-4-7", + "opus-4.7", + "opus-4-8", + "opus-4.8", + "opus-5", + "opus.5", + "sonnet-5", + "sonnet.5", + "haiku-5", + "haiku.5", +]; + +/** + * Adaptive thinking effort levels (per Anthropic adaptive thinking docs): + * "max" is available on all adaptive-thinking Claude models; "xhigh" only on + * Opus 4.7/4.8/5, Sonnet 5, Haiku 5.5, and Fable 5. + */ +export function applyClaudeFamilyRules(model: ChatModel): void { + if (idIncludesAny(model.id, MAX_EFFORT_ONLY)) mergeThinking(model, { max: "max" }); + if (idIncludesAny(model.id, XHIGH_AND_MAX_EFFORT)) mergeThinking(model, { xhigh: "xhigh", max: "max" }); + if (model.id.includes("fable-5")) mergeThinking(model, { off: null, xhigh: "xhigh", max: "max" }); + if (model.api !== "anthropic-messages") return; + const messagesModel = model as Model<"anthropic-messages">; + if (isClaudeLatestAlias(model.id) || idIncludesAny(model.id, ADAPTIVE_THINKING)) { + mergeCompat(messagesModel, { forceAdaptiveThinking: true }); + } + if (idIncludesAny(model.id.toLowerCase(), TEMPERATURE_UNSUPPORTED)) { + mergeCompat(messagesModel, { supportsTemperature: false }); + } +} + +/** + * models.dev may list Opus 5.5, Sonnet 5.5, and Haiku 5.5 before their effort + * metadata is complete. Haiku 5.5 accepts disabled thinking; Opus 5.5 and + * Sonnet 5.5 reject it. + */ +export function claude55ThinkingLevels(modelId: string): ThinkingLevelMap { + return { + ...(modelId.includes("haiku") ? {} : { off: null }), + minimal: null, + low: "low", + medium: "medium", + high: "high", + xhigh: "xhigh", + max: "max", + }; +} diff --git a/packages/ai/catalog/families/deepseek.ts b/packages/ai/catalog/families/deepseek.ts new file mode 100644 index 000000000..630ed51c9 --- /dev/null +++ b/packages/ai/catalog/families/deepseek.ts @@ -0,0 +1,67 @@ +// DeepSeek V4 models on OpenAI-compatible hosts. + +import type { Model, OpenAICompletionsCompat } from "../../src/types.ts"; +import { type ChatModel, mergeCompat, mergeThinking } from "../define.ts"; + +export const DEEPSEEK_COMPAT: OpenAICompletionsCompat = { + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "deepseek", +}; + +export const DEEPSEEK_V4_THINKING_LEVEL_MAP = { + minimal: null, + low: null, + medium: null, + high: "high", + max: "max", +} as const; +export const DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP = { + ...DEEPSEEK_V4_THINKING_LEVEL_MAP, + low: "low", +} as const; +// Azure Foundry rejects DeepSeek's own max effort. +const AZURE_DEEPSEEK_V4_THINKING_LEVEL_MAP = { + minimal: null, + low: "low", + medium: "medium", + high: "high", + xhigh: null, + max: null, +} as const; + +// Qwen Token Plan serves DeepSeek V4 through Qwen's own thinking format. +const QWEN_FORMAT_PROVIDERS = new Set(["qwen-token-plan", "qwen-token-plan-cn", "qwen-token-plan-individual"]); +// These hosts accept native reasoning efforts instead of DeepSeek's thinking toggle. +const NATIVE_EFFORT_PROVIDERS = new Set(["openrouter", "opencode"]); + +function isDeepSeekV4(model: ChatModel): model is Model<"openai-completions"> { + return model.api === "openai-completions" && model.id.includes("deepseek-v4"); +} + +/** DeepSeek V4 needs its reasoning content replayed on every host. */ +export function applyDeepSeekCompatRules(model: ChatModel): void { + if (!isDeepSeekV4(model) || QWEN_FORMAT_PROVIDERS.has(model.provider)) return; + mergeCompat( + model, + NATIVE_EFFORT_PROVIDERS.has(model.provider) + ? { requiresReasoningContentOnAssistantMessages: DEEPSEEK_COMPAT.requiresReasoningContentOnAssistantMessages } + : DEEPSEEK_COMPAT, + ); +} + +/** DeepSeek V4 effort levels when the host did not report its own. */ +export function applyDeepSeekFamilyRules(model: ChatModel): void { + if (!isDeepSeekV4(model) || model.thinkingLevelMap !== undefined) return; + if (model.provider === "openrouter") { + mergeThinking(model, { ...DEEPSEEK_V4_THINKING_LEVEL_MAP, xhigh: "xhigh", max: null }); + } else if (model.provider === "azure") { + mergeThinking(model, AZURE_DEEPSEEK_V4_THINKING_LEVEL_MAP); + } else if ( + (model.provider === "deepseek" || model.provider === "opencode" || model.provider === "opencode-go") && + model.id.includes("deepseek-v4-flash") + ) { + mergeThinking(model, DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP); + } else { + mergeThinking(model, DEEPSEEK_V4_THINKING_LEVEL_MAP); + } +} diff --git a/packages/ai/catalog/families/google.ts b/packages/ai/catalog/families/google.ts new file mode 100644 index 000000000..8e6bd9936 --- /dev/null +++ b/packages/ai/catalog/families/google.ts @@ -0,0 +1,22 @@ +// Google Gemini and Gemma models. + +import type { ThinkingLevelMap } from "../../src/types.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; +import type { ModelsDevReasoningOption } from "../sources/models-dev.ts"; + +function isGemma4Model(modelId: string): boolean { + return /gemma-?4/.test(modelId.toLowerCase()); +} + +/** Gemini effort levels from models.dev, with a Gemma 4 fallback. */ +export function googleThinkingLevels( + modelId: string, + reasoningOptions: readonly ModelsDevReasoningOption[] = [], +): ThinkingLevelMap | undefined { + const effortMap = getEffortThinkingLevelMap(reasoningOptions); + if (effortMap) return effortMap; + if (isGemma4Model(modelId)) { + return { off: null, minimal: "MINIMAL", low: null, medium: null, high: "HIGH" }; + } + return undefined; +} diff --git a/packages/ai/catalog/families/kimi.ts b/packages/ai/catalog/families/kimi.ts new file mode 100644 index 000000000..01e9f9067 --- /dev/null +++ b/packages/ai/catalog/families/kimi.ts @@ -0,0 +1,38 @@ +// Moonshot Kimi models, shared by Moonshot, Kimi Coding, and gateways. + +import type { ModelCost, OpenAICompletionsCompat } from "../../src/types.ts"; + +/** + * Kimi K3 and its point releases (kimi-k3.1-pro, kimi-k3.1-pro-highspeed) share + * K3's request format. https://api.moonshot.ai/v1/models reports effort levels + * and dynamic tool loading for all of them. + */ +export function isKimiK3FamilyId(modelId: string): boolean { + return /^kimi-k3(?:\.\d+)?(?:-|$)/.test(modelId); +} + +export const KIMI_K3_MAX_TOKENS = 131072; +export const KIMI_K3_COST: ModelCost = { + input: 3, + output: 15, + cacheRead: 0.3, + cacheWrite: 0, +}; + +// Kimi Chat Completions settings shared by Moonshot and Kimi Coding. +export const KIMI_OPENAI_COMPAT: OpenAICompletionsCompat = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsStrictMode: false, + thinkingFormat: "deepseek", +}; + +// Kimi K3-style models take reasoning_effort and need earlier reasoning sent back. +export const KIMI_K3_OPENAI_COMPAT: OpenAICompletionsCompat = { + ...KIMI_OPENAI_COMPAT, + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "openai", + supportsReasoningEffort: true, +}; diff --git a/packages/ai/catalog/families/openai.ts b/packages/ai/catalog/families/openai.ts new file mode 100644 index 000000000..3f3a6fa11 --- /dev/null +++ b/packages/ai/catalog/families/openai.ts @@ -0,0 +1,141 @@ +// OpenAI GPT models: facts that hold on every host that serves them. + +import type { Api, Model, ModelCost } from "../../src/types.ts"; +import { type ChatModel, mergeCompat, mergeThinking } from "../define.ts"; +import { roundCost } from "../sources/models-dev.ts"; + +export const OPENAI_LONG_CONTEXT_INPUT_THRESHOLD = 272000; + +/** OpenAI bills prompts above 272K input tokens at 2x input and 1.5x output rates. */ +export function withOpenAiLongContextPricing(cost: ModelCost): ModelCost { + return { + ...cost, + tiers: [ + { + inputTokensAbove: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, + input: roundCost(cost.input * 2), + output: roundCost(cost.output * 1.5), + cacheRead: roundCost(cost.cacheRead * 2), + cacheWrite: roundCost(cost.cacheWrite * 2), + }, + ], + }; +} + +// Keep current OpenAI prices authoritative until models.dev and passthrough +// catalogs catch up. +// https://developers.openai.com/api/docs/pricing +export const OPENAI_STANDARD_COSTS: Record = { + "gpt-5.6-luna": { input: 0.2, output: 1.2, cacheRead: 0.02, cacheWrite: 0.25 }, + "gpt-5.6-sol": { input: 4, output: 20, cacheRead: 0.4, cacheWrite: 5 }, + "gpt-5.6-terra": { input: 2, output: 12, cacheRead: 0.2, cacheWrite: 2.5 }, + "gpt-6-astra": { input: 10, output: 50, cacheRead: 1, cacheWrite: 12.5 }, + "gpt-6-luna": { input: 0.1, output: 0.5, cacheRead: 0.01, cacheWrite: 0.125 }, + "gpt-6-sol": { input: 2, output: 10, cacheRead: 0.2, cacheWrite: 2.5 }, + "gpt-6.1-sol": { input: 2, output: 10, cacheRead: 0.1, cacheWrite: 2.5 }, + "gpt-6.1-terra": { input: 1, output: 5, cacheRead: 0.05, cacheWrite: 1.25 }, +}; + +/** The standard price of an OpenAI model with long-context pricing. */ +export function openAiStandardCost(id: string): ModelCost { + return withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS[id]); +} + +// GPT-5.5 and older put tools loaded through tool search or additional_tools in a namespace named +// after the tool and reject replayed calls without it, so history from other models fails there +// (#11415). Like Codex, dynamic tool loading therefore starts with GPT-5.6, which does not do this. +/** Models that accept tool search and `additional_tools`. */ +export const OPENAI_TOOL_SEARCH_MODEL_IDS = new Set([ + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-6-astra", + "gpt-6-sol", + "gpt-6-luna", + "gpt-6.1-sol", + "gpt-6.1-terra", +]); +/** Models that accept mid-conversation developer messages. */ +export const OPENAI_MID_CONVO_MODEL_IDS = new Set([ + "gpt-5.4", + "gpt-5.4-mini", + "gpt-5.4-pro", + "gpt-5.5", + ...OPENAI_TOOL_SEARCH_MODEL_IDS, +]); + +const RESPONSES_APIS = new Set(["openai-responses", "azure-openai-responses", "openai-codex-responses"]); + +function isResponsesApi(model: ChatModel): boolean { + return RESPONSES_APIS.has(model.api); +} + +export function supportsOpenAiXhigh(modelId: string): boolean { + return ( + modelId.includes("gpt-5.2") || + modelId.includes("gpt-5.3") || + modelId.includes("gpt-5.4") || + modelId.includes("gpt-5.5") || + modelId.includes("gpt-5.6") || + modelId.includes("gpt-6") + ); +} + +const GPT_6_MODEL_IDS = new Set(["gpt-6-astra", "gpt-6-sol", "gpt-6-luna", "gpt-6.1-sol", "gpt-6.1-terra"]); + +/** Thinking levels of GPT models on any host. */ +export function applyOpenAiFamilyRules(model: ChatModel): void { + if ((model.api === "openai-responses" || model.api === "azure-openai-responses") && model.id.startsWith("gpt-5")) { + mergeThinking(model, { off: null }); + } + if (GPT_6_MODEL_IDS.has(model.id) && isResponsesApi(model)) { + mergeThinking(model, { + // GPT-6 Astra and GPT-6.1 Sol reject reasoning.effort "none"; GPT-6.1 Terra accepts it. All reject + // "minimal" (verified on Codex, October 2026). + off: model.id === "gpt-6-astra" || model.id === "gpt-6.1-sol" ? null : "none", + minimal: null, + low: "low", + medium: "medium", + high: "high", + xhigh: "xhigh", + max: "max", + }); + } + if (supportsOpenAiXhigh(model.id)) { + mergeThinking(model, { xhigh: "xhigh" }); + } + if ( + (model.id.includes("gpt-5.6") || model.id.includes("gpt-6")) && + (isResponsesApi(model) || model.api === "openai-completions") + ) { + mergeThinking(model, { max: "max" }); + } + if (model.id.endsWith("gpt-5.5-pro")) { + mergeThinking(model, { off: null, minimal: null, low: null }); + } +} + +/** + * OpenAI custom grammar tools on Responses endpoints. OpenAI rejects + * `type: "custom"` tools for pre-GPT-5 models (gpt-4.x, gpt-4o, o-series). + */ +export function enableOpenAiGrammarTools(model: ChatModel): void { + if (!isResponsesApi(model)) return; + const match = /^gpt-(\d+)/.exec(model.id); + if (!match || Number(match[1]) < 5) return; + mergeCompat(model as Model<"openai-responses">, { supportsOpenAIGrammarTools: true }); +} + +/** + * Newer OpenAI Responses models accept developer messages after the + * conversation has started. Proxies that pass them through also pass + * `additional_tools` items through unchanged, for the models that accept them. + */ +export function enableOpenAiMidConvoSystemMessages(model: ChatModel, options: { proxy: boolean }): void { + if (model.api !== "openai-responses" && model.api !== "openai-codex-responses") return; + if (!OPENAI_MID_CONVO_MODEL_IDS.has(model.id)) return; + mergeCompat(model as Model<"openai-responses">, { + supportsMidConvoSystemMessages: true, + ...(options.proxy && OPENAI_TOOL_SEARCH_MODEL_IDS.has(model.id) ? { supportsAdditionalTools: true } : {}), + }); +} diff --git a/packages/ai/catalog/generate.ts b/packages/ai/catalog/generate.ts new file mode 100644 index 000000000..718159674 --- /dev/null +++ b/packages/ai/catalog/generate.ts @@ -0,0 +1,308 @@ +#!/usr/bin/env node + +import { + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + renameSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { + createModelDataManifest, + groupProviderModelData, + MODEL_DATA_MANIFEST_FILE, + type ModelDataStructure, + readModelDataProviderIds, + validateGeneratedModelData, + validateModelDataDirectory, +} from "../scripts/model-data.ts"; +import type { + AnyModel, + Api, + ClassifierApi, + ClassifierModel, + ImageApi, + ImageModel, + Model, + ModelType, +} from "../src/types.ts"; +import { buildCatalog, type ProviderCatalog } from "./build.ts"; +import { PROVIDERS } from "./providers/index.ts"; +import { loadCatalogSources } from "./sources/index.ts"; + +/** A catalog entry with its model type spelled out. */ +type CatalogEntry = AnyModel & { type: ModelType }; + +const packageRoot = join(dirname(fileURLToPath(import.meta.url)), ".."); + +interface GeneratorOptions { + strict: boolean; + dataOnly: boolean; + jsonOnly: boolean; + jsonOutputDir: string | undefined; + pretty: boolean; +} + +function readGeneratorOptions(args: string[]): GeneratorOptions { + let strict = false; + let dataOnly = false; + let jsonOnly = false; + let jsonOutputDir: string | undefined; + let pretty = false; + + for (let index = 0; index < args.length; index++) { + const arg = args[index]; + if (arg === "--strict") { + strict = true; + continue; + } + if (arg === "--data-only") { + dataOnly = true; + continue; + } + if (arg === "--json-only") { + jsonOnly = true; + continue; + } + if (arg === "--pretty") { + pretty = true; + continue; + } + if (arg === "--json-output") { + const value = args[++index]; + if (!value) throw new Error("--json-output requires a directory"); + jsonOutputDir = resolve(value); + continue; + } + throw new Error(`Unknown argument: ${arg}`); + } + + if (jsonOnly && !jsonOutputDir) throw new Error("--json-only requires --json-output"); + if (dataOnly && (jsonOnly || jsonOutputDir)) + throw new Error("--data-only cannot be combined with JSON catalog output"); + return { strict, dataOnly, jsonOnly, jsonOutputDir, pretty }; +} + +/** Key models by id, keeping the first entry per id, sorted by id. */ +function sortedById(models: readonly T[]): [string, T][] { + const byId = new Map(); + for (const model of models) if (!byId.has(model.id)) byId.set(model.id, model); + return [...byId].sort(([left], [right]) => left.localeCompare(right)); +} + +async function generateModels(generatorOptions: GeneratorOptions): Promise { + const sources = await loadCatalogSources(generatorOptions.strict); + const catalogs = buildCatalog(PROVIDERS, sources, generatorOptions.strict); + // Providers without any models are left out, as if they did not exist. + const providers: Record = Object.fromEntries( + Object.entries(catalogs).filter( + ([, catalog]) => catalog.chat.length + catalog.image.length + catalog.classifier.length > 0, + ), + ); + + const sortedProviderIds = Object.keys(providers).sort(); + const jsonChatProviders: Record & { type: "chat" }>> = {}; + const jsonImageProviders: Record>> = {}; + const jsonClassifierProviders: Record>> = {}; + const jsonAllProviders: Record = {}; + for (const providerId of sortedProviderIds) { + jsonChatProviders[providerId] = Object.fromEntries(sortedById(providers[providerId].chat)); + jsonImageProviders[providerId] = Object.fromEntries(sortedById(providers[providerId].image)); + jsonClassifierProviders[providerId] = Object.fromEntries(sortedById(providers[providerId].classifier)); + jsonAllProviders[providerId] = [ + ...Object.values(jsonChatProviders[providerId]), + ...Object.values(jsonImageProviders[providerId]), + ...Object.values(jsonClassifierProviders[providerId]), + ]; + } + + const serializeJson = (value: unknown) => + `${JSON.stringify(value, null, generatorOptions.pretty ? 2 : undefined)}\n`; + const writeJson = (path: string, value: unknown) => writeFileSync(path, serializeJson(value)); + const generatedDataProviderIds = generatorOptions.dataOnly + ? readModelDataProviderIds(packageRoot) + : sortedProviderIds; + const missingProviderIds = generatedDataProviderIds.filter((providerId) => !jsonAllProviders[providerId]); + if (missingProviderIds.length > 0) { + throw new Error(`Cannot hydrate missing providers: ${missingProviderIds.join(", ")}`); + } + + // Only the ignored internal data is grouped by API for type derivation. + const generatedDataProviders: Record>> = {}; + const modelDataStructure: ModelDataStructure = {}; + for (const providerId of generatedDataProviderIds) { + const { groups, structure } = groupProviderModelData(providerId, jsonAllProviders[providerId]); + generatedDataProviders[providerId] = groups; + modelDataStructure[providerId] = structure; + } + + const generatedAt = new Date().toISOString(); + + if (!generatorOptions.jsonOnly) { + // Stage and validate all provider values before replacing the current generated data. + const providersDir = join(packageRoot, "src/providers"); + const dataDir = join(providersDir, "data"); + const stagingRoot = mkdtempSync(join(providersDir, ".model-generation-")); + const stagedDataDir = join(stagingRoot, "data"); + const previousDataDir = join(stagingRoot, "previous-data"); + let restoreGeneratedCatalog: (() => void) | undefined; + try { + mkdirSync(stagedDataDir, { recursive: true }); + const fileContents: Record = {}; + for (const providerId of generatedDataProviderIds) { + const filename = `${providerId}.json`; + const content = serializeJson(generatedDataProviders[providerId]); + fileContents[filename] = content; + writeFileSync(join(stagedDataDir, filename), content); + } + writeJson( + join(stagedDataDir, MODEL_DATA_MANIFEST_FILE), + createModelDataManifest(modelDataStructure, fileContents, generatedAt), + ); + validateModelDataDirectory(modelDataStructure, stagedDataDir); + + if (!generatorOptions.dataOnly) { + const previousShardContents = new Map( + readdirSync(providersDir) + .filter((entry) => entry.endsWith(".models.ts")) + .map((entry) => [entry, readFileSync(join(providersDir, entry), "utf8")] as const), + ); + const aggregatorPath = join(packageRoot, "src/models.generated.ts"); + const previousAggregator = readFileSync(aggregatorPath, "utf8"); + restoreGeneratedCatalog = () => { + for (const entry of readdirSync(providersDir)) { + if (entry.endsWith(".models.ts")) rmSync(join(providersDir, entry)); + } + for (const [entry, content] of previousShardContents) { + writeFileSync(join(providersDir, entry), content); + } + writeFileSync(aggregatorPath, previousAggregator); + }; + + const generatedHeader = `// This file is auto-generated by catalog/generate.ts +// Do not edit manually - run 'npm run generate-models' to update + +`; + const catalogConstName = (providerId: string) => + `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_MODELS`; + const imageCatalogConstName = (providerId: string) => + `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_IMAGE_MODELS`; + const classifierCatalogConstName = (providerId: string) => + `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_CLASSIFIER_MODELS`; + const generatedShardFiles = new Set(); + for (const providerId of sortedProviderIds) { + let output = generatedHeader; + output += `import values from "./data/${providerId}.json" with { type: "json" };\n`; + output += `import { flattenChatModelCatalog, flattenClassifierModelCatalog, flattenImageModelCatalog, type ChatModelCatalog, type ClassifierModelCatalog, type ImageModelCatalog } from "../model-catalog.ts";\n\n`; + output += `export const ${catalogConstName(providerId)}: ChatModelCatalog =\n`; + output += `\tflattenChatModelCatalog(${JSON.stringify(providerId)}, values);\n\n`; + output += `export const ${imageCatalogConstName(providerId)}: ImageModelCatalog =\n`; + output += `\tflattenImageModelCatalog(${JSON.stringify(providerId)}, values);\n\n`; + output += `export const ${classifierCatalogConstName(providerId)}: ClassifierModelCatalog =\n`; + output += `\tflattenClassifierModelCatalog(${JSON.stringify(providerId)}, values);\n`; + const filename = `${providerId}.models.ts`; + generatedShardFiles.add(filename); + writeFileSync(join(providersDir, filename), output); + } + for (const entry of readdirSync(providersDir)) { + if (entry.endsWith(".models.ts") && !generatedShardFiles.has(entry)) rmSync(join(providersDir, entry)); + } + + let output = generatedHeader; + for (const providerId of sortedProviderIds) { + output += `import { ${classifierCatalogConstName(providerId)}, ${imageCatalogConstName(providerId)}, ${catalogConstName(providerId)} } from "./providers/${providerId}.models.ts";\n`; + } + output += `\nexport const MODELS: {\n`; + for (const providerId of sortedProviderIds) { + output += `\treadonly ${JSON.stringify(providerId)}: typeof ${catalogConstName(providerId)};\n`; + } + output += `} = {\n`; + for (const providerId of sortedProviderIds) { + output += `\t${JSON.stringify(providerId)}: ${catalogConstName(providerId)},\n`; + } + output += `};\n\nexport const IMAGE_MODELS: {\n`; + for (const providerId of sortedProviderIds) { + output += `\treadonly ${JSON.stringify(providerId)}: typeof ${imageCatalogConstName(providerId)};\n`; + } + output += `} = {\n`; + for (const providerId of sortedProviderIds) { + output += `\t${JSON.stringify(providerId)}: ${imageCatalogConstName(providerId)},\n`; + } + output += `};\n\nexport const CLASSIFIER_MODELS: {\n`; + for (const providerId of sortedProviderIds) { + output += `\treadonly ${JSON.stringify(providerId)}: typeof ${classifierCatalogConstName(providerId)};\n`; + } + output += `} = {\n`; + for (const providerId of sortedProviderIds) { + output += `\t${JSON.stringify(providerId)}: ${classifierCatalogConstName(providerId)},\n`; + } + output += `};\n`; + writeFileSync(aggregatorPath, output); + console.log("Generated provider catalogs and src/models.generated.ts"); + } + + const hadPreviousData = existsSync(dataDir); + if (hadPreviousData) renameSync(dataDir, previousDataDir); + try { + renameSync(stagedDataDir, dataDir); + validateGeneratedModelData(packageRoot); + } catch (error) { + rmSync(dataDir, { recursive: true, force: true }); + if (hadPreviousData && existsSync(previousDataDir)) renameSync(previousDataDir, dataDir); + throw error; + } + restoreGeneratedCatalog = undefined; + console.log( + generatorOptions.dataOnly + ? "Hydrated JSON model values under src/providers/data/" + : "Generated JSON model values under src/providers/data/", + ); + } catch (error) { + restoreGeneratedCatalog?.(); + throw error; + } finally { + rmSync(stagingRoot, { recursive: true, force: true }); + } + } + + if (generatorOptions.jsonOutputDir) { + // `models.json` and `providers/{id}.json` retain the legacy keyed chat catalog. + // The `.all` variants are arrays so the same upstream id can appear once per type. + const providerOutputDir = join(generatorOptions.jsonOutputDir, "providers"); + rmSync(generatorOptions.jsonOutputDir, { recursive: true, force: true }); + mkdirSync(providerOutputDir, { recursive: true }); + writeJson(join(generatorOptions.jsonOutputDir, "models.json"), jsonChatProviders); + writeJson(join(generatorOptions.jsonOutputDir, "models.all.json"), jsonAllProviders); + writeJson(join(generatorOptions.jsonOutputDir, "providers.json"), sortedProviderIds); + for (const providerId of sortedProviderIds) { + writeJson(join(providerOutputDir, `${providerId}.json`), jsonChatProviders[providerId]); + writeJson(join(providerOutputDir, `${providerId}.all.json`), jsonAllProviders[providerId]); + } + console.log(`Generated JSON model catalog under ${generatorOptions.jsonOutputDir}`); + } + + // Print statistics + const chatModels = Object.values(providers).flatMap((catalog) => catalog.chat); + const totalModels = chatModels.length; + const reasoningModels = chatModels.filter((model) => model.reasoning).length; + + console.log(`\nModel Statistics:`); + console.log(` Total tool-capable models: ${totalModels}`); + console.log(` Reasoning-capable models: ${reasoningModels}`); + + for (const [provider, models] of Object.entries(providers)) { + console.log( + ` ${provider}: ${models.chat.length} chat models, ${models.image.length} image models, ${models.classifier.length} classifier models`, + ); + } +} + +generateModels(readGeneratorOptions(process.argv.slice(2))).catch((error) => { + console.error(error); + process.exitCode = 1; +}); diff --git a/packages/ai/catalog/providers/amazon-bedrock.ts b/packages/ai/catalog/providers/amazon-bedrock.ts new file mode 100644 index 000000000..28bc0c71c --- /dev/null +++ b/packages/ai/catalog/providers/amazon-bedrock.ts @@ -0,0 +1,32 @@ +import { defineProvider, modelsDevModels, withInputLimits } from "../define.ts"; + +// Opus 5 is only callable through cross-region inference profiles. +const INFERENCE_PROFILE_ONLY_MODEL_IDS = new Set(["anthropic.claude-opus-5"]); + +function bedrockBaseUrl(modelId: string): string { + return modelId.startsWith("eu.") + ? "https://bedrock-runtime.eu-central-1.amazonaws.com" + : "https://bedrock-runtime.us-east-1.amazonaws.com"; +} + +export default defineProvider({ + id: "amazon-bedrock", + models: (ctx) => + modelsDevModels(ctx, "amazon-bedrock", { + api: "bedrock-converse-stream", + baseUrl: "", + include: (source) => + !INFERENCE_PROFILE_ONLY_MODEL_IDS.has(source.id) && + // Jamba does not support tool use in streaming mode. + !source.id.startsWith("ai21.jamba") && + // Mistral 7B Instruct does not support system messages. + !source.id.startsWith("mistral.mistral-7b-instruct-v0"), + // Includes models.dev pricing tiers, e.g. the long-context tier for OpenAI models (#10326). + map: (model, source) => ({ + ...model, + baseUrl: bedrockBaseUrl(model.id), + ...(source.structuredOutput ? { compat: { supportsStrictMode: true } } : {}), + }), + }), + finalize: (model) => withInputLimits(model, { images: { maxPerMessage: 20 } }), +}); diff --git a/packages/ai/catalog/providers/ant-ling.ts b/packages/ai/catalog/providers/ant-ling.ts new file mode 100644 index 000000000..b9cf58cc2 --- /dev/null +++ b/packages/ai/catalog/providers/ant-ling.ts @@ -0,0 +1,52 @@ +import type { Model, OpenAICompletionsCompat } from "../../src/types.ts"; +import { defineProvider, mergeThinking } from "../define.ts"; + +const COMPAT: OpenAICompletionsCompat = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsLongCacheRetention: false, +}; + +function model(id: string, name: string, reasoning: boolean, cost: Model<"openai-completions">["cost"]) { + return { + id, + name, + api: "openai-completions", + baseUrl: "https://api.ant-ling.com/v1", + provider: "ant-ling", + reasoning, + input: ["text"], + cost, + contextWindow: 262144, + maxTokens: 65536, + compat: reasoning ? { ...COMPAT, thinkingFormat: "ant-ling" } : COMPAT, + } satisfies Model<"openai-completions">; +} + +export default defineProvider({ + id: "ant-ling", + models: () => [ + model("Ling-2.6-flash", "Ling 2.6 Flash", false, { input: 0.01, output: 0.02, cacheRead: 0, cacheWrite: 0 }), + model("Ling-2.6-1T", "Ling 2.6 1T", false, { input: 0.06, output: 0.25, cacheRead: 0, cacheWrite: 0 }), + model("Ring-2.6-1T", "Ring 2.6 1T", true, { input: 0.06, output: 0.25, cacheRead: 0, cacheWrite: 0 }), + ], + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + thinkingFormat: "ant-ling", + supportsStrictMode: true, + supportsLongCacheRetention: false, + }, + }, + finalize: (model) => { + // Ring reasons by default. Only high/xhigh have documented explicit effort controls. + if (model.reasoning) { + mergeThinking(model, { off: null, minimal: null, low: null, medium: null, high: "high", xhigh: "xhigh" }); + } + }, +}); diff --git a/packages/ai/catalog/providers/anthropic.ts b/packages/ai/catalog/providers/anthropic.ts new file mode 100644 index 000000000..5b97aff2d --- /dev/null +++ b/packages/ai/catalog/providers/anthropic.ts @@ -0,0 +1,66 @@ +import type { Model } from "../../src/types.ts"; +import { defineProvider, mergeCompat, mergeThinking, modelsDevModels, withInputLimits } from "../define.ts"; +import { + claude55ThinkingLevels, + enableClaudeMidConvoEffort, + supportsClaudeMidConvoEffort, + supportsClaudeMidConvoSystemMessages, +} from "../families/anthropic.ts"; + +// Models the API may fall back to, with the price the caller is billed. +const ALLOWED_FALLBACK_MODELS: Record = { + "claude-fable-5": ["claude-opus-4-8", "claude-opus-5"], + "claude-opus-5": ["claude-opus-4-8"], +}; + +const MODELS_WITH_1M_CONTEXT = new Set(["claude-opus-4-6", "claude-sonnet-4-6"]); +const MODELS_55 = new Set(["claude-opus-5-5", "claude-sonnet-5-5", "claude-haiku-5-5"]); + +export default defineProvider({ + id: "anthropic", + models: (ctx) => + modelsDevModels(ctx, "anthropic", { + api: "anthropic-messages", + baseUrl: "https://api.anthropic.com", + map: (model) => { + if (MODELS_55.has(model.id)) mergeThinking(model, claude55ThinkingLevels(model.id)); + if (MODELS_WITH_1M_CONTEXT.has(model.id)) model.contextWindow = 1000000; + return model; + }, + }), + finalize: (model) => { + if (model.api !== "anthropic-messages") return; + const messagesModel = model as Model<"anthropic-messages">; + if (supportsClaudeMidConvoEffort(model.id)) enableClaudeMidConvoEffort(messagesModel); + if (supportsClaudeMidConvoSystemMessages(model.id)) { + mergeCompat(messagesModel, { supportsMidConvoSystemMessages: true, supportsMidConvoToolChanges: true }); + } + mergeCompat(messagesModel, { supportsStrictTools: true }); + // Ephemeral entries have a hard five-minute lifetime; `ttl: "1h"` extends + // it to one hour. Only direct Anthropic is annotated so cache warming does + // not assume equivalent behavior through proxies. + // https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching + model.promptCache = { short: 300, long: 3600 }; + withInputLimits(model, { + maxRequestBytes: 32 * 1024 * 1024, + images: { maxPerRequest: model.contextWindow === 200000 ? 100 : 600 }, + }); + }, + finalizeAll: (models) => { + const modelsById = new Map(models.map((model) => [model.id, model as Model<"anthropic-messages">])); + for (const [modelId, fallbackModelIds] of Object.entries(ALLOWED_FALLBACK_MODELS)) { + const model = modelsById.get(modelId); + if (!model) continue; + const compatibleFallbackModelIds = model.compat?.supportsMidConvoEffort + ? fallbackModelIds.filter(supportsClaudeMidConvoEffort) + : fallbackModelIds; + const allowedFallbackModels = compatibleFallbackModelIds.flatMap((fallbackModelId) => { + const fallbackModel = modelsById.get(fallbackModelId); + return fallbackModel + ? [{ provider: fallbackModel.provider, model: fallbackModel.id, cost: fallbackModel.cost }] + : []; + }); + if (allowedFallbackModels.length > 0) mergeCompat(model, { allowedFallbackModels }); + } + }, +}); diff --git a/packages/ai/catalog/providers/azure.ts b/packages/ai/catalog/providers/azure.ts new file mode 100644 index 000000000..94bd77a65 --- /dev/null +++ b/packages/ai/catalog/providers/azure.ts @@ -0,0 +1,61 @@ +import type { Model, ModelCost } from "../../src/types.ts"; +import { type ChatModel, defineProvider, mergeCompat } from "../define.ts"; +import { enableOpenAiGrammarTools } from "../families/openai.ts"; + +// Azure Foundry deploys these with larger context windows than OpenAI's own short-tier defaults. +// See models-sold-directly-by-azure docs. +const CONTEXT_WINDOW_OVERRIDES: Record = { + "gpt-5.4": 1050000, + "gpt-5.5": 1050000, + "gpt-5.6-luna": 1050000, + "gpt-5.6-sol": 1050000, + "gpt-5.6-terra": 1050000, +}; + +// Azure resells DeepSeek at its own rates. US data zone, checked 2026-09-16. +// https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/deepseek/ +const DEEPSEEK_V4_PRO_COST: ModelCost = { input: 1.925, output: 3.828, cacheRead: 0.165, cacheWrite: 0 }; + +/** OpenAI models as Azure deploys them. Azure bills without OpenAI's long-context tier. */ +function openAiModels(models: ChatModel[]): ChatModel[] { + return models + .filter((model) => model.api === "openai-responses") + .map((model) => ({ + ...model, + api: "azure-openai-responses", + provider: "azure", + baseUrl: "", + cost: { + input: model.cost.input, + output: model.cost.output, + cacheRead: model.cost.cacheRead, + cacheWrite: model.cost.cacheWrite, + }, + contextWindow: CONTEXT_WINDOW_OVERRIDES[model.id] ?? model.contextWindow, + })); +} + +export default defineProvider({ + id: "azure", + models: (ctx) => [ + ...openAiModels(ctx.providerModels("openai")), + ...ctx + .providerModels("deepseek") + .filter((model) => model.id === "deepseek-v4-pro") + .map((model) => ({ ...model, provider: "azure", baseUrl: "", cost: DEEPSEEK_V4_PRO_COST })), + ], + compat: { "openai-completions": { supportsStrictMode: true } }, + finalize: (model) => { + enableOpenAiGrammarTools(model); + if (model.api === "openai-completions") { + // Azure 400s on DeepSeek's `thinking` field and on every prompt cache parameter, discards a + // `developer` system message unbilled once reasoning_effort is set, and honours mid-convo ones (#9645). + mergeCompat(model as Model<"openai-completions">, { + supportsDeveloperRole: false, + supportsMidConvoSystemMessages: true, + thinkingFormat: "openai", + supportsLongCacheRetention: false, + }); + } + }, +}); diff --git a/packages/ai/catalog/providers/baseten.ts b/packages/ai/catalog/providers/baseten.ts new file mode 100644 index 000000000..4aaaaca13 --- /dev/null +++ b/packages/ai/catalog/providers/baseten.ts @@ -0,0 +1,89 @@ +import type { OpenAICompletionsCompat, ThinkingLevelMap } from "../../src/types.ts"; +import { defineProvider, modelsDevModels } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +const BASE_COMPAT: OpenAICompletionsCompat = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + supportsUsageInStreaming: true, + maxTokensField: "max_tokens", + supportsStrictMode: true, + // Baseten automatic prompt caching needs session affinity so related + // requests land on the same replica. See: + // https://docs.baseten.co/inference/model-apis/pricing-and-limits + sendSessionAffinityHeaders: true, + supportsLongCacheRetention: false, +}; +const REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { + ...BASE_COMPAT, + supportsReasoningEffort: true, + thinkingFormat: "openai", +}; +const TOGGLE_REASONING_COMPAT: OpenAICompletionsCompat = { + ...BASE_COMPAT, + thinkingFormat: "baseten", + chatTemplateArgs: { enable_thinking: { $var: "thinking.enabled" } }, +}; +const TOGGLE_REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { + ...REASONING_EFFORT_COMPAT, + thinkingFormat: "baseten", + chatTemplateArgs: { enable_thinking: { $var: "thinking.enabled" } }, +}; +const TOGGLE_LEVELS: ThinkingLevelMap = { + off: "off", + minimal: null, + low: null, + medium: null, + high: "high", + xhigh: null, + max: null, +}; +const GLM_52_LEVELS: ThinkingLevelMap = { + off: "none", + minimal: null, + low: null, + medium: null, + high: "high", + xhigh: null, + max: "max", +}; +const GLM_52_MODEL_IDS = new Set(["zai-org/GLM-5.2", "zai-org/GLM-5.2-Fast"]); + +export default defineProvider({ + id: "baseten", + models: (ctx) => + modelsDevModels(ctx, "baseten", { + api: "openai-completions", + baseUrl: "https://inference.baseten.co/v1", + skipDeprecated: true, + recordReasoning: false, + map: (model, source) => { + const options = source.reasoningOptions ?? []; + const isGlm52 = GLM_52_MODEL_IDS.has(model.id); + const supportsToggle = isGlm52 || options.some((option) => option.type === "toggle"); + const supportsEffort = isGlm52 || options.some((option) => option.type === "effort"); + const compat = + supportsToggle && supportsEffort + ? TOGGLE_REASONING_EFFORT_COMPAT + : supportsToggle + ? TOGGLE_REASONING_COMPAT + : supportsEffort + ? REASONING_EFFORT_COMPAT + : BASE_COMPAT; + const thinkingLevelMap = isGlm52 + ? GLM_52_LEVELS + : supportsToggle + ? TOGGLE_LEVELS + : getEffortThinkingLevelMap(options); + return { + ...model, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + // Baseten's GLM-5.2 endpoints are text-only despite models.dev reporting image input. + input: !isGlm52 && source.imageInput ? ["text", "image"] : ["text"], + compat, + }; + }, + }), + compat: { "openai-completions": { supportsStrictMode: true } }, +}); diff --git a/packages/ai/catalog/providers/cerebras.ts b/packages/ai/catalog/providers/cerebras.ts new file mode 100644 index 000000000..18340cbd7 --- /dev/null +++ b/packages/ai/catalog/providers/cerebras.ts @@ -0,0 +1,8 @@ +import { defineProvider, modelsDevModels } from "../define.ts"; + +export default defineProvider({ + id: "cerebras", + models: (ctx) => + modelsDevModels(ctx, "cerebras", { api: "openai-completions", baseUrl: "https://api.cerebras.ai/v1" }), + compat: { "openai-completions": { supportsStore: false, supportsDeveloperRole: false } }, +}); diff --git a/packages/ai/catalog/providers/cloudflare-ai-gateway.ts b/packages/ai/catalog/providers/cloudflare-ai-gateway.ts new file mode 100644 index 000000000..ed4fb95d5 --- /dev/null +++ b/packages/ai/catalog/providers/cloudflare-ai-gateway.ts @@ -0,0 +1,84 @@ +import { + CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL, + CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL, + CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL, +} from "../../src/api/cloudflare.ts"; +import type { Model } from "../../src/types.ts"; +import { type ChatModel, defineProvider, mergeCompat, modelsDevModels } from "../define.ts"; +import { enableOpenAiGrammarTools, OPENAI_STANDARD_COSTS, withOpenAiLongContextPricing } from "../families/openai.ts"; + +/** models.dev lists gateway models as `/`; route each upstream to its passthrough. */ +function routeGatewayModel(model: ChatModel): ChatModel | undefined { + const slash = model.id.indexOf("/"); + if (slash === -1) return undefined; + const upstream = model.id.slice(0, slash); + const nativeId = model.id.slice(slash + 1); + if (upstream === "openai") { + // Cloudflare AI Gateway passes OpenAI usage through at OpenAI list prices. + const standardCost = OPENAI_STANDARD_COSTS[nativeId]; + return { + ...model, + id: nativeId, + api: "openai-responses", + baseUrl: CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL, + ...(standardCost ? { cost: withOpenAiLongContextPricing(standardCost) } : {}), + }; + } + if (upstream === "anthropic") { + // The /anthropic passthrough forwards the model ID to Anthropic unchanged. + // models.dev lists dotted versions (claude-opus-5.5), but Anthropic only + // accepts dashed IDs (claude-opus-5-5). + return { + ...model, + id: nativeId.replaceAll(".", "-"), + api: "anthropic-messages", + baseUrl: CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL, + // Gateway passthroughs forward session affinity headers to upstreams that + // use them for cache/routing affinity. + compat: { sendSessionAffinityHeaders: true }, + }; + } + if (upstream === "workers-ai") { + return { ...model, compat: { sendSessionAffinityHeaders: true } }; + } + return undefined; +} + +export default defineProvider({ + id: "cloudflare-ai-gateway", + models: (ctx) => { + const models = modelsDevModels(ctx, "cloudflare-ai-gateway", { + api: "openai-completions", + baseUrl: CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL, + map: routeGatewayModel, + }); + // The gateway proxies Workers AI through its OpenAI-compatible /compat endpoint, + // but models.dev may omit or intermittently drop those `workers-ai/*` entries + // from the AI Gateway catalog. Mirror the Workers AI catalog under the documented + // prefix so the gateway keeps its OpenAI-compatible models stable. + const ids = new Set(models.map((model) => model.id)); + const mirrored = modelsDevModels(ctx, "cloudflare-workers-ai", { + api: "openai-completions", + baseUrl: CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL, + compat: { sendSessionAffinityHeaders: true }, + include: (source) => !ids.has(`workers-ai/${source.id}`), + map: (model) => ({ ...model, id: `workers-ai/${model.id}` }), + }); + return [...models, ...mirrored]; + }, + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsLongCacheRetention: false, + }, + }, + finalize: (model) => { + if (model.api === "openai-responses") { + mergeCompat(model as Model<"openai-responses">, { supportsStrictMode: true }); + } + enableOpenAiGrammarTools(model); + }, +}); diff --git a/packages/ai/catalog/providers/cloudflare-workers-ai.ts b/packages/ai/catalog/providers/cloudflare-workers-ai.ts new file mode 100644 index 000000000..6274661b6 --- /dev/null +++ b/packages/ai/catalog/providers/cloudflare-workers-ai.ts @@ -0,0 +1,61 @@ +import { CLOUDFLARE_WORKERS_AI_BASE_URL, CLOUDFLARE_WORKERS_AI_REST_BASE_URL } from "../../src/api/cloudflare.ts"; +import type { ClassifierModel } from "../../src/types.ts"; +import { defineProvider, modelsDevModels } from "../define.ts"; + +function classifier( + id: string, + name: string, + input: number, + contextWindow: number, + imageInput: "images" | "assets", +): ClassifierModel<"cloudflare-workers-ai-system-one"> { + return { + type: "classifier", + id, + name, + api: "cloudflare-workers-ai-system-one", + provider: "cloudflare-workers-ai", + baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, + input: ["text", "image"], + cost: { input, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow, + compat: { imageInput }, + }; +} + +// Workers AI has no unauthenticated catalog, so its System One models are listed here from +// Cloudflare's model pages. +const CLASSIFIER_MODELS = [ + // Cloudflare-hosted Clef decision models. They take up to four PNG, JPEG, or WebP images as data URLs. + // Clef Omni also takes audio and video, which classifier contexts do not carry. + // Pricing and context: https://developers.cloudflare.com/workers-ai/models/clef/, + // https://developers.cloudflare.com/workers-ai/models/clef-flash/, + // and https://developers.cloudflare.com/workers-ai/models/clef-omni/ + classifier("@cf/cloudflare/clef", "Clef", 0.24, 65536, "images"), + classifier("@cf/cloudflare/clef-flash", "Clef Flash", 0.038, 24576, "images"), + classifier("@cf/cloudflare/clef-omni", "Clef Omni", 0.15, 64000, "images"), + // TypeSafe's Jev models take PNG and JPEG images as System One assets. + // https://developers.cloudflare.com/ai/models/typesafe/jev/ (jev-latest) + // and https://developers.cloudflare.com/ai/models/typesafe/jev-1.14.0/ + classifier("typesafe/jev", "Jev", 0.042, 32000, "assets"), + classifier("typesafe/jev-1.14.0", "Jev 1.14", 0.042, 32000, "assets"), +]; + +export default defineProvider({ + id: "cloudflare-workers-ai", + models: (ctx) => + modelsDevModels(ctx, "cloudflare-workers-ai", { + api: "openai-completions", + baseUrl: CLOUDFLARE_WORKERS_AI_BASE_URL, + compat: { sendSessionAffinityHeaders: true }, + }), + classifiers: () => CLASSIFIER_MODELS, + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + supportsStrictMode: true, + supportsLongCacheRetention: false, + }, + }, +}); diff --git a/packages/ai/catalog/providers/deepseek.ts b/packages/ai/catalog/providers/deepseek.ts new file mode 100644 index 000000000..af048844d --- /dev/null +++ b/packages/ai/catalog/providers/deepseek.ts @@ -0,0 +1,57 @@ +import type { Model } from "../../src/types.ts"; +import { defineProvider, mergeCompat } from "../define.ts"; +import { DEEPSEEK_COMPAT, DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP } from "../families/deepseek.ts"; + +// models.dev's DeepSeek entries lag the API; pricing from +// https://api-docs.deepseek.com/quick_start/pricing. DeepSeek also offers +// time-based off-peak rates, which the cost schema cannot represent yet. +const MODELS: Model<"openai-completions">[] = [ + { + id: "deepseek-flash", + name: "DeepSeek V4.1 Flash", + api: "openai-completions", + baseUrl: "https://api.deepseek.com", + provider: "deepseek", + reasoning: true, + thinkingLevelMap: DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP, + input: ["text", "image"], + cost: { input: 0.3, output: 1.2, cacheRead: 0.006, cacheWrite: 0 }, + contextWindow: 1000000, + maxTokens: 384000, + compat: DEEPSEEK_COMPAT, + }, + { + id: "deepseek-v4-pro", + name: "DeepSeek V4 Pro", + api: "openai-completions", + baseUrl: "https://api.deepseek.com", + provider: "deepseek", + reasoning: true, + input: ["text"], + cost: { input: 1.32, output: 3.96, cacheRead: 0.044, cacheWrite: 0 }, + contextWindow: 1000000, + maxTokens: 384000, + compat: DEEPSEEK_COMPAT, + }, +]; + +export default defineProvider({ + id: "deepseek", + models: () => structuredClone(MODELS), + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + maxTokensField: "max_tokens", + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "deepseek", + supportsStrictMode: true, + }, + }, + finalize: (model) => { + // DeepSeek V4 Pro accepts plain system text in place mid-conversation. + if (model.id === "deepseek-v4-pro") { + mergeCompat(model as Model<"openai-completions">, { supportsMidConvoSystemMessages: true }); + } + }, +}); diff --git a/packages/ai/catalog/providers/fireworks.ts b/packages/ai/catalog/providers/fireworks.ts new file mode 100644 index 000000000..9751c022b --- /dev/null +++ b/packages/ai/catalog/providers/fireworks.ts @@ -0,0 +1,100 @@ +import type { AnthropicMessagesCompat, Model, OpenAICompletionsCompat } from "../../src/types.ts"; +import { defineProvider, mergeCompat, mergeThinking, modelsDevModels } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +// Fireworks Anthropic-compatible API. Prompt caching uses automatic prefix +// matching and session affinity: x-session-affinity routes requests to the same +// replica for cache hits. cache_control on tools and eager_input_streaming are +// not supported. +// https://docs.fireworks.ai/tools-sdks/anthropic-compatibility +const ANTHROPIC_COMPAT: AnthropicMessagesCompat = { + allowEmptySignature: true, + sendSessionAffinityHeaders: true, + supportsEagerToolInputStreaming: false, + supportsCacheControlOnTools: false, + supportsLongCacheRetention: false, +}; +const OPENAI_COMPAT: OpenAICompletionsCompat = { + supportsStore: false, + supportsDeveloperRole: false, + sendSessionAffinityHeaders: true, + supportsLongCacheRetention: false, +}; +const KIMI_K3_COMPAT: OpenAICompletionsCompat = { + ...OPENAI_COMPAT, + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "openai", +}; + +// Verified against Fireworks Messages raw_output on 2026-09-10 (#9323). +// Fall back to verified support when models.dev omits effort metadata; this is +// not an allowlist. Any Fireworks Messages model advertising effort uses adaptive thinking. +const ADAPTIVE_THINKING_FALLBACK_MODELS = new Set([ + "accounts/fireworks/models/deepseek-v4-flash-0731", + "accounts/fireworks/models/deepseek-v4-flash-vision-exp", + "accounts/fireworks/models/deepseek-v4-pro-0813", + "accounts/fireworks/models/qwen3p8-max", + "accounts/fireworks/models/qwen3p8-2p4t-a95b", +]); + +export default defineProvider({ + id: "fireworks", + models: (ctx) => + modelsDevModels(ctx, "fireworks-ai", { + api: "anthropic-messages", + // SDK appends /v1/messages. + baseUrl: "https://api.fireworks.ai/inference", + map: (model, source) => { + if (model.id.includes("glm-") || model.id.includes("kimi-k3")) { + return { + ...model, + api: "openai-completions", + baseUrl: "https://api.fireworks.ai/inference/v1", + compat: model.id.includes("kimi-k3") ? KIMI_K3_COMPAT : OPENAI_COMPAT, + }; + } + // Use adaptive thinking for cataloged effort controls, with verified + // fallbacks where models.dev is incomplete. New models need no allowlist entry. + const adaptive = + source.reasoningOptions?.some((option) => option.type === "effort") || + ADAPTIVE_THINKING_FALLBACK_MODELS.has(model.id); + return { ...model, compat: { ...ANTHROPIC_COMPAT, ...(adaptive ? { forceAdaptiveThinking: true } : {}) } }; + }, + }), + compat: { "openai-completions": { supportsStrictMode: true } }, + finalize: (model, ctx) => { + if (model.api === "anthropic-messages" && (model as Model<"anthropic-messages">).compat?.forceAdaptiveThinking) { + // Qwen Max currently advertises only a toggle. Prefer upstream effort + // metadata once available instead of replacing it with this fallback. + if (model.id === "accounts/fireworks/models/qwen3p8-max" && !model.thinkingLevelMap) { + model.thinkingLevelMap = getEffortThinkingLevelMap([ + { type: "effort", values: ["low", "medium", "xhigh"] }, + ]); + } + if ( + ctx.modelsDevSource(model)?.reasoningOptions?.some((option) => option.type === "toggle") || + // The 2.4T alias omits the verified toggle in models.dev. + model.id === "accounts/fireworks/models/qwen3p8-2p4t-a95b" + ) { + mergeThinking(model, { off: "none" }); + } + if (model.id === "accounts/fireworks/models/deepseek-v4-pro-0813") mergeThinking(model, { low: "low" }); + } + if (model.id.includes("glm-5p2")) { + // GLM 5.2 and its fast router support off/high/max. Fireworks maps low + // and medium to high, so do not expose those aliases as distinct levels. + mergeThinking(model, { off: "none", minimal: null, low: null, medium: null, max: "max" }); + } + if (model.id.includes("kimi-k3")) { + // Fireworks maps medium to high on both APIs; do not expose it as a distinct level. + mergeThinking(model, { medium: null }); + // Kimi K3 accepts tool-bearing system messages; Fireworks passes them through. + if (model.api === "openai-completions") { + mergeCompat(model as Model<"openai-completions">, { + supportsMidConvoSystemMessages: true, + supportsMidConvoToolAdditions: true, + }); + } + } + }, +}); diff --git a/packages/ai/catalog/providers/github-copilot.ts b/packages/ai/catalog/providers/github-copilot.ts new file mode 100644 index 000000000..2bea3586b --- /dev/null +++ b/packages/ai/catalog/providers/github-copilot.ts @@ -0,0 +1,122 @@ +import type { Api, Model, ThinkingLevelMap } from "../../src/types.ts"; +import { addMissing, type ChatModel, defineProvider, mergeCompat, mergeThinking, modelsDevModels } from "../define.ts"; +import { claude55ThinkingLevels, supportsClaudeMidConvoSystemMessages } from "../families/anthropic.ts"; +import { + enableOpenAiGrammarTools, + enableOpenAiMidConvoSystemMessages, + openAiStandardCost, +} from "../families/openai.ts"; + +const BASE_URL = "https://api.individual.githubcopilot.com"; +const STATIC_HEADERS = { + "User-Agent": "GitHubCopilotChat/0.35.0", + "Editor-Version": "vscode/1.107.0", + "Editor-Plugin-Version": "copilot-chat/0.35.0", + "Copilot-Integration-Id": "vscode-chat", +} as const; + +// GitHub's "Models with extended capabilities" table lists these Copilot models as supporting +// the extended 1 million token context window. +const EXTENDED_CONTEXT_MODEL_IDS = new Set([ + "claude-fable-5", + "claude-opus-4.8", + "claude-opus-5", + "claude-opus-5.5", + "claude-sonnet-4.6", + "claude-sonnet-5", + "gpt-5.3-codex", + "gpt-6-astra", + "gpt-6-luna", + "gpt-6-sol", +]); + +const EAGER_TOOL_INPUT_STREAMING_UNSUPPORTED_MODEL_IDS = new Set([ + "claude-haiku-4.5", + "claude-sonnet-4", + "claude-sonnet-4.5", +]); + +// Checked manually against the authenticated GitHub Copilot /models endpoint on 2026-06-15. +// Keep this to narrow corrections over models.dev metadata instead of snapshotting Copilot's catalog. +const THINKING_LEVEL_OVERRIDES: Record = { + "claude-opus-4.8": { minimal: "low" }, + "claude-opus-5": { minimal: "low" }, + "claude-sonnet-4.6": { minimal: "low", max: "max" }, +}; + +/** Claude 4.x and 5.x use Messages; GPT, Grok, OSWE, and MAI-Code are only served on /responses. */ +function copilotApi(modelId: string): Api { + if (/^claude-(haiku|sonnet|opus|fable)-[45]([.-]|$)/.test(modelId)) return "anthropic-messages"; + if (/^(gpt-|grok-|oswe|mai-)/.test(modelId)) return "openai-responses"; + return "openai-completions"; +} + +// The authenticated Copilot catalog advertised these models on 2026-09-22, +// but models.dev did not include them yet. +const GPT_6_LUNA: Model<"openai-responses"> = { + id: "gpt-6-luna", + name: "GPT-6 Luna", + api: "openai-responses", + provider: "github-copilot", + baseUrl: BASE_URL, + reasoning: true, + input: ["text", "image"], + cost: openAiStandardCost("gpt-6-luna"), + contextWindow: 1000000, + maxTokens: 128000, + headers: { ...STATIC_HEADERS }, +}; + +export default defineProvider({ + id: "github-copilot", + models: (ctx) => + addMissing( + modelsDevModels(ctx, "github-copilot", { + api: "openai-completions", + baseUrl: BASE_URL, + headers: STATIC_HEADERS, + skipDeprecated: true, + defaultContextWindow: 128000, + defaultMaxTokens: 8192, + map: (model): ChatModel => { + const api = copilotApi(model.id); + const result: ChatModel = { ...model, api }; + if (api === "openai-completions") { + result.compat = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + }; + } + if (model.id === "claude-opus-5.5") mergeThinking(result, claude55ThinkingLevels(model.id)); + return result; + }, + }), + [GPT_6_LUNA], + ).map((model) => (EXTENDED_CONTEXT_MODEL_IDS.has(model.id) ? { ...model, contextWindow: 1000000 } : model)), + compat: { "openai-completions": { supportsStrictMode: true } }, + finalize: (model) => { + if (model.id.startsWith("gpt-5")) mergeThinking(model, { minimal: "low" }); + const override = THINKING_LEVEL_OVERRIDES[model.id]; + if (override) mergeThinking(model, override); + if (model.api === "anthropic-messages") { + const messagesModel = model as Model<"anthropic-messages">; + // Copilot forwards mid-conversation system messages but rejects + // `tool_addition`/`tool_removal` blocks, so tool changes stay top-level. + if (supportsClaudeMidConvoSystemMessages(model.id)) { + mergeCompat(messagesModel, { supportsMidConvoSystemMessages: true }); + } + if (EAGER_TOOL_INPUT_STREAMING_UNSUPPORTED_MODEL_IDS.has(model.id)) { + mergeCompat(messagesModel, { supportsEagerToolInputStreaming: false }); + } + } + // Copilot forwards Kimi K3's system text but silently drops its tool-bearing message. + if (model.api === "openai-completions" && model.id === "kimi-k3") { + mergeCompat(model as Model<"openai-completions">, { supportsMidConvoSystemMessages: true }); + } + // Copilot passes mid-conversation developer messages and `additional_tools` + // through to OpenAI unchanged; tool search is not verified through it. + enableOpenAiMidConvoSystemMessages(model, { proxy: true }); + enableOpenAiGrammarTools(model); + }, +}); diff --git a/packages/ai/catalog/providers/google-vertex.ts b/packages/ai/catalog/providers/google-vertex.ts new file mode 100644 index 000000000..e74a4ed5f --- /dev/null +++ b/packages/ai/catalog/providers/google-vertex.ts @@ -0,0 +1,43 @@ +import { defineProvider, modelsDevModels } from "../define.ts"; +import { googleThinkingLevels } from "../families/google.ts"; +import { modelsDevChatModels } from "../sources/models-dev.ts"; +import { GOOGLE_ALIAS_TARGETS } from "./google.ts"; + +export default defineProvider({ + id: "google-vertex", + models: (ctx) => { + const sources = new Map( + modelsDevChatModels(ctx.sources.modelsDev, "google-vertex").map((model) => [model.id, model]), + ); + return modelsDevModels(ctx, "google-vertex", { + api: "google-vertex", + baseUrl: "https://{location}-aiplatform.googleapis.com", + recordReasoning: false, + // The models.dev catalog also includes Claude, OpenAI, and other MaaS models + // that do not use the @google/genai Gemini streaming path. + include: (source) => source.id.startsWith("gemini-") && source.id !== "gemini-3.1-flash-lite-preview", + map: (model, aliasSource) => { + const target = GOOGLE_ALIAS_TARGETS[model.id]; + const source = (target && sources.get(target)) || aliasSource; + const thinkingLevelMap = googleThinkingLevels(model.id, source.reasoningOptions); + return { + ...model, + reasoning: source.reasoning, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + input: source.imageInput ? ["text", "image"] : ["text"], + cost: { + input: source.cost.input || 0, + output: source.cost.output || 0, + // models.dev reports Vertex cache_read/cache_write values for Gemini 2.5 Flash that + // do not match the official Gemini API standard pricing table. pi only accounts + // cachedContentTokenCount as cacheRead. + cacheRead: model.id === "gemini-2.5-flash" ? 0.03 : source.cost.cacheRead || 0, + cacheWrite: 0, + }, + contextWindow: source.contextWindow || 4096, + maxTokens: source.maxTokens || 4096, + }; + }, + }); + }, +}); diff --git a/packages/ai/catalog/providers/google.ts b/packages/ai/catalog/providers/google.ts new file mode 100644 index 000000000..15cac1f93 --- /dev/null +++ b/packages/ai/catalog/providers/google.ts @@ -0,0 +1,36 @@ +import { defineProvider, modelsDevModels, withInputLimits } from "../define.ts"; +import { googleThinkingLevels } from "../families/google.ts"; +import { modelsDevChatModels, modelsDevCost } from "../sources/models-dev.ts"; + +// Rolling aliases take their metadata from the model they currently point to. +export const GOOGLE_ALIAS_TARGETS: Record = { + "gemini-flash-latest": "gemini-3.5-flash", + "gemini-flash-lite-latest": "gemini-3.1-flash-lite", +}; + +export default defineProvider({ + id: "google", + models: (ctx) => { + const sources = new Map(modelsDevChatModels(ctx.sources.modelsDev, "google").map((model) => [model.id, model])); + return modelsDevModels(ctx, "google", { + api: "google-generative-ai", + baseUrl: "https://generativelanguage.googleapis.com/v1beta", + recordReasoning: false, + map: (model, aliasSource) => { + const target = GOOGLE_ALIAS_TARGETS[model.id]; + const source = (target && sources.get(target)) || aliasSource; + const thinkingLevelMap = googleThinkingLevels(model.id, source.reasoningOptions); + return { + ...model, + reasoning: source.reasoning, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + input: source.imageInput ? ["text", "image"] : ["text"], + cost: modelsDevCost(source.cost), + contextWindow: source.contextWindow || 4096, + maxTokens: source.maxTokens || 4096, + }; + }, + }); + }, + finalize: (model) => withInputLimits(model, { maxRequestBytes: 20 * 1024 * 1024, images: { maxPerRequest: 3600 } }), +}); diff --git a/packages/ai/catalog/providers/groq.ts b/packages/ai/catalog/providers/groq.ts new file mode 100644 index 000000000..7f6e15d76 --- /dev/null +++ b/packages/ai/catalog/providers/groq.ts @@ -0,0 +1,13 @@ +import { defineProvider, mergeThinking, modelsDevModels } from "../define.ts"; + +export default defineProvider({ + id: "groq", + models: (ctx) => + modelsDevModels(ctx, "groq", { api: "openai-completions", baseUrl: "https://api.groq.com/openai/v1" }), + compat: { "openai-completions": { supportsStrictMode: true } }, + finalize: (model) => { + if (model.id === "qwen/qwen3.6-27b") { + mergeThinking(model, { minimal: null, low: null, medium: null, high: "default" }); + } + }, +}); diff --git a/packages/ai/catalog/providers/huggingface.ts b/packages/ai/catalog/providers/huggingface.ts new file mode 100644 index 000000000..69ab9104c --- /dev/null +++ b/packages/ai/catalog/providers/huggingface.ts @@ -0,0 +1,12 @@ +import { defineProvider, modelsDevModels } from "../define.ts"; + +export default defineProvider({ + id: "huggingface", + models: (ctx) => + modelsDevModels(ctx, "huggingface", { + api: "openai-completions", + baseUrl: "https://router.huggingface.co/v1", + compat: { supportsDeveloperRole: false }, + }), + compat: { "openai-completions": { supportsStrictMode: true } }, +}); diff --git a/packages/ai/catalog/providers/index.ts b/packages/ai/catalog/providers/index.ts new file mode 100644 index 000000000..60b29cbd5 --- /dev/null +++ b/packages/ai/catalog/providers/index.ts @@ -0,0 +1,90 @@ +// Every built-in provider catalog. Add new providers here. + +import type { ProviderDefinition } from "../define.ts"; +import amazonBedrock from "./amazon-bedrock.ts"; +import antLing from "./ant-ling.ts"; +import anthropic from "./anthropic.ts"; +import azure from "./azure.ts"; +import baseten from "./baseten.ts"; +import cerebras from "./cerebras.ts"; +import cloudflareAiGateway from "./cloudflare-ai-gateway.ts"; +import cloudflareWorkersAi from "./cloudflare-workers-ai.ts"; +import deepseek from "./deepseek.ts"; +import fireworks from "./fireworks.ts"; +import githubCopilot from "./github-copilot.ts"; +import google from "./google.ts"; +import googleVertex from "./google-vertex.ts"; +import groq from "./groq.ts"; +import huggingface from "./huggingface.ts"; +import kimiCoding from "./kimi-coding.ts"; +import meta from "./meta.ts"; +import minimax from "./minimax.ts"; +import minimaxCn from "./minimax-cn.ts"; +import mistral from "./mistral.ts"; +import moonshotai from "./moonshotai.ts"; +import moonshotaiCn from "./moonshotai-cn.ts"; +import nvidia from "./nvidia.ts"; +import openai from "./openai.ts"; +import openaiCodex from "./openai-codex.ts"; +import opencode from "./opencode.ts"; +import opencodeGo from "./opencode-go.ts"; +import openrouter from "./openrouter.ts"; +import qwenTokenPlan from "./qwen-token-plan.ts"; +import qwenTokenPlanCn from "./qwen-token-plan-cn.ts"; +import qwenTokenPlanIndividual from "./qwen-token-plan-individual.ts"; +import radius from "./radius.ts"; +import together from "./together.ts"; +import typesafe from "./typesafe.ts"; +import vercelAiGateway from "./vercel-ai-gateway.ts"; +import xai from "./xai.ts"; +import xiaomi from "./xiaomi.ts"; +import xiaomiTokenPlanAms from "./xiaomi-token-plan-ams.ts"; +import xiaomiTokenPlanCn from "./xiaomi-token-plan-cn.ts"; +import xiaomiTokenPlanSgp from "./xiaomi-token-plan-sgp.ts"; +import zai from "./zai.ts"; +import zaiCodingCn from "./zai-coding-cn.ts"; + +export const PROVIDERS: readonly ProviderDefinition[] = [ + amazonBedrock, + antLing, + anthropic, + azure, + baseten, + cerebras, + cloudflareAiGateway, + cloudflareWorkersAi, + deepseek, + fireworks, + githubCopilot, + googleVertex, + google, + groq, + huggingface, + kimiCoding, + meta, + minimaxCn, + minimax, + mistral, + moonshotaiCn, + moonshotai, + nvidia, + openaiCodex, + openai, + opencodeGo, + opencode, + openrouter, + qwenTokenPlanCn, + qwenTokenPlanIndividual, + qwenTokenPlan, + radius, + together, + typesafe, + vercelAiGateway, + xai, + xiaomiTokenPlanAms, + xiaomiTokenPlanCn, + xiaomiTokenPlanSgp, + xiaomi, + zaiCodingCn, + zai, +]; diff --git a/packages/ai/catalog/providers/kimi-coding.ts b/packages/ai/catalog/providers/kimi-coding.ts new file mode 100644 index 000000000..782b44455 --- /dev/null +++ b/packages/ai/catalog/providers/kimi-coding.ts @@ -0,0 +1,55 @@ +import type { Model, ModelCost } from "../../src/types.ts"; +import { defineProvider, modelsDevModels } from "../define.ts"; +import { KIMI_K3_COST, KIMI_K3_OPENAI_COMPAT, KIMI_OPENAI_COMPAT } from "../families/kimi.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +// Kimi Coding is subscription-backed, so models.dev reports zero cost. Use the +// equivalent Moonshot API rates to estimate the value of subscription usage. +const IMPLIED_COSTS: Record = { + k3: KIMI_K3_COST, + "k3-256k": KIMI_K3_COST, + "k3.1-pro": { input: 2, output: 10, cacheRead: 0.2, cacheWrite: 0 }, + "k3.1-pro-highspeed": { input: 4, output: 20, cacheRead: 0.2, cacheWrite: 0 }, + "kimi-for-coding": { input: 0.95, output: 4, cacheRead: 0.19, cacheWrite: 0 }, + "kimi-for-coding-highspeed": { input: 1.9, output: 8, cacheRead: 0.38, cacheWrite: 0 }, +}; + +// Kimi Coding uses Chat Completions, the same protocol as the official Kimi Code client. +export default defineProvider({ + id: "kimi-coding", + models: (ctx) => + modelsDevModels(ctx, "kimi-code-plan-global", { + api: "openai-completions", + baseUrl: "https://api.kimi.com/coding/v1", + recordReasoning: false, + map: (model, source): Model<"openai-completions"> => { + // K3-style models report effort levels and dynamic tool loading (tools declared in + // later system messages) in https://api.kimi.com/coding/v1/models; models.dev mirrors + // the effort levels. kimi-for-coding-highspeed reports neither and rejects + // tool-bearing system messages with a 400. + const isK3Style = source.reasoningOptions?.some((option) => option.type === "effort") === true; + const impliedCost = IMPLIED_COSTS[model.id]; + // K3-style models send reasoning_effort, so expose the effort values models.dev lists. + const effortLevels = + isK3Style && source.reasoningOptions ? getEffortThinkingLevelMap(source.reasoningOptions) : undefined; + return { + ...model, + compat: { + ...(isK3Style ? KIMI_K3_OPENAI_COMPAT : KIMI_OPENAI_COMPAT), + // All models accept system text after the conversation starts. + supportsMidConvoSystemMessages: true, + ...(isK3Style ? { supportsMidConvoToolAdditions: true } : {}), + }, + cost: { + input: source.cost.input || impliedCost?.input || 0, + output: source.cost.output || impliedCost?.output || 0, + cacheRead: source.cost.cacheRead || impliedCost?.cacheRead || 0, + cacheWrite: source.cost.cacheWrite || impliedCost?.cacheWrite || 0, + }, + // Kimi Code reports every model as always-thinking. + thinkingLevelMap: { ...effortLevels, off: null }, + }; + }, + }), + compat: { "openai-completions": { supportsStrictMode: true } }, +}); diff --git a/packages/ai/catalog/providers/meta.ts b/packages/ai/catalog/providers/meta.ts new file mode 100644 index 000000000..7d4bba68d --- /dev/null +++ b/packages/ai/catalog/providers/meta.ts @@ -0,0 +1,6 @@ +import { defineProvider, modelsDevModels } from "../define.ts"; + +export default defineProvider({ + id: "meta", + models: (ctx) => modelsDevModels(ctx, "meta", { api: "openai-responses", baseUrl: "https://api.meta.ai/v1" }), +}); diff --git a/packages/ai/catalog/providers/minimax-cn.ts b/packages/ai/catalog/providers/minimax-cn.ts new file mode 100644 index 000000000..c34cd2316 --- /dev/null +++ b/packages/ai/catalog/providers/minimax-cn.ts @@ -0,0 +1,7 @@ +import { defineProvider } from "../define.ts"; +import { minimaxModels } from "./minimax.ts"; + +export default defineProvider({ + id: "minimax-cn", + models: (ctx) => minimaxModels(ctx, "minimax-cn", "https://api.minimaxi.com/anthropic"), +}); diff --git a/packages/ai/catalog/providers/minimax.ts b/packages/ai/catalog/providers/minimax.ts new file mode 100644 index 000000000..7504b7683 --- /dev/null +++ b/packages/ai/catalog/providers/minimax.ts @@ -0,0 +1,26 @@ +import { defineProvider, modelsDevModels, type ProviderContext } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +// MiniMax asked to hide older models from its direct providers (#2596). +const HIDDEN_MODEL_IDS = new Set(["MiniMax-M2", "MiniMax-M2.1", "MiniMax-M2.5", "MiniMax-M2.5-highspeed"]); + +/** MiniMax's Anthropic-compatible API. The SDK appends /v1/messages. */ +export function minimaxModels(ctx: ProviderContext, source: string, baseUrl: string) { + return modelsDevModels(ctx, source, { + api: "anthropic-messages", + baseUrl, + recordReasoning: false, + include: (model) => !HIDDEN_MODEL_IDS.has(model.id), + map: (model, source) => { + // Models with effort levels (e.g. MiniMax-M3.1-Flash) take adaptive thinking with + // output_config.effort and reject thinking.type="disabled" (no "none" effort, so off is null). + const thinkingLevelMap = getEffortThinkingLevelMap(source.reasoningOptions ?? []); + return thinkingLevelMap ? { ...model, thinkingLevelMap, compat: { forceAdaptiveThinking: true } } : model; + }, + }); +} + +export default defineProvider({ + id: "minimax", + models: (ctx) => minimaxModels(ctx, "minimax", "https://api.minimax.io/anthropic"), +}); diff --git a/packages/ai/catalog/providers/mistral.ts b/packages/ai/catalog/providers/mistral.ts new file mode 100644 index 000000000..8009e4650 --- /dev/null +++ b/packages/ai/catalog/providers/mistral.ts @@ -0,0 +1,46 @@ +import { addMissing, defineProvider, modelsDevModels } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; +import { roundCost } from "../sources/models-dev.ts"; + +export default defineProvider({ + id: "mistral", + models: (ctx) => + addMissing( + modelsDevModels(ctx, "mistral", { + api: "mistral-conversations", + baseUrl: "https://api.mistral.ai", + recordReasoning: false, + map: (model, source) => { + // Models with effort values use `reasoning_effort` with these levels. + // Reasoning models without them (Magistral) use `prompt_mode`. + const thinkingLevelMap = getEffortThinkingLevelMap(source.reasoningOptions ?? []); + return { + ...model, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + cost: { + input: source.cost.input || 0, + output: source.cost.output || 0, + // Mistral bills cached input at 10% when models.dev does not say otherwise. + cacheRead: source.cost.cacheRead ?? (source.cost.input ? roundCost(source.cost.input * 0.1) : 0), + cacheWrite: source.cost.cacheWrite || 0, + }, + }; + }, + }), + [ + { + id: "mistral-medium-3.5", + name: "Mistral Medium 3.5", + api: "mistral-conversations", + provider: "mistral", + baseUrl: "https://api.mistral.ai", + reasoning: true, + thinkingLevelMap: getEffortThinkingLevelMap([{ type: "effort", values: ["none", "high"] }]), + input: ["text", "image"], + cost: { input: 1.5, output: 7.5, cacheRead: 0, cacheWrite: 0 }, + contextWindow: 262144, + maxTokens: 262144, + }, + ], + ), +}); diff --git a/packages/ai/catalog/providers/moonshotai-cn.ts b/packages/ai/catalog/providers/moonshotai-cn.ts new file mode 100644 index 000000000..e8f3d91e1 --- /dev/null +++ b/packages/ai/catalog/providers/moonshotai-cn.ts @@ -0,0 +1,9 @@ +import { defineProvider } from "../define.ts"; +import { finalizeMoonshotModel, MOONSHOT_COMPLETIONS_COMPAT, moonshotModels } from "./moonshotai.ts"; + +export default defineProvider({ + id: "moonshotai-cn", + models: (ctx) => moonshotModels(ctx, "moonshotai-cn", "https://api.moonshot.cn/v1"), + compat: { "openai-completions": MOONSHOT_COMPLETIONS_COMPAT }, + finalize: finalizeMoonshotModel, +}); diff --git a/packages/ai/catalog/providers/moonshotai.ts b/packages/ai/catalog/providers/moonshotai.ts new file mode 100644 index 000000000..11ba2a5cc --- /dev/null +++ b/packages/ai/catalog/providers/moonshotai.ts @@ -0,0 +1,71 @@ +import type { Model } from "../../src/types.ts"; +import { + type ChatModel, + defineProvider, + mergeCompat, + mergeThinking, + modelsDevModels, + type ProviderContext, +} from "../define.ts"; +import { isKimiK3FamilyId, KIMI_K3_COST, KIMI_K3_OPENAI_COMPAT, KIMI_OPENAI_COMPAT } from "../families/kimi.ts"; + +const ALWAYS_THINKING_MODEL_IDS = new Set(["kimi-k2.7-code", "kimi-k2.7-code-highspeed"]); +// Kimi K2.6/K2.7 accept system text after the conversation starts but reject +// tool-bearing system messages. +const TEXT_ONLY_MID_CONVO_MODEL_IDS = new Set(["kimi-k2.6", ...ALWAYS_THINKING_MODEL_IDS]); + +export function moonshotModels(ctx: ProviderContext, source: string, baseUrl: string) { + return modelsDevModels(ctx, source, { + api: "openai-completions", + baseUrl, + map: (model, source): Model<"openai-completions"> => { + const isKimiK3 = isKimiK3FamilyId(model.id); + return { + ...model, + reasoning: isKimiK3 || source.reasoning, + // Moonshot does not bill cache writes; models.dev lists the input rate as cache_write for the K3 family. + cost: + model.id === "kimi-k3" + ? { ...KIMI_K3_COST } + : { + input: source.cost.input || 0, + output: source.cost.output || 0, + cacheRead: source.cost.cacheRead || 0, + cacheWrite: isKimiK3 ? 0 : source.cost.cacheWrite || 0, + }, + compat: { ...(isKimiK3 ? KIMI_K3_OPENAI_COMPAT : KIMI_OPENAI_COMPAT) }, + }; + }, + }); +} + +export const MOONSHOT_COMPLETIONS_COMPAT = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", +} as const; + +export function finalizeMoonshotModel(model: ChatModel): void { + if (ALWAYS_THINKING_MODEL_IDS.has(model.id)) { + // Kimi K2.7 Code is always-thinking. Official docs say + // `thinking: { type: "disabled" }` is rejected, and callers can omit + // the thinking parameter to use the enabled default. + mergeThinking(model, { off: null }); + } + const isKimiK3 = isKimiK3FamilyId(model.id); + if (isKimiK3 || TEXT_ONLY_MID_CONVO_MODEL_IDS.has(model.id)) { + mergeCompat(model as Model<"openai-completions">, { + supportsMidConvoSystemMessages: true, + // The Kimi K3 family also accepts tool-bearing system messages. + ...(isKimiK3 ? { supportsMidConvoToolAdditions: true } : {}), + }); + } +} + +export default defineProvider({ + id: "moonshotai", + models: (ctx) => moonshotModels(ctx, "moonshotai", "https://api.moonshot.ai/v1"), + compat: { "openai-completions": MOONSHOT_COMPLETIONS_COMPAT }, + finalize: finalizeMoonshotModel, +}); diff --git a/packages/ai/catalog/providers/nvidia.ts b/packages/ai/catalog/providers/nvidia.ts new file mode 100644 index 000000000..1d492ff29 --- /dev/null +++ b/packages/ai/catalog/providers/nvidia.ts @@ -0,0 +1,64 @@ +import { defineProvider, modelsDevModels } from "../define.ts"; +import { NVIDIA_BASE_URL, normalizeNvidiaModelId } from "../sources/index.ts"; + +// Listed by NIM but unusable through pi (no tool calls, broken streaming, or retired). +const UNSUPPORTED_MODEL_IDS = new Set([ + "abacusai/dracarys-llama-3.1-70b-instruct", + "bytedance/seed-oss-36b-instruct", + "deepseek-ai/deepseek-v4-flash", + "deepseek-ai/deepseek-v4-pro", + "google/gemma-2-2b-it", + "google/gemma-3n-e2b-it", + "google/gemma-3n-e4b-it", + "google/gemma-4-31b-it", + "meta/llama-3.2-1b-instruct", + "meta/llama-4-maverick-17b-128e-instruct", + "microsoft/phi-4-mini-instruct", + "minimaxai/minimax-m2.7", + "mistralai/mistral-nemotron", + "nvidia/nemotron-mini-4b-instruct", + "qwen/qwen3-next-80b-a3b-instruct", + "qwen/qwen3.5-397b-a17b", + "sarvamai/sarvam-m", + "upstage/solar-10.7b-instruct", +]); + +export default defineProvider({ + id: "nvidia", + models: (ctx) => { + const liveIds = ctx.sources.nvidiaModelIds; + const liveId = (id: string) => liveIds.get(id) ?? liveIds.get(normalizeNvidiaModelId(id)); + return modelsDevModels(ctx, "nvidia", { + api: "openai-completions", + baseUrl: NVIDIA_BASE_URL, + headers: { "NVCF-POLL-SECONDS": "3600" }, + compat: { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsStrictMode: false, + supportsLongCacheRetention: false, + }, + // Only models the live NIM API serves, under the id it uses. + include: (source) => { + if (!source.textInput || !source.textOutput) return false; + const id = liveId(source.id); + return id !== undefined && !UNSUPPORTED_MODEL_IDS.has(id); + }, + map: (model) => { + const id = liveId(model.id) ?? model.id; + return { ...model, id, name: model.name === model.id ? id : model.name }; + }, + }); + }, + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsLongCacheRetention: false, + }, + }, +}); diff --git a/packages/ai/catalog/providers/openai-codex.ts b/packages/ai/catalog/providers/openai-codex.ts new file mode 100644 index 000000000..fc120d039 --- /dev/null +++ b/packages/ai/catalog/providers/openai-codex.ts @@ -0,0 +1,68 @@ +import type { Model } from "../../src/types.ts"; +import { defineProvider, mergeCompat, mergeThinking } from "../define.ts"; +import { + enableOpenAiGrammarTools, + enableOpenAiMidConvoSystemMessages, + OPENAI_TOOL_SEARCH_MODEL_IDS, + openAiStandardCost, + supportsOpenAiXhigh, +} from "../families/openai.ts"; + +// ChatGPT OAuth models. models.dev does not list them; keep a small, explicit +// list without aliases. Only list models the Codex backend accepts for ChatGPT +// accounts: GPT-5.5 and GPT-5.3 Codex Spark are rejected with "not supported +// when using Codex with a ChatGPT account". +const MODELS = [ + // GPT-6.1 uses the 500k default returned by /backend-api/codex/models. + { id: "gpt-6.1-sol", name: "GPT-6.1 Sol", contextWindow: 500000 }, + { id: "gpt-6.1-terra", name: "GPT-6.1 Terra", contextWindow: 500000 }, + // GPT-5.6 and GPT-6 use Codex's 272k default catalog limit. + { id: "gpt-6-astra", name: "GPT-6 Astra", contextWindow: 272000 }, + { id: "gpt-6-sol", name: "GPT-6 Sol", contextWindow: 272000 }, + { id: "gpt-6-luna", name: "GPT-6 Luna", contextWindow: 272000 }, + { id: "gpt-5.6-luna", name: "GPT-5.6 Luna", contextWindow: 272000 }, + { id: "gpt-5.6-sol", name: "GPT-5.6 Sol", contextWindow: 272000 }, + { id: "gpt-5.6-terra", name: "GPT-5.6 Terra", contextWindow: 272000 }, +]; + +// Codex CLI keeps additional_tools behind its own model list. +const ADDITIONAL_TOOLS_MODEL_IDS = new Set([ + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-6-astra", + "gpt-6-sol", + "gpt-6-luna", + "gpt-6.1-sol", + "gpt-6.1-terra", +]); + +export default defineProvider({ + id: "openai-codex", + models: () => + MODELS.map( + ({ id, name, contextWindow }): Model<"openai-codex-responses"> => ({ + id, + name, + api: "openai-codex-responses", + provider: "openai-codex", + baseUrl: "https://chatgpt.com/backend-api", + reasoning: true, + input: ["text", "image"], + cost: openAiStandardCost(id), + contextWindow, + maxTokens: 128000, + }), + ), + finalize: (model) => { + if (supportsOpenAiXhigh(model.id)) mergeThinking(model, { minimal: "low" }); + enableOpenAiGrammarTools(model); + if (OPENAI_TOOL_SEARCH_MODEL_IDS.has(model.id)) { + mergeCompat(model as Model<"openai-codex-responses">, { + ...(ADDITIONAL_TOOLS_MODEL_IDS.has(model.id) ? { supportsAdditionalTools: true } : {}), + supportsToolSearch: true, + }); + } + enableOpenAiMidConvoSystemMessages(model, { proxy: false }); + }, +}); diff --git a/packages/ai/catalog/providers/openai.ts b/packages/ai/catalog/providers/openai.ts new file mode 100644 index 000000000..873dc11ec --- /dev/null +++ b/packages/ai/catalog/providers/openai.ts @@ -0,0 +1,141 @@ +import type { ClassifierModel, Model } from "../../src/types.ts"; +import { + addMissing, + type ChatModel, + defineProvider, + mergeCompat, + mergeThinking, + modelsDevModels, + withInputLimits, +} from "../define.ts"; +import { + enableOpenAiGrammarTools, + enableOpenAiMidConvoSystemMessages, + OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, + OPENAI_STANDARD_COSTS, + OPENAI_TOOL_SEARCH_MODEL_IDS, + withOpenAiLongContextPricing, +} from "../families/openai.ts"; + +const BASE_URL = "https://api.openai.com/v1"; + +// models.dev lists this alias, but it is not accepted by OpenAI APIs. +const UNSUPPORTED_MODEL_IDS = new Set(["gpt-5.6"]); + +// Keep direct OpenAI requests in the short-context pricing tier by default. Users can opt into the +// larger context through model overrides, so retain long-context cost metadata on the capped models. +const SHORT_CONTEXT_CAPPED_MODEL_IDS = new Set([ + "gpt-5.4", + "gpt-5.5", + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-6-astra", + "gpt-6-sol", + "gpt-6-luna", + "gpt-6.1-sol", + "gpt-6.1-terra", +]); +const LONG_CONTEXT_PRICING_MODEL_IDS = new Set([...SHORT_CONTEXT_CAPPED_MODEL_IDS, "gpt-5.4-pro", "gpt-5.5-pro"]); + +const NONE_REASONING_MODEL_IDS = new Set([ + "gpt-5.1", + "gpt-5.2", + "gpt-5.3-codex", + "gpt-5.4", + "gpt-5.4-mini", + "gpt-5.4-nano", + "gpt-5.5", + "gpt-5.6-sol", + "gpt-5.6-terra", + "gpt-5.6-luna", + "gpt-6-sol", + "gpt-6-luna", +]); +// OpenAI documents configuration_update for the GPT-6 family. +// https://developers.openai.com/api/docs/guides/reasoning +const MID_CONVO_EFFORT_MODEL_IDS = new Set(["gpt-6-astra", "gpt-6-sol", "gpt-6-luna", "gpt-6.1-sol", "gpt-6.1-terra"]); + +// Not listed by models.dev. +const GPT_5_CHAT_LATEST: Model<"openai-responses"> = { + id: "gpt-5-chat-latest", + name: "GPT-5 Chat Latest", + api: "openai-responses", + baseUrl: BASE_URL, + provider: "openai", + reasoning: false, + input: ["text", "image"], + cost: { input: 1.25, output: 10, cacheRead: 0.125, cacheWrite: 0 }, + contextWindow: 128000, + maxTokens: 16384, +}; + +// OpenAI Decisions API (public beta): gpt-6-luna is its only model. It bills input tokens only, +// with the same long-context multiplier as chat requests. Only API keys work: Sign in with ChatGPT +// tokens are rejected on /v1/decisions, and the Codex backend has no Decisions route. +// The endpoint rejects inputs above 922K tokens (the model's documented maximum input), but +// requests running longer than about five seconds, currently above roughly 600K input tokens, +// fail with a gateway timeout. +// https://developers.openai.com/api/docs/guides/decisions +const CLASSIFIER_MODELS: ClassifierModel<"openai-decisions">[] = [ + { + type: "classifier", + id: "gpt-6-luna", + name: "GPT-6 Luna", + api: "openai-decisions", + provider: "openai", + baseUrl: BASE_URL, + input: ["text", "image"], + cost: withOpenAiLongContextPricing({ input: 0.1, output: 0, cacheRead: 0, cacheWrite: 0 }), + contextWindow: 922000, + }, +]; + +function correctUpstream(model: ChatModel): ChatModel { + if (SHORT_CONTEXT_CAPPED_MODEL_IDS.has(model.id)) { + model.contextWindow = OPENAI_LONG_CONTEXT_INPUT_THRESHOLD; + model.maxTokens = 128000; + } + if (LONG_CONTEXT_PRICING_MODEL_IDS.has(model.id)) { + model.cost = withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS[model.id] ?? model.cost); + } + // models.dev reports gpt-5-pro output as 272000 (a duplicate of the input sub-limit); + // the actual max output is 128000. Also propagates to the derived Azure clone. + if (model.id === "gpt-5-pro") model.maxTokens = 128000; + return model; +} + +export default defineProvider({ + id: "openai", + models: (ctx) => + addMissing( + modelsDevModels(ctx, "openai", { + api: "openai-responses", + baseUrl: BASE_URL, + // Audio models are only served on Chat Completions. + protocol: "responses", + include: (source) => !UNSUPPORTED_MODEL_IDS.has(source.id), + map: correctUpstream, + }), + [GPT_5_CHAT_LATEST], + ), + classifiers: () => CLASSIFIER_MODELS, + finalize: (model) => { + if (model.api !== "openai-responses") return; + const responsesModel = model as Model<"openai-responses">; + if (NONE_REASONING_MODEL_IDS.has(model.id)) mergeThinking(model, { off: "none" }); + if (model.id === "gpt-5.5") mergeThinking(model, { minimal: null }); + mergeCompat(responsesModel, { supportsStrictMode: true }); + enableOpenAiGrammarTools(model); + if (OPENAI_TOOL_SEARCH_MODEL_IDS.has(model.id)) { + mergeCompat(responsesModel, { supportsAdditionalTools: true, supportsToolSearch: true }); + } + enableOpenAiMidConvoSystemMessages(model, { proxy: false }); + // OpenAI charges prompt-cache writes starting with the GPT-5.6 family, and exactly + // those models accept `prompt_cache_options`; older models reject the parameter. + // https://developers.openai.com/api/docs/guides/prompt-caching + if (model.cost.cacheWrite > 0) mergeCompat(responsesModel, { supportsExplicitPromptCacheMode: true }); + if (MID_CONVO_EFFORT_MODEL_IDS.has(model.id)) mergeCompat(responsesModel, { supportsMidConvoEffort: true }); + withInputLimits(model, { maxRequestBytes: 512 * 1024 * 1024, images: { maxPerRequest: 1500 } }); + }, +}); diff --git a/packages/ai/catalog/providers/opencode-go.ts b/packages/ai/catalog/providers/opencode-go.ts new file mode 100644 index 000000000..b13b778e7 --- /dev/null +++ b/packages/ai/catalog/providers/opencode-go.ts @@ -0,0 +1,9 @@ +import { defineProvider } from "../define.ts"; +import { finalizeOpencodeModel, OPENCODE_COMPLETIONS_COMPAT, opencodeModels } from "./opencode.ts"; + +export default defineProvider({ + id: "opencode-go", + models: opencodeModels, + compat: { "openai-completions": OPENCODE_COMPLETIONS_COMPAT }, + finalize: finalizeOpencodeModel, +}); diff --git a/packages/ai/catalog/providers/opencode.ts b/packages/ai/catalog/providers/opencode.ts new file mode 100644 index 000000000..36057c764 --- /dev/null +++ b/packages/ai/catalog/providers/opencode.ts @@ -0,0 +1,206 @@ +import type { Api, ClassifierModel, Model, OpenAICompletionsCompat, OpenAIResponsesCompat } from "../../src/types.ts"; +import { + type ChatModel, + defineProvider, + mergeCompat, + mergeThinking, + modelsDevModels, + type ProviderContext, +} from "../define.ts"; +import { supportsClaudeMidConvoSystemMessages } from "../families/anthropic.ts"; +import { googleThinkingLevels } from "../families/google.ts"; +import { isKimiK3FamilyId } from "../families/kimi.ts"; +import { enableOpenAiGrammarTools, enableOpenAiMidConvoSystemMessages } from "../families/openai.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; +import type { ModelsDevModel } from "../sources/models-dev.ts"; + +// OpenAI-compatible endpoints reject the long prompt-cache retention parameter for these models. +const LONG_CACHE_RETENTION_UNSUPPORTED_MODELS = new Set([ + "opencode:deepseek-v4-flash", + "opencode:deepseek-v4.1-flash", + "opencode:deepseek-v4-pro", + "opencode:kimi-k2.5", + "opencode:kimi-k2.6", + "opencode:minimax-m2.7", + "opencode-go:kimi-k2.6", +]); + +// OpenCode serves each model on one protocol-specific endpoint. Native protocols +// come before generic Chat Completions if models.dev ever lists several. +const OPENCODE_PROTOCOLS = ["responses", "messages", "generate-content", "chat-completions"] as const; + +interface OpencodeApi { + api: Api; + baseUrl: string; + compat?: OpenAICompletionsCompat | OpenAIResponsesCompat; +} + +/** + * The API and base URL models.dev lists for a model. Models served only on a + * protocol pi does not support fail strict generation and are skipped otherwise. + */ +function opencodeApi(ctx: ProviderContext, source: ModelsDevModel): OpencodeApi | undefined { + const entry = OPENCODE_PROTOCOLS.map((protocol) => + source.apis.find((candidate) => candidate.protocol === protocol), + ).find((candidate) => candidate !== undefined); + if (!entry) { + const protocols = source.apis.map((candidate) => candidate.protocol).join(", ") || "none"; + const message = `models.dev lists no supported API for ${ctx.providerId}/${source.id} (${protocols})`; + if (ctx.strict) throw new Error(message); + console.warn(`Skipping model: ${message}`); + return undefined; + } + switch (entry.protocol) { + case "responses": + return { + api: "openai-responses", + baseUrl: entry.baseUrl, + compat: { sessionAffinityFormat: "openai-nosession" }, + }; + case "messages": + // The Anthropic SDK appends /v1/messages itself. + return { api: "anthropic-messages", baseUrl: entry.baseUrl.replace(/\/v1\/?$/, "") }; + case "generate-content": + return { api: "google-generative-ai", baseUrl: entry.baseUrl }; + case "chat-completions": + return { + api: "openai-completions", + baseUrl: entry.baseUrl, + ...(entry.aisdk === "@ai-sdk/alibaba" ? { compat: { cacheControlFormat: "anthropic" } } : {}), + }; + } +} + +/** OpenCode Zen and Go models, on the endpoints models.dev lists for them. */ +export function opencodeModels(ctx: ProviderContext): ChatModel[] { + const provider = ctx.providerId; + return modelsDevModels(ctx, provider, { + api: "openai-completions", + // Replaced by the base URL of the API models.dev lists for each model. + baseUrl: "", + skipDeprecated: true, + // Codex Spark is not usable through OpenCode. + include: (source) => source.id !== "gpt-5.3-codex-spark", + map: (model, source) => { + const served = opencodeApi(ctx, source); + if (!served) return undefined; + let { api, baseUrl, compat } = served; + if (provider === "opencode" && model.id === "grok-build-0.1") { + compat = { ...compat, supportsReasoningEffort: false }; + } + if (model.id === "kimi-k2.6") { + // OpenCode Kimi K2.6 accepts Anthropic-style thinking objects + // and rejects string thinking values or combined reasoning_effort. + compat = { ...compat, thinkingFormat: "deepseek", supportsReasoningEffort: false }; + } + // models.dev reports these as @ai-sdk/anthropic, but OpenCode Go serves + // them through the OpenAI-compatible /openai/v1/chat/completions path. + if ( + provider === "opencode-go" && + api === "anthropic-messages" && + (model.id === "qwen3.5-plus" || model.id === "qwen3.6-plus") + ) { + api = "openai-completions"; + baseUrl = baseUrl.replace(/\/anthropic$/, "/openai/v1"); + // Qwen/DashScope uses enable_thinking at the top level. + compat = { ...compat, thinkingFormat: "qwen" }; + } + if (api === "openai-completions") { + compat = { + ...compat, + maxTokensField: "max_tokens", + ...(LONG_CACHE_RETENTION_UNSUPPORTED_MODELS.has(`${provider}:${model.id}`) + ? { supportsLongCacheRetention: false } + : {}), + }; + } + const thinkingLevelMap = + api === "google-generative-ai" + ? googleThinkingLevels(model.id, source.reasoningOptions) + : provider === "opencode-go" && model.id === "deepseek-v4.1-flash" + ? getEffortThinkingLevelMap(source.reasoningOptions ?? []) + : undefined; + const result: ChatModel = { ...model, api, baseUrl, ...(thinkingLevelMap ? { thinkingLevelMap } : {}) }; + if (compat) result.compat = compat; + // OpenCode variants list Claude Sonnet 4/4.5 with 1M context, actual limit is 200K. + if (model.id === "claude-sonnet-4-5" || model.id === "claude-sonnet-4") result.contextWindow = 200000; + if (["claude-opus-4-6", "claude-sonnet-4-6", "claude-opus-4.6", "claude-sonnet-4.6"].includes(model.id)) { + result.contextWindow = 1000000; + } + if (model.id === "gpt-5.4") { + result.contextWindow = 272000; + result.maxTokens = 128000; + } + return result; + }, + }); +} + +export const OPENCODE_COMPLETIONS_COMPAT = { + supportsStore: false, + supportsDeveloperRole: false, + supportsStrictMode: true, +} as const; + +/** Behavior shared by OpenCode Zen and Go. */ +export function finalizeOpencodeModel(model: ChatModel): void { + if (model.provider === "opencode-go" && model.id === "glm-5.2") { + mergeThinking(model, { off: null, minimal: null, low: null, medium: null, high: "high", max: "max" }); + } + if (model.provider === "opencode-go" && model.id === "kimi-k2.6") { + // OpenCode Go exposes Kimi K2.6 thinking as on/off, not distinct effort tiers. + mergeThinking(model, { minimal: null, low: null, medium: null }); + } + if (model.provider === "opencode" && model.id === "grok-build-0.1") { + // OpenCode Zen Grok Build reasons by default but rejects explicit reasoningEffort. + mergeThinking(model, { off: null, minimal: null, low: null, medium: null }); + } + if (model.api === "anthropic-messages") { + const messagesModel = model as Model<"anthropic-messages">; + // OpenCode Zen forwards mid-conversation system messages but rejects + // `tool_addition`/`tool_removal` blocks, so tool changes stay top-level. + if (model.provider === "opencode" && supportsClaudeMidConvoSystemMessages(model.id)) { + mergeCompat(messagesModel, { supportsMidConvoSystemMessages: true }); + } + // OpenCode Qwen 3.8 Flash emits and accepts thinking blocks with empty signatures. + if (model.id === "qwen3.8-flash") mergeCompat(messagesModel, { allowEmptySignature: true }); + } + // The Kimi K3 family accepts tool-bearing system messages; OpenCode passes them through. + if (model.api === "openai-completions" && isKimiK3FamilyId(model.id)) { + mergeCompat(model as Model<"openai-completions">, { + supportsMidConvoSystemMessages: true, + supportsMidConvoToolAdditions: true, + }); + } + // OpenCode passes mid-conversation developer messages and `additional_tools` + // through to OpenAI unchanged; tool search is not verified through it. + enableOpenAiMidConvoSystemMessages(model, { proxy: true }); +} + +// OpenCode Zen serves Jev through its TypeSafe-compatible System One endpoint. +// Neither its /inference/openai/v1/models listing nor models.dev carries metadata for it. +// https://opencode.ai/docs/zen +function jev(id: string, name: string, input: number): ClassifierModel<"typesafe-system-one"> { + return { + type: "classifier", + id, + name, + api: "typesafe-system-one", + provider: "opencode", + baseUrl: "https://opencode.ai/inference/systemone/v1", + input: ["text"], + cost: { input, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow: 32000, + }; +} + +export default defineProvider({ + id: "opencode", + models: opencodeModels, + classifiers: () => [jev("jev-1.13", "Jev 1.13", 0.042), jev("jev-1.13-free", "Jev 1.13 Free", 0)], + compat: { "openai-completions": OPENCODE_COMPLETIONS_COMPAT }, + finalize: (model) => { + finalizeOpencodeModel(model); + enableOpenAiGrammarTools(model); + }, +}); diff --git a/packages/ai/catalog/providers/openrouter.ts b/packages/ai/catalog/providers/openrouter.ts new file mode 100644 index 000000000..6d705af93 --- /dev/null +++ b/packages/ai/catalog/providers/openrouter.ts @@ -0,0 +1,108 @@ +import type { Model, OpenAICompletionsCompat } from "../../src/types.ts"; +import { addMissing, type ChatModel, defineProvider, mergeCompat, mergeThinking } from "../define.ts"; +import { enableClaudeMidConvoEffort, supportsClaudeMidConvoEffort } from "../families/anthropic.ts"; +import { isKimiK3FamilyId, KIMI_K3_MAX_TOKENS } from "../families/kimi.ts"; +import { OPENAI_MID_CONVO_MODEL_IDS } from "../families/openai.ts"; + +const BASE_URL = "https://openrouter.ai/api/v1"; +// OpenRouter rejects `configuration_update` system messages on Opus 5 ("Mid-conversation +// reasoning effort (configuration_update) is not supported on anthropic/claude-opus-5-20260723") +// while accepting them on Fable 5.1. +const MID_CONVO_EFFORT_UNSUPPORTED_MODEL_IDS = new Set(["anthropic/claude-opus-5"]); + +/** Keep selected model metadata stable until upstream settles. */ +function correctUpstream(model: ChatModel): ChatModel { + // Keep Kimi K3's canonical output limit when gateway metadata is missing or incorrect. + if ( + model.id === "~moonshotai/kimi-latest" || + (model.id.startsWith("moonshotai/") && isKimiK3FamilyId(model.id.slice("moonshotai/".length))) + ) { + model.maxTokens = KIMI_K3_MAX_TOKENS; + } + if (model.id === "moonshotai/kimi-k2.5") { + model.cost = { ...model.cost, input: 0.41, output: 2.06, cacheRead: 0.07 }; + model.maxTokens = 4096; + } + if (model.id.startsWith("moonshotai/kimi-k2.6")) { + mergeCompat(model as Model<"openai-completions">, { + supportsDeveloperRole: false, + requiresReasoningContentOnAssistantMessages: true, + }); + } + if (model.id === "z-ai/glm-5") model.cost = { ...model.cost, input: 0.6, output: 1.9, cacheRead: 0.119 }; + return model; +} + +// Router aliases. Their price depends on the model they route to, so it is unknown here. +const ROUTER_MODELS: Model<"openai-completions">[] = [ + { + id: "auto", + name: "Auto", + api: "openai-completions", + provider: "openrouter", + baseUrl: BASE_URL, + reasoning: true, + input: ["text", "image"], + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow: 2000000, + maxTokens: 30000, + }, + // OpenRouter exposes Fusion as a router alias/plugin entry point; its model + // metadata does not advertise tools, but the alias resolves to a concrete model + // that can invoke caller tools and has the openrouter:fusion server tool auto-injected. + { + id: "openrouter/fusion", + name: "OpenRouter: Fusion", + api: "openai-completions", + provider: "openrouter", + baseUrl: BASE_URL, + reasoning: true, + input: ["text"], + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow: 1000000, + maxTokens: 30000, + }, +]; + +export default defineProvider({ + id: "openrouter", + models: (ctx) => + addMissing(structuredClone(ctx.sources.openRouter.chat).map(correctUpstream), structuredClone(ROUTER_MODELS)), + images: (ctx) => structuredClone(ctx.sources.openRouter.images), + classifiers: (ctx) => structuredClone(ctx.sources.openRouter.classifiers), + compat: { + "openai-completions": (model): OpenAICompletionsCompat => ({ + // OpenRouter passes developer messages through to Anthropic and OpenAI models only. + ...(/^(anthropic|openai)\//.test(model.id) ? {} : { supportsDeveloperRole: false }), + thinkingFormat: "openrouter", + supportsStrictMode: true, + ...(/^~?anthropic\//.test(model.id) ? { cacheControlFormat: "anthropic" } : {}), + sendSessionAffinityHeaders: true, + }), + }, + finalize: (model) => { + if ( + model.api === "anthropic-messages" && + supportsClaudeMidConvoEffort(model.id) && + !MID_CONVO_EFFORT_UNSUPPORTED_MODEL_IDS.has(model.id) + ) { + enableClaudeMidConvoEffort(model as Model<"anthropic-messages">); + } + if (model.id.startsWith("inception/mercury-2")) { + // Mercury 2 in instant mode (reasoning_effort: "none") disables tool calling. + // Mark "off" unsupported so the openai-completions provider omits the reasoning param + // instead of defaulting to {reasoning:{effort:"none"}}. + // Pi's low/medium/high pass through verbatim; OpenRouter normalizes to Mercury's vocabulary. + mergeThinking(model, { off: null }); + } + if (model.id === "z-ai/glm-5.2") mergeThinking(model, { xhigh: "xhigh" }); + // OpenAI models behind OpenRouter accept plain system text in place. + if ( + model.api === "openai-completions" && + model.id.startsWith("openai/") && + OPENAI_MID_CONVO_MODEL_IDS.has(model.id.slice("openai/".length)) + ) { + mergeCompat(model as Model<"openai-completions">, { supportsMidConvoSystemMessages: true }); + } + }, +}); diff --git a/packages/ai/catalog/providers/qwen-token-plan-cn.ts b/packages/ai/catalog/providers/qwen-token-plan-cn.ts new file mode 100644 index 000000000..33b210dd9 --- /dev/null +++ b/packages/ai/catalog/providers/qwen-token-plan-cn.ts @@ -0,0 +1,13 @@ +import { defineProvider } from "../define.ts"; +import { QWEN_COMPLETIONS_COMPAT, qwenTokenPlanModels } from "./qwen-token-plan.ts"; + +export default defineProvider({ + id: "qwen-token-plan-cn", + models: (ctx) => + qwenTokenPlanModels( + ctx, + "alibaba-token-plan-cn", + "https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1", + ), + compat: { "openai-completions": QWEN_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/qwen-token-plan-individual.ts b/packages/ai/catalog/providers/qwen-token-plan-individual.ts new file mode 100644 index 000000000..c172748d1 --- /dev/null +++ b/packages/ai/catalog/providers/qwen-token-plan-individual.ts @@ -0,0 +1,34 @@ +import { assertExactModelIds } from "../../scripts/model-data.ts"; +import { defineProvider } from "../define.ts"; +import { INTERNATIONAL_BASE_URL, QWEN_COMPLETIONS_COMPAT, qwenTokenPlanModels } from "./qwen-token-plan.ts"; + +// QwenCloud Token Plan Individual text-model allowlist, verified 2026-09-03. +// It shares the international endpoint with a narrower catalog. +// https://docs.qwencloud.com/token-plan/personal/token-plan-personal-overview +const MODEL_IDS = new Set([ + "deepseek-v4-flash-0731", + "deepseek-v4-pro", + "deepseek-v4-pro-0813", + "glm-5.2", + "qwen3.6-flash", + "qwen3.7-max", + "qwen3.7-plus", + "qwen3.8-flash", + "qwen3.8-max", +]); + +export default defineProvider({ + id: "qwen-token-plan-individual", + models: (ctx) => { + const models = qwenTokenPlanModels(ctx, "alibaba-token-plan", INTERNATIONAL_BASE_URL, MODEL_IDS); + if (ctx.strict) { + assertExactModelIds( + ctx.providerId, + MODEL_IDS, + models.map((model) => model.id), + ); + } + return models; + }, + compat: { "openai-completions": QWEN_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/qwen-token-plan.ts b/packages/ai/catalog/providers/qwen-token-plan.ts new file mode 100644 index 000000000..b526a9e28 --- /dev/null +++ b/packages/ai/catalog/providers/qwen-token-plan.ts @@ -0,0 +1,53 @@ +import type { Model, OpenAICompletionsCompat } from "../../src/types.ts"; +import { defineProvider, modelsDevModels, type ProviderContext } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +// Alibaba Cloud Model Studio Token Plan. International and China use separate +// endpoints and API keys (sk-sp- prefix). models.dev lists them as +// "alibaba-token-plan[-cn]". +export const INTERNATIONAL_BASE_URL = "https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"; + +const COMPAT: OpenAICompletionsCompat = { + thinkingFormat: "qwen", + supportsDeveloperRole: false, + supportsStore: false, + supportsReasoningEffort: true, +}; +const FALLBACK_THINKING_LEVELS = { + minimal: null, + low: null, + medium: null, + high: "high", + xhigh: null, + max: "max", +} as const; +const REASONING_EFFORT_FALLBACK_MODEL_IDS = new Set(["glm-5", "glm-5.1"]); +// Retired preview id — models.dev may still list it after GA ships. +const EXCLUDED_MODEL_IDS = new Set(["qwen3.8-max-preview"]); + +export function qwenTokenPlanModels(ctx: ProviderContext, source: string, baseUrl: string, modelIds?: Set) { + return modelsDevModels(ctx, source, { + api: "openai-completions", + baseUrl, + recordReasoning: false, + include: (model) => !EXCLUDED_MODEL_IDS.has(model.id) && (!modelIds || modelIds.has(model.id)), + map: (model, source): Model<"openai-completions"> => { + const thinkingLevelMap = + getEffortThinkingLevelMap(source.reasoningOptions ?? []) ?? + (REASONING_EFFORT_FALLBACK_MODEL_IDS.has(model.id) ? FALLBACK_THINKING_LEVELS : undefined); + return { + ...model, + compat: thinkingLevelMap ? COMPAT : { ...COMPAT, supportsReasoningEffort: false }, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + }; + }, + }); +} + +export const QWEN_COMPLETIONS_COMPAT = { supportsStrictMode: true } as const; + +export default defineProvider({ + id: "qwen-token-plan", + models: (ctx) => qwenTokenPlanModels(ctx, "alibaba-token-plan", INTERNATIONAL_BASE_URL), + compat: { "openai-completions": QWEN_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/radius.ts b/packages/ai/catalog/providers/radius.ts new file mode 100644 index 000000000..3c1b9deb4 --- /dev/null +++ b/packages/ai/catalog/providers/radius.ts @@ -0,0 +1,7 @@ +import { defineProvider } from "../define.ts"; + +// The unauthenticated public catalog; authenticated clients overlay it at runtime. +export default defineProvider({ + id: "radius", + models: (ctx) => structuredClone(ctx.sources.radius), +}); diff --git a/packages/ai/catalog/providers/together.ts b/packages/ai/catalog/providers/together.ts new file mode 100644 index 000000000..30cea723b --- /dev/null +++ b/packages/ai/catalog/providers/together.ts @@ -0,0 +1,58 @@ +import type { OpenAICompletionsCompat, ThinkingLevelMap } from "../../src/types.ts"; +import { defineProvider, modelsDevModels } from "../define.ts"; + +const BASE_COMPAT: OpenAICompletionsCompat = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + supportsStrictMode: false, + supportsLongCacheRetention: false, +}; +const TOGGLE_REASONING_COMPAT: OpenAICompletionsCompat = { ...BASE_COMPAT, thinkingFormat: "together" }; +const REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { + ...BASE_COMPAT, + supportsReasoningEffort: true, + thinkingFormat: "openai", +}; +const TOGGLE_REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { + ...TOGGLE_REASONING_COMPAT, + supportsReasoningEffort: true, +}; + +const REASONING_EFFORT_MODELS = new Set(["openai/gpt-oss-20b", "openai/gpt-oss-120b"]); +const TOGGLE_REASONING_EFFORT_MODELS = new Set(["deepseek-ai/DeepSeek-V4-Pro-0813"]); + +const REASONING_EFFORT_LEVELS: ThinkingLevelMap = { off: null, minimal: null }; +const DEEPSEEK_V4_LEVELS: ThinkingLevelMap = { minimal: null, low: null, medium: null, high: "high", xhigh: null }; +const TOGGLE_REASONING_LEVELS: ThinkingLevelMap = { minimal: null, low: null, medium: null }; + +export default defineProvider({ + id: "together", + models: (ctx) => + modelsDevModels(ctx, "togetherai", { + api: "openai-completions", + baseUrl: "https://api.together.ai/v1", + skipDeprecated: true, + map: (model) => { + if (!model.reasoning) return { ...model, compat: BASE_COMPAT }; + if (REASONING_EFFORT_MODELS.has(model.id)) { + return { ...model, compat: REASONING_EFFORT_COMPAT, thinkingLevelMap: { ...REASONING_EFFORT_LEVELS } }; + } + if (TOGGLE_REASONING_EFFORT_MODELS.has(model.id)) { + return { ...model, compat: TOGGLE_REASONING_EFFORT_COMPAT, thinkingLevelMap: { ...DEEPSEEK_V4_LEVELS } }; + } + return { ...model, compat: TOGGLE_REASONING_COMPAT, thinkingLevelMap: { ...TOGGLE_REASONING_LEVELS } }; + }, + }), + compat: { + "openai-completions": { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + thinkingFormat: "together", + supportsLongCacheRetention: false, + }, + }, +}); diff --git a/packages/ai/catalog/providers/typesafe.ts b/packages/ai/catalog/providers/typesafe.ts new file mode 100644 index 000000000..384c19831 --- /dev/null +++ b/packages/ai/catalog/providers/typesafe.ts @@ -0,0 +1,32 @@ +import type { ClassifierModel } from "../../src/types.ts"; +import { defineProvider } from "../define.ts"; + +export default defineProvider({ + id: "typesafe", + classifiers: (ctx): ClassifierModel<"typesafe-system-one">[] => { + const metadata = ctx.sources.modelsDevDecisions["typesafe/jev-latest"]; + if (!metadata || metadata.type !== "decision") { + if (ctx.strict) throw new Error("models.dev did not return decision model typesafe/jev-latest"); + return []; + } + return [ + { + type: "classifier", + id: "jev-latest", + name: metadata.name, + api: "typesafe-system-one", + provider: "typesafe", + baseUrl: "https://api.typesafe.ai/v1/", + // models.dev lists text input only, but the API accepts image assets as an experimental + // feature behind an opt-in header, which the typesafe-system-one API sends. + // https://github.com/typesafe-ai/typesafe-sdk-js (v0.7.0 changelog) + input: ["text", "image"], + // The canonical models.dev entry has no direct-provider pricing. System One reports token usage, + // so classify() results carry token counts but price them at zero. + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow: metadata.limit?.context || 64000, + compat: { imageInput: "assets", sendImageOptInHeader: true }, + }, + ]; + }, +}); diff --git a/packages/ai/catalog/providers/vercel-ai-gateway.ts b/packages/ai/catalog/providers/vercel-ai-gateway.ts new file mode 100644 index 000000000..e414e235c --- /dev/null +++ b/packages/ai/catalog/providers/vercel-ai-gateway.ts @@ -0,0 +1,14 @@ +import { defineProvider } from "../define.ts"; +import { isKimiK3FamilyId, KIMI_K3_MAX_TOKENS } from "../families/kimi.ts"; + +export default defineProvider({ + id: "vercel-ai-gateway", + models: (ctx) => + structuredClone(ctx.sources.aiGateway.chat).map((model) => + // Keep Kimi K3's canonical output limit when gateway metadata is missing or incorrect. + model.id.startsWith("moonshotai/") && isKimiK3FamilyId(model.id.slice("moonshotai/".length)) + ? { ...model, maxTokens: KIMI_K3_MAX_TOKENS } + : model, + ), + classifiers: (ctx) => structuredClone(ctx.sources.aiGateway.classifiers), +}); diff --git a/packages/ai/catalog/providers/xai.ts b/packages/ai/catalog/providers/xai.ts new file mode 100644 index 000000000..1a189f82b --- /dev/null +++ b/packages/ai/catalog/providers/xai.ts @@ -0,0 +1,29 @@ +import { defineProvider, mergeThinking, modelsDevModels } from "../define.ts"; + +// Retired or unsuitable for pi's built-in catalog. +const EXCLUDED_MODEL_IDS = new Set([ + "grok-3", + "grok-3-fast", + "grok-4.20-0309-non-reasoning", + "grok-4.20-0309-reasoning", + "grok-build-0.1", + "grok-code-fast-1", +]); + +export default defineProvider({ + id: "xai", + models: (ctx) => + modelsDevModels(ctx, "xai", { + api: "openai-responses", + baseUrl: "https://api.x.ai/v1", + compat: { supportsLongCacheRetention: false }, + include: (source) => !EXCLUDED_MODEL_IDS.has(source.id), + }), + finalize: (model) => { + // Models without verified effort options must not send the undocumented + // "none"/"minimal" efforts. + if (model.api === "openai-responses" && model.thinkingLevelMap === undefined) { + mergeThinking(model, { off: null, minimal: null }); + } + }, +}); diff --git a/packages/ai/catalog/providers/xiaomi-token-plan-ams.ts b/packages/ai/catalog/providers/xiaomi-token-plan-ams.ts new file mode 100644 index 000000000..6f8c6ce12 --- /dev/null +++ b/packages/ai/catalog/providers/xiaomi-token-plan-ams.ts @@ -0,0 +1,8 @@ +import { defineProvider } from "../define.ts"; +import { XIAOMI_COMPLETIONS_COMPAT, xiaomiModels } from "./xiaomi.ts"; + +export default defineProvider({ + id: "xiaomi-token-plan-ams", + models: (ctx) => xiaomiModels(ctx, "https://token-plan-ams.xiaomimimo.com/v1"), + compat: { "openai-completions": XIAOMI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/xiaomi-token-plan-cn.ts b/packages/ai/catalog/providers/xiaomi-token-plan-cn.ts new file mode 100644 index 000000000..a7a371860 --- /dev/null +++ b/packages/ai/catalog/providers/xiaomi-token-plan-cn.ts @@ -0,0 +1,8 @@ +import { defineProvider } from "../define.ts"; +import { XIAOMI_COMPLETIONS_COMPAT, xiaomiModels } from "./xiaomi.ts"; + +export default defineProvider({ + id: "xiaomi-token-plan-cn", + models: (ctx) => xiaomiModels(ctx, "https://token-plan-cn.xiaomimimo.com/v1"), + compat: { "openai-completions": XIAOMI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/xiaomi-token-plan-sgp.ts b/packages/ai/catalog/providers/xiaomi-token-plan-sgp.ts new file mode 100644 index 000000000..d3614b3ad --- /dev/null +++ b/packages/ai/catalog/providers/xiaomi-token-plan-sgp.ts @@ -0,0 +1,8 @@ +import { defineProvider } from "../define.ts"; +import { XIAOMI_COMPLETIONS_COMPAT, xiaomiModels } from "./xiaomi.ts"; + +export default defineProvider({ + id: "xiaomi-token-plan-sgp", + models: (ctx) => xiaomiModels(ctx, "https://token-plan-sgp.xiaomimimo.com/v1"), + compat: { "openai-completions": XIAOMI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/xiaomi.ts b/packages/ai/catalog/providers/xiaomi.ts new file mode 100644 index 000000000..3129bf139 --- /dev/null +++ b/packages/ai/catalog/providers/xiaomi.ts @@ -0,0 +1,46 @@ +import type { Model } from "../../src/types.ts"; +import { defineProvider, modelsDevModels, type ProviderContext } from "../define.ts"; + +/** + * Xiaomi MiMo. The `xiaomi` provider targets the API billing endpoint; the + * `xiaomi-token-plan-*` providers cover prepaid Token Plan endpoints per region. + * MiMo Chat Completions only documents thinking.type enabled/disabled and + * rejects some reasoning_effort values (e.g. "minimal"). Xiaomi documents all + * effort levels as identical, so expose only off and high. + */ +export function xiaomiModels(ctx: ProviderContext, baseUrl: string) { + return modelsDevModels(ctx, ctx.providerId, { + api: "openai-completions", + baseUrl, + skipDeprecated: true, + recordReasoning: false, + compat: { + supportsReasoningEffort: false, + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "deepseek", + }, + map: (model): Model<"openai-completions"> => + model.reasoning + ? { + ...model, + thinkingLevelMap: { + off: "off", + minimal: null, + low: null, + medium: null, + high: "high", + xhigh: null, + max: null, + }, + } + : model, + }); +} + +export const XIAOMI_COMPLETIONS_COMPAT = { supportsStrictMode: true } as const; + +export default defineProvider({ + id: "xiaomi", + models: (ctx) => xiaomiModels(ctx, "https://api.xiaomimimo.com/v1"), + compat: { "openai-completions": XIAOMI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/zai-coding-cn.ts b/packages/ai/catalog/providers/zai-coding-cn.ts new file mode 100644 index 000000000..2321572ec --- /dev/null +++ b/packages/ai/catalog/providers/zai-coding-cn.ts @@ -0,0 +1,8 @@ +import { defineProvider } from "../define.ts"; +import { ZAI_COMPLETIONS_COMPAT, zaiCodingPlanModels } from "./zai.ts"; + +export default defineProvider({ + id: "zai-coding-cn", + models: (ctx) => zaiCodingPlanModels(ctx, "zhipuai-coding-plan", "https://open.bigmodel.cn/api/coding/paas/v4"), + compat: { "openai-completions": ZAI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/catalog/providers/zai.ts b/packages/ai/catalog/providers/zai.ts new file mode 100644 index 000000000..540d30d79 --- /dev/null +++ b/packages/ai/catalog/providers/zai.ts @@ -0,0 +1,50 @@ +import type { Model } from "../../src/types.ts"; +import { defineProvider, modelsDevModels, type ProviderContext } from "../define.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; +import { modelsDevChatModels, modelsDevCost } from "../sources/models-dev.ts"; + +// Tool-call streaming (`tool_stream`) is rejected by the GLM-4.5 generation. +const TOOL_STREAM_UNSUPPORTED_MODEL_IDS = new Set(["glm-4.5", "glm-4.5-air", "glm-4.5-flash", "glm-4.5v"]); + +/** The Z.AI coding plans. The international and China plans list the same models. */ +export function zaiCodingPlanModels(ctx: ProviderContext, source: string, baseUrl: string) { + // Coding-plan entries carry plan prices; show the pay-as-you-go API rates. + const apiModels = new Map(modelsDevChatModels(ctx.sources.modelsDev, "zai").map((model) => [model.id, model])); + return modelsDevModels(ctx, source, { + api: "openai-completions", + baseUrl, + map: (model, source): Model<"openai-completions"> => { + const thinkingLevelMap = getEffortThinkingLevelMap(source.reasoningOptions ?? []); + if (thinkingLevelMap && (model.id === "glm-5.2" || model.id === "glm-5.2-highspeed")) { + thinkingLevelMap.off = "none"; + } + const apiModel = apiModels.get(model.id); + return { + ...model, + ...(thinkingLevelMap ? { thinkingLevelMap } : {}), + cost: modelsDevCost(apiModel?.source.cost ? apiModel.cost : source.cost), + compat: { + supportsDeveloperRole: false, + thinkingFormat: "zai", + ...(thinkingLevelMap ? { supportsReasoningEffort: true } : {}), + ...(!TOOL_STREAM_UNSUPPORTED_MODEL_IDS.has(model.id) ? { zaiToolStream: true } : {}), + }, + }; + }, + }); +} + +export const ZAI_COMPLETIONS_COMPAT = { + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + maxTokensField: "max_tokens", + thinkingFormat: "zai", + supportsStrictMode: true, +} as const; + +export default defineProvider({ + id: "zai", + models: (ctx) => zaiCodingPlanModels(ctx, "zai-coding-plan", "https://api.z.ai/api/coding/paas/v4"), + compat: { "openai-completions": ZAI_COMPLETIONS_COMPAT }, +}); diff --git a/packages/ai/scripts/models-dev-reasoning-options.ts b/packages/ai/catalog/reasoning.ts similarity index 79% rename from packages/ai/scripts/models-dev-reasoning-options.ts rename to packages/ai/catalog/reasoning.ts index 9d0711395..389f39382 100644 --- a/packages/ai/scripts/models-dev-reasoning-options.ts +++ b/packages/ai/catalog/reasoning.ts @@ -1,12 +1,5 @@ import type { ThinkingLevel, ThinkingLevelMap } from "../src/types.ts"; - -export type ModelsDevReasoningOption = - | { type: "toggle" } - | { - type: "effort"; - values: Array<"none" | "minimal" | "low" | "medium" | "high" | "xhigh" | "max" | "default" | null>; - } - | { type: "budget_tokens"; min?: number; max?: number }; +import type { ModelsDevReasoningOption } from "./sources/models-dev.ts"; const THINKING_LEVELS: readonly ThinkingLevel[] = ["minimal", "low", "medium", "high", "xhigh", "max"]; diff --git a/packages/ai/catalog/sources/index.ts b/packages/ai/catalog/sources/index.ts new file mode 100644 index 000000000..b043778b9 --- /dev/null +++ b/packages/ai/catalog/sources/index.ts @@ -0,0 +1,95 @@ +import { + DEFAULT_RADIUS_GATEWAY, + getRadiusModelsFromConfig, + loadRadiusGatewayConfig, +} from "../../src/providers/radius-config.ts"; +import type { Model } from "../../src/types.ts"; +import { + MODELS_DEV_DECISIONS_URL, + MODELS_DEV_URL, + type ModelsDevDecisionMetadata, + type ModelsDevV2Catalog, +} from "./models-dev.ts"; +import { fetchOpenRouterCatalog, type OpenRouterCatalog } from "./openrouter.ts"; +import { type AiGatewayCatalog, fetchAiGatewayCatalog } from "./vercel-ai-gateway.ts"; + +export const NVIDIA_BASE_URL = "https://integrate.api.nvidia.com/v1"; + +/** Everything fetched from upstream before the catalog is built. */ +export interface CatalogSources { + modelsDev: ModelsDevV2Catalog; + modelsDevDecisions: Record; + openRouter: OpenRouterCatalog; + aiGateway: AiGatewayCatalog; + /** Live NVIDIA NIM ids, keyed by both the raw and the normalized id. */ + nvidiaModelIds: Map; + radius: Model<"pi-messages">[]; +} + +async function fetchJson(url: string, label: string): Promise { + const response = await fetch(url); + if (!response.ok) throw new Error(`${label} returned ${response.status}`); + return (await response.json()) as T; +} + +export function normalizeNvidiaModelId(modelId: string): string { + return modelId.toLowerCase().replaceAll("_", "."); +} + +async function fetchNvidiaModelIds(): Promise> { + const data = await fetchJson<{ data?: { id: string }[] }>(`${NVIDIA_BASE_URL}/models`, "NVIDIA NIM API"); + const modelIds = new Map(); + for (const model of data.data ?? []) { + modelIds.set(model.id, model.id); + modelIds.set(normalizeNvidiaModelId(model.id), model.id); + } + return modelIds; +} + +async function fetchRadiusModels(): Promise[]> { + const config = await loadRadiusGatewayConfig(DEFAULT_RADIUS_GATEWAY); + const models = getRadiusModelsFromConfig("radius", config); + if (models.length === 0) throw new Error("Radius API returned no models"); + return models; +} + +/** + * Fetch every upstream source. In strict mode any failure aborts generation; + * otherwise a failed source contributes nothing. + */ +export async function loadCatalogSources(strict: boolean): Promise { + async function load(label: string, fetcher: () => Promise, fallback: T): Promise { + try { + console.log(`Fetching ${label}...`); + return await fetcher(); + } catch (error) { + console.error(`Failed to fetch ${label}:`, error); + if (strict) throw error; + return fallback; + } + } + + const modelsDev = await load( + "models.dev catalog", + () => fetchJson(MODELS_DEV_URL, "models.dev API"), + {}, + ); + const modelsDevDecisions = await load( + "models.dev decision models", + () => fetchJson>(MODELS_DEV_DECISIONS_URL, "models.dev decision API"), + {}, + ); + const nvidiaModelIds = modelsDev.nvidia?.models + ? await load("NVIDIA NIM models", fetchNvidiaModelIds, new Map()) + : new Map(); + const openRouter = await load("OpenRouter models", fetchOpenRouterCatalog, { + chat: [], + images: [], + classifiers: [], + }); + if (strict && openRouter.images.length === 0) throw new Error("OpenRouter API returned no usable image models"); + const aiGateway = await load("Vercel AI Gateway models", fetchAiGatewayCatalog, { chat: [], classifiers: [] }); + const radius = await load("Radius models", fetchRadiusModels, []); + + return { modelsDev, modelsDevDecisions, openRouter, aiGateway, nvidiaModelIds, radius }; +} diff --git a/packages/ai/catalog/sources/models-dev.ts b/packages/ai/catalog/sources/models-dev.ts new file mode 100644 index 000000000..3928dc438 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev.ts @@ -0,0 +1,242 @@ +import type { ModelCost } from "../../src/types.ts"; + +// models.dev v2 catalog. The schema is published as experimental, so everything +// the catalog reads goes through `normalizeModelsDevModel()` below. +// https://github.com/anomalyco/models.dev/blob/dev/packages/core/src/schema-v2.ts +export const MODELS_DEV_URL = "https://models.dev/experimental/v2.0/api.json"; +// v2 has no provider-agnostic decision metadata endpoint; v1 still serves it. +export const MODELS_DEV_DECISIONS_URL = "https://models.dev/models.json?type=decision"; + +export type ModelsDevReasoningOption = + | { type: "toggle" } + | { + type: "effort"; + values: Array<"none" | "minimal" | "low" | "medium" | "high" | "xhigh" | "max" | "default" | null>; + } + | { type: "budget_tokens"; min?: number; max?: number }; + +type ModelsDevEffort = Extract["values"][number]; + +interface V2CachePrices { + read?: number; + write?: number | Record; + explicit?: { read?: number; write?: number | Record }; +} + +interface V2CostOverride { + when: Record; + price?: number; + cache?: V2CachePrices; +} + +interface V2ModalityCost { + input?: { unit?: string; price: number; cache?: V2CachePrices; overrides?: V2CostOverride[] }; + output?: { unit?: string; price: number; overrides?: V2CostOverride[] }; +} + +interface V2ApiEntry { + base_url: string; + path?: string; + sdk?: { aisdk?: string }; + compatibility?: Record; + capabilities?: Record; +} + +/** The subset of a models.dev v2 model the catalog reads. */ +export interface ModelsDevV2Model { + id: string; + type: string; + name: string; + status?: string; + modalities: { input: string[]; output: string[] }; + capabilities: { + tools: { supported: boolean }; + reasoning: { + supported: boolean; + toggle?: boolean; + effort?: ModelsDevEffort[]; + budget?: { min?: number; max?: number }; + }; + structured_output?: boolean; + }; + limit?: { context?: number; input?: number; output?: number }; + cost?: { text?: V2ModalityCost }; + api: Record; +} + +export type ModelsDevV2Catalog = Record }>; + +/** Prices as models.dev lists them. Missing prices stay undefined. */ +export interface ModelsDevCost { + input?: number; + output?: number; + cacheRead?: number; + cacheWrite?: number; + /** Context-size tiers. Unlisted rates fall back to the base rates. */ + tiers?: Array<{ above: number; input?: number; output?: number; cacheRead?: number; cacheWrite?: number }>; +} + +/** One API a model is served on, in the order models.dev lists them. */ +export interface ModelsDevApi { + protocol: string; + baseUrl: string; + aisdk?: string; +} + +/** A models.dev model normalized for the catalog. */ +export interface ModelsDevModel { + id: string; + name: string; + type: string; + status?: string; + toolCall: boolean; + structuredOutput: boolean; + reasoning: boolean; + /** Undefined when the model does not reason. */ + reasoningOptions?: ModelsDevReasoningOption[]; + imageInput: boolean; + textInput: boolean; + textOutput: boolean; + contextWindow?: number; + maxTokens?: number; + cost: ModelsDevCost; + apis: ModelsDevApi[]; + /** The full v2 entry, for fields not normalized above. */ + source: ModelsDevV2Model; +} + +/** Provider-agnostic decision model metadata from models.dev. */ +export interface ModelsDevDecisionMetadata { + id: string; + type?: string; + name: string; + limit?: { context?: number }; +} + +function firstPrice(write: number | Record | undefined): number | undefined { + return typeof write === "object" ? Object.values(write)[0] : write; +} + +// models.dev lists one cache write price per TTL; the first is the host's default. +function cacheWritePrice(cache: V2CachePrices | undefined): number | undefined { + return firstPrice(cache?.write) ?? firstPrice(cache?.explicit?.write); +} + +function contextThreshold(when: Record): number | undefined { + const keys = Object.keys(when); + if (keys.length !== 1 || keys[0] !== "context") return undefined; + const context = when.context as { gt?: unknown; gte?: unknown }; + const size = context.gt ?? context.gte; + return typeof size === "number" ? size : undefined; +} + +function normalizeCost(cost: ModelsDevV2Model["cost"]): ModelsDevCost { + const input = cost?.text?.input; + const output = cost?.text?.output; + const perToken = (unit: string | undefined) => unit === undefined || unit === "token"; + if (!input || !output || !perToken(input.unit) || !perToken(output.unit)) return {}; + + // Only context-size overrides map onto pi's cost tiers. Region, service-tier, + // and time-based prices have no pi equivalent yet. + const tiers = new Map[number]>(); + const tier = (above: number) => { + let entry = tiers.get(above); + if (!entry) { + entry = { above }; + tiers.set(above, entry); + } + return entry; + }; + for (const override of input.overrides ?? []) { + const above = contextThreshold(override.when); + if (above === undefined) continue; + const entry = tier(above); + if (override.price !== undefined) entry.input = override.price; + if (override.cache?.read !== undefined) entry.cacheRead = override.cache.read; + const cacheWrite = cacheWritePrice(override.cache); + if (cacheWrite !== undefined) entry.cacheWrite = cacheWrite; + } + for (const override of output.overrides ?? []) { + const above = contextThreshold(override.when); + if (above === undefined) continue; + if (override.price !== undefined) tier(above).output = override.price; + } + + return { + input: input.price, + output: output.price, + cacheRead: input.cache?.read, + cacheWrite: cacheWritePrice(input.cache), + ...(tiers.size > 0 ? { tiers: [...tiers.values()] } : {}), + }; +} + +function reasoningOptions(reasoning: ModelsDevV2Model["capabilities"]["reasoning"]): ModelsDevReasoningOption[] { + const options: ModelsDevReasoningOption[] = []; + if (reasoning.toggle) options.push({ type: "toggle" }); + if (reasoning.effort !== undefined) options.push({ type: "effort", values: reasoning.effort }); + if (reasoning.budget !== undefined) options.push({ type: "budget_tokens", ...reasoning.budget }); + return options; +} + +export function normalizeModelsDevModel(model: ModelsDevV2Model): ModelsDevModel { + const input = model.modalities?.input ?? []; + const reasoning = model.capabilities?.reasoning; + return { + id: model.id, + name: model.name, + type: model.type, + status: model.status, + toolCall: model.capabilities?.tools?.supported === true, + structuredOutput: model.capabilities?.structured_output === true, + reasoning: reasoning?.supported === true, + reasoningOptions: reasoning?.supported ? reasoningOptions(reasoning) : undefined, + imageInput: input.some((modality) => modality === "image" || modality.startsWith("image/")), + textInput: input.includes("text"), + textOutput: (model.modalities?.output ?? []).includes("text"), + contextWindow: model.limit?.context, + maxTokens: model.limit?.output, + cost: normalizeCost(model.cost), + apis: Object.entries(model.api ?? {}).map(([protocol, entry]) => ({ + protocol, + baseUrl: entry.base_url, + aisdk: entry.sdk?.aisdk, + })), + source: model, + }; +} + +export function roundCost(value: number): number { + return Number(value.toFixed(6)); +} + +/** Converts models.dev prices to pi's cost shape. Missing rates are free. */ +export function modelsDevCost(cost: ModelsDevCost): ModelCost { + const base = { + input: cost.input || 0, + output: cost.output || 0, + cacheRead: cost.cacheRead || 0, + cacheWrite: cost.cacheWrite || 0, + }; + const tiers = cost.tiers?.map((tier) => ({ + inputTokensAbove: tier.above, + input: tier.input ?? base.input, + output: tier.output ?? base.output, + cacheRead: tier.cacheRead ?? base.cacheRead, + cacheWrite: tier.cacheWrite ?? base.cacheWrite, + })); + return tiers && tiers.length > 0 ? { ...base, tiers } : base; +} + +/** + * Chat models of one models.dev provider. Specialized types (image, speech, + * realtime, decision, ...) only appear in v2 and are left out here. + */ +export function modelsDevChatModels(catalog: ModelsDevV2Catalog, provider: string): ModelsDevModel[] { + return ( + Object.values(catalog[provider]?.models ?? {}) + // Unlike pi models, models.dev v2 always states the type. + .filter(({ type }) => type === "chat") + .map(normalizeModelsDevModel) + ); +} diff --git a/packages/ai/scripts/openrouter-catalog.ts b/packages/ai/catalog/sources/openrouter.ts similarity index 66% rename from packages/ai/scripts/openrouter-catalog.ts rename to packages/ai/catalog/sources/openrouter.ts index f0b7c0b03..a7e82a735 100644 --- a/packages/ai/scripts/openrouter-catalog.ts +++ b/packages/ai/catalog/sources/openrouter.ts @@ -1,6 +1,29 @@ -import { type OpenRouterModel, openRouterCost } from "../src/providers/openrouter-api.ts"; -import type { ClassifierModel, ImageModel, Model } from "../src/types.ts"; -import { getOpenRouterThinkingLevelMap, type OpenRouterReasoningMetadata } from "./openrouter-reasoning-options.ts"; +import { type OpenRouterModel, openRouterCost } from "../../src/providers/openrouter-api.ts"; +import type { ClassifierModel, ImageModel, Model, ThinkingLevel, ThinkingLevelMap } from "../../src/types.ts"; +import { getEffortThinkingLevelMap } from "../reasoning.ts"; + +const OPENROUTER_MODELS_URL = "https://openrouter.ai/api/v1/models"; + +export interface OpenRouterReasoningMetadata { + mandatory?: boolean; + default_enabled?: boolean; + supported_efforts?: Array; + default_effort?: ThinkingLevel | "none"; +} + +/** Convert OpenRouter's reasoning metadata into Pi model capabilities. */ +export function getOpenRouterThinkingLevelMap( + reasoning: OpenRouterReasoningMetadata | undefined, +): ThinkingLevelMap | undefined { + if (!reasoning) return undefined; + if (!reasoning.supported_efforts?.length) return reasoning.mandatory === true ? { off: null } : undefined; + + // OpenRouter's supported_efforts uses the same effort values as models.dev reasoning_options, + // so both sources can share the same Pi thinking-level conversion. + const map = getEffortThinkingLevelMap([{ type: "effort", values: reasoning.supported_efforts }]); + if (!map) return reasoning.mandatory === true ? { off: null } : undefined; + return { ...map, off: reasoning.mandatory === true ? null : "none" }; +} export interface OpenRouterModelListItem extends OpenRouterModel { name: string; @@ -106,3 +129,20 @@ export function buildOpenRouterCatalog( return { chat, images, classifiers }; } + +async function fetchOpenRouterList(query: string): Promise { + const response = await fetch(`${OPENROUTER_MODELS_URL}${query}`); + if (!response.ok) throw new Error(`OpenRouter API returned ${response.status}`); + const data = (await response.json()) as { data?: OpenRouterModelListItem[] }; + return data.data ?? []; +} + +/** Fetch the tool-capable chat, image, and decision listings. */ +export async function fetchOpenRouterCatalog(): Promise { + const [listed, imageListed, decisionListed] = await Promise.all([ + fetchOpenRouterList(""), + fetchOpenRouterList("?output_modalities=image"), + fetchOpenRouterList("?output_modalities=decisions"), + ]); + return buildOpenRouterCatalog(listed, imageListed, decisionListed); +} diff --git a/packages/ai/catalog/sources/vercel-ai-gateway.ts b/packages/ai/catalog/sources/vercel-ai-gateway.ts new file mode 100644 index 000000000..3da44b2d8 --- /dev/null +++ b/packages/ai/catalog/sources/vercel-ai-gateway.ts @@ -0,0 +1,140 @@ +import type { ClassifierModel, Model, ModelCost, ModelCostRates, ModelCostTier } from "../../src/types.ts"; + +const AI_GATEWAY_MODELS_URL = "https://ai-gateway.vercel.sh/v1/models"; +const AI_GATEWAY_BASE_URL = "https://ai-gateway.vercel.sh"; +// TypeSafe-compatible System One endpoint for evaluation models. +// https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe +const AI_GATEWAY_TYPESAFE_BASE_URL = "https://ai-gateway.vercel.sh/typesafe/v1"; + +/** A prompt-length bracket. `min` is inclusive and `max` exclusive, both in prompt tokens. */ +export interface AiGatewayPriceTier { + cost?: string | number; + min?: number; + max?: number; +} + +/** Vercel AI Gateway pricing in $/token. */ +export interface AiGatewayPricing { + input?: string | number; + output?: string | number; + input_cache_read?: string | number; + input_cache_write?: string | number; + input_tiers?: AiGatewayPriceTier[]; + output_tiers?: AiGatewayPriceTier[]; + input_cache_read_tiers?: AiGatewayPriceTier[]; + input_cache_write_tiers?: AiGatewayPriceTier[]; +} + +const RATE_FIELDS = [ + { rate: "input", tiers: "input_tiers" }, + { rate: "output", tiers: "output_tiers" }, + { rate: "cacheRead", tiers: "input_cache_read_tiers" }, + { rate: "cacheWrite", tiers: "input_cache_write_tiers" }, +] as const; + +function perMillion(value: string | number | undefined): number { + const parsed = typeof value === "number" ? value : parseFloat(value ?? "0"); + return Number.isFinite(parsed) ? Number((parsed * 1_000_000).toFixed(6)) : 0; +} + +/** + * Convert AI Gateway pricing to $/million tokens. Each rate lists its own prompt-length brackets; + * every bracket start above zero becomes a request-wide tier with the rates in effect there. + */ +export function getAiGatewayCost(pricing: AiGatewayPricing | undefined): ModelCost { + const base: ModelCostRates = { + input: perMillion(pricing?.input), + output: perMillion(pricing?.output), + cacheRead: perMillion(pricing?.input_cache_read), + cacheWrite: perMillion(pricing?.input_cache_write), + }; + + const starts = new Set(); + for (const field of RATE_FIELDS) { + for (const bracket of pricing?.[field.tiers] ?? []) { + if (bracket.min !== undefined && bracket.min > 0) starts.add(bracket.min); + } + } + + const tiers = [...starts] + .sort((a, b) => a - b) + .map((start): ModelCostTier => { + const tier: ModelCostTier = { inputTokensAbove: start - 1, ...base }; + for (const field of RATE_FIELDS) { + const bracket = pricing?.[field.tiers]?.find( + (candidate) => (candidate.min ?? 0) <= start && (candidate.max === undefined || start < candidate.max), + ); + if (bracket?.cost !== undefined) tier[field.rate] = perMillion(bracket.cost); + } + return tier; + }); + + return tiers.length > 0 ? { ...base, tiers } : base; +} + +interface AiGatewayModel { + id: string; + name?: string; + type?: string; + context_window?: number; + max_tokens?: number; + tags?: string[]; + pricing?: AiGatewayPricing; +} + +export interface AiGatewayCatalog { + chat: Model<"anthropic-messages">[]; + classifiers: ClassifierModel<"typesafe-system-one">[]; +} + +/** Fetch tool-capable chat models and evaluation (classifier) models. */ +export async function fetchAiGatewayCatalog(): Promise { + const response = await fetch(AI_GATEWAY_MODELS_URL); + if (!response.ok) throw new Error(`Vercel AI Gateway API returned ${response.status}`); + const data = (await response.json()) as { data?: unknown }; + const items = Array.isArray(data.data) ? (data.data as AiGatewayModel[]) : []; + const chat: AiGatewayCatalog["chat"] = []; + const classifiers: AiGatewayCatalog["classifiers"] = []; + + for (const model of items) { + // Evaluation models such as TypeSafe's Jev are served through the + // TypeSafe-compatible System One endpoint. + if (model.type === "evaluation") { + classifiers.push({ + type: "classifier", + id: model.id, + name: model.name || model.id, + api: "typesafe-system-one", + provider: "vercel-ai-gateway", + baseUrl: AI_GATEWAY_TYPESAFE_BASE_URL, + input: ["text"], + cost: { + input: perMillion(model.pricing?.input), + output: perMillion(model.pricing?.output), + cacheRead: 0, + cacheWrite: 0, + }, + contextWindow: model.context_window || 4096, + }); + continue; + } + const tags = Array.isArray(model.tags) ? model.tags : []; + // Only include models that support tools + if (!tags.includes("tool-use")) continue; + + chat.push({ + id: model.id, + name: model.name || model.id, + api: "anthropic-messages", + baseUrl: AI_GATEWAY_BASE_URL, + provider: "vercel-ai-gateway", + reasoning: tags.includes("reasoning"), + input: tags.includes("vision") ? ["text", "image"] : ["text"], + compat: { allowEmptySignature: true }, + cost: getAiGatewayCost(model.pricing), + contextWindow: model.context_window || 4096, + maxTokens: model.max_tokens || 4096, + }); + } + return { chat, classifiers }; +} diff --git a/packages/ai/package.json b/packages/ai/package.json index ac6f865c6..6e04980f0 100644 --- a/packages/ai/package.json +++ b/packages/ai/package.json @@ -57,9 +57,9 @@ ], "scripts": { "clean": "shx rm -rf dist", - "generate-models": "node scripts/generate-models.ts --strict", - "hydrate-model-data": "node scripts/generate-models.ts --strict --data-only", - "generate-model-catalog": "node scripts/generate-models.ts --strict --json-only --json-output ../../.artifacts/model-catalog", + "generate-models": "node catalog/generate.ts --strict", + "hydrate-model-data": "node catalog/generate.ts --strict --data-only", + "generate-model-catalog": "node catalog/generate.ts --strict --json-only --json-output ../../.artifacts/model-catalog", "check:model-data": "node scripts/check-model-data.ts", "build": "npm run generate-models && npm run build:offline", "build:offline": "npm run check:model-data && tsc -p tsconfig.build.json && shx rm -rf dist/providers/data && shx cp -r src/providers/data dist/providers/data", diff --git a/packages/ai/scripts/ai-gateway-pricing.ts b/packages/ai/scripts/ai-gateway-pricing.ts deleted file mode 100644 index 74ac54b84..000000000 --- a/packages/ai/scripts/ai-gateway-pricing.ts +++ /dev/null @@ -1,67 +0,0 @@ -import type { ModelCost, ModelCostRates, ModelCostTier } from "../src/types.ts"; - -/** A prompt-length bracket. `min` is inclusive and `max` exclusive, both in prompt tokens. */ -export interface AiGatewayPriceTier { - cost?: string | number; - min?: number; - max?: number; -} - -/** Vercel AI Gateway pricing in $/token. */ -export interface AiGatewayPricing { - input?: string | number; - output?: string | number; - input_cache_read?: string | number; - input_cache_write?: string | number; - input_tiers?: AiGatewayPriceTier[]; - output_tiers?: AiGatewayPriceTier[]; - input_cache_read_tiers?: AiGatewayPriceTier[]; - input_cache_write_tiers?: AiGatewayPriceTier[]; -} - -const RATE_FIELDS = [ - { rate: "input", tiers: "input_tiers" }, - { rate: "output", tiers: "output_tiers" }, - { rate: "cacheRead", tiers: "input_cache_read_tiers" }, - { rate: "cacheWrite", tiers: "input_cache_write_tiers" }, -] as const; - -function perMillion(value: string | number | undefined): number { - const parsed = typeof value === "number" ? value : parseFloat(value ?? "0"); - return Number.isFinite(parsed) ? Number((parsed * 1_000_000).toFixed(6)) : 0; -} - -/** - * Convert AI Gateway pricing to $/million tokens. Each rate lists its own prompt-length brackets; - * every bracket start above zero becomes a request-wide tier with the rates in effect there. - */ -export function getAiGatewayCost(pricing: AiGatewayPricing | undefined): ModelCost { - const base: ModelCostRates = { - input: perMillion(pricing?.input), - output: perMillion(pricing?.output), - cacheRead: perMillion(pricing?.input_cache_read), - cacheWrite: perMillion(pricing?.input_cache_write), - }; - - const starts = new Set(); - for (const field of RATE_FIELDS) { - for (const bracket of pricing?.[field.tiers] ?? []) { - if (bracket.min !== undefined && bracket.min > 0) starts.add(bracket.min); - } - } - - const tiers = [...starts] - .sort((a, b) => a - b) - .map((start): ModelCostTier => { - const tier: ModelCostTier = { inputTokensAbove: start - 1, ...base }; - for (const field of RATE_FIELDS) { - const bracket = pricing?.[field.tiers]?.find( - (candidate) => (candidate.min ?? 0) <= start && (candidate.max === undefined || start < candidate.max), - ); - if (bracket?.cost !== undefined) tier[field.rate] = perMillion(bracket.cost); - } - return tier; - }); - - return tiers.length > 0 ? { ...base, tiers } : base; -} diff --git a/packages/ai/scripts/generate-models.ts b/packages/ai/scripts/generate-models.ts deleted file mode 100644 index c63afb6e5..000000000 --- a/packages/ai/scripts/generate-models.ts +++ /dev/null @@ -1,3688 +0,0 @@ -#!/usr/bin/env node - -import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync } from "fs"; -import { dirname, join, resolve } from "path"; -import { fileURLToPath } from "url"; -import { getEffortThinkingLevelMap, type ModelsDevReasoningOption } from "./models-dev-reasoning-options.ts"; -import { type AiGatewayPricing, getAiGatewayCost } from "./ai-gateway-pricing.ts"; -import { buildOpenRouterCatalog, type OpenRouterCatalog, type OpenRouterModelListItem } from "./openrouter-catalog.ts"; -import { - CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL, - CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL, - CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL, - CLOUDFLARE_WORKERS_AI_BASE_URL, - CLOUDFLARE_WORKERS_AI_REST_BASE_URL, -} from "../src/api/cloudflare.ts"; -import type { - AnthropicMessagesCompat, - AnyModel, - Api, - ClassifierApi, - ClassifierModel, - ImageApi, - ImageModel, - KnownProvider, - Model, - ModelCost, - ModelPromptCache, - OpenAICompletionsCompat, - OpenAIResponsesCompat, -} from "../src/types.ts"; -import { - assertExactModelIds, - createModelDataManifest, - groupProviderModelData, - type ModelDataStructure, - MODEL_DATA_MANIFEST_FILE, - readModelDataProviderIds, - validateGeneratedModelData, - validateModelDataDirectory, -} from "./model-data.ts"; -import { - DEFAULT_RADIUS_GATEWAY, - getRadiusModelsFromConfig, - loadRadiusGatewayConfig, -} from "../src/providers/radius-config.ts"; - -const __filename = fileURLToPath(import.meta.url); -const __dirname = dirname(__filename); -const packageRoot = join(__dirname, ".."); - -function readGeneratorOptions(args: string[]): { - strict: boolean; - dataOnly: boolean; - jsonOnly: boolean; - jsonOutputDir: string | undefined; - pretty: boolean; -} { - let strict = false; - let dataOnly = false; - let jsonOnly = false; - let jsonOutputDir: string | undefined; - let pretty = false; - - for (let index = 0; index < args.length; index++) { - const arg = args[index]; - if (arg === "--strict") { - strict = true; - continue; - } - if (arg === "--data-only") { - dataOnly = true; - continue; - } - if (arg === "--json-only") { - jsonOnly = true; - continue; - } - if (arg === "--pretty") { - pretty = true; - continue; - } - if (arg === "--json-output") { - const value = args[++index]; - if (!value) throw new Error("--json-output requires a directory"); - jsonOutputDir = resolve(value); - continue; - } - throw new Error(`Unknown argument: ${arg}`); - } - - if (jsonOnly && !jsonOutputDir) throw new Error("--json-only requires --json-output"); - if (dataOnly && (jsonOnly || jsonOutputDir)) throw new Error("--data-only cannot be combined with JSON catalog output"); - return { strict, dataOnly, jsonOnly, jsonOutputDir, pretty }; -} - -const generatorOptions = readGeneratorOptions(process.argv.slice(2)); - -interface ModelsDevModel { - id: string; - name: string; - tool_call?: boolean; - structured_output?: boolean; - reasoning?: boolean; - reasoning_options?: ModelsDevReasoningOption[]; - status?: string; - limit?: { - context?: number; - output?: number; - }; - cost?: { - input?: number; - output?: number; - cache_read?: number; - cache_write?: number; - tiers?: { - input?: number; - output?: number; - cache_read?: number; - cache_write?: number; - tier?: { - type?: string; - size?: number; - }; - }[]; - }; - modalities?: { - input?: string[]; - output?: string[]; - }; - provider?: { - npm?: string; - }; -} - -interface ModelsDevProvider { - models?: Record; -} - -interface ModelsDevMetadata { - id: string; - type?: string; - name: string; - limit?: { context?: number }; - modalities?: { input?: string[] }; -} - -type ModelsDevCatalog = Record; - -interface NvidiaNimModelListItem { - id: string; -} - -interface AiGatewayModel { - id: string; - name?: string; - type?: string; - context_window?: number; - max_tokens?: number; - tags?: string[]; - pricing?: AiGatewayPricing; -} - -const COPILOT_STATIC_HEADERS = { - "User-Agent": "GitHubCopilotChat/0.35.0", - "Editor-Version": "vscode/1.107.0", - "Editor-Plugin-Version": "copilot-chat/0.35.0", - "Copilot-Integration-Id": "vscode-chat", -} as const; - -const TOGETHER_BASE_URL = "https://api.together.ai/v1"; -const TOGETHER_BASE_COMPAT: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - maxTokensField: "max_tokens", - supportsStrictMode: false, - supportsLongCacheRetention: false, -}; -const TOGETHER_TOGGLE_REASONING_COMPAT: OpenAICompletionsCompat = { - ...TOGETHER_BASE_COMPAT, - thinkingFormat: "together", -}; -const TOGETHER_REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { - ...TOGETHER_BASE_COMPAT, - supportsReasoningEffort: true, - thinkingFormat: "openai", -}; -const TOGETHER_TOGGLE_REASONING_EFFORT_COMPAT: OpenAICompletionsCompat = { - ...TOGETHER_TOGGLE_REASONING_COMPAT, - supportsReasoningEffort: true, -}; -const TOGETHER_REASONING_EFFORT_MODELS = new Set(["openai/gpt-oss-20b", "openai/gpt-oss-120b"]); -const TOGETHER_TOGGLE_REASONING_EFFORT_MODELS = new Set(["deepseek-ai/DeepSeek-V4-Pro-0813"]); -const TOGETHER_REASONING_EFFORT_LEVEL_MAP = { - off: null, - minimal: null, -} as const; -const TOGETHER_DEEPSEEK_V4_THINKING_LEVEL_MAP = { - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: null, -} as const; -const TOGETHER_TOGGLE_REASONING_LEVEL_MAP = { - minimal: null, - low: null, - medium: null, -} as const; - -const AI_GATEWAY_MODELS_URL = "https://ai-gateway.vercel.sh/v1"; -const AI_GATEWAY_BASE_URL = "https://ai-gateway.vercel.sh"; -// TypeSafe-compatible System One endpoint for evaluation models. -// https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe -const AI_GATEWAY_TYPESAFE_BASE_URL = "https://ai-gateway.vercel.sh/typesafe/v1"; -const VERTEX_BASE_URL = "https://{location}-aiplatform.googleapis.com"; -const NVIDIA_BASE_URL = "https://integrate.api.nvidia.com/v1"; -const NVIDIA_HEADERS = { - "NVCF-POLL-SECONDS": "3600", -} as const; -const NVIDIA_OPENAI_COMPAT: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - maxTokensField: "max_tokens", - supportsStrictMode: false, - supportsLongCacheRetention: false, -}; -const NVIDIA_NIM_UNSUPPORTED_MODELS = new Set([ - "abacusai/dracarys-llama-3.1-70b-instruct", - "bytedance/seed-oss-36b-instruct", - "deepseek-ai/deepseek-v4-flash", - "deepseek-ai/deepseek-v4-pro", - "google/gemma-2-2b-it", - "google/gemma-3n-e2b-it", - "google/gemma-3n-e4b-it", - "google/gemma-4-31b-it", - "meta/llama-3.2-1b-instruct", - "meta/llama-4-maverick-17b-128e-instruct", - "microsoft/phi-4-mini-instruct", - "minimaxai/minimax-m2.7", - "mistralai/mistral-nemotron", - "nvidia/nemotron-mini-4b-instruct", - "qwen/qwen3-next-80b-a3b-instruct", - "qwen/qwen3.5-397b-a17b", - "sarvamai/sarvam-m", - "upstage/solar-10.7b-instruct", -]); -const ZAI_TOOL_STREAM_UNSUPPORTED_MODELS = new Set(["glm-4.5", "glm-4.5-air", "glm-4.5-flash", "glm-4.5v"]); -const OPENCODE_GO_GLM52_THINKING_LEVEL_MAP = { - off: null, - minimal: null, - low: null, - medium: null, - high: "high", - max: "max", -} as const; -const EAGER_TOOL_INPUT_STREAMING_UNSUPPORTED_ANTHROPIC_MODELS = new Set([ - "github-copilot:claude-haiku-4.5", - "github-copilot:claude-sonnet-4", - "github-copilot:claude-sonnet-4.5", -]); -const ANTHROPIC_ALLOWED_FALLBACK_MODELS = { - "claude-fable-5": ["claude-opus-4-8", "claude-opus-5"], - "claude-opus-5": ["claude-opus-4-8"], -} satisfies Record; - -const DEEPSEEK_V4_THINKING_LEVEL_MAP = { - minimal: null, - low: null, - medium: null, - high: "high", - max: "max", -} as const; -const DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP = { - ...DEEPSEEK_V4_THINKING_LEVEL_MAP, - low: "low", -} as const; -// Azure Foundry rejects DeepSeek's own max effort. -const AZURE_DEEPSEEK_V4_THINKING_LEVEL_MAP = { - minimal: null, - low: "low", - medium: "medium", - high: "high", - xhigh: null, - max: null, -} as const; -// Verified against Fireworks Messages raw_output on 2026-09-10 (#9323). -// Fall back to verified support when models.dev omits effort metadata; this is -// not an allowlist. Any Fireworks Messages model advertising effort uses adaptive thinking. -const FIREWORKS_ADAPTIVE_THINKING_FALLBACK_MODELS = new Set([ - "accounts/fireworks/models/deepseek-v4-flash-0731", - "accounts/fireworks/models/deepseek-v4-flash-vision-exp", - "accounts/fireworks/models/deepseek-v4-pro-0813", - "accounts/fireworks/models/qwen3p8-max", - "accounts/fireworks/models/qwen3p8-2p4t-a95b", -]); -const QWEN_TOKEN_PLAN_FALLBACK_THINKING_LEVEL_MAP = { - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: null, - max: "max", -} as const; -const QWEN_TOKEN_PLAN_REASONING_EFFORT_FALLBACK_MODEL_IDS = new Set(["glm-5", "glm-5.1"]); -// Retired preview id — models.dev may still list it after GA ships. -const QWEN_TOKEN_PLAN_EXCLUDED_MODEL_IDS = new Set(["qwen3.8-max-preview"]); -const QWEN_TOKEN_PLAN_PROVIDER_IDS = new Set([ - "qwen-token-plan", - "qwen-token-plan-cn", - "qwen-token-plan-individual", -]); -// QwenCloud Token Plan Individual text-model allowlist, verified 2026-09-03. -// Retired models remain excluded above even if the public catalog lags. -// https://docs.qwencloud.com/token-plan/personal/token-plan-personal-overview -const QWEN_TOKEN_PLAN_INDIVIDUAL_MODEL_IDS = new Set([ - "deepseek-v4-flash-0731", - "deepseek-v4-pro", - "deepseek-v4-pro-0813", - "glm-5.2", - "qwen3.6-flash", - "qwen3.7-max", - "qwen3.7-plus", - "qwen3.8-flash", - "qwen3.8-max", -]); - -const KIMI_K3_MAX_TOKENS = 131072; -const KIMI_K3_COST = { - input: 3, - output: 15, - cacheRead: 0.3, - cacheWrite: 0, -} as const; -// Kimi Chat Completions settings shared by Moonshot and Kimi Coding. -const KIMI_OPENAI_COMPAT: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - maxTokensField: "max_tokens", - supportsStrictMode: false, - thinkingFormat: "deepseek", -}; -// Kimi K3-style models take reasoning_effort and need earlier reasoning sent back. -const KIMI_K3_OPENAI_COMPAT: OpenAICompletionsCompat = { - ...KIMI_OPENAI_COMPAT, - requiresReasoningContentOnAssistantMessages: true, - thinkingFormat: "openai", - supportsReasoningEffort: true, -}; -// Kimi K3 and its point releases (kimi-k3.1-pro, kimi-k3.1-pro-highspeed) share K3's request -// format. https://api.moonshot.ai/v1/models reports effort levels and dynamic tool loading for all. -function isKimiK3FamilyId(modelId: string): boolean { - return /^kimi-k3(?:\.\d+)?(?:-|$)/.test(modelId); -} - -const ANT_LING_RING_THINKING_LEVEL_MAP = { - off: null, - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: "xhigh", -} as const; - -const BEDROCK_INFERENCE_PROFILE_ONLY_MODEL_IDS = new Set(["anthropic.claude-opus-5"]); -const MODELS_DEV_OPENAI_UNSUPPORTED_MODEL_IDS = new Set(["gpt-5.6"]); -// GPT-5.5 and older put tools loaded through tool search or additional_tools in a namespace named -// after the tool and reject replayed calls without it, so history from other models fails there -// (#11415). Like Codex, dynamic tool loading therefore starts with GPT-5.6, which does not do this. -const OPENAI_TOOL_SEARCH_MODEL_IDS = new Set([ - "gpt-5.6-sol", - "gpt-5.6-terra", - "gpt-5.6-luna", - "gpt-6-astra", - "gpt-6-sol", - "gpt-6-luna", - "gpt-6.1-sol", - "gpt-6.1-terra", -]); -const OPENAI_ADDITIONAL_TOOLS_MODEL_IDS = OPENAI_TOOL_SEARCH_MODEL_IDS; -const OPENAI_MID_CONVO_SYSTEM_MESSAGE_MODEL_IDS = new Set([ - "gpt-5.4", - "gpt-5.4-mini", - "gpt-5.4-pro", - "gpt-5.5", - ...OPENAI_TOOL_SEARCH_MODEL_IDS, -]); -const OPENAI_CODEX_ADDITIONAL_TOOLS_MODEL_IDS = new Set([ - "gpt-5.6-sol", - "gpt-5.6-terra", - "gpt-5.6-luna", - "gpt-6-astra", - "gpt-6-sol", - "gpt-6-luna", - "gpt-6.1-sol", - "gpt-6.1-terra", -]); -// OpenAI documents configuration_update for the GPT-6 family. -// https://developers.openai.com/api/docs/guides/reasoning -const OPENAI_MID_CONVO_EFFORT_MODEL_IDS = new Set([ - "gpt-6-astra", - "gpt-6-sol", - "gpt-6-luna", - "gpt-6.1-sol", - "gpt-6.1-terra", -]); -const OPENAI_LONG_CONTEXT_INPUT_THRESHOLD = 272000; -const OPENAI_SHORT_CONTEXT_CAPPED_MODEL_IDS = new Set([ - "gpt-5.4", - "gpt-5.5", - "gpt-5.6-sol", - "gpt-5.6-terra", - "gpt-5.6-luna", - "gpt-6-astra", - "gpt-6-sol", - "gpt-6-luna", - "gpt-6.1-sol", - "gpt-6.1-terra", -]); -const OPENAI_LONG_CONTEXT_PRICING_MODEL_IDS = new Set([ - "gpt-5.4", - "gpt-5.4-pro", - "gpt-5.5", - "gpt-5.5-pro", - "gpt-5.6-sol", - "gpt-5.6-terra", - "gpt-5.6-luna", - "gpt-6-astra", - "gpt-6-sol", - "gpt-6-luna", - "gpt-6.1-sol", - "gpt-6.1-terra", -]); - -// Keep the generated default no less restrictive than coding-agent's historical -// image preprocessing. Provider limits can narrow this profile, but unknown -// providers retain the cache-safe 2000px / 4.5 MiB behavior. -const DEFAULT_IMAGE_RESIZE = { - maxWidth: 2000, - maxHeight: 2000, - maxBytes: 4.5 * 1024 * 1024, - jpegQuality: 80, -} as const; - -function withOpenAiLongContextPricing(cost: Model["cost"]): Model["cost"] { - return { - ...cost, - tiers: [ - { - inputTokensAbove: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - input: roundCost(cost.input * 2), - output: roundCost(cost.output * 1.5), - cacheRead: roundCost(cost.cacheRead * 2), - cacheWrite: roundCost(cost.cacheWrite * 2), - }, - ], - }; -} - -// Keep current OpenAI prices authoritative until models.dev and passthrough -// catalogs catch up. -// https://developers.openai.com/api/docs/pricing -const OPENAI_STANDARD_COSTS: Record = { - "gpt-5.6-luna": { input: 0.2, output: 1.2, cacheRead: 0.02, cacheWrite: 0.25 }, - "gpt-5.6-sol": { input: 4, output: 20, cacheRead: 0.4, cacheWrite: 5 }, - "gpt-5.6-terra": { input: 2, output: 12, cacheRead: 0.2, cacheWrite: 2.5 }, - "gpt-6-astra": { input: 10, output: 50, cacheRead: 1, cacheWrite: 12.5 }, - "gpt-6-luna": { input: 0.1, output: 0.5, cacheRead: 0.01, cacheWrite: 0.125 }, - "gpt-6-sol": { input: 2, output: 10, cacheRead: 0.2, cacheWrite: 2.5 }, - "gpt-6.1-sol": { input: 2, output: 10, cacheRead: 0.1, cacheWrite: 2.5 }, - "gpt-6.1-terra": { input: 1, output: 5, cacheRead: 0.05, cacheWrite: 1.25 }, -}; - -const OPENAI_RESPONSES_NONE_REASONING_MODELS = new Set([ - "gpt-5.1", - "gpt-5.2", - "gpt-5.3-codex", - "gpt-5.4", - "gpt-5.4-mini", - "gpt-5.4-nano", - "gpt-5.5", - "gpt-5.6-sol", - "gpt-5.6-terra", - "gpt-5.6-luna", - "gpt-6-sol", - "gpt-6-luna", -]); -const XAI_BUILTIN_EXCLUDED_MODEL_IDS = new Set([ - "grok-3", - "grok-3-fast", - "grok-4.20-0309-non-reasoning", - "grok-4.20-0309-reasoning", - "grok-build-0.1", - "grok-code-fast-1", -]); -const XAI_RESPONSES_COMPAT: OpenAIResponsesCompat = { - supportsLongCacheRetention: false, -}; - -const OPENCODE_OPENAI_COMPLETIONS_LONG_CACHE_RETENTION_UNSUPPORTED_MODELS = new Set([ - "opencode:deepseek-v4-flash", - "opencode:deepseek-v4.1-flash", - "opencode:deepseek-v4-pro", - "opencode:kimi-k2.5", - "opencode:kimi-k2.6", - "opencode:minimax-m2.7", - "opencode-go:kimi-k2.6", -]); - -// GitHub's "Models with extended capabilities" table lists these Copilot models as supporting -// the extended 1 million token context window. -const GITHUB_COPILOT_EXTENDED_CONTEXT_MODELS = new Set([ - "claude-fable-5", - "claude-opus-4.8", - "claude-opus-5", - "claude-opus-5.5", - "claude-sonnet-4.6", - "claude-sonnet-5", - "gpt-5.3-codex", - "gpt-6-astra", - "gpt-6-luna", - "gpt-6-sol", -]); - -// Checked manually against the authenticated GitHub Copilot /models endpoint on 2026-06-15. -// Keep this to narrow corrections over models.dev metadata instead of snapshotting Copilot's catalog. -const GITHUB_COPILOT_THINKING_LEVEL_OVERRIDES = { - "claude-opus-4.8": { minimal: "low" }, - "claude-opus-5": { minimal: "low" }, - "claude-sonnet-4.6": { minimal: "low", max: "max" }, -} satisfies Record["thinkingLevelMap"]>>; - -function mergeThinkingLevelMap(model: Model, map: NonNullable["thinkingLevelMap"]>): void { - model.thinkingLevelMap = { ...model.thinkingLevelMap, ...map }; -} - -const modelsDevReasoningOptions = new Map(); - -function getModelKey(model: Pick, "provider" | "id">): string { - return `${model.provider}:${model.id}`; -} - -function recordModelsDevReasoningOptions(provider: string, id: string, sourceModel: ModelsDevModel): void { - if (sourceModel.reasoning_options !== undefined) { - modelsDevReasoningOptions.set(`${provider}:${id}`, sourceModel.reasoning_options); - } -} - -function supportsDirectReasoningEffort(model: Model): boolean { - if (model.api === "anthropic-messages") return model.compat?.forceAdaptiveThinking === true; - if ( - model.api === "openai-responses" || - model.api === "azure-openai-responses" || - model.api === "openai-codex-responses" - ) { - return true; - } - if (model.api !== "openai-completions") return false; - - const compat = { - ...detectOpenAICompletionsCompat(model as Model<"openai-completions">), - ...(model.compat as OpenAICompletionsCompat | undefined), - }; - return compat.thinkingFormat === "openai" && compat.supportsReasoningEffort; -} - -function applyModelsDevReasoningOptionMetadata(model: Model): void { - const reasoningOptions = modelsDevReasoningOptions.get(getModelKey(model)); - if (!reasoningOptions || !supportsDirectReasoningEffort(model)) return; - const thinkingLevelMap = getEffortThinkingLevelMap(reasoningOptions); - if (thinkingLevelMap) mergeThinkingLevelMap(model, thinkingLevelMap); -} - -function getTogetherCompat(modelId: string, reasoning: boolean): OpenAICompletionsCompat { - if (!reasoning) return TOGETHER_BASE_COMPAT; - if (TOGETHER_REASONING_EFFORT_MODELS.has(modelId)) return TOGETHER_REASONING_EFFORT_COMPAT; - if (TOGETHER_TOGGLE_REASONING_EFFORT_MODELS.has(modelId)) return TOGETHER_TOGGLE_REASONING_EFFORT_COMPAT; - return TOGETHER_TOGGLE_REASONING_COMPAT; -} - -function getTogetherThinkingLevelMap( - modelId: string, - reasoning: boolean, -): NonNullable["thinkingLevelMap"]> | undefined { - if (!reasoning) return undefined; - if (TOGETHER_REASONING_EFFORT_MODELS.has(modelId)) return { ...TOGETHER_REASONING_EFFORT_LEVEL_MAP }; - if (TOGETHER_TOGGLE_REASONING_EFFORT_MODELS.has(modelId)) return { ...TOGETHER_DEEPSEEK_V4_THINKING_LEVEL_MAP }; - return { ...TOGETHER_TOGGLE_REASONING_LEVEL_MAP }; -} - -function supportsOpenAiXhigh(modelId: string): boolean { - return ( - modelId.includes("gpt-5.2") || - modelId.includes("gpt-5.3") || - modelId.includes("gpt-5.4") || - modelId.includes("gpt-5.5") || - modelId.includes("gpt-5.6") || - modelId.includes("gpt-6") - ); -} - -function supportsOpenAiMax(model: Model): boolean { - return ( - (model.id.includes("gpt-5.6") || model.id.includes("gpt-6")) && - (model.api === "openai-responses" || - model.api === "azure-openai-responses" || - model.api === "openai-codex-responses" || - model.api === "openai-completions") - ); -} - -const VERIFIED_ANTHROPIC_MID_CONVO_EFFORT_PROVIDERS = new Set(["anthropic", "openrouter"]); -// OpenRouter rejects `configuration_update` system messages on Opus 5 ("Mid-conversation -// reasoning effort (configuration_update) is not supported on anthropic/claude-opus-5-20260723") -// while accepting them on Fable 5.1, so gate that model there. -const MID_CONVO_EFFORT_UNSUPPORTED_ANTHROPIC_MODELS = new Set(["openrouter:anthropic/claude-opus-5"]); - -function supportsAnthropicMidConvoEffort(modelId: string): boolean { - const id = modelId.toLowerCase().replace(/^~?anthropic\//, ""); - return ( - /^claude-opus-(?:5|5[.-]5)(?:-\d{8})?$/.test(id) || - /^claude-(?:sonnet|haiku)-5[.-]5(?:-\d{8})?$/.test(id) || - /^claude-(?:fable|mythos)-5(?:[.-]1)(?:-\d{8})?$/.test(id) - ); -} - -// Claude models that accept `thinking: {type: "disabled"}` alongside per-message effort. -// Opus 5.5, Sonnet 5.5, Fable 5.1, and Mythos 5.1 reject it. -// https://platform.claude.com/docs/en/build-with-claude/thinking#configuring-thinking -const ANTHROPIC_DISABLED_THINKING_MODELS = new Set(["claude-opus-5", "claude-haiku-5-5", "claude-haiku-5.5"]); - -function supportsAnthropicDisabledThinkingWithMidConvoEffort(modelId: string): boolean { - // Strip OpenRouter's "anthropic/" or "~anthropic/" prefix and any dated snapshot suffix. - const id = modelId - .toLowerCase() - .replace(/^~?anthropic\//, "") - .replace(/-\d{8}$/, ""); - return ANTHROPIC_DISABLED_THINKING_MODELS.has(id); -} - -function supportsAnthropicMidConvoSystemMessages(modelId: string): boolean { - return ( - /^claude-opus-(?:4[.-]8|5(?:[.-]5)?)(?:-\d{8})?$/.test(modelId) || - /^claude-(?:sonnet|haiku)-5[.-]5(?:-\d{8})?$/.test(modelId) || - /^claude-(?:fable|mythos)-5(?:[.-]1)?(?:-\d{8})?$/.test(modelId) - ); -} - -function isAnthropicAdaptiveThinkingModel(modelId: string): boolean { - return ( - // OpenRouter's `~anthropic/claude-*-latest` aliases follow the newest model of each family, - // all of which use adaptive thinking. - /^~anthropic\/claude-[a-z]+-latest$/.test(modelId) || - modelId.includes("opus-4-6") || - modelId.includes("opus-4.6") || - modelId.includes("opus-4-7") || - modelId.includes("opus-4.7") || - modelId.includes("opus-4-8") || - modelId.includes("opus-4.8") || - modelId.includes("opus-5") || - modelId.includes("opus.5") || - modelId.includes("sonnet-4-6") || - modelId.includes("sonnet-4.6") || - modelId.includes("sonnet-5") || - modelId.includes("sonnet.5") || - modelId.includes("haiku-5") || - modelId.includes("haiku.5") || - modelId.includes("fable-5") || - modelId.includes("mythos-5") - ); -} - -function isAnthropicTemperatureUnsupportedModel(modelId: string): boolean { - const id = modelId.toLowerCase(); - return ( - id.includes("opus-4-7") || - id.includes("opus-4.7") || - id.includes("opus-4-8") || - id.includes("opus-4.8") || - id.includes("opus-5") || - id.includes("opus.5") || - id.includes("sonnet-5-5") || - id.includes("sonnet-5.5") || - id.includes("haiku-5-5") || - id.includes("haiku-5.5") - ); -} - -const OPENAI_COMPLETIONS_DEFAULT_COMPAT = { - supportsStore: true, - supportsDeveloperRole: true, - supportsReasoningEffort: true, - supportsUsageInStreaming: true, - supportsFinishReason: true, - maxTokensField: "max_completion_tokens", - requiresToolResultName: false, - requiresAssistantAfterToolResult: false, - requiresThinkingAsText: false, - requiresReasoningContentOnAssistantMessages: false, - thinkingFormat: "openai", - openRouterRouting: {}, - vercelGatewayRouting: {}, - chatTemplateKwargs: {}, - chatTemplateArgs: {}, - zaiToolStream: false, - supportsStrictMode: false, - supportsOpenAIGrammarTools: false, - supportsMidConvoSystemMessages: false, - supportsMidConvoToolAdditions: false, - sendSessionAffinityHeaders: false, - supportsLongCacheRetention: true, -} satisfies Required< - Omit< - OpenAICompletionsCompat, - "cacheControlFormat" | "supportsThinkingTokenBudget" | "thinkingTokenBudgetField" - > -> & { - cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"]; -}; - -type OpenAICompletionsResolvedCompat = typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT & { - cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"]; -}; - -function mergeAnthropicMessagesCompat(model: Model, compat: AnthropicMessagesCompat): void { - model.compat = { ...(model.compat as AnthropicMessagesCompat | undefined), ...compat }; -} - -function detectOpenAICompletionsCompat(model: Model<"openai-completions">): OpenAICompletionsResolvedCompat { - const provider = model.provider; - const baseUrl = model.baseUrl; - - const isZai = - provider === "zai" || - provider === "zai-coding-cn" || - baseUrl.includes("api.z.ai") || - baseUrl.includes("open.bigmodel.cn"); - const isTogether = - provider === "together" || baseUrl.includes("api.together.ai") || baseUrl.includes("api.together.xyz"); - const isMoonshot = provider === "moonshotai" || provider === "moonshotai-cn" || baseUrl.includes("api.moonshot."); - const isOpenRouter = provider === "openrouter" || baseUrl.includes("openrouter.ai"); - const isCloudflareWorkersAI = provider === "cloudflare-workers-ai" || baseUrl.includes("api.cloudflare.com"); - const isCloudflareAiGateway = provider === "cloudflare-ai-gateway" || baseUrl.includes("gateway.ai.cloudflare.com"); - const isNvidia = provider === "nvidia" || baseUrl.includes("integrate.api.nvidia.com"); - const isAntLing = provider === "ant-ling" || baseUrl.includes("api.ant-ling.com"); - const isCerebras = provider === "cerebras" || baseUrl.includes("cerebras.ai"); - const isDeepSeek = provider === "deepseek" || baseUrl.toLowerCase().includes("deepseek.com"); - - const isNonStandard = - isNvidia || - isCerebras || - provider === "xai" || - baseUrl.includes("api.x.ai") || - isTogether || - baseUrl.includes("chutes.ai") || - isDeepSeek || - isZai || - isMoonshot || - provider === "opencode" || - baseUrl.includes("opencode.ai") || - isCloudflareWorkersAI || - isCloudflareAiGateway || - isAntLing; - - const useMaxTokens = - baseUrl.includes("chutes.ai") || - isDeepSeek || - isMoonshot || - isCloudflareAiGateway || - isTogether || - isNvidia || - isAntLing || - isZai; - - const isGrok = provider === "xai" || baseUrl.includes("api.x.ai"); - const isOpenRouterDeveloperRoleModel = - isOpenRouter && (model.id.startsWith("anthropic/") || model.id.startsWith("openai/")); - const cacheControlFormat = - provider === "openrouter" && /^~?anthropic\//.test(model.id) ? "anthropic" : undefined; - - return { - supportsStore: !isNonStandard, - supportsDeveloperRole: isOpenRouterDeveloperRoleModel || (!isNonStandard && !isOpenRouter), - supportsReasoningEffort: - !isGrok && !isZai && !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isAntLing, - supportsUsageInStreaming: true, - supportsFinishReason: true, - maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens", - requiresToolResultName: false, - requiresAssistantAfterToolResult: false, - requiresThinkingAsText: false, - requiresReasoningContentOnAssistantMessages: isDeepSeek, - thinkingFormat: isDeepSeek - ? "deepseek" - : isZai - ? "zai" - : isTogether - ? "together" - : isAntLing - ? "ant-ling" - : isOpenRouter - ? "openrouter" - : "openai", - openRouterRouting: {}, - vercelGatewayRouting: {}, - chatTemplateKwargs: {}, - chatTemplateArgs: {}, - zaiToolStream: false, - // Preserve built-in behavior as explicit metadata against the conservative runtime default. - supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isCerebras, - supportsOpenAIGrammarTools: false, - supportsMidConvoSystemMessages: false, - supportsMidConvoToolAdditions: false, - ...(cacheControlFormat ? { cacheControlFormat } : {}), - sendSessionAffinityHeaders: isOpenRouter, - supportsLongCacheRetention: !( - isTogether || - isCloudflareWorkersAI || - isCloudflareAiGateway || - isNvidia || - isAntLing - ), - }; -} - -function isPlainEmptyObject(value: unknown): boolean { - return typeof value === "object" && value !== null && !Array.isArray(value) && Object.keys(value).length === 0; -} - -function openAICompletionsCompatDelta(compat: OpenAICompletionsResolvedCompat): OpenAICompletionsCompat { - const delta: OpenAICompletionsCompat = {}; - for (const [key, value] of Object.entries(compat)) { - const defaultValue = OPENAI_COMPLETIONS_DEFAULT_COMPAT[key as keyof typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT]; - if (isPlainEmptyObject(value) && isPlainEmptyObject(defaultValue)) continue; - if (value !== defaultValue) { - (delta as Record)[key] = value; - } - } - return delta; -} - -function mergeOpenAICompletionsCompat(model: Model, compat: OpenAICompletionsCompat): void { - model.compat = { ...(model.compat as OpenAICompletionsCompat | undefined), ...compat }; -} - -function applyOpenAICompletionsCompatMetadata(model: Model): void { - if (model.api !== "openai-completions") return; - const detected = openAICompletionsCompatDelta(detectOpenAICompletionsCompat(model as Model<"openai-completions">)); - model.compat = { ...detected, ...(model.compat as OpenAICompletionsCompat | undefined) }; - if (Object.keys(model.compat).length === 0) { - delete model.compat; - } -} - -function applyAnthropicMessagesCompatMetadata(model: Model): void { - if (model.api !== "anthropic-messages") return; - const compat = getAnthropicMessagesCompat(model.provider, model.id); - if (compat) { - mergeAnthropicMessagesCompat(model, compat); - if (compat.supportsMidConvoEffort && !supportsAnthropicDisabledThinkingWithMidConvoEffort(model.id)) { - mergeThinkingLevelMap(model, { off: null }); - } - } -} - -function isAnthropicFallbackMetadataModel(model: Model): model is Model<"anthropic-messages"> { - if (model.provider !== "anthropic" || model.api !== "anthropic-messages") return false; - return ( - model.id in ANTHROPIC_ALLOWED_FALLBACK_MODELS || - Object.values(ANTHROPIC_ALLOWED_FALLBACK_MODELS).some((fallbackModelIds) => fallbackModelIds.includes(model.id)) - ); -} - -function applyAnthropicAllowedFallbackModelMetadata(models: readonly Model<"anthropic-messages">[]): void { - const modelsById = new Map(models.map((model) => [model.id, model])); - for (const [modelId, fallbackModelIds] of Object.entries(ANTHROPIC_ALLOWED_FALLBACK_MODELS)) { - const model = modelsById.get(modelId); - if (!model) continue; - - const compatibleFallbackModelIds = model.compat?.supportsMidConvoEffort - ? fallbackModelIds.filter(supportsAnthropicMidConvoEffort) - : fallbackModelIds; - const allowedFallbackModels = compatibleFallbackModelIds.flatMap((fallbackModelId) => { - const fallbackModel = modelsById.get(fallbackModelId); - return fallbackModel - ? [{ provider: fallbackModel.provider, model: fallbackModel.id, cost: fallbackModel.cost }] - : []; - }); - if (allowedFallbackModels.length > 0) { - mergeAnthropicMessagesCompat(model, { allowedFallbackModels }); - } - } -} - -function applyStrictToolCompatMetadata(model: Model): void { - if ( - (model.provider === "openai" || model.provider === "cloudflare-ai-gateway") && - model.api === "openai-responses" - ) { - model.compat = { ...(model.compat as OpenAIResponsesCompat | undefined), supportsStrictMode: true }; - } else if (model.provider === "anthropic" && model.api === "anthropic-messages") { - mergeAnthropicMessagesCompat(model, { supportsStrictTools: true }); - } -} - -// Responses endpoints verified (OpenAI, ChatGPT Codex backend, GitHub Copilot, -// opencode zen) or documented (Azure OpenAI, Cloudflare AI Gateway) to pass -// OpenAI custom grammar tools through. OpenAI rejects `type: "custom"` tools -// for pre-GPT-5 models (gpt-4.x, gpt-4o, o-series). -const OPENAI_GRAMMAR_TOOL_PROVIDERS = new Set([ - "openai", - "openai-codex", - "azure", - "github-copilot", - "opencode", - "cloudflare-ai-gateway", -]); -const OPENAI_GRAMMAR_TOOL_APIS = new Set([ - "openai-responses", - "azure-openai-responses", - "openai-codex-responses", -]); - -function applyOpenAIGrammarToolCompatMetadata(model: Model): void { - if (!OPENAI_GRAMMAR_TOOL_APIS.has(model.api) || !OPENAI_GRAMMAR_TOOL_PROVIDERS.has(model.provider)) return; - const match = /^gpt-(\d+)/.exec(model.id); - if (!match || Number(match[1]) < 5) return; - model.compat = { ...(model.compat as OpenAIResponsesCompat | undefined), supportsOpenAIGrammarTools: true }; -} - -function applyOpenAIToolSearchMetadata(model: Model): void { - const isOpenAIResponses = model.provider === "openai" && model.api === "openai-responses"; - const isOpenAICodex = model.provider === "openai-codex" && model.api === "openai-codex-responses"; - if (!(isOpenAIResponses || isOpenAICodex) || !OPENAI_TOOL_SEARCH_MODEL_IDS.has(model.id)) return; - const supportsAdditionalTools = - (isOpenAIResponses && OPENAI_ADDITIONAL_TOOLS_MODEL_IDS.has(model.id)) || - (isOpenAICodex && OPENAI_CODEX_ADDITIONAL_TOOLS_MODEL_IDS.has(model.id)); - model.compat = { - ...(model.compat as OpenAIResponsesCompat | undefined), - ...(supportsAdditionalTools ? { supportsAdditionalTools: true } : {}), - supportsToolSearch: true, - }; -} - -// Moonshot Kimi K2.6/K2.7 accept system text after the conversation starts but reject -// tool-bearing system messages. Kimi K3 accepts both forms; Fireworks and OpenCode pass -// its tool-bearing form through. GitHub Copilot forwards K3 text but silently drops its -// tool-bearing message. DeepSeek V4 Pro and OpenAI models behind OpenRouter also accept -// plain system text in place. Kimi Coding sets these flags in createKimiCodingModels(). -function applyOpenAICompletionsTranscriptMetadata(model: Model): void { - if (model.api !== "openai-completions") return; - const isKimiK3 = - (model.provider.startsWith("moonshot") && isKimiK3FamilyId(model.id)) || - (model.provider === "fireworks" && model.id.includes("kimi-k3")) || - ((model.provider === "opencode" || model.provider === "opencode-go") && isKimiK3FamilyId(model.id)); - const isMoonshotKimiK2 = - model.provider.startsWith("moonshot") && - (model.id === "kimi-k2.6" || model.id === "kimi-k2.7-code" || model.id === "kimi-k2.7-code-highspeed"); - const isTextOnly = - isMoonshotKimiK2 || - (model.provider === "github-copilot" && model.id === "kimi-k3") || - (model.provider === "deepseek" && model.id === "deepseek-v4-pro") || - (model.provider === "openrouter" && - model.id.startsWith("openai/") && - OPENAI_MID_CONVO_SYSTEM_MESSAGE_MODEL_IDS.has(model.id.slice("openai/".length))); - if (!isKimiK3 && !isTextOnly) return; - model.compat = { - ...(model.compat as OpenAICompletionsCompat | undefined), - supportsMidConvoSystemMessages: true, - ...(isKimiK3 ? { supportsMidConvoToolAdditions: true } : {}), - }; -} - -// Newer OpenAI Responses models accept developer messages after the conversation has started. -// OpenCode Zen, OpenCode Go, and GitHub Copilot pass both those messages and -// `additional_tools` items through to OpenAI unchanged; tool search is not verified -// through those proxies. -const OPENAI_RESPONSES_PROXY_PROVIDERS = new Set(["opencode", "opencode-go", "github-copilot"]); - -function applyOpenAIResponsesTranscriptMetadata(model: Model): void { - const isOpenAIResponses = model.provider === "openai" && model.api === "openai-responses"; - const isOpenAICodex = model.provider === "openai-codex" && model.api === "openai-codex-responses"; - const isProxiedResponses = - OPENAI_RESPONSES_PROXY_PROVIDERS.has(model.provider) && model.api === "openai-responses"; - if ( - !(isOpenAIResponses || isOpenAICodex || isProxiedResponses) || - !OPENAI_MID_CONVO_SYSTEM_MESSAGE_MODEL_IDS.has(model.id) - ) { - return; - } - model.compat = { - ...(model.compat as OpenAIResponsesCompat | undefined), - supportsMidConvoSystemMessages: true, - ...(isProxiedResponses && OPENAI_ADDITIONAL_TOOLS_MODEL_IDS.has(model.id) ? { supportsAdditionalTools: true } : {}), - }; -} - -// OpenAI charges prompt-cache writes starting with the GPT-5.6 family, and exactly -// those models accept `prompt_cache_options`; older models reject the parameter. -// https://developers.openai.com/api/docs/guides/prompt-caching -function applyOpenAIExplicitPromptCacheMetadata(model: Model): void { - if (model.provider !== "openai" || model.api !== "openai-responses") return; - if (!(model.cost.cacheWrite > 0)) return; - model.compat = { - ...(model.compat as OpenAIResponsesCompat | undefined), - supportsExplicitPromptCacheMode: true, - }; -} - -// GPT-6 models accept `configuration_update` input items that change the reasoning effort -// without changing the cached prompt prefix. Not enabled for openai-codex (Codex CLI keeps it -// behind a flag that is off by default) or proxies. -function applyOpenAIMidConvoEffortMetadata(model: Model): void { - if (model.provider !== "openai" || model.api !== "openai-responses") return; - if (!OPENAI_MID_CONVO_EFFORT_MODEL_IDS.has(model.id)) return; - model.compat = { - ...(model.compat as OpenAIResponsesCompat | undefined), - supportsMidConvoEffort: true, - }; -} - -// Anthropic ephemeral entries have a hard five-minute lifetime; `ttl: "1h"` -// extends it to one hour. Only direct Anthropic is annotated so cache warming -// does not assume equivalent behavior through proxies. -// https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching -const ANTHROPIC_PROMPT_CACHE: ModelPromptCache = { short: 300, long: 3600 }; - -function applyPromptCacheMetadata(model: Model): void { - if (model.provider === "anthropic" && model.api === "anthropic-messages") { - model.promptCache = ANTHROPIC_PROMPT_CACHE; - } - // Do not add OpenAI lifetimes yet. Before enabling warming for explicit - // OpenAI caches, re-evaluate it using observed expiry, replay, and billing - // behavior; a documented TTL alone does not establish full cache loss. -} - -function applyImageInputMetadata(model: AnyModel): void { - if (!model.input.includes("image")) return; - - const providerLimits: AnyModel["inputLimits"] = - model.provider === "anthropic" - ? { - maxRequestBytes: 32 * 1024 * 1024, - images: { maxPerRequest: model.type !== "image" && model.contextWindow === 200000 ? 100 : 600 }, - } - : model.provider === "amazon-bedrock" - ? { images: { maxPerMessage: 20 } } - : model.provider === "openai" - ? { maxRequestBytes: 512 * 1024 * 1024, images: { maxPerRequest: 1500 } } - : model.provider === "google" - ? { maxRequestBytes: 20 * 1024 * 1024, images: { maxPerRequest: 3600 } } - : undefined; - const configuredImages = model.inputLimits?.images; - model.inputLimits = { - ...providerLimits, - ...model.inputLimits, - images: { - ...providerLimits?.images, - ...configuredImages, - resize: { ...DEFAULT_IMAGE_RESIZE, ...configuredImages?.resize }, - }, - }; -} - -function isGemma4Model(modelId: string): boolean { - return /gemma-?4/.test(modelId.toLowerCase()); -} - -function getGoogleThinkingLevelMap( - modelId: string, - reasoningOptions: readonly ModelsDevReasoningOption[], -): NonNullable["thinkingLevelMap"]> | undefined { - const effortMap = getEffortThinkingLevelMap(reasoningOptions); - if (effortMap) return effortMap; - if (isGemma4Model(modelId)) { - return { off: null, minimal: "MINIMAL", low: null, medium: null, high: "HIGH" }; - } - return undefined; -} - -function applyThinkingLevelMetadata(model: Model): void { - if ( - (model.api === "openai-responses" || model.api === "azure-openai-responses") && - model.id.startsWith("gpt-5") - ) { - mergeThinkingLevelMap(model, { off: null }); - } - if ( - (model.id === "gpt-6-astra" || - model.id === "gpt-6-sol" || - model.id === "gpt-6-luna" || - model.id === "gpt-6.1-sol" || - model.id === "gpt-6.1-terra") && - (model.api === "openai-responses" || - model.api === "azure-openai-responses" || - model.api === "openai-codex-responses") - ) { - mergeThinkingLevelMap(model, { - // GPT-6 Astra and GPT-6.1 Sol reject reasoning.effort "none"; GPT-6.1 Terra accepts it. All reject - // "minimal" (verified on Codex, October 2026). - off: model.id === "gpt-6-astra" || model.id === "gpt-6.1-sol" ? null : "none", - minimal: null, - low: "low", - medium: "medium", - high: "high", - xhigh: "xhigh", - max: "max", - }); - } - if (model.provider === "github-copilot" && model.id.startsWith("gpt-5")) { - mergeThinkingLevelMap(model, { minimal: "low" }); - } - if ( - model.api === "openai-responses" && - model.provider === "openai" && - OPENAI_RESPONSES_NONE_REASONING_MODELS.has(model.id) - ) { - mergeThinkingLevelMap(model, { off: "none" }); - } - // xAI models without verified effort options must not send the undocumented - // "none"/"minimal" efforts. - if (model.provider === "xai" && model.api === "openai-responses" && model.thinkingLevelMap === undefined) { - mergeThinkingLevelMap(model, { off: null, minimal: null }); - } - if (supportsOpenAiXhigh(model.id)) { - mergeThinkingLevelMap(model, { xhigh: "xhigh" }); - } - if (supportsOpenAiMax(model)) { - mergeThinkingLevelMap(model, { max: "max" }); - } - if (model.provider === "openai" && model.id === "gpt-5.5") { - mergeThinkingLevelMap(model, { minimal: null }); - } - if (model.id.endsWith("gpt-5.5-pro")) { - mergeThinkingLevelMap(model, { off: null, minimal: null, low: null }); - } - // Anthropic adaptive-thinking effort support (per Anthropic adaptive thinking docs): - // - "max" is available on all adaptive-thinking Claude models. - // - "xhigh" is only available on Opus 4.7/4.8/5, Sonnet 5, Haiku 5.5, and Fable 5. - if ( - model.id.includes("opus-4-6") || - model.id.includes("opus-4.6") || - model.id.includes("sonnet-4-6") || - model.id.includes("sonnet-4.6") - ) { - mergeThinkingLevelMap(model, { max: "max" }); - } - if ( - model.id.includes("opus-4-7") || - model.id.includes("opus-4.7") || - model.id.includes("opus-4-8") || - model.id.includes("opus-4.8") || - model.id.includes("opus-5") || - model.id.includes("opus.5") || - model.id.includes("sonnet-5") || - model.id.includes("sonnet.5") || - model.id.includes("haiku-5") || - model.id.includes("haiku.5") - ) { - mergeThinkingLevelMap(model, { xhigh: "xhigh", max: "max" }); - } - if (model.id.includes("fable-5")) { - mergeThinkingLevelMap(model, { off: null, xhigh: "xhigh", max: "max" }); - } - if (model.api === "anthropic-messages" && isAnthropicAdaptiveThinkingModel(model.id)) { - mergeAnthropicMessagesCompat(model, { forceAdaptiveThinking: true }); - } - if (model.api === "anthropic-messages" && isAnthropicTemperatureUnsupportedModel(model.id)) { - mergeAnthropicMessagesCompat(model, { supportsTemperature: false }); - } - if ( - model.api === "openai-completions" && - model.id.includes("deepseek-v4") && - model.thinkingLevelMap === undefined - ) { - mergeThinkingLevelMap( - model, - model.provider === "openrouter" - ? { ...DEEPSEEK_V4_THINKING_LEVEL_MAP, xhigh: "xhigh", max: null } - : model.provider === "azure" - ? AZURE_DEEPSEEK_V4_THINKING_LEVEL_MAP - : (model.provider === "deepseek" || model.provider === "opencode" || model.provider === "opencode-go") && - model.id.includes("deepseek-v4-flash") - ? DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP - : DEEPSEEK_V4_THINKING_LEVEL_MAP, - ); - } - if (model.provider === "groq" && model.id === "qwen/qwen3.6-27b") { - mergeThinkingLevelMap(model, { minimal: null, low: null, medium: null, high: "default" }); - } - if (model.provider === "openai-codex" && supportsOpenAiXhigh(model.id)) { - mergeThinkingLevelMap(model, { minimal: "low" }); - } - if ( - (model.provider === "moonshotai" || model.provider === "moonshotai-cn") && - (model.id === "kimi-k2.7-code" || model.id === "kimi-k2.7-code-highspeed") - ) { - // Kimi K2.7 Code is always-thinking. Official docs say - // `thinking: { type: "disabled" }` is rejected, and callers can omit - // the thinking parameter to use the enabled default. - mergeThinkingLevelMap(model, { off: null }); - } - if (model.provider === "openrouter" && model.id.startsWith("inception/mercury-2")) { - // Mercury 2 in instant mode (reasoning_effort: "none") disables tool calling. - // Mark "off" unsupported so the openai-completions provider omits the reasoning param - // instead of defaulting to {reasoning:{effort:"none"}} (see openai-completions.ts:575). - // Pi's low/medium/high pass through verbatim; OpenRouter normalizes to Mercury's vocabulary. - mergeThinkingLevelMap(model, { off: null }); - } - if (model.provider === "openrouter" && model.id === "z-ai/glm-5.2") { - mergeThinkingLevelMap(model, { xhigh: "xhigh" }); - } - if (model.provider === "fireworks") { - if (model.api === "anthropic-messages" && model.compat?.forceAdaptiveThinking) { - // Qwen Max currently advertises only a toggle. Prefer upstream effort - // metadata once available instead of replacing it with this fallback. - if (model.id === "accounts/fireworks/models/qwen3p8-max" && !model.thinkingLevelMap) { - model.thinkingLevelMap = getEffortThinkingLevelMap([ - { type: "effort", values: ["low", "medium", "xhigh"] }, - ]); - } - const reasoningOptions = modelsDevReasoningOptions.get(getModelKey(model)); - if ( - reasoningOptions?.some((option) => option.type === "toggle") || - // The 2.4T alias omits the verified toggle in models.dev. - model.id === "accounts/fireworks/models/qwen3p8-2p4t-a95b" - ) { - mergeThinkingLevelMap(model, { off: "none" }); - } - if (model.id === "accounts/fireworks/models/deepseek-v4-pro-0813") { - mergeThinkingLevelMap(model, { low: "low" }); - } - } - if (model.id.includes("glm-5p2")) { - // GLM 5.2 and its fast router support off/high/max. Fireworks maps low - // and medium to high, so do not expose those aliases as distinct levels. - mergeThinkingLevelMap(model, { off: "none", minimal: null, low: null, medium: null, max: "max" }); - } - if (model.id.includes("kimi-k3")) { - // Fireworks maps medium to high on both APIs; do not expose it as a distinct level. - mergeThinkingLevelMap(model, { medium: null }); - } - } - if (model.provider === "opencode-go" && model.id === "glm-5.2") { - mergeThinkingLevelMap(model, OPENCODE_GO_GLM52_THINKING_LEVEL_MAP); - } - if (model.provider === "opencode-go" && model.id === "kimi-k2.6") { - // OpenCode Go exposes Kimi K2.6 thinking as on/off, not distinct effort tiers. - mergeThinkingLevelMap(model, { minimal: null, low: null, medium: null }); - } - if (model.provider === "opencode" && model.id === "grok-build-0.1") { - // OpenCode Zen Grok Build reasons by default but rejects explicit reasoningEffort. - mergeThinkingLevelMap(model, { off: null, minimal: null, low: null, medium: null }); - } - if (model.provider === "ant-ling" && model.reasoning) { - // Ring reasons by default. Only high/xhigh have documented explicit effort controls. - mergeThinkingLevelMap(model, ANT_LING_RING_THINKING_LEVEL_MAP); - } - if (model.provider === "github-copilot") { - const override = GITHUB_COPILOT_THINKING_LEVEL_OVERRIDES[model.id]; - if (override) { - mergeThinkingLevelMap(model, override); - } - } -} - -function getAnthropicMessagesCompat(provider: string, modelId: string): AnthropicMessagesCompat | undefined { - const compat: AnthropicMessagesCompat = {}; - if ( - VERIFIED_ANTHROPIC_MID_CONVO_EFFORT_PROVIDERS.has(provider) && - supportsAnthropicMidConvoEffort(modelId) && - !MID_CONVO_EFFORT_UNSUPPORTED_ANTHROPIC_MODELS.has(`${provider}:${modelId}`) - ) { - compat.supportsMidConvoEffort = true; - } - if (provider === "anthropic" && supportsAnthropicMidConvoSystemMessages(modelId)) { - compat.supportsMidConvoSystemMessages = true; - compat.supportsMidConvoToolChanges = true; - } - // OpenCode Zen and GitHub Copilot forward mid-conversation system messages but reject - // `tool_addition`/`tool_removal` blocks, so tool changes stay top-level there. - if ((provider === "opencode" || provider === "github-copilot") && supportsAnthropicMidConvoSystemMessages(modelId)) { - compat.supportsMidConvoSystemMessages = true; - } - if (EAGER_TOOL_INPUT_STREAMING_UNSUPPORTED_ANTHROPIC_MODELS.has(`${provider}:${modelId}`)) { - compat.supportsEagerToolInputStreaming = false; - } - if (provider === "xiaomi" || provider.startsWith("xiaomi-token-plan-")) { - compat.allowEmptySignature = true; - } - // OpenCode Qwen 3.8 Flash emits and accepts thinking blocks with empty signatures. - if ((provider === "opencode" || provider === "opencode-go") && modelId === "qwen3.8-flash") { - compat.allowEmptySignature = true; - } - return Object.keys(compat).length > 0 ? compat : undefined; -} - -function getBedrockBaseUrl(modelId: string): string { - return modelId.startsWith("eu.") - ? "https://bedrock-runtime.eu-central-1.amazonaws.com" - : "https://bedrock-runtime.us-east-1.amazonaws.com"; -} - -function normalizeNvidiaModelId(modelId: string): string { - return modelId.toLowerCase().replaceAll("_", "."); -} - -function roundCost(value: number): number { - return Number(value.toFixed(6)); -} - -function getModelsDevCost(cost: ModelsDevModel["cost"]): ModelCost { - const base = { - input: cost?.input || 0, - output: cost?.output || 0, - cacheRead: cost?.cache_read || 0, - cacheWrite: cost?.cache_write || 0, - }; - // Rates a tier does not list keep the base price. - const tiers = cost?.tiers?.flatMap((tier) => { - const context = tier.tier; - if (context?.type !== "context" || context.size === undefined) return []; - return [ - { - inputTokensAbove: context.size, - input: tier.input ?? base.input, - output: tier.output ?? base.output, - cacheRead: tier.cache_read ?? base.cacheRead, - cacheWrite: tier.cache_write ?? base.cacheWrite, - }, - ]; - }); - - return tiers && tiers.length > 0 ? { ...base, tiers } : base; -} - -async function fetchNvidiaNimModelIds(): Promise> { - try { - console.log("Fetching models from NVIDIA NIM API..."); - const response = await fetch(`${NVIDIA_BASE_URL}/models`); - if (!response.ok) throw new Error(`NVIDIA NIM API returned ${response.status}`); - const data = (await response.json()) as { data?: NvidiaNimModelListItem[] }; - const modelIds = new Map(); - - for (const model of data.data ?? []) { - modelIds.set(model.id, model.id); - modelIds.set(normalizeNvidiaModelId(model.id), model.id); - } - - console.log(`Fetched ${data.data?.length ?? 0} model IDs from NVIDIA NIM`); - return modelIds; - } catch (error) { - console.error("Failed to fetch NVIDIA NIM models:", error); - if (generatorOptions.strict) throw error; - return new Map(); - } -} - -async function fetchOpenRouterList(query: string): Promise { - const response = await fetch(`https://openrouter.ai/api/v1/models${query}`); - if (!response.ok) throw new Error(`OpenRouter API returned ${response.status}`); - const data = (await response.json()) as { data?: OpenRouterModelListItem[] }; - return data.data ?? []; -} - -async function fetchOpenRouterModels(): Promise { - try { - console.log("Fetching models from OpenRouter API..."); - const [listed, imageListed, decisionListed] = await Promise.all([ - fetchOpenRouterList(""), - fetchOpenRouterList("?output_modalities=image"), - fetchOpenRouterList("?output_modalities=decisions"), - ]); - const catalog = buildOpenRouterCatalog(listed, imageListed, decisionListed); - console.log( - `Fetched ${catalog.chat.length} tool-capable, ${catalog.images.length} image, and ${catalog.classifiers.length} classifier models from OpenRouter`, - ); - if (generatorOptions.strict && catalog.images.length === 0) { - throw new Error("OpenRouter API returned no usable image models"); - } - return catalog; - } catch (error) { - console.error("Failed to fetch OpenRouter models:", error); - if (generatorOptions.strict) throw error; - return { chat: [], images: [], classifiers: [] }; - } -} - -async function fetchRadiusModels(): Promise[]> { - try { - console.log("Fetching models from Radius API..."); - const config = await loadRadiusGatewayConfig(DEFAULT_RADIUS_GATEWAY); - const models = getRadiusModelsFromConfig("radius", config); - if (models.length === 0) throw new Error("Radius API returned no models"); - console.log(`Fetched ${models.length} models from Radius`); - return models; - } catch (error) { - console.error("Failed to fetch Radius models:", error); - if (generatorOptions.strict) throw error; - return []; - } -} - -async function fetchAiGatewayModels(): Promise<{ - chat: Model[]; - classifiers: ClassifierModel<"typesafe-system-one">[]; -}> { - try { - console.log("Fetching models from Vercel AI Gateway API..."); - const response = await fetch(`${AI_GATEWAY_MODELS_URL}/models`); - if (!response.ok) throw new Error(`Vercel AI Gateway API returned ${response.status}`); - const data = await response.json(); - const models: Model[] = []; - const classifiers: ClassifierModel<"typesafe-system-one">[] = []; - - const toNumber = (value: string | number | undefined): number => { - if (typeof value === "number") { - return Number.isFinite(value) ? value : 0; - } - const parsed = parseFloat(value ?? "0"); - return Number.isFinite(parsed) ? parsed : 0; - }; - - const items = Array.isArray(data.data) ? (data.data as AiGatewayModel[]) : []; - for (const model of items) { - // Evaluation models such as TypeSafe's Jev are served through the - // TypeSafe-compatible System One endpoint. - if (model.type === "evaluation") { - classifiers.push({ - type: "classifier", - id: model.id, - name: model.name || model.id, - api: "typesafe-system-one", - provider: "vercel-ai-gateway", - baseUrl: AI_GATEWAY_TYPESAFE_BASE_URL, - input: ["text"], - cost: { - input: roundCost(toNumber(model.pricing?.input) * 1_000_000), - output: roundCost(toNumber(model.pricing?.output) * 1_000_000), - cacheRead: 0, - cacheWrite: 0, - }, - contextWindow: model.context_window || 4096, - }); - continue; - } - const tags = Array.isArray(model.tags) ? model.tags : []; - // Only include models that support tools - if (!tags.includes("tool-use")) continue; - - const input: ("text" | "image")[] = ["text"]; - if (tags.includes("vision")) { - input.push("image"); - } - - models.push({ - id: model.id, - name: model.name || model.id, - api: "anthropic-messages", - baseUrl: AI_GATEWAY_BASE_URL, - provider: "vercel-ai-gateway", - reasoning: tags.includes("reasoning"), - input, - compat: { allowEmptySignature: true }, - cost: getAiGatewayCost(model.pricing), - contextWindow: model.context_window || 4096, - maxTokens: model.max_tokens || 4096, - }); - } - - console.log( - `Fetched ${models.length} tool-capable and ${classifiers.length} classifier models from Vercel AI Gateway`, - ); - return { chat: models, classifiers }; - } catch (error) { - console.error("Failed to fetch Vercel AI Gateway models:", error); - if (generatorOptions.strict) throw error; - return { chat: [], classifiers: [] }; - } -} - -function processZaiModels(data: ModelsDevCatalog): Model[] { - const variants = [ - { - source: "zai-coding-plan", - provider: "zai", - baseUrl: "https://api.z.ai/api/coding/paas/v4", - }, - { - source: "zhipuai-coding-plan", - provider: "zai-coding-cn", - baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", - }, - ] as const; - const models: Model[] = []; - - for (const { source, provider, baseUrl } of variants) { - for (const [modelId, model] of Object.entries(data[source]?.models ?? {})) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - const supportsImage = m.modalities?.input?.includes("image"); - - const thinkingLevelMap = getEffortThinkingLevelMap(m.reasoning_options ?? []); - const isGlm52 = modelId === "glm-5.2" || modelId === "glm-5.2-highspeed"; - if (thinkingLevelMap && isGlm52) { - thinkingLevelMap.off = "none"; - } - const supportsReasoningEffort = thinkingLevelMap !== undefined; - const referenceCost = data.zai?.models[modelId]?.cost ?? m.cost; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider, - baseUrl, - reasoning: m.reasoning === true, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: supportsImage ? ["text", "image"] : ["text"], - cost: getModelsDevCost(referenceCost), - compat: { - supportsDeveloperRole: false, - thinkingFormat: "zai", - ...(supportsReasoningEffort ? { supportsReasoningEffort: true } : {}), - ...(!ZAI_TOOL_STREAM_UNSUPPORTED_MODELS.has(modelId) ? { zaiToolStream: true } : {}), - }, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions(provider, modelId, m); - } - } - - return models; -} - -function processBasetenModels(provider: ModelsDevProvider | undefined): Model[] { - if (!provider?.models) return []; - - const baseUrl = "https://inference.baseten.co/v1"; - const baseCompat: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - supportsUsageInStreaming: true, - maxTokensField: "max_tokens", - supportsStrictMode: true, - // Baseten automatic prompt caching needs session affinity so related - // requests land on the same replica. See: - // https://docs.baseten.co/inference/model-apis/pricing-and-limits - sendSessionAffinityHeaders: true, - supportsLongCacheRetention: false, - }; - const reasoningEffortCompat: OpenAICompletionsCompat = { - ...baseCompat, - supportsReasoningEffort: true, - thinkingFormat: "openai", - }; - const toggleReasoningCompat: OpenAICompletionsCompat = { - ...baseCompat, - thinkingFormat: "baseten", - chatTemplateArgs: { enable_thinking: { $var: "thinking.enabled" } }, - }; - const toggleReasoningEffortCompat: OpenAICompletionsCompat = { - ...reasoningEffortCompat, - thinkingFormat: "baseten", - chatTemplateArgs: { enable_thinking: { $var: "thinking.enabled" } }, - }; - const toggleThinkingLevelMap = { - off: "off", - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: null, - max: null, - } as const; - const glm52ThinkingLevelMap = { - off: "none", - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: null, - max: "max", - } as const; - const models: Model[] = []; - - for (const [modelId, model] of Object.entries(provider.models)) { - if (model.status === "deprecated") continue; - - const reasoning = model.reasoning === true; - const reasoningOptions = model.reasoning_options ?? []; - const isGlm52 = modelId === "zai-org/GLM-5.2" || modelId === "zai-org/GLM-5.2-Fast"; - const supportsToggle = reasoningOptions.some((option) => option.type === "toggle") || isGlm52; - const supportsEffort = reasoningOptions.some((option) => option.type === "effort") || isGlm52; - const compat = - supportsToggle && supportsEffort - ? toggleReasoningEffortCompat - : supportsToggle - ? toggleReasoningCompat - : supportsEffort - ? reasoningEffortCompat - : baseCompat; - const thinkingLevelMap = isGlm52 - ? glm52ThinkingLevelMap - : supportsToggle - ? toggleThinkingLevelMap - : getEffortThinkingLevelMap(reasoningOptions); - // Baseten's GLM-5.2 endpoints are text-only despite models.dev reporting image input. - const supportsImageInput = !isGlm52 && model.modalities?.input?.includes("image"); - - models.push({ - id: modelId, - name: model.name || modelId, - api: "openai-completions", - provider: "baseten", - baseUrl, - reasoning, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: supportsImageInput ? ["text", "image"] : ["text"], - cost: getModelsDevCost(model.cost), - compat, - contextWindow: model.limit?.context || 4096, - maxTokens: model.limit?.output || 4096, - }); - } - - return models; -} - -function processGoogleModels(data: ModelsDevCatalog): Model[] { - const models: Model[] = []; - const googleModels = data.google?.models; - if (googleModels) { - for (const [modelId, model] of Object.entries(googleModels)) { - if (model.tool_call !== true) continue; - const source = - modelId === "gemini-flash-latest" - ? (googleModels["gemini-3.5-flash"] ?? model) - : modelId === "gemini-flash-lite-latest" - ? (googleModels["gemini-3.1-flash-lite"] ?? model) - : model; - const thinkingLevelMap = getGoogleThinkingLevelMap(modelId, source.reasoning_options ?? []); - - models.push({ - id: modelId, - name: model.name || modelId, - api: "google-generative-ai", - provider: "google", - baseUrl: "https://generativelanguage.googleapis.com/v1beta", - reasoning: source.reasoning === true, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: source.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(source.cost), - contextWindow: source.limit?.context || 4096, - maxTokens: source.limit?.output || 4096, - }); - } - } - - // The google-vertex models.dev catalog also includes Claude, OpenAI, and other - // MaaS models that do not use the @google/genai Gemini streaming path. - const vertexModels = data["google-vertex"]?.models; - if (vertexModels) { - for (const [modelId, model] of Object.entries(vertexModels)) { - if (model.tool_call !== true || !modelId.startsWith("gemini-")) continue; - if (modelId === "gemini-3.1-flash-lite-preview") continue; - const source = - modelId === "gemini-flash-latest" - ? (vertexModels["gemini-3.5-flash"] ?? model) - : modelId === "gemini-flash-lite-latest" - ? (vertexModels["gemini-3.1-flash-lite"] ?? model) - : model; - const thinkingLevelMap = getGoogleThinkingLevelMap(modelId, source.reasoning_options ?? []); - // models.dev reports Vertex cache_read/cache_write values for Gemini 2.5 Flash that - // do not match the official Gemini API standard pricing table. pi only accounts - // cachedContentTokenCount as cacheRead. - const cacheRead = modelId === "gemini-2.5-flash" ? 0.03 : source.cost?.cache_read || 0; - - models.push({ - id: modelId, - name: model.name || modelId, - api: "google-vertex", - provider: "google-vertex", - baseUrl: VERTEX_BASE_URL, - reasoning: source.reasoning === true, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: source.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: { - input: source.cost?.input || 0, - output: source.cost?.output || 0, - cacheRead, - cacheWrite: 0, - }, - contextWindow: source.limit?.context || 4096, - maxTokens: source.limit?.output || 4096, - }); - } - } - - return models; -} - -function processFireworksModels(provider: ModelsDevProvider | undefined): Model[] { - if (!provider?.models) return []; - - const anthropicCompat: AnthropicMessagesCompat = { - allowEmptySignature: true, - sendSessionAffinityHeaders: true, - supportsEagerToolInputStreaming: false, - supportsCacheControlOnTools: false, - supportsLongCacheRetention: false, - }; - const openAICompat: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - sendSessionAffinityHeaders: true, - supportsLongCacheRetention: false, - }; - const kimiK3Compat: OpenAICompletionsCompat = { - ...openAICompat, - requiresReasoningContentOnAssistantMessages: true, - thinkingFormat: "openai", - }; - const models: Model[] = []; - - for (const [modelId, model] of Object.entries(provider.models)) { - if (model.tool_call !== true) continue; - - const input: ("text" | "image")[] = model.modalities?.input?.includes("image") - ? ["text", "image"] - : ["text"]; - const common = { - id: modelId, - name: model.name || modelId, - provider: "fireworks", - reasoning: model.reasoning === true, - input, - cost: getModelsDevCost(model.cost), - contextWindow: model.limit?.context || 4096, - maxTokens: model.limit?.output || 4096, - }; - - if (modelId.includes("glm-")) { - models.push({ - ...common, - api: "openai-completions", - baseUrl: "https://api.fireworks.ai/inference/v1", - compat: openAICompat, - }); - } else if (modelId.includes("kimi-k3")) { - models.push({ - ...common, - api: "openai-completions", - baseUrl: "https://api.fireworks.ai/inference/v1", - compat: kimiK3Compat, - }); - } else { - models.push({ - ...common, - api: "anthropic-messages", - // Fireworks Anthropic-compatible API - SDK appends /v1/messages. - baseUrl: "https://api.fireworks.ai/inference", - // Fireworks prompt caching uses automatic prefix matching + session affinity. - // x-session-affinity routes requests to the same replica for cache hits. - // cache_control on tools and eager_input_streaming are not supported. - // See: https://docs.fireworks.ai/tools-sdks/anthropic-compatibility - // Use adaptive thinking for cataloged effort controls, with verified - // fallbacks where models.dev is incomplete. New models need no allowlist entry. - compat: { - ...anthropicCompat, - ...(model.reasoning_options?.some((option) => option.type === "effort") || - FIREWORKS_ADAPTIVE_THINKING_FALLBACK_MODELS.has(modelId) - ? { forceAdaptiveThinking: true } - : {}), - }, - }); - } - recordModelsDevReasoningOptions("fireworks", modelId, model); - } - - return models; -} - -// Kimi Coding is subscription-backed, so models.dev reports zero cost. Use the -// equivalent Moonshot API rates to estimate the value of subscription usage. -const KIMI_CODING_IMPLIED_COSTS: Record["cost"]> = { - k3: KIMI_K3_COST, - "k3-256k": KIMI_K3_COST, - "k3.1-pro": { input: 2, output: 10, cacheRead: 0.2, cacheWrite: 0 }, - "k3.1-pro-highspeed": { input: 4, output: 20, cacheRead: 0.2, cacheWrite: 0 }, - "kimi-for-coding": { input: 0.95, output: 4, cacheRead: 0.19, cacheWrite: 0 }, - "kimi-for-coding-highspeed": { input: 1.9, output: 8, cacheRead: 0.38, cacheWrite: 0 }, -}; - -// Kimi Coding uses Chat Completions, the same protocol as the official Kimi Code client. -function createKimiCodingModels(kimiModels: Record): Model<"openai-completions">[] { - const models: Model<"openai-completions">[] = []; - for (const [modelId, m] of Object.entries(kimiModels)) { - if (m.tool_call !== true) continue; - - // K3-style models report effort levels and dynamic tool loading (tools declared in - // later system messages) in https://api.kimi.com/coding/v1/models; models.dev mirrors - // the effort levels. kimi-for-coding-highspeed reports neither and rejects - // tool-bearing system messages with a 400. - const isK3Style = m.reasoning_options?.some((option) => option.type === "effort") === true; - const impliedCost = KIMI_CODING_IMPLIED_COSTS[modelId]; - // K3-style models send reasoning_effort, so expose the effort values models.dev lists. - const effortLevels = isK3Style && m.reasoning_options ? getEffortThinkingLevelMap(m.reasoning_options) : undefined; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "kimi-coding", - baseUrl: "https://api.kimi.com/coding/v1", - compat: { - ...(isK3Style ? KIMI_K3_OPENAI_COMPAT : KIMI_OPENAI_COMPAT), - // All models accept system text after the conversation starts. - supportsMidConvoSystemMessages: true, - ...(isK3Style ? { supportsMidConvoToolAdditions: true } : {}), - }, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: { - input: m.cost?.input || impliedCost?.input || 0, - output: m.cost?.output || impliedCost?.output || 0, - cacheRead: m.cost?.cache_read || impliedCost?.cacheRead || 0, - cacheWrite: m.cost?.cache_write || impliedCost?.cacheWrite || 0, - }, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - // Kimi Code reports every model as always-thinking. - thinkingLevelMap: { ...effortLevels, off: null }, - }); - } - return models; -} - -async function loadModelsDevData(): Promise[]> { - try { - console.log("Fetching models from models.dev API..."); - const response = await fetch("https://models.dev/api.json"); - if (!response.ok) throw new Error(`models.dev API returned ${response.status}`); - const data = (await response.json()) as ModelsDevCatalog; - - const models: Model[] = []; - const nvidiaNimModelIds = data.nvidia?.models ? await fetchNvidiaNimModelIds() : new Map(); - - // Process Amazon Bedrock models - if (data["amazon-bedrock"]?.models) { - for (const [modelId, model] of Object.entries(data["amazon-bedrock"].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (BEDROCK_INFERENCE_PROFILE_ONLY_MODEL_IDS.has(modelId)) continue; - - let id = modelId; - - if (id.startsWith("ai21.jamba")) { - // These models doesn't support tool use in streaming mode - continue; - } - - if (id.startsWith("mistral.mistral-7b-instruct-v0")) { - // These models doesn't support system messages - continue; - } - - models.push({ - id, - name: m.name || id, - api: "bedrock-converse-stream" as const, - provider: "amazon-bedrock" as const, - baseUrl: getBedrockBaseUrl(id), - reasoning: m.reasoning === true, - input: (m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"]) as ("text" | "image")[], - // Includes models.dev pricing tiers, e.g. the long-context tier for OpenAI models (#10326). - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - ...(m.structured_output === true && { compat: { supportsStrictMode: true } }), - }); - recordModelsDevReasoningOptions("amazon-bedrock" as const, id, m); - } - } - - // Process Anthropic models - if (data.anthropic?.models) { - for (const [modelId, model] of Object.entries(data.anthropic.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "anthropic-messages", - provider: "anthropic", - baseUrl: "https://api.anthropic.com", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("anthropic", modelId, m); - } - } - - models.push(...processGoogleModels(data)); - - // Process OpenAI models - if (data.openai?.models) { - for (const [modelId, model] of Object.entries(data.openai.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - // models.dev lists this alias, but it is not accepted by OpenAI APIs. - if (MODELS_DEV_OPENAI_UNSUPPORTED_MODEL_IDS.has(modelId)) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-responses", - provider: "openai", - baseUrl: "https://api.openai.com/v1", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("openai", modelId, m); - } - } - - // Process Groq models - if (data.groq?.models) { - for (const [modelId, model] of Object.entries(data.groq.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "groq", - baseUrl: "https://api.groq.com/openai/v1", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("groq", modelId, m); - } - } - - // Process Cerebras models - if (data.cerebras?.models) { - for (const [modelId, model] of Object.entries(data.cerebras.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "cerebras", - baseUrl: "https://api.cerebras.ai/v1", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("cerebras", modelId, m); - } - } - - // Process Cloudflare Workers AI models - if (data["cloudflare-workers-ai"]?.models) { - for (const [modelId, model] of Object.entries(data["cloudflare-workers-ai"].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_BASE_URL, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - compat: { sendSessionAffinityHeaders: true }, - }); - recordModelsDevReasoningOptions("cloudflare-workers-ai", modelId, m); - } - } - - // Process Cloudflare AI Gateway models - const cloudflareAIGatewayModelIds = new Set(); - if (data["cloudflare-ai-gateway"]?.models) { - for (const [prefixedId, model] of Object.entries(data["cloudflare-ai-gateway"].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - const slashIdx = prefixedId.indexOf("/"); - if (slashIdx === -1) continue; - const upstream = prefixedId.slice(0, slashIdx); - const nativeId = prefixedId.slice(slashIdx + 1); - - let api: "anthropic-messages" | "openai-completions" | "openai-responses"; - let baseUrl: string; - let id: string; - if (upstream === "openai") { - api = "openai-responses"; - baseUrl = CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL; - id = nativeId; - } else if (upstream === "anthropic") { - api = "anthropic-messages"; - baseUrl = CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL; - // The /anthropic passthrough forwards the model ID to Anthropic unchanged. - // models.dev lists dotted versions (claude-opus-5.5), but Anthropic only - // accepts dashed IDs (claude-opus-5-5). - id = nativeId.replaceAll(".", "-"); - } else if (upstream === "workers-ai") { - api = "openai-completions"; - baseUrl = CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL; - id = prefixedId; - } else { - continue; - } - - // Gateway passthroughs forward session affinity headers to upstreams that - // use them for cache/routing affinity. - const compat = - upstream === "anthropic" || upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined; - - cloudflareAIGatewayModelIds.add(id); - models.push({ - id, - name: m.name || id, - api, - provider: "cloudflare-ai-gateway", - baseUrl, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - ...(compat ? { compat } : {}), - }); - recordModelsDevReasoningOptions("cloudflare-ai-gateway", id, m); - } - } - - // The gateway proxies Workers AI through its OpenAI-compatible /compat endpoint, - // but models.dev may omit or intermittently drop those `workers-ai/*` entries - // from the AI Gateway catalog. Mirror the Workers AI catalog under the documented - // prefix so the gateway keeps its OpenAI-compatible models stable. - if (data["cloudflare-workers-ai"]?.models) { - for (const [modelId, model] of Object.entries(data["cloudflare-workers-ai"].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - const id = `workers-ai/${modelId}`; - if (cloudflareAIGatewayModelIds.has(id)) continue; - cloudflareAIGatewayModelIds.add(id); - - models.push({ - id, - name: m.name || id, - api: "openai-completions", - provider: "cloudflare-ai-gateway", - baseUrl: CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - compat: { sendSessionAffinityHeaders: true }, - }); - recordModelsDevReasoningOptions("cloudflare-ai-gateway", id, m); - } - } - - // Process xAi models - if (data.xai?.models) { - for (const [modelId, model] of Object.entries(data.xai.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-responses", - provider: "xai", - baseUrl: "https://api.x.ai/v1", - compat: { ...XAI_RESPONSES_COMPAT }, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("xai", modelId, m); - } - } - - // Process Meta models - if (data.meta?.models) { - for (const [modelId, model] of Object.entries(data.meta.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-responses", - provider: "meta", - baseUrl: "https://api.meta.ai/v1", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("meta", modelId, m); - } - } - - models.push(...processZaiModels(data)); - - // Process Mistral models - if (data.mistral?.models) { - for (const [modelId, model] of Object.entries(data.mistral.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - // Models with effort values use `reasoning_effort` with these levels. - // Reasoning models without them (Magistral) use `prompt_mode`. - const thinkingLevelMap = getEffortThinkingLevelMap(m.reasoning_options ?? []); - - models.push({ - id: modelId, - name: m.name || modelId, - api: "mistral-conversations", - provider: "mistral", - baseUrl: "https://api.mistral.ai", - reasoning: m.reasoning === true, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: { - input: m.cost?.input || 0, - output: m.cost?.output || 0, - cacheRead: m.cost?.cache_read ?? (m.cost?.input ? roundCost(m.cost.input * 0.1) : 0), - cacheWrite: m.cost?.cache_write || 0, - }, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - } - } - - // Process Hugging Face models - if (data.huggingface?.models) { - for (const [modelId, model] of Object.entries(data.huggingface.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "huggingface", - baseUrl: "https://router.huggingface.co/v1", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - compat: { - supportsDeveloperRole: false, - }, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("huggingface", modelId, m); - } - } - - models.push(...processFireworksModels(data["fireworks-ai"])); - - // Process NVIDIA NIM models - if (data.nvidia?.models) { - for (const [modelId, model] of Object.entries(data.nvidia.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (!m.modalities?.input?.includes("text")) continue; - if (!m.modalities?.output?.includes("text")) continue; - - const liveModelId = nvidiaNimModelIds.get(modelId) ?? nvidiaNimModelIds.get(normalizeNvidiaModelId(modelId)); - if (!liveModelId) continue; - if (NVIDIA_NIM_UNSUPPORTED_MODELS.has(liveModelId)) continue; - - models.push({ - id: liveModelId, - name: m.name || liveModelId, - api: "openai-completions", - provider: "nvidia", - baseUrl: NVIDIA_BASE_URL, - headers: { ...NVIDIA_HEADERS }, - reasoning: m.reasoning === true, - input: m.modalities.input.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - compat: NVIDIA_OPENAI_COMPAT, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("nvidia", liveModelId, m); - } - } - - // Process Together AI models - const togetherProvider = data.together ?? data.togetherai ?? data["together-ai"]; - if (togetherProvider?.models) { - for (const [modelId, model] of Object.entries(togetherProvider.models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (m.status === "deprecated") continue; - - const reasoning = m.reasoning === true; - const thinkingLevelMap = getTogetherThinkingLevelMap(modelId, reasoning); - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider: "together", - baseUrl: TOGETHER_BASE_URL, - reasoning, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - compat: getTogetherCompat(modelId, reasoning), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions("together", modelId, m); - } - } - - models.push(...processBasetenModels(data.baseten)); - - // Process OpenCode models (Zen and Go) - // API mapping based on provider.npm field: - // - @ai-sdk/openai → openai-responses - // - @ai-sdk/anthropic → anthropic-messages - // - @ai-sdk/google → google-generative-ai - // - null/undefined/@ai-sdk/openai-compatible → openai-completions - const opencodeVariants = [ - { key: "opencode", provider: "opencode", basePath: "https://opencode.ai/inference" }, - { key: "opencode-go", provider: "opencode-go", basePath: "https://opencode.ai/inference/go" }, - ] as const; - - for (const variant of opencodeVariants) { - if (!data[variant.key]?.models) continue; - - for (const [modelId, model] of Object.entries(data[variant.key].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (m.status === "deprecated") continue; - - const npm = m.provider?.npm; - let api: Api; - let baseUrl: string; - let compat: OpenAICompletionsCompat | OpenAIResponsesCompat | undefined; - - if (npm === "@ai-sdk/openai") { - api = "openai-responses"; - baseUrl = `${variant.basePath}/openai/v1`; - compat = { sessionAffinityFormat: "openai-nosession" }; - } else if (npm === "@ai-sdk/anthropic") { - api = "anthropic-messages"; - // Anthropic SDK appends /v1/messages to baseURL - baseUrl = `${variant.basePath}/anthropic`; - } else if (npm === "@ai-sdk/google") { - api = "google-generative-ai"; - baseUrl = `${variant.basePath}/google/v1beta`; - } else if (npm === "@ai-sdk/alibaba") { - api = "openai-completions"; - baseUrl = `${variant.basePath}/openai/v1`; - compat = { cacheControlFormat: "anthropic" }; - } else { - // null, undefined, or @ai-sdk/openai-compatible - api = "openai-completions"; - baseUrl = `${variant.basePath}/openai/v1`; - } - - if (variant.provider === "opencode" && modelId === "grok-build-0.1") { - compat = { ...(compat ?? {}), supportsReasoningEffort: false }; - } - - if ((variant.provider === "opencode" || variant.provider === "opencode-go") && modelId === "kimi-k2.6") { - // OpenCode Kimi K2.6 accepts Anthropic-style thinking objects - // and rejects string thinking values or combined reasoning_effort. - compat = { ...(compat ?? {}), thinkingFormat: "deepseek", supportsReasoningEffort: false }; - } - - // Fix known mismatches between models.dev npm data and actual - // OpenCode Go endpoint behaviour. models.dev reports these models - // as @ai-sdk/anthropic, but OpenCode Go serves them through the - // OpenAI-compatible /openai/v1/chat/completions path. - if (variant.provider === "opencode-go") { - if (modelId === "qwen3.5-plus" || modelId === "qwen3.6-plus") { - api = "openai-completions"; - baseUrl = `${variant.basePath}/openai/v1`; - // Qwen/DashScope uses enable_thinking at the top level. - compat = { ...(compat ?? {}), thinkingFormat: "qwen" }; - } - } - - if (api === "openai-completions") { - compat = { ...(compat ?? {}), maxTokensField: "max_tokens" }; - if ( - OPENCODE_OPENAI_COMPLETIONS_LONG_CACHE_RETENTION_UNSUPPORTED_MODELS.has( - `${variant.provider}:${modelId}`, - ) - ) { - compat = { ...compat, supportsLongCacheRetention: false }; - } - } - - let thinkingLevelMap: NonNullable["thinkingLevelMap"]> | undefined; - if (api === "google-generative-ai") { - thinkingLevelMap = getGoogleThinkingLevelMap(modelId, m.reasoning_options ?? []); - } else if (variant.provider === "opencode-go" && modelId === "deepseek-v4.1-flash") { - thinkingLevelMap = getEffortThinkingLevelMap(m.reasoning_options ?? []); - } - models.push({ - id: modelId, - name: m.name || modelId, - api, - provider: variant.provider, - baseUrl, - reasoning: m.reasoning === true, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - ...(compat ? { compat } : {}), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - recordModelsDevReasoningOptions(variant.provider, modelId, m); - } - } - - // Process GitHub Copilot models - if (data["github-copilot"]?.models) { - for (const [modelId, model] of Object.entries(data["github-copilot"].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (m.status === "deprecated") continue; - - // Claude 4.x and 5.x models route to Anthropic Messages API - const isCopilotClaude = /^claude-(haiku|sonnet|opus|fable)-[45]([.\-]|$)/.test(modelId); - // GPT, Grok, OSWE, and MAI-Code models are only served through - // the Copilot /responses endpoint. - const needsResponsesApi = - modelId.startsWith("gpt-") || - modelId.startsWith("grok-") || - modelId.startsWith("oswe") || - modelId.startsWith("mai-"); - - const api: Api = isCopilotClaude - ? "anthropic-messages" - : needsResponsesApi - ? "openai-responses" - : "openai-completions"; - - const anthropicCompat = - api === "anthropic-messages" ? getAnthropicMessagesCompat("github-copilot", modelId) : undefined; - - const copilotModel: Model = { - id: modelId, - name: m.name || modelId, - api, - provider: "github-copilot", - baseUrl: "https://api.individual.githubcopilot.com", - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 128000, - maxTokens: m.limit?.output || 8192, - headers: { ...COPILOT_STATIC_HEADERS }, - ...(anthropicCompat ? { compat: anthropicCompat } : {}), - // compat only applies to openai-completions - ...(api === "openai-completions" ? { - compat: { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - }, - } : {}), - }; - - models.push(copilotModel); - recordModelsDevReasoningOptions("github-copilot", modelId, m); - } - } - - // Process MiniMax models - const minimaxVariants = [ - { key: "minimax", provider: "minimax", baseUrl: "https://api.minimax.io/anthropic" }, - { key: "minimax-cn", provider: "minimax-cn", baseUrl: "https://api.minimaxi.com/anthropic" }, - ] as const; - // MiniMax asked to hide older models from its direct providers (#2596). - const minimaxHiddenIds = new Set(["MiniMax-M2", "MiniMax-M2.1", "MiniMax-M2.5", "MiniMax-M2.5-highspeed"]); - - for (const { key, provider, baseUrl } of minimaxVariants) { - if (data[key]?.models) { - for (const [modelId, model] of Object.entries(data[key].models)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true || minimaxHiddenIds.has(modelId)) continue; - // Models with effort levels (e.g. MiniMax-M3.1-Flash) take adaptive thinking with - // output_config.effort and reject thinking.type="disabled" (no "none" effort, so off is null). - const thinkingLevelMap = getEffortThinkingLevelMap(m.reasoning_options ?? []); - - models.push({ - id: modelId, - name: m.name || modelId, - api: "anthropic-messages", - provider, - // MiniMax's Anthropic-compatible API - SDK appends /v1/messages - baseUrl, - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - ...(thinkingLevelMap ? { thinkingLevelMap, compat: { forceAdaptiveThinking: true } } : {}), - }); - } - } - } - - // Process Kimi For Coding models - if (data["kimi-code-plan-global"]?.models) { - models.push(...createKimiCodingModels(data["kimi-code-plan-global"].models as Record)); - } - - // Process Moonshot AI models - const moonshotVariants = [ - { key: "moonshotai", provider: "moonshotai", baseUrl: "https://api.moonshot.ai/v1" }, - { key: "moonshotai-cn", provider: "moonshotai-cn", baseUrl: "https://api.moonshot.cn/v1" }, - ] as const; - const getMoonshotProviderModels = (key: "moonshotai" | "moonshotai-cn"): Record => { - const providerModels = data[key]?.models as Record | undefined; - return providerModels ? { ...providerModels } : {}; - }; - const moonshotModels = { - moonshotai: getMoonshotProviderModels("moonshotai"), - "moonshotai-cn": getMoonshotProviderModels("moonshotai-cn"), - }; - - for (const { key, provider, baseUrl } of moonshotVariants) { - for (const [modelId, m] of Object.entries(moonshotModels[key])) { - if (m.tool_call !== true) continue; - - const isKimiK3 = isKimiK3FamilyId(modelId); - const compat = { ...(isKimiK3 ? KIMI_K3_OPENAI_COMPAT : KIMI_OPENAI_COMPAT) }; - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider, - baseUrl, - reasoning: isKimiK3 || m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - // Moonshot does not bill cache writes; models.dev lists the input rate as cache_write for Kimi K3. - cost: - modelId === "kimi-k3" - ? { ...KIMI_K3_COST } - : { - input: m.cost?.input || 0, - output: m.cost?.output || 0, - cacheRead: m.cost?.cache_read || 0, - cacheWrite: isKimiK3 ? 0 : m.cost?.cache_write || 0, - }, - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - compat, - }); - recordModelsDevReasoningOptions(provider, modelId, m); - } - } - - // Process Xiaomi MiMo models - // Built-in `xiaomi` targets the API billing endpoint (single stable URL, - // keys from platform.xiaomimimo.com). The three `xiaomi-token-plan-*` - // providers cover prepaid Token Plan endpoints in cn / ams / sgp. - // MiMo Chat Completions only documents thinking.type enabled/disabled and - // rejects some reasoning_effort values (e.g. "minimal"). Xiaomi documents - // all effort levels as identical, so expose only off and high. - const xiaomiCompat: OpenAICompletionsCompat = { - supportsReasoningEffort: false, - requiresReasoningContentOnAssistantMessages: true, - thinkingFormat: "deepseek", - }; - const xiaomiThinkingLevelMap = { - off: "off", - minimal: null, - low: null, - medium: null, - high: "high", - xhigh: null, - max: null, - } as const; - const xiaomiVariants = [ - { source: "xiaomi", provider: "xiaomi", baseUrl: "https://api.xiaomimimo.com/v1" }, - { - source: "xiaomi-token-plan-cn", - provider: "xiaomi-token-plan-cn", - baseUrl: "https://token-plan-cn.xiaomimimo.com/v1", - }, - { - source: "xiaomi-token-plan-ams", - provider: "xiaomi-token-plan-ams", - baseUrl: "https://token-plan-ams.xiaomimimo.com/v1", - }, - { - source: "xiaomi-token-plan-sgp", - provider: "xiaomi-token-plan-sgp", - baseUrl: "https://token-plan-sgp.xiaomimimo.com/v1", - }, - ] as const; - - for (const { source, provider, baseUrl } of xiaomiVariants) { - const providerModels = data[source]?.models; - if (!providerModels) continue; - - for (const [modelId, model] of Object.entries(providerModels)) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (m.status === "deprecated") continue; - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider, - baseUrl, - compat: xiaomiCompat, - reasoning: m.reasoning === true, - ...(m.reasoning === true ? { thinkingLevelMap: xiaomiThinkingLevelMap } : {}), - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - } - } - - // Process Alibaba Cloud Model Studio Token Plan models. International and - // China use separate endpoints and API keys (sk-sp- prefix). The Individual - // provider reuses the international source and endpoint with a narrower catalog. - // models.dev keys are "alibaba-token-plan[-cn]"; pi exposes them as - // "qwen-token-plan[-cn]" plus the Individual catalog view. - const qwenTokenPlanCompat: OpenAICompletionsCompat = { - thinkingFormat: "qwen", - supportsDeveloperRole: false, - supportsStore: false, - supportsReasoningEffort: true, - }; - const qwenTokenPlanVariants = [ - { - source: "alibaba-token-plan", - provider: "qwen-token-plan", - baseUrl: "https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1", - modelIds: undefined, - }, - { - source: "alibaba-token-plan", - provider: "qwen-token-plan-individual", - baseUrl: "https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1", - modelIds: QWEN_TOKEN_PLAN_INDIVIDUAL_MODEL_IDS, - }, - { - source: "alibaba-token-plan-cn", - provider: "qwen-token-plan-cn", - baseUrl: "https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1", - modelIds: undefined, - }, - ] as const; - - for (const { source, provider, baseUrl, modelIds } of qwenTokenPlanVariants) { - const providerModels = data[source]?.models; - const emittedModelIds = modelIds ? new Set() : undefined; - - for (const [modelId, model] of Object.entries(providerModels ?? {})) { - const m = model as ModelsDevModel; - if (m.tool_call !== true) continue; - if (QWEN_TOKEN_PLAN_EXCLUDED_MODEL_IDS.has(modelId)) continue; - if (modelIds && !modelIds.has(modelId)) continue; - const thinkingLevelMap = - getEffortThinkingLevelMap(m.reasoning_options ?? []) ?? - (QWEN_TOKEN_PLAN_REASONING_EFFORT_FALLBACK_MODEL_IDS.has(modelId) - ? QWEN_TOKEN_PLAN_FALLBACK_THINKING_LEVEL_MAP - : undefined); - - models.push({ - id: modelId, - name: m.name || modelId, - api: "openai-completions", - provider, - baseUrl, - compat: thinkingLevelMap - ? qwenTokenPlanCompat - : { ...qwenTokenPlanCompat, supportsReasoningEffort: false }, - ...(thinkingLevelMap ? { thinkingLevelMap } : {}), - reasoning: m.reasoning === true, - input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"], - cost: getModelsDevCost(m.cost), - contextWindow: m.limit?.context || 4096, - maxTokens: m.limit?.output || 4096, - }); - emittedModelIds?.add(modelId); - } - - if (modelIds && emittedModelIds && generatorOptions.strict) { - assertExactModelIds(provider, modelIds, emittedModelIds); - } - } - - console.log(`Loaded ${models.length} tool-capable models from models.dev`); - return models; - } catch (error) { - console.error("Failed to load models.dev data:", error); - if (generatorOptions.strict) throw error; - return []; - } -} - -async function loadModelsDevClassifierModels(): Promise[]> { - try { - console.log("Fetching classifier models from models.dev API..."); - const response = await fetch("https://models.dev/models.json?type=decision"); - if (!response.ok) throw new Error(`models.dev classifier API returned ${response.status}`); - const data = (await response.json()) as Record; - const metadata = data["typesafe/jev-latest"]; - if (!metadata || metadata.type !== "decision") { - throw new Error("models.dev did not return decision model typesafe/jev-latest"); - } - return [ - { - type: "classifier", - id: "jev-latest", - name: metadata.name, - api: "typesafe-system-one", - provider: "typesafe", - baseUrl: "https://api.typesafe.ai/v1/", - // models.dev lists text input only, but the API accepts image assets as an experimental - // feature behind an opt-in header, which the typesafe-system-one API sends. - // https://github.com/typesafe-ai/typesafe-sdk-js (v0.7.0 changelog) - input: ["text", "image"], - // The canonical models.dev entry has no direct-provider pricing. System One reports token usage, - // so classify() results carry token counts but price them at zero. - cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: metadata.limit?.context || 64000, - compat: { imageInput: "assets", sendImageOptInHeader: true }, - }, - ]; - } catch (error) { - console.error("Failed to load models.dev classifier data:", error); - if (generatorOptions.strict) throw error; - return []; - } -} - -// OpenCode Zen serves Jev through its TypeSafe-compatible System One endpoint. -// Neither its /inference/v1/models listing nor models.dev carries metadata for it. -// https://opencode.ai/docs/zen -const OPENCODE_CLASSIFIER_MODELS: ClassifierModel<"typesafe-system-one">[] = [ - { - type: "classifier", - id: "jev-1.13", - name: "Jev 1.13", - api: "typesafe-system-one", - provider: "opencode", - baseUrl: "https://opencode.ai/inference/systemone/v1", - input: ["text"], - cost: { input: 0.042, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 32000, - }, - { - type: "classifier", - id: "jev-1.13-free", - name: "Jev 1.13 Free", - api: "typesafe-system-one", - provider: "opencode", - baseUrl: "https://opencode.ai/inference/systemone/v1", - input: ["text"], - cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 32000, - }, -]; - -// Workers AI has no unauthenticated catalog, so its System One models are listed here from -// Cloudflare's model pages. -const CLOUDFLARE_WORKERS_AI_CLASSIFIER_MODELS: ClassifierModel<"cloudflare-workers-ai-system-one">[] = [ - // Cloudflare-hosted Clef decision models. They take up to four PNG, JPEG, or WebP images as data URLs. - // Clef Omni also takes audio and video, which classifier contexts do not carry. - // Pricing and context: https://developers.cloudflare.com/workers-ai/models/clef/, - // https://developers.cloudflare.com/workers-ai/models/clef-flash/, - // and https://developers.cloudflare.com/workers-ai/models/clef-omni/ - { - type: "classifier", - id: "@cf/cloudflare/clef", - name: "Clef", - api: "cloudflare-workers-ai-system-one", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, - input: ["text", "image"], - cost: { input: 0.24, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 65536, - compat: { imageInput: "images" }, - }, - { - type: "classifier", - id: "@cf/cloudflare/clef-flash", - name: "Clef Flash", - api: "cloudflare-workers-ai-system-one", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, - input: ["text", "image"], - cost: { input: 0.038, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 24576, - compat: { imageInput: "images" }, - }, - { - type: "classifier", - id: "@cf/cloudflare/clef-omni", - name: "Clef Omni", - api: "cloudflare-workers-ai-system-one", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, - input: ["text", "image"], - cost: { input: 0.15, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 64000, - compat: { imageInput: "images" }, - }, - // TypeSafe's Jev models take PNG and JPEG images as System One assets. - // https://developers.cloudflare.com/ai/models/typesafe/jev/ (jev-latest) - // and https://developers.cloudflare.com/ai/models/typesafe/jev-1.14.0/ - { - type: "classifier", - id: "typesafe/jev", - name: "Jev", - api: "cloudflare-workers-ai-system-one", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, - input: ["text", "image"], - cost: { input: 0.042, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 32000, - compat: { imageInput: "assets" }, - }, - { - type: "classifier", - id: "typesafe/jev-1.14.0", - name: "Jev 1.14", - api: "cloudflare-workers-ai-system-one", - provider: "cloudflare-workers-ai", - baseUrl: CLOUDFLARE_WORKERS_AI_REST_BASE_URL, - input: ["text", "image"], - cost: { input: 0.042, output: 0, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 32000, - compat: { imageInput: "assets" }, - }, -]; - -// OpenAI Decisions API (public beta): gpt-6-luna is its only model. It bills input tokens only, -// with the same long-context multiplier as chat requests. Only API keys work: Sign in with ChatGPT -// tokens are rejected on /v1/decisions, and the Codex backend has no Decisions route. -// The endpoint rejects inputs above 922K tokens (the model's documented maximum input), but -// requests running longer than about five seconds, currently above roughly 600K input tokens, -// fail with a gateway timeout. -// https://developers.openai.com/api/docs/guides/decisions -const OPENAI_CLASSIFIER_MODELS: ClassifierModel<"openai-decisions">[] = [ - { - type: "classifier", - id: "gpt-6-luna", - name: "GPT-6 Luna", - api: "openai-decisions", - provider: "openai", - baseUrl: "https://api.openai.com/v1", - input: ["text", "image"], - cost: withOpenAiLongContextPricing({ input: 0.1, output: 0, cacheRead: 0, cacheWrite: 0 }), - contextWindow: 922000, - }, -]; - -async function generateModels() { - // Fetch models from all upstream catalogs. - // models.dev: Anthropic, Google, OpenAI, Groq, Cerebras, and others - // OpenRouter: its tool-capable routed catalog - // AI Gateway: OpenAI-compatible catalog with tool-capable models - // Radius: its unauthenticated public catalog; authenticated clients overlay it at runtime - const modelsDevModels = await loadModelsDevData(); - const modelsDevClassifierModels = await loadModelsDevClassifierModels(); - const openRouterCatalog = await fetchOpenRouterModels(); - const aiGatewayCatalog = await fetchAiGatewayModels(); - const radiusModels = await fetchRadiusModels(); - - // Combine chat models (models.dev has priority where sources overlap). - const allModels = [...modelsDevModels, ...openRouterCatalog.chat, ...aiGatewayCatalog.chat, ...radiusModels].filter( - (model) => - !(model.provider === "xai" && XAI_BUILTIN_EXCLUDED_MODEL_IDS.has(model.id)) && - !((model.provider === "opencode" || model.provider === "opencode-go") && model.id === "gpt-5.3-codex-spark"), - ); - - // The authenticated Copilot catalog advertised these models on 2026-09-22, - // but models.dev did not include them yet. - const missingCopilotModels: Model[] = [ - { - id: "claude-opus-5.5", - name: "Claude Opus 5.5", - api: "anthropic-messages", - provider: "github-copilot", - baseUrl: "https://api.individual.githubcopilot.com", - reasoning: true, - thinkingLevelMap: { - off: null, - minimal: null, - low: "low", - medium: "medium", - high: "high", - xhigh: "xhigh", - max: "max", - }, - input: ["text", "image"], - cost: { input: 4, output: 20, cacheRead: 0.2, cacheWrite: 5 }, - contextWindow: 1000000, - maxTokens: 128000, - headers: { ...COPILOT_STATIC_HEADERS }, - }, - ...(["gpt-6-sol", "gpt-6-luna"] as const).map((id) => ({ - id, - name: id === "gpt-6-sol" ? "GPT-6 Sol" : "GPT-6 Luna", - api: "openai-responses" as const, - provider: "github-copilot" as const, - baseUrl: "https://api.individual.githubcopilot.com", - reasoning: true, - input: ["text", "image"] as ("text" | "image")[], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS[id]), - contextWindow: 1000000, - maxTokens: 128000, - headers: { ...COPILOT_STATIC_HEADERS }, - })), - ]; - for (const model of missingCopilotModels) { - if (!allModels.some((candidate) => candidate.provider === model.provider && candidate.id === model.id)) { - allModels.push(model); - } - } - - // Temporary overrides until upstream model metadata is corrected. - for (const candidate of allModels) { - if (candidate.provider === "github-copilot" && GITHUB_COPILOT_EXTENDED_CONTEXT_MODELS.has(candidate.id)) { - candidate.contextWindow = 1000000; - } - - // models.dev may list Opus 5.5, Sonnet 5.5, and Haiku 5.5 before their effort metadata is complete. - if ( - (candidate.provider === "anthropic" && - (candidate.id === "claude-opus-5-5" || - candidate.id === "claude-sonnet-5-5" || - candidate.id === "claude-haiku-5-5")) || - (candidate.provider === "github-copilot" && candidate.id === "claude-opus-5.5") - ) { - mergeThinkingLevelMap(candidate, { - // Haiku 5.5 accepts disabled thinking; Opus 5.5 and Sonnet 5.5 reject it. - ...(candidate.id === "claude-haiku-5-5" ? {} : { off: null }), - minimal: null, - low: "low", - medium: "medium", - high: "high", - xhigh: "xhigh", - max: "max", - }); - } - - if ( - (candidate.provider === "anthropic" || - candidate.provider === "opencode" || - candidate.provider === "opencode-go") && - (candidate.id === "claude-opus-4-6" || - candidate.id === "claude-sonnet-4-6" || - candidate.id === "claude-opus-4.6" || - candidate.id === "claude-sonnet-4.6") - ) { - candidate.contextWindow = 1000000; - } - - // OpenCode variants list Claude Sonnet 4/4.5 with 1M context, actual limit is 200K - if ( - (candidate.provider === "opencode" || candidate.provider === "opencode-go") && - (candidate.id === "claude-sonnet-4-5" || candidate.id === "claude-sonnet-4") - ) { - candidate.contextWindow = 200000; - } - if ((candidate.provider === "opencode" || candidate.provider === "opencode-go") && candidate.id === "gpt-5.4") { - candidate.contextWindow = 272000; - candidate.maxTokens = 128000; - } - // Keep direct OpenAI requests in the short-context pricing tier by default. Users can opt into the - // larger context through model overrides, so retain long-context cost metadata on the capped models. - if (candidate.provider === "openai" && OPENAI_SHORT_CONTEXT_CAPPED_MODEL_IDS.has(candidate.id)) { - candidate.contextWindow = OPENAI_LONG_CONTEXT_INPUT_THRESHOLD; - candidate.maxTokens = 128000; - } - if (candidate.provider === "openai" && OPENAI_LONG_CONTEXT_PRICING_MODEL_IDS.has(candidate.id)) { - const standardCost = OPENAI_STANDARD_COSTS[candidate.id]; - candidate.cost = withOpenAiLongContextPricing(standardCost ?? candidate.cost); - } - // Cloudflare AI Gateway passes OpenAI usage through at OpenAI list prices. - if (candidate.provider === "cloudflare-ai-gateway") { - const standardCost = OPENAI_STANDARD_COSTS[candidate.id]; - if (standardCost) candidate.cost = withOpenAiLongContextPricing(standardCost); - } - // models.dev reports gpt-5-pro output as 272000 (a duplicate of the input sub-limit); - // the actual max output is 128000. Also propagates to the derived Azure clone. - if (candidate.provider === "openai" && candidate.id === "gpt-5-pro") { - candidate.maxTokens = 128000; - } - // Keep Kimi K3's canonical output limit when gateway metadata is missing or incorrect. - if ( - (candidate.provider === "openrouter" && candidate.id === "~moonshotai/kimi-latest") || - ((candidate.provider === "openrouter" || candidate.provider === "vercel-ai-gateway") && - candidate.id.startsWith("moonshotai/") && - isKimiK3FamilyId(candidate.id.slice("moonshotai/".length))) - ) { - candidate.maxTokens = KIMI_K3_MAX_TOKENS; - } - // Keep selected OpenRouter model metadata stable until upstream settles. - if (candidate.provider === "openrouter" && candidate.id === "moonshotai/kimi-k2.5") { - candidate.cost.input = 0.41; - candidate.cost.output = 2.06; - candidate.cost.cacheRead = 0.07; - candidate.maxTokens = 4096; - } - if (candidate.provider === "openrouter" && candidate.id.startsWith("moonshotai/kimi-k2.6")) { - candidate.compat = { - ...candidate.compat, - supportsDeveloperRole: false, - requiresReasoningContentOnAssistantMessages: true, - }; - } - if (candidate.provider === "openrouter" && candidate.id === "z-ai/glm-5") { - candidate.cost.input = 0.6; - candidate.cost.output = 1.9; - candidate.cost.cacheRead = 0.119; - } - } - - // Add missing gpt models - const missingOpenAiModels: Model<"openai-responses">[] = [ - { - id: "gpt-6.1-sol", - name: "GPT-6.1 Sol", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6.1-sol"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-6.1-terra", - name: "GPT-6.1 Terra", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6.1-terra"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-6-astra", - name: "GPT-6 Astra", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-astra"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-6-sol", - name: "GPT-6 Sol", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-sol"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-6-luna", - name: "GPT-6 Luna", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-luna"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-5.6-sol", - name: "GPT-5.6 Sol", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-sol"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-5.6-terra", - name: "GPT-5.6 Terra", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-terra"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-5.6-luna", - name: "GPT-5.6 Luna", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-luna"]), - contextWindow: OPENAI_LONG_CONTEXT_INPUT_THRESHOLD, - maxTokens: 128000, - }, - { - id: "gpt-5-chat-latest", - name: "GPT-5 Chat Latest", - api: "openai-responses", - baseUrl: "https://api.openai.com/v1", - provider: "openai", - reasoning: false, - input: ["text", "image"], - cost: { - input: 1.25, - output: 10, - cacheRead: 0.125, - cacheWrite: 0, - }, - contextWindow: 128000, - maxTokens: 16384, - }, - ]; - for (const model of missingOpenAiModels) { - if (!allModels.some((m) => m.provider === model.provider && m.id === model.id)) { - allModels.push(model); - } - } - - const deepseekCompat: OpenAICompletionsCompat = { - requiresReasoningContentOnAssistantMessages: true, - thinkingFormat: "deepseek", - }; - const deepseekModels: Model<"openai-completions">[] = [ - { - id: "deepseek-flash", - name: "DeepSeek V4.1 Flash", - api: "openai-completions", - baseUrl: "https://api.deepseek.com", - provider: "deepseek", - reasoning: true, - thinkingLevelMap: DEEPSEEK_V4_FLASH_THINKING_LEVEL_MAP, - input: ["text", "image"], - cost: { - // DeepSeek also offers time-based off-peak rates, which the cost schema cannot represent yet. - input: 0.3, - output: 1.2, - cacheRead: 0.006, - cacheWrite: 0, - }, - contextWindow: 1000000, - maxTokens: 384000, - compat: deepseekCompat, - }, - { - id: "deepseek-v4-pro", - name: "DeepSeek V4 Pro", - api: "openai-completions", - baseUrl: "https://api.deepseek.com", - provider: "deepseek", - reasoning: true, - input: ["text"], - cost: { - // DeepSeek also offers time-based off-peak rates, which the cost schema cannot represent yet. - input: 1.32, - output: 3.96, - cacheRead: 0.044, - cacheWrite: 0, - }, - contextWindow: 1000000, - maxTokens: 384000, - compat: deepseekCompat, - }, - ]; - allModels.push(...deepseekModels); - - const antLingCompat: OpenAICompletionsCompat = { - supportsStore: false, - supportsDeveloperRole: false, - supportsReasoningEffort: false, - maxTokensField: "max_tokens", - supportsLongCacheRetention: false, - }; - const antLingModels: Model<"openai-completions">[] = [ - { - id: "Ling-2.6-flash", - name: "Ling 2.6 Flash", - api: "openai-completions", - baseUrl: "https://api.ant-ling.com/v1", - provider: "ant-ling", - reasoning: false, - input: ["text"], - cost: { input: 0.01, output: 0.02, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 262144, - maxTokens: 65536, - compat: antLingCompat, - }, - { - id: "Ling-2.6-1T", - name: "Ling 2.6 1T", - api: "openai-completions", - baseUrl: "https://api.ant-ling.com/v1", - provider: "ant-ling", - reasoning: false, - input: ["text"], - cost: { input: 0.06, output: 0.25, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 262144, - maxTokens: 65536, - compat: antLingCompat, - }, - { - id: "Ring-2.6-1T", - name: "Ring 2.6 1T", - api: "openai-completions", - baseUrl: "https://api.ant-ling.com/v1", - provider: "ant-ling", - reasoning: true, - input: ["text"], - cost: { input: 0.06, output: 0.25, cacheRead: 0, cacheWrite: 0 }, - contextWindow: 262144, - maxTokens: 65536, - compat: { ...antLingCompat, thinkingFormat: "ant-ling" }, - }, - ]; - allModels.push(...antLingModels); - - for (const candidate of allModels) { - if ( - candidate.api === "openai-completions" && - candidate.id.includes("deepseek-v4") && - !QWEN_TOKEN_PLAN_PROVIDER_IDS.has(candidate.provider) - ) { - const preservesNativeReasoningEffort = candidate.provider === "openrouter" || candidate.provider === "opencode"; - candidate.compat = { - ...candidate.compat, - ...(preservesNativeReasoningEffort - ? { - requiresReasoningContentOnAssistantMessages: - deepseekCompat.requiresReasoningContentOnAssistantMessages, - } - : deepseekCompat), - }; - } - } - - // OpenAI Codex (ChatGPT OAuth) models - // NOTE: These are not fetched from models.dev; we keep a small, explicit list to avoid aliases. - // Only list models the Codex backend accepts for ChatGPT accounts. GPT-5.5 and GPT-5.3 Codex Spark are rejected - // with "not supported when using Codex with a ChatGPT account". - // GPT-5.6 and GPT-6 use Codex's 272k default catalog limit. - // GPT-6.1 Sol uses the 500k default returned by /backend-api/codex/models. - const CODEX_BASE_URL = "https://chatgpt.com/backend-api"; - const CODEX_CONTEXT = 272000; - const CODEX_GPT_61_CONTEXT = 500000; - const CODEX_GPT_56_CONTEXT = 272000; - const CODEX_MAX_TOKENS = 128000; - const codexModels: Model<"openai-codex-responses">[] = [ - { - id: "gpt-6.1-sol", - name: "GPT-6.1 Sol", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6.1-sol"]), - contextWindow: CODEX_GPT_61_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-6.1-terra", - name: "GPT-6.1 Terra", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6.1-terra"]), - contextWindow: CODEX_GPT_61_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-6-astra", - name: "GPT-6 Astra", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-astra"]), - contextWindow: CODEX_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-6-sol", - name: "GPT-6 Sol", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-sol"]), - contextWindow: CODEX_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-6-luna", - name: "GPT-6 Luna", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-6-luna"]), - contextWindow: CODEX_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-5.6-luna", - name: "GPT-5.6 Luna", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-luna"]), - contextWindow: CODEX_GPT_56_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-5.6-sol", - name: "GPT-5.6 Sol", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-sol"]), - contextWindow: CODEX_GPT_56_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - { - id: "gpt-5.6-terra", - name: "GPT-5.6 Terra", - api: "openai-codex-responses", - provider: "openai-codex", - baseUrl: CODEX_BASE_URL, - reasoning: true, - input: ["text", "image"], - cost: withOpenAiLongContextPricing(OPENAI_STANDARD_COSTS["gpt-5.6-terra"]), - contextWindow: CODEX_GPT_56_CONTEXT, - maxTokens: CODEX_MAX_TOKENS, - }, - ]; - allModels.push(...codexModels); - - // Add missing Mistral Medium 3.5 model until models.dev includes it - if (!allModels.some(m => m.provider === "mistral" && m.id === "mistral-medium-3.5")) { - allModels.push({ - id: "mistral-medium-3.5", - name: "Mistral Medium 3.5", - api: "mistral-conversations", - provider: "mistral", - baseUrl: "https://api.mistral.ai", - reasoning: true, - thinkingLevelMap: getEffortThinkingLevelMap([{ type: "effort", values: ["none", "high"] }]), - input: ["text", "image"], - cost: { - input: 1.5, - output: 7.5, - cacheRead: 0, - cacheWrite: 0, - }, - contextWindow: 262144, // 256k tokens - maxTokens: 262144, - }); - } - - // Add "auto" alias for openrouter/auto - if (!allModels.some(m => m.provider === "openrouter" && m.id === "auto")) { - allModels.push({ - id: "auto", - name: "Auto", - api: "openai-completions", - provider: "openrouter", - baseUrl: "https://openrouter.ai/api/v1", - reasoning: true, - input: ["text", "image"], - cost: { - // we dont know about the costs because OpenRouter auto routes to different models - // and then charges you for the underlying used model - input:0, - output:0, - cacheRead:0, - cacheWrite:0, - }, - contextWindow: 2000000, - maxTokens: 30000, - }); - } - - // Add "fusion" alias for openrouter/fusion. OpenRouter exposes Fusion as a - // router alias/plugin entry point; its model metadata does not advertise - // tools, but the alias resolves to a concrete model that can invoke caller - // tools and has the openrouter:fusion server tool auto-injected. - if (!allModels.some(m => m.provider === "openrouter" && m.id === "openrouter/fusion")) { - allModels.push({ - id: "openrouter/fusion", - name: "OpenRouter: Fusion", - api: "openai-completions", - provider: "openrouter", - baseUrl: "https://openrouter.ai/api/v1", - reasoning: true, - input: ["text"], - cost: { - // we dont know about the costs because Fusion routes to multiple models - // and then charges you for the underlying used models - input: 0, - output: 0, - cacheRead: 0, - cacheWrite: 0, - }, - contextWindow: 1000000, - maxTokens: 30000, - }); - } - - // Azure Foundry deploys these with larger context windows than OpenAI's own short-tier defaults. - // See models-sold-directly-by-azure docs. - const AZURE_CONTEXT_WINDOW_OVERRIDES: Record = { - "gpt-5.4": 1050000, - "gpt-5.5": 1050000, - "gpt-5.6-luna": 1050000, - "gpt-5.6-sol": 1050000, - "gpt-5.6-terra": 1050000, - }; - const azureOpenAiModels: Model[] = allModels - .filter((model) => model.provider === "openai" && model.api === "openai-responses") - .map((model) => ({ - ...model, - api: "azure-openai-responses", - provider: "azure", - baseUrl: "", - cost: { - input: model.cost.input, - output: model.cost.output, - cacheRead: model.cost.cacheRead, - cacheWrite: model.cost.cacheWrite, - }, - contextWindow: AZURE_CONTEXT_WINDOW_OVERRIDES[model.id] ?? model.contextWindow, - })); - allModels.push(...azureOpenAiModels); - - // Azure resells DeepSeek at its own rates. US data zone, checked 2026-09-16. - // https://azure.microsoft.com/en-us/pricing/details/ai-foundry-models/deepseek/ - const AZURE_DEEPSEEK_V4_PRO_COST: ModelCost = { input: 1.925, output: 3.828, cacheRead: 0.165, cacheWrite: 0 }; - // Azure 400s on DeepSeek's `thinking` field and on every prompt cache parameter, discards a - // `developer` system message unbilled once reasoning_effort is set, and honours mid-convo ones (#9645). - const azureDeepSeekModels: Model[] = allModels - .filter((model) => model.provider === "deepseek" && model.id === "deepseek-v4-pro") - .map((model) => ({ - ...model, - provider: "azure", - baseUrl: "", - cost: AZURE_DEEPSEEK_V4_PRO_COST, - compat: { - ...(model.compat as OpenAICompletionsCompat), - supportsDeveloperRole: false, - supportsMidConvoSystemMessages: true, - thinkingFormat: "openai", - supportsLongCacheRetention: false, - }, - })); - allModels.push(...azureDeepSeekModels); - - for (const model of allModels) { - applyOpenAICompletionsCompatMetadata(model); - applyAnthropicMessagesCompatMetadata(model); - applyModelsDevReasoningOptionMetadata(model); - applyThinkingLevelMetadata(model); - applyStrictToolCompatMetadata(model); - applyOpenAIGrammarToolCompatMetadata(model); - applyOpenAIToolSearchMetadata(model); - applyOpenAICompletionsTranscriptMetadata(model); - applyOpenAIResponsesTranscriptMetadata(model); - applyOpenAIExplicitPromptCacheMetadata(model); - applyOpenAIMidConvoEffortMetadata(model); - applyPromptCacheMetadata(model); - applyImageInputMetadata(model); - } - applyAnthropicAllowedFallbackModelMetadata(allModels.filter(isAnthropicFallbackMetadataModel)); - - // Keep chat and image catalogs separate so one upstream ID can expose both - // operations with different API implementations. - type ProviderCatalog = { - chat: Record>; - image: Record>; - classifier: Record>; - }; - const providers: Record = {}; - for (const model of allModels) { - providers[model.provider] ??= { chat: {}, image: {}, classifier: {} }; - // Only add if not already present (models.dev takes priority over OpenRouter). - providers[model.provider].chat[model.id] ??= { ...model, type: "chat" }; - } - for (const model of openRouterCatalog.images) { - applyImageInputMetadata(model); - providers[model.provider] ??= { chat: {}, image: {}, classifier: {} }; - providers[model.provider].image[model.id] ??= model; - } - const classifierModels: ClassifierModel[] = [ - ...modelsDevClassifierModels, - ...openRouterCatalog.classifiers, - ...aiGatewayCatalog.classifiers, - ...OPENCODE_CLASSIFIER_MODELS, - ...CLOUDFLARE_WORKERS_AI_CLASSIFIER_MODELS, - ...OPENAI_CLASSIFIER_MODELS, - ]; - for (const model of classifierModels) { - providers[model.provider] ??= { chat: {}, image: {}, classifier: {} }; - providers[model.provider].classifier[model.id] ??= model; - } - - const sortedProviderIds = Object.keys(providers).sort(); - const jsonChatProviders: Record>> = {}; - const jsonImageProviders: Record>> = {}; - const jsonClassifierProviders: Record>> = {}; - const jsonAllProviders: Record = {}; - for (const providerId of sortedProviderIds) { - jsonChatProviders[providerId] = Object.fromEntries( - Object.entries(providers[providerId].chat).sort(([left], [right]) => left.localeCompare(right)), - ); - jsonImageProviders[providerId] = Object.fromEntries( - Object.entries(providers[providerId].image).sort(([left], [right]) => left.localeCompare(right)), - ); - jsonClassifierProviders[providerId] = Object.fromEntries( - Object.entries(providers[providerId].classifier).sort(([left], [right]) => left.localeCompare(right)), - ); - jsonAllProviders[providerId] = [ - ...Object.values(jsonChatProviders[providerId]), - ...Object.values(jsonImageProviders[providerId]), - ...Object.values(jsonClassifierProviders[providerId]), - ]; - } - - const serializeJson = (value: unknown) => `${JSON.stringify(value, null, generatorOptions.pretty ? 2 : undefined)}\n`; - const writeJson = (path: string, value: unknown) => writeFileSync(path, serializeJson(value)); - const generatedDataProviderIds = generatorOptions.dataOnly - ? readModelDataProviderIds(packageRoot) - : sortedProviderIds; - const missingProviderIds = generatedDataProviderIds.filter((providerId) => !jsonAllProviders[providerId]); - if (missingProviderIds.length > 0) { - throw new Error(`Cannot hydrate missing providers: ${missingProviderIds.join(", ")}`); - } - - // Only the ignored internal data is grouped by API for type derivation. - const generatedDataProviders: Record>> = {}; - const modelDataStructure: ModelDataStructure = {}; - for (const providerId of generatedDataProviderIds) { - const { groups, structure } = groupProviderModelData(providerId, jsonAllProviders[providerId]); - generatedDataProviders[providerId] = groups; - modelDataStructure[providerId] = structure; - } - - const generatedAt = new Date().toISOString(); - - if (!generatorOptions.jsonOnly) { - // Stage and validate all provider values before replacing the current generated data. - const providersDir = join(packageRoot, "src/providers"); - const dataDir = join(providersDir, "data"); - const stagingRoot = mkdtempSync(join(providersDir, ".model-generation-")); - const stagedDataDir = join(stagingRoot, "data"); - const previousDataDir = join(stagingRoot, "previous-data"); - let restoreGeneratedCatalog: (() => void) | undefined; - try { - mkdirSync(stagedDataDir, { recursive: true }); - const fileContents: Record = {}; - for (const providerId of generatedDataProviderIds) { - const filename = `${providerId}.json`; - const content = serializeJson(generatedDataProviders[providerId]); - fileContents[filename] = content; - writeFileSync(join(stagedDataDir, filename), content); - } - writeJson( - join(stagedDataDir, MODEL_DATA_MANIFEST_FILE), - createModelDataManifest(modelDataStructure, fileContents, generatedAt), - ); - validateModelDataDirectory(modelDataStructure, stagedDataDir); - - if (!generatorOptions.dataOnly) { - const previousShardContents = new Map( - readdirSync(providersDir) - .filter((entry) => entry.endsWith(".models.ts")) - .map((entry) => [entry, readFileSync(join(providersDir, entry), "utf8")] as const), - ); - const aggregatorPath = join(packageRoot, "src/models.generated.ts"); - const previousAggregator = readFileSync(aggregatorPath, "utf8"); - restoreGeneratedCatalog = () => { - for (const entry of readdirSync(providersDir)) { - if (entry.endsWith(".models.ts")) rmSync(join(providersDir, entry)); - } - for (const [entry, content] of previousShardContents) { - writeFileSync(join(providersDir, entry), content); - } - writeFileSync(aggregatorPath, previousAggregator); - }; - - const generatedHeader = `// This file is auto-generated by scripts/generate-models.ts -// Do not edit manually - run 'npm run generate-models' to update - -`; - const catalogConstName = (providerId: string) => - `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_MODELS`; - const imageCatalogConstName = (providerId: string) => - `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_IMAGE_MODELS`; - const classifierCatalogConstName = (providerId: string) => - `${providerId.toUpperCase().replace(/[^A-Z0-9]+/g, "_")}_CLASSIFIER_MODELS`; - const generatedShardFiles = new Set(); - for (const providerId of sortedProviderIds) { - let output = generatedHeader; - output += `import values from "./data/${providerId}.json" with { type: "json" };\n`; - output += `import { flattenChatModelCatalog, flattenClassifierModelCatalog, flattenImageModelCatalog, type ChatModelCatalog, type ClassifierModelCatalog, type ImageModelCatalog } from "../model-catalog.ts";\n\n`; - output += `export const ${catalogConstName(providerId)}: ChatModelCatalog =\n`; - output += `\tflattenChatModelCatalog(${JSON.stringify(providerId)}, values);\n\n`; - output += `export const ${imageCatalogConstName(providerId)}: ImageModelCatalog =\n`; - output += `\tflattenImageModelCatalog(${JSON.stringify(providerId)}, values);\n\n`; - output += `export const ${classifierCatalogConstName(providerId)}: ClassifierModelCatalog =\n`; - output += `\tflattenClassifierModelCatalog(${JSON.stringify(providerId)}, values);\n`; - const filename = `${providerId}.models.ts`; - generatedShardFiles.add(filename); - writeFileSync(join(providersDir, filename), output); - } - for (const entry of readdirSync(providersDir)) { - if (entry.endsWith(".models.ts") && !generatedShardFiles.has(entry)) rmSync(join(providersDir, entry)); - } - - let output = generatedHeader; - for (const providerId of sortedProviderIds) { - output += `import { ${classifierCatalogConstName(providerId)}, ${imageCatalogConstName(providerId)}, ${catalogConstName(providerId)} } from "./providers/${providerId}.models.ts";\n`; - } - output += `\nexport const MODELS: {\n`; - for (const providerId of sortedProviderIds) { - output += `\treadonly ${JSON.stringify(providerId)}: typeof ${catalogConstName(providerId)};\n`; - } - output += `} = {\n`; - for (const providerId of sortedProviderIds) { - output += `\t${JSON.stringify(providerId)}: ${catalogConstName(providerId)},\n`; - } - output += `};\n\nexport const IMAGE_MODELS: {\n`; - for (const providerId of sortedProviderIds) { - output += `\treadonly ${JSON.stringify(providerId)}: typeof ${imageCatalogConstName(providerId)};\n`; - } - output += `} = {\n`; - for (const providerId of sortedProviderIds) { - output += `\t${JSON.stringify(providerId)}: ${imageCatalogConstName(providerId)},\n`; - } - output += `};\n\nexport const CLASSIFIER_MODELS: {\n`; - for (const providerId of sortedProviderIds) { - output += `\treadonly ${JSON.stringify(providerId)}: typeof ${classifierCatalogConstName(providerId)};\n`; - } - output += `} = {\n`; - for (const providerId of sortedProviderIds) { - output += `\t${JSON.stringify(providerId)}: ${classifierCatalogConstName(providerId)},\n`; - } - output += `};\n`; - writeFileSync(aggregatorPath, output); - console.log("Generated provider catalogs and src/models.generated.ts"); - } - - const hadPreviousData = existsSync(dataDir); - if (hadPreviousData) renameSync(dataDir, previousDataDir); - try { - renameSync(stagedDataDir, dataDir); - validateGeneratedModelData(packageRoot); - } catch (error) { - rmSync(dataDir, { recursive: true, force: true }); - if (hadPreviousData && existsSync(previousDataDir)) renameSync(previousDataDir, dataDir); - throw error; - } - restoreGeneratedCatalog = undefined; - console.log( - generatorOptions.dataOnly - ? "Hydrated JSON model values under src/providers/data/" - : "Generated JSON model values under src/providers/data/", - ); - } catch (error) { - restoreGeneratedCatalog?.(); - throw error; - } finally { - rmSync(stagingRoot, { recursive: true, force: true }); - } - } - - if (generatorOptions.jsonOutputDir) { - // `models.json` and `providers/{id}.json` retain the legacy keyed chat catalog. - // The `.all` variants are arrays so the same upstream id can appear once per type. - const providerOutputDir = join(generatorOptions.jsonOutputDir, "providers"); - rmSync(generatorOptions.jsonOutputDir, { recursive: true, force: true }); - mkdirSync(providerOutputDir, { recursive: true }); - writeJson(join(generatorOptions.jsonOutputDir, "models.json"), jsonChatProviders); - writeJson(join(generatorOptions.jsonOutputDir, "models.all.json"), jsonAllProviders); - writeJson(join(generatorOptions.jsonOutputDir, "providers.json"), sortedProviderIds); - for (const providerId of sortedProviderIds) { - writeJson(join(providerOutputDir, `${providerId}.json`), jsonChatProviders[providerId]); - writeJson(join(providerOutputDir, `${providerId}.all.json`), jsonAllProviders[providerId]); - } - console.log(`Generated JSON model catalog under ${generatorOptions.jsonOutputDir}`); - } - - // Print statistics - const totalModels = allModels.length; - const reasoningModels = allModels.filter((model) => model.reasoning).length; - - console.log(`\nModel Statistics:`); - console.log(` Total tool-capable models: ${totalModels}`); - console.log(` Reasoning-capable models: ${reasoningModels}`); - - for (const [provider, models] of Object.entries(providers)) { - console.log( - ` ${provider}: ${Object.keys(models.chat).length} chat models, ${Object.keys(models.image).length} image models, ${Object.keys(models.classifier).length} classifier models`, - ); - } -} - -// Run the generator -generateModels().catch((error) => { - console.error(error); - process.exitCode = 1; -}); diff --git a/packages/ai/scripts/openrouter-reasoning-options.ts b/packages/ai/scripts/openrouter-reasoning-options.ts deleted file mode 100644 index 6fc4a5c6c..000000000 --- a/packages/ai/scripts/openrouter-reasoning-options.ts +++ /dev/null @@ -1,23 +0,0 @@ -import type { ThinkingLevel, ThinkingLevelMap } from "../src/types.ts"; -import { getEffortThinkingLevelMap } from "./models-dev-reasoning-options.ts"; - -export interface OpenRouterReasoningMetadata { - mandatory?: boolean; - default_enabled?: boolean; - supported_efforts?: Array; - default_effort?: ThinkingLevel | "none"; -} - -/** Convert OpenRouter's reasoning metadata into Pi model capabilities. */ -export function getOpenRouterThinkingLevelMap( - reasoning: OpenRouterReasoningMetadata | undefined, -): ThinkingLevelMap | undefined { - if (!reasoning) return undefined; - if (!reasoning.supported_efforts?.length) return reasoning.mandatory === true ? { off: null } : undefined; - - // OpenRouter's supported_efforts uses the same effort values as models.dev reasoning_options, - // so both sources can share the same Pi thinking-level conversion. - const map = getEffortThinkingLevelMap([{ type: "effort", values: reasoning.supported_efforts }]); - if (!map) return reasoning.mandatory === true ? { off: null } : undefined; - return { ...map, off: reasoning.mandatory === true ? null : "none" }; -} diff --git a/packages/ai/src/models.generated.ts b/packages/ai/src/models.generated.ts index 0890c4cee..93f2889fd 100644 --- a/packages/ai/src/models.generated.ts +++ b/packages/ai/src/models.generated.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import { AMAZON_BEDROCK_CLASSIFIER_MODELS, AMAZON_BEDROCK_IMAGE_MODELS, AMAZON_BEDROCK_MODELS } from "./providers/amazon-bedrock.models.ts"; diff --git a/packages/ai/src/providers/amazon-bedrock.models.ts b/packages/ai/src/providers/amazon-bedrock.models.ts index 37b348ef7..0d4030b6a 100644 --- a/packages/ai/src/providers/amazon-bedrock.models.ts +++ b/packages/ai/src/providers/amazon-bedrock.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/amazon-bedrock.json" with { type: "json" }; diff --git a/packages/ai/src/providers/ant-ling.models.ts b/packages/ai/src/providers/ant-ling.models.ts index bc42a8c62..4f66c20ba 100644 --- a/packages/ai/src/providers/ant-ling.models.ts +++ b/packages/ai/src/providers/ant-ling.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/ant-ling.json" with { type: "json" }; diff --git a/packages/ai/src/providers/anthropic.models.ts b/packages/ai/src/providers/anthropic.models.ts index c70b3160c..656c9fabb 100644 --- a/packages/ai/src/providers/anthropic.models.ts +++ b/packages/ai/src/providers/anthropic.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/anthropic.json" with { type: "json" }; diff --git a/packages/ai/src/providers/azure.models.ts b/packages/ai/src/providers/azure.models.ts index f86701758..0fb15ec40 100644 --- a/packages/ai/src/providers/azure.models.ts +++ b/packages/ai/src/providers/azure.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/azure.json" with { type: "json" }; diff --git a/packages/ai/src/providers/baseten.models.ts b/packages/ai/src/providers/baseten.models.ts index 864eb1f99..d68edd3e1 100644 --- a/packages/ai/src/providers/baseten.models.ts +++ b/packages/ai/src/providers/baseten.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/baseten.json" with { type: "json" }; diff --git a/packages/ai/src/providers/cerebras.models.ts b/packages/ai/src/providers/cerebras.models.ts index 39b1d2900..e3454f2fa 100644 --- a/packages/ai/src/providers/cerebras.models.ts +++ b/packages/ai/src/providers/cerebras.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/cerebras.json" with { type: "json" }; diff --git a/packages/ai/src/providers/cloudflare-ai-gateway.models.ts b/packages/ai/src/providers/cloudflare-ai-gateway.models.ts index 95e656542..ff524e062 100644 --- a/packages/ai/src/providers/cloudflare-ai-gateway.models.ts +++ b/packages/ai/src/providers/cloudflare-ai-gateway.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/cloudflare-ai-gateway.json" with { type: "json" }; diff --git a/packages/ai/src/providers/cloudflare-workers-ai.models.ts b/packages/ai/src/providers/cloudflare-workers-ai.models.ts index d84e49d16..0d34f6fae 100644 --- a/packages/ai/src/providers/cloudflare-workers-ai.models.ts +++ b/packages/ai/src/providers/cloudflare-workers-ai.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/cloudflare-workers-ai.json" with { type: "json" }; diff --git a/packages/ai/src/providers/deepseek.models.ts b/packages/ai/src/providers/deepseek.models.ts index 5ece8a190..9c92fb885 100644 --- a/packages/ai/src/providers/deepseek.models.ts +++ b/packages/ai/src/providers/deepseek.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/deepseek.json" with { type: "json" }; diff --git a/packages/ai/src/providers/fireworks.models.ts b/packages/ai/src/providers/fireworks.models.ts index 44cfa0856..53144268b 100644 --- a/packages/ai/src/providers/fireworks.models.ts +++ b/packages/ai/src/providers/fireworks.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/fireworks.json" with { type: "json" }; diff --git a/packages/ai/src/providers/github-copilot.models.ts b/packages/ai/src/providers/github-copilot.models.ts index 4b853d477..a71210730 100644 --- a/packages/ai/src/providers/github-copilot.models.ts +++ b/packages/ai/src/providers/github-copilot.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/github-copilot.json" with { type: "json" }; diff --git a/packages/ai/src/providers/google-vertex.models.ts b/packages/ai/src/providers/google-vertex.models.ts index a1778eecd..b081a3c71 100644 --- a/packages/ai/src/providers/google-vertex.models.ts +++ b/packages/ai/src/providers/google-vertex.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/google-vertex.json" with { type: "json" }; diff --git a/packages/ai/src/providers/google.models.ts b/packages/ai/src/providers/google.models.ts index 726a47440..ef5bec067 100644 --- a/packages/ai/src/providers/google.models.ts +++ b/packages/ai/src/providers/google.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/google.json" with { type: "json" }; diff --git a/packages/ai/src/providers/groq.models.ts b/packages/ai/src/providers/groq.models.ts index 75a84c36f..0af13338e 100644 --- a/packages/ai/src/providers/groq.models.ts +++ b/packages/ai/src/providers/groq.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/groq.json" with { type: "json" }; diff --git a/packages/ai/src/providers/huggingface.models.ts b/packages/ai/src/providers/huggingface.models.ts index 3bc971b6a..b8028a480 100644 --- a/packages/ai/src/providers/huggingface.models.ts +++ b/packages/ai/src/providers/huggingface.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/huggingface.json" with { type: "json" }; diff --git a/packages/ai/src/providers/kimi-coding.models.ts b/packages/ai/src/providers/kimi-coding.models.ts index 5b6ea5404..af36f3b47 100644 --- a/packages/ai/src/providers/kimi-coding.models.ts +++ b/packages/ai/src/providers/kimi-coding.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/kimi-coding.json" with { type: "json" }; diff --git a/packages/ai/src/providers/meta.models.ts b/packages/ai/src/providers/meta.models.ts index 81cd83917..d51b53db5 100644 --- a/packages/ai/src/providers/meta.models.ts +++ b/packages/ai/src/providers/meta.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/meta.json" with { type: "json" }; diff --git a/packages/ai/src/providers/minimax-cn.models.ts b/packages/ai/src/providers/minimax-cn.models.ts index b6cab80c6..db78342f4 100644 --- a/packages/ai/src/providers/minimax-cn.models.ts +++ b/packages/ai/src/providers/minimax-cn.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/minimax-cn.json" with { type: "json" }; diff --git a/packages/ai/src/providers/minimax.models.ts b/packages/ai/src/providers/minimax.models.ts index 4a46241a8..0adf27526 100644 --- a/packages/ai/src/providers/minimax.models.ts +++ b/packages/ai/src/providers/minimax.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/minimax.json" with { type: "json" }; diff --git a/packages/ai/src/providers/mistral.models.ts b/packages/ai/src/providers/mistral.models.ts index 0f7bed430..b59abdde8 100644 --- a/packages/ai/src/providers/mistral.models.ts +++ b/packages/ai/src/providers/mistral.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/mistral.json" with { type: "json" }; diff --git a/packages/ai/src/providers/moonshotai-cn.models.ts b/packages/ai/src/providers/moonshotai-cn.models.ts index e1cde46dc..b9183981a 100644 --- a/packages/ai/src/providers/moonshotai-cn.models.ts +++ b/packages/ai/src/providers/moonshotai-cn.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/moonshotai-cn.json" with { type: "json" }; diff --git a/packages/ai/src/providers/moonshotai.models.ts b/packages/ai/src/providers/moonshotai.models.ts index 19ac3d66e..41bd93e39 100644 --- a/packages/ai/src/providers/moonshotai.models.ts +++ b/packages/ai/src/providers/moonshotai.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/moonshotai.json" with { type: "json" }; diff --git a/packages/ai/src/providers/nvidia.models.ts b/packages/ai/src/providers/nvidia.models.ts index f7dfc374f..73e3a7c18 100644 --- a/packages/ai/src/providers/nvidia.models.ts +++ b/packages/ai/src/providers/nvidia.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/nvidia.json" with { type: "json" }; diff --git a/packages/ai/src/providers/openai-codex.models.ts b/packages/ai/src/providers/openai-codex.models.ts index 996f1fd2f..b5e910527 100644 --- a/packages/ai/src/providers/openai-codex.models.ts +++ b/packages/ai/src/providers/openai-codex.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/openai-codex.json" with { type: "json" }; diff --git a/packages/ai/src/providers/openai.models.ts b/packages/ai/src/providers/openai.models.ts index 06e0d4123..32faca489 100644 --- a/packages/ai/src/providers/openai.models.ts +++ b/packages/ai/src/providers/openai.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/openai.json" with { type: "json" }; diff --git a/packages/ai/src/providers/opencode-go.models.ts b/packages/ai/src/providers/opencode-go.models.ts index cb1dcc343..6231ee805 100644 --- a/packages/ai/src/providers/opencode-go.models.ts +++ b/packages/ai/src/providers/opencode-go.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/opencode-go.json" with { type: "json" }; diff --git a/packages/ai/src/providers/opencode.models.ts b/packages/ai/src/providers/opencode.models.ts index 812c584e3..c671fa275 100644 --- a/packages/ai/src/providers/opencode.models.ts +++ b/packages/ai/src/providers/opencode.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/opencode.json" with { type: "json" }; diff --git a/packages/ai/src/providers/openrouter.models.ts b/packages/ai/src/providers/openrouter.models.ts index 75d808b59..80bdc1c08 100644 --- a/packages/ai/src/providers/openrouter.models.ts +++ b/packages/ai/src/providers/openrouter.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/openrouter.json" with { type: "json" }; diff --git a/packages/ai/src/providers/qwen-token-plan-cn.models.ts b/packages/ai/src/providers/qwen-token-plan-cn.models.ts index 6b609faf7..4c6907031 100644 --- a/packages/ai/src/providers/qwen-token-plan-cn.models.ts +++ b/packages/ai/src/providers/qwen-token-plan-cn.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/qwen-token-plan-cn.json" with { type: "json" }; diff --git a/packages/ai/src/providers/qwen-token-plan-individual.models.ts b/packages/ai/src/providers/qwen-token-plan-individual.models.ts index 8d48afd89..bb2175425 100644 --- a/packages/ai/src/providers/qwen-token-plan-individual.models.ts +++ b/packages/ai/src/providers/qwen-token-plan-individual.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/qwen-token-plan-individual.json" with { type: "json" }; diff --git a/packages/ai/src/providers/qwen-token-plan.models.ts b/packages/ai/src/providers/qwen-token-plan.models.ts index c546c44d3..bb035d2e3 100644 --- a/packages/ai/src/providers/qwen-token-plan.models.ts +++ b/packages/ai/src/providers/qwen-token-plan.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/qwen-token-plan.json" with { type: "json" }; diff --git a/packages/ai/src/providers/radius.models.ts b/packages/ai/src/providers/radius.models.ts index c8f3b12d7..81070bd0a 100644 --- a/packages/ai/src/providers/radius.models.ts +++ b/packages/ai/src/providers/radius.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/radius.json" with { type: "json" }; diff --git a/packages/ai/src/providers/together.models.ts b/packages/ai/src/providers/together.models.ts index d8d5c919b..3e1c5d2e3 100644 --- a/packages/ai/src/providers/together.models.ts +++ b/packages/ai/src/providers/together.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/together.json" with { type: "json" }; diff --git a/packages/ai/src/providers/typesafe.models.ts b/packages/ai/src/providers/typesafe.models.ts index 05429c335..2be16f483 100644 --- a/packages/ai/src/providers/typesafe.models.ts +++ b/packages/ai/src/providers/typesafe.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/typesafe.json" with { type: "json" }; diff --git a/packages/ai/src/providers/vercel-ai-gateway.models.ts b/packages/ai/src/providers/vercel-ai-gateway.models.ts index e1c25832b..d9aff7dd9 100644 --- a/packages/ai/src/providers/vercel-ai-gateway.models.ts +++ b/packages/ai/src/providers/vercel-ai-gateway.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/vercel-ai-gateway.json" with { type: "json" }; diff --git a/packages/ai/src/providers/xai.models.ts b/packages/ai/src/providers/xai.models.ts index b1fb5d302..654b7e524 100644 --- a/packages/ai/src/providers/xai.models.ts +++ b/packages/ai/src/providers/xai.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/xai.json" with { type: "json" }; diff --git a/packages/ai/src/providers/xiaomi-token-plan-ams.models.ts b/packages/ai/src/providers/xiaomi-token-plan-ams.models.ts index 72b691d02..6c514b5d1 100644 --- a/packages/ai/src/providers/xiaomi-token-plan-ams.models.ts +++ b/packages/ai/src/providers/xiaomi-token-plan-ams.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/xiaomi-token-plan-ams.json" with { type: "json" }; diff --git a/packages/ai/src/providers/xiaomi-token-plan-cn.models.ts b/packages/ai/src/providers/xiaomi-token-plan-cn.models.ts index cae4bc12a..df833e743 100644 --- a/packages/ai/src/providers/xiaomi-token-plan-cn.models.ts +++ b/packages/ai/src/providers/xiaomi-token-plan-cn.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/xiaomi-token-plan-cn.json" with { type: "json" }; diff --git a/packages/ai/src/providers/xiaomi-token-plan-sgp.models.ts b/packages/ai/src/providers/xiaomi-token-plan-sgp.models.ts index 13db578e9..e965bd8e4 100644 --- a/packages/ai/src/providers/xiaomi-token-plan-sgp.models.ts +++ b/packages/ai/src/providers/xiaomi-token-plan-sgp.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/xiaomi-token-plan-sgp.json" with { type: "json" }; diff --git a/packages/ai/src/providers/xiaomi.models.ts b/packages/ai/src/providers/xiaomi.models.ts index 0ade9200d..8fd6f6949 100644 --- a/packages/ai/src/providers/xiaomi.models.ts +++ b/packages/ai/src/providers/xiaomi.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/xiaomi.json" with { type: "json" }; diff --git a/packages/ai/src/providers/zai-coding-cn.models.ts b/packages/ai/src/providers/zai-coding-cn.models.ts index 7dbd6a27c..bb30aba5d 100644 --- a/packages/ai/src/providers/zai-coding-cn.models.ts +++ b/packages/ai/src/providers/zai-coding-cn.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/zai-coding-cn.json" with { type: "json" }; diff --git a/packages/ai/src/providers/zai.models.ts b/packages/ai/src/providers/zai.models.ts index 7de6890f7..8bdb1ca58 100644 --- a/packages/ai/src/providers/zai.models.ts +++ b/packages/ai/src/providers/zai.models.ts @@ -1,4 +1,4 @@ -// This file is auto-generated by scripts/generate-models.ts +// This file is auto-generated by catalog/generate.ts // Do not edit manually - run 'npm run generate-models' to update import values from "./data/zai.json" with { type: "json" }; diff --git a/packages/ai/test/catalog-generation-utils.ts b/packages/ai/test/catalog-generation-utils.ts new file mode 100644 index 000000000..4e68f390d --- /dev/null +++ b/packages/ai/test/catalog-generation-utils.ts @@ -0,0 +1,52 @@ +import { writeFileSync } from "node:fs"; +import type { ModelsDevV2Catalog } from "../catalog/sources/models-dev.ts"; + +/** + * Write a `node --import` preload that serves every upstream catalog the + * generator fetches: the given models.dev v2 catalog, and minimal responses for + * the other sources. Any other request fails. + */ +export function writeUpstreamMock(preloadPath: string, modelsDev: ModelsDevV2Catalog): void { + const responses = { + "https://models.dev/experimental/v2.0/api.json": modelsDev, + "https://models.dev/models.json?type=decision": { + "typesafe/jev-latest": { id: "typesafe/jev-latest", name: "Jev", type: "decision", limit: { context: 64000 } }, + }, + "https://openrouter.ai/api/v1/models": { data: [] }, + "https://openrouter.ai/api/v1/models?output_modalities=image": { + data: [ + { + id: "test/image", + name: "Test Image", + architecture: { input_modalities: ["text"], output_modalities: ["image"] }, + pricing: {}, + }, + ], + }, + "https://openrouter.ai/api/v1/models?output_modalities=decisions": { data: [] }, + "https://ai-gateway.vercel.sh/v1/models": { data: [] }, + "https://radius.pi.dev/v1/config": { + baseUrl: "https://radius.pi.dev", + models: [ + { + id: "test", + name: "Test", + reasoning: false, + input: ["text"], + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, + contextWindow: 4096, + maxTokens: 4096, + }, + ], + }, + }; + writeFileSync( + preloadPath, + `const responses = ${JSON.stringify(responses)};\n` + + `globalThis.fetch = async (input) => {\n` + + ` const url = String(input);\n` + + ` if (url in responses) return Response.json(responses[url]);\n` + + ` throw new Error(\`Unexpected fetch: \${url}\`);\n` + + `};\n`, + ); +} diff --git a/packages/ai/test/fireworks-model-generation.test.ts b/packages/ai/test/fireworks-model-generation.test.ts index 4b497bb77..189f80ac5 100644 --- a/packages/ai/test/fireworks-model-generation.test.ts +++ b/packages/ai/test/fireworks-model-generation.test.ts @@ -1,14 +1,15 @@ import { spawnSync } from "node:child_process"; -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; import { afterEach, describe, expect, it } from "vitest"; -import type { ModelsDevReasoningOption } from "../scripts/models-dev-reasoning-options.ts"; +import type { ModelsDevReasoningOption, ModelsDevV2Model } from "../catalog/sources/models-dev.ts"; import { streamSimple } from "../src/api/anthropic-messages.ts"; import { getSupportedThinkingLevels, hasApi } from "../src/models.ts"; import type { Api, Model } from "../src/types.ts"; import { normalizeContext } from "../src/utils/transcript.ts"; +import { writeUpstreamMock } from "./catalog-generation-utils.ts"; const packageRoot = fileURLToPath(new URL("..", import.meta.url)); const temporaryRoots: string[] = []; @@ -24,38 +25,32 @@ function generateFireworksModels( temporaryRoots.push(root); const preloadPath = join(root, "mock-catalog.mjs"); const outputPath = join(root, "catalog"); - const catalog = { - "fireworks-ai": { - models: Object.fromEntries( - Object.entries(options).map(([id, reasoning_options]) => [ - `accounts/fireworks/models/${id}`, - { id, tool_call: true, reasoning: true, reasoning_options }, - ]), - ), - }, - }; - writeFileSync( - preloadPath, - `const catalog = ${JSON.stringify(catalog)};\n` + - `globalThis.fetch = async (input) => {\n` + - ` const url = String(input);\n` + - ` if (url === "https://models.dev/api.json") return Response.json(catalog);\n` + - ` if (url === "https://models.dev/models.json?type=decision") return Response.json({ "typesafe/jev-latest": { name: "Jev", type: "decision", limit: { context: 64000, output: 0 } } });\n` + - ` if (url.startsWith("https://openrouter.ai/api/v1/models") || url === "https://ai-gateway.vercel.sh/v1/models") return Response.json({ data: [] });\n` + - ` if (url === "https://radius.pi.dev/v1/config") return Response.json({ baseUrl: "https://radius.pi.dev", models: [{ id: "test", name: "Test", reasoning: false, input: ["text"], cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, contextWindow: 4096, maxTokens: 4096 }] });\n` + - ` throw new Error(\`Unexpected fetch: \${url}\`);\n` + - `};\n`, - ); + const models = Object.entries(options).map(([id, reasoningOptions]): ModelsDevV2Model => { + const effort = reasoningOptions?.find((option) => option.type === "effort"); + const budget = reasoningOptions?.find((option) => option.type === "budget_tokens"); + return { + id: `accounts/fireworks/models/${id}`, + type: "chat", + name: id, + modalities: { input: ["text"], output: ["text"] }, + capabilities: { + tools: { supported: true }, + reasoning: { + supported: true, + ...(reasoningOptions?.some((option) => option.type === "toggle") ? { toggle: true } : {}), + ...(effort ? { effort: effort.values } : {}), + ...(budget ? { budget: { min: budget.min, max: budget.max } } : {}), + }, + }, + api: {}, + }; + }); + writeUpstreamMock(preloadPath, { + "fireworks-ai": { models: Object.fromEntries(models.map((model) => [model.id, model])) }, + }); const result = spawnSync( process.execPath, - [ - "--import", - pathToFileURL(preloadPath).href, - "scripts/generate-models.ts", - "--json-only", - "--json-output", - outputPath, - ], + ["--import", pathToFileURL(preloadPath).href, "catalog/generate.ts", "--json-only", "--json-output", outputPath], { cwd: packageRoot, encoding: "utf8", timeout: 10_000 }, ); expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0); diff --git a/packages/ai/test/generate-models-strict.test.ts b/packages/ai/test/generate-models-strict.test.ts index 78d27f4ec..a5f3ffa8e 100644 --- a/packages/ai/test/generate-models-strict.test.ts +++ b/packages/ai/test/generate-models-strict.test.ts @@ -1,9 +1,11 @@ import { spawnSync } from "node:child_process"; -import { cpSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { cpSync, mkdirSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; import { afterEach, describe, expect, it } from "vitest"; +import type { ModelsDevV2Model } from "../catalog/sources/models-dev.ts"; +import { writeUpstreamMock } from "./catalog-generation-utils.ts"; const packageRoot = fileURLToPath(new URL("..", import.meta.url)); const temporaryRoots: string[] = []; @@ -18,7 +20,7 @@ describe("strict model generation", () => { temporaryRoots.push(fixtureRoot); const isolatedPackageRoot = join(fixtureRoot, "package"); mkdirSync(isolatedPackageRoot); - for (const entry of ["package.json", "scripts", "src"]) { + for (const entry of ["package.json", "catalog", "scripts", "src"]) { cpSync(join(packageRoot, entry), join(isolatedPackageRoot, entry), { recursive: true }); } const preloadPath = join(fixtureRoot, "mock-models-dev.mjs"); @@ -35,26 +37,22 @@ describe("strict model generation", () => { "qwen3.8-max-preview", ]; const sourceModels = Object.fromEntries( - modelIds.map((id) => [ + modelIds.map((id): [string, ModelsDevV2Model] => [ id, { id, + type: "chat", name: id, - tool_call: id !== "deepseek-v4-flash-0731", + modalities: { input: ["text"], output: ["text"] }, + capabilities: { + tools: { supported: id !== "deepseek-v4-flash-0731" }, + reasoning: { supported: false }, + }, + api: {}, }, ]), ); - const catalog = { "alibaba-token-plan": { models: sourceModels } }; - writeFileSync( - preloadPath, - `const catalog = ${JSON.stringify(catalog)};\n` + - `globalThis.fetch = async (input) => {\n` + - ` if (String(input) === "https://models.dev/api.json") {\n` + - ` return new Response(JSON.stringify(catalog), { status: 200 });\n` + - ` }\n` + - ` throw new Error(\`Unexpected fetch: \${String(input)}\`);\n` + - `};\n`, - ); + writeUpstreamMock(preloadPath, { "alibaba-token-plan": { models: sourceModels } }); const generatedPaths = [ "src/models.generated.ts", @@ -67,7 +65,7 @@ describe("strict model generation", () => { const result = spawnSync( process.execPath, - ["--import", pathToFileURL(preloadPath).href, "scripts/generate-models.ts", "--strict"], + ["--import", pathToFileURL(preloadPath).href, "catalog/generate.ts", "--strict"], { cwd: isolatedPackageRoot, encoding: "utf8", diff --git a/packages/ai/test/image-model-data.test.ts b/packages/ai/test/image-model-data.test.ts index 21545fe8f..645fdc242 100644 --- a/packages/ai/test/image-model-data.test.ts +++ b/packages/ai/test/image-model-data.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { buildOpenRouterCatalog, type OpenRouterModelListItem } from "../scripts/openrouter-catalog.ts"; +import { buildOpenRouterCatalog, type OpenRouterModelListItem } from "../catalog/sources/openrouter.ts"; const imageOnly: OpenRouterModelListItem = { id: "example/image-model", diff --git a/packages/ai/test/model-cost-tiers.test.ts b/packages/ai/test/model-cost-tiers.test.ts index ac74ea7c6..9eade534e 100644 --- a/packages/ai/test/model-cost-tiers.test.ts +++ b/packages/ai/test/model-cost-tiers.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; -import { getAiGatewayCost } from "../scripts/ai-gateway-pricing.ts"; -import { buildOpenRouterCatalog, type OpenRouterModelListItem } from "../scripts/openrouter-catalog.ts"; +import { buildOpenRouterCatalog, type OpenRouterModelListItem } from "../catalog/sources/openrouter.ts"; +import { getAiGatewayCost } from "../catalog/sources/vercel-ai-gateway.ts"; function openRouterChatCost(pricing: OpenRouterModelListItem["pricing"]) { const catalog = buildOpenRouterCatalog( diff --git a/packages/ai/test/opencode-model-generation.test.ts b/packages/ai/test/opencode-model-generation.test.ts new file mode 100644 index 000000000..6bf861697 --- /dev/null +++ b/packages/ai/test/opencode-model-generation.test.ts @@ -0,0 +1,97 @@ +import { spawnSync } from "node:child_process"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { afterEach, describe, expect, it } from "vitest"; +import type { ModelsDevV2Model } from "../catalog/sources/models-dev.ts"; +import type { Api, Model } from "../src/types.ts"; +import { writeUpstreamMock } from "./catalog-generation-utils.ts"; + +const packageRoot = fileURLToPath(new URL("..", import.meta.url)); +const temporaryRoots: string[] = []; + +afterEach(() => { + for (const root of temporaryRoots.splice(0)) rmSync(root, { force: true, recursive: true }); +}); + +function sourceModel(id: string, api: ModelsDevV2Model["api"]): ModelsDevV2Model { + return { + id, + type: "chat", + name: id, + modalities: { input: ["text"], output: ["text"] }, + capabilities: { tools: { supported: true }, reasoning: { supported: false } }, + api, + }; +} + +function generateOpencodeModels(models: ModelsDevV2Model[], strict = false) { + const root = mkdtempSync(join(tmpdir(), "pi-opencode-generation-")); + temporaryRoots.push(root); + const preloadPath = join(root, "mock-catalog.mjs"); + const outputPath = join(root, "catalog"); + writeUpstreamMock(preloadPath, { + opencode: { models: Object.fromEntries(models.map((model) => [model.id, model])) }, + }); + const result = spawnSync( + process.execPath, + [ + "--import", + pathToFileURL(preloadPath).href, + "catalog/generate.ts", + ...(strict ? ["--strict"] : []), + "--json-only", + "--json-output", + outputPath, + ], + { cwd: packageRoot, encoding: "utf8", timeout: 10_000 }, + ); + return { + result, + models: () => + JSON.parse(readFileSync(join(outputPath, "providers/opencode.json"), "utf8")) as Record>, + }; +} + +describe("OpenCode model generation", () => { + it("uses the protocol and base URL models.dev lists for each model", () => { + const base = "https://example.test/zen"; + const { result, models } = generateOpencodeModels([ + sourceModel("chat", { "chat-completions": { base_url: `${base}/openai/v1` } }), + sourceModel("responses", { responses: { base_url: `${base}/openai/v1` } }), + sourceModel("messages", { messages: { base_url: `${base}/anthropic/v1` } }), + sourceModel("gemini", { "generate-content": { base_url: `${base}/google/v1beta` } }), + // Native protocols win over Chat Completions regardless of listing order. + sourceModel("both", { + "chat-completions": { base_url: `${base}/openai/v1` }, + messages: { base_url: `${base}/anthropic/v1` }, + }), + sourceModel("live", { "bidi-generate-content": { base_url: `${base}/google/v1beta` } }), + ]); + expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0); + expect(result.stderr).toContain("models.dev lists no supported API for opencode/live (bidi-generate-content)"); + + const generated = models(); + const endpoints = Object.fromEntries( + Object.values(generated).map((model) => [model.id, { api: model.api, baseUrl: model.baseUrl }]), + ); + expect(endpoints).toEqual({ + chat: { api: "openai-completions", baseUrl: `${base}/openai/v1` }, + responses: { api: "openai-responses", baseUrl: `${base}/openai/v1` }, + // The Anthropic SDK appends /v1/messages itself. + messages: { api: "anthropic-messages", baseUrl: `${base}/anthropic` }, + gemini: { api: "google-generative-ai", baseUrl: `${base}/google/v1beta` }, + both: { api: "anthropic-messages", baseUrl: `${base}/anthropic` }, + }); + }); + + it("fails strict generation for a model without a supported protocol", () => { + const { result } = generateOpencodeModels( + [sourceModel("live", { "bidi-generate-content": { base_url: "https://example.test/google/v1beta" } })], + true, + ); + expect(result.status).toBe(1); + expect(result.stderr).toContain("models.dev lists no supported API for opencode/live (bidi-generate-content)"); + }); +}); diff --git a/packages/ai/test/openrouter-reasoning-options.test.ts b/packages/ai/test/openrouter-reasoning-options.test.ts index dbeb1140e..743f58d6c 100644 --- a/packages/ai/test/openrouter-reasoning-options.test.ts +++ b/packages/ai/test/openrouter-reasoning-options.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { getOpenRouterThinkingLevelMap } from "../scripts/openrouter-reasoning-options.ts"; +import { getOpenRouterThinkingLevelMap } from "../catalog/sources/openrouter.ts"; import { streamSimple } from "../src/api/openai-completions.ts"; import type { Model, ThinkingLevelMap } from "../src/types.ts"; import { normalizeContext } from "../src/utils/transcript.ts"; diff --git a/packages/ai/test/reasoning-options.test.ts b/packages/ai/test/reasoning-options.test.ts index fafa9fa5e..135a0b9ec 100644 --- a/packages/ai/test/reasoning-options.test.ts +++ b/packages/ai/test/reasoning-options.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { getEffortThinkingLevelMap } from "../scripts/models-dev-reasoning-options.ts"; +import { getEffortThinkingLevelMap } from "../catalog/reasoning.ts"; describe("getEffortThinkingLevelMap", () => { it("exposes only verified effort values and none", () => { diff --git a/scripts/diff-model-catalog.mjs b/scripts/diff-model-catalog.mjs index e00563f76..1d2f10a4e 100644 --- a/scripts/diff-model-catalog.mjs +++ b/scripts/diff-model-catalog.mjs @@ -63,7 +63,11 @@ const currentThinkingOutput = join(temporaryRoot, "after-thinking"); let worktreeAdded = false; function generateCatalog(cwd, outputDir, pretty = false) { - const args = ["packages/ai/scripts/generate-models.ts", "--strict", "--json-only", "--json-output", outputDir]; + // Baselines from before the catalog moved still have the old generator. + const generator = existsSync(join(cwd, "packages/ai/catalog/generate.ts")) + ? "packages/ai/catalog/generate.ts" + : "packages/ai/scripts/generate-models.ts"; + const args = [generator, "--strict", "--json-only", "--json-output", outputDir]; if (pretty) args.push("--pretty"); run(process.execPath, args, { cwd, capture: true }); } diff --git a/scripts/issues/categories/models.md b/scripts/issues/categories/models.md index cc9d32953..774f24e99 100644 --- a/scripts/issues/categories/models.md +++ b/scripts/issues/categories/models.md @@ -6,7 +6,7 @@ owner: davidbrai Which models exist and which one is used. In: -- Built-in model catalog (generate-models, models.dev, pi.dev registry). +- Built-in model catalog (packages/ai/catalog, models.dev, pi.dev registry). - Model metadata: context window, max tokens, thinking level map, pricing. - models.json custom providers and models. - Model resolution: --model, --provider, defaultModel, startup fallback, scoped models, virtual models. diff --git a/tsconfig.json b/tsconfig.json index b530a76e4..8a90126b1 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -49,6 +49,6 @@ "@earendil-works/pi-agent-old/*": ["./packages/agent-old/src/*"] } }, - "include": ["scripts/model-catalog-protocol.ts", "scripts/model-catalog-protocol.test.ts", "packages/*/src/**/*", "packages/*/test/**/*", "packages/durable/test-workers/**/*", "packages/coding-agent/examples/**/*", "packages/agent/examples/**/*"], + "include": ["scripts/model-catalog-protocol.ts", "scripts/model-catalog-protocol.test.ts", "packages/*/src/**/*", "packages/*/test/**/*", "packages/ai/catalog/**/*", "packages/durable/test-workers/**/*", "packages/coding-agent/examples/**/*", "packages/agent/examples/**/*"], "exclude": ["**/dist/**", "packages/coding-agent/examples/extensions/gondolin/**"] }