mirror of
https://github.com/badlogic/pi-mono.git
synced 2026-11-05 23:37:25 +00:00
feat(ai,agent,coding-agent): validate tool arguments with pi-schema, drop TypeBox
pi-schema: opt-in coercion, compile(schema, { coerce: true }) with coerce() and
parse(), the superset of TypeBox's Value.Convert rules and pi-ai's old coercion;
unresolvedRefs option; url and json-pointer-uri-fragment formats.
pi-ai: Tool takes any JSON Schema; validation through pi-schema; Type, Static
and TSchema are no longer exported (breaking). pi-agent-core and pi-durable:
tools on pi-schema, no TypeBox dependency. Coding agent: extensions keep Type,
Static, TSchema and StringEnum through an extension view of pi-ai; built-in
tools stay on TypeBox.
This commit is contained in:
Mario Zechner
parent
2c8d1feb14
commit
6536f80129
139 changed files with 5115 additions and 1305 deletions
|
|
@ -4,6 +4,31 @@
|
|||
|
||||
### Breaking Changes
|
||||
|
||||
- The main entry no longer re-exports TypeBox (`Type`, `Static`, `TSchema`), and `typebox` is no longer a dependency. `Tool.parameters` is any JSON Schema object (`Tool<TParameters extends object>`): `@earendil-works/pi-schema`'s builders, which also type the arguments through pi-schema's `Static`, TypeBox schemas, or plain objects. `validateToolCall()`/`validateToolArguments()` validate with pi-schema (`compile(schema, { coerce: true }).parse()`): every schema gets the same coercion, the superset of TypeBox's `Value.Convert` rules (which only applied to TypeBox-built schemas) and pi-ai's own, so plain JSON Schema tools (MCP tools) now also get, for example, `null` to `"null"` for a required string, `"3.5"` to 3 for an integer and a single value wrapped into an array. The error text lists pi-schema's messages (`- edits.0.oldText: missing`, `- limit: expected number, got string`) instead of TypeBox's. `StringEnum()` returns a pi-schema `Schema<T>` with the same JSON (`{ type: "string", enum }`). The model and compat schemas (`providers/model-schema`, `providers/compat-schema`) are built with pi-schema, with the same JSON Schema, and their types come from pi-schema's `Static`; `ProviderCompat` is exported. Migration:
|
||||
|
||||
```ts
|
||||
// Before
|
||||
import { Type, type Static, StringEnum } from "@earendil-works/pi-ai";
|
||||
const params = Type.Object({ path: Type.String(), mode: StringEnum(["a", "b"]) });
|
||||
type Params = Static<typeof params>;
|
||||
|
||||
// After: pi-schema builders (closed objects; pass { open: true } to allow other properties)
|
||||
import { StringEnum } from "@earendil-works/pi-ai";
|
||||
import { obj, type Static, str } from "@earendil-works/pi-schema";
|
||||
const params = obj({ path: str(), mode: StringEnum(["a", "b"]) });
|
||||
type Params = Static<typeof params>;
|
||||
|
||||
// Or keep TypeBox, imported from typebox itself (its schemas are JSON Schema)
|
||||
import { Type, type Static } from "typebox";
|
||||
```
|
||||
- Tool arguments are validated as JSON Schema only; nothing runs through TypeBox's own validator any more, so TypeBox features outside JSON Schema no longer apply to tool arguments:
|
||||
- `Type.Refine()` refinements are not run: arguments they would reject are accepted.
|
||||
- `Type.BigInt()`, `Type.Undefined()` and `Type.Void()` reject every JSON argument (TypeBox converted to them).
|
||||
- Codecs (`Type.Codec()`) do not decode: the tool receives the encoded JSON value.
|
||||
- Formats registered with TypeBox's `Format.Set()` are not checked; the standard formats plus `url` and `json-pointer-uri-fragment` are.
|
||||
- Property names are names, not regular expressions: `"a.b"` no longer also converts the argument `axb`.
|
||||
- `additionalProperties` converts only undeclared properties; TypeBox also converted declared ones with it when an object had more than one property.
|
||||
- A `$ref` that names nothing (a missing definition, an external document) rejects only arguments at its place (`- t: unresolved reference #/definitions/Missing`) instead of failing every call of the tool, as TypeBox's rejection did before.
|
||||
- `appendGrammarToolInputJsonDelta()` (`api/constrained-sampling`) takes the input delta instead of the whole input so far and no longer closes the input; close it with the new `closeGrammarToolInputJson()` ([#9265](https://github.com/earendil-works/pi/issues/9265))
|
||||
|
||||
### Added
|
||||
|
|
|
|||
|
|
@ -103,15 +103,16 @@ Unified LLM API with provider collections, automatic auth resolution, token and
|
|||
npm install @earendil-works/pi-ai
|
||||
```
|
||||
|
||||
TypeBox exports are re-exported from `@earendil-works/pi-ai`: `Type`, `Static`, and `TSchema`.
|
||||
Tool parameters are JSON Schema. The examples build them with `@earendil-works/pi-schema`, which also types the arguments (`npm install @earendil-works/pi-schema`); TypeBox schemas and plain JSON Schema objects work as well.
|
||||
|
||||
## Quick Start
|
||||
|
||||
You build a `Models` collection of providers and stream through it. The quickest start registers every built-in provider; apps that care about bundle size register individual providers instead (see [Provider Factories](#provider-factories) and [Bundling and Tree Shaking](#bundling-and-tree-shaking)).
|
||||
|
||||
```typescript
|
||||
import { Type, type Context, type Tool } from '@earendil-works/pi-ai';
|
||||
import { type Context, type Tool } from '@earendil-works/pi-ai';
|
||||
import { builtinModels } from '@earendil-works/pi-ai/providers/all';
|
||||
import { obj, opt, str } from '@earendil-works/pi-schema';
|
||||
|
||||
// A Models collection with every built-in provider registered
|
||||
const models = builtinModels();
|
||||
|
|
@ -119,12 +120,12 @@ const models = builtinModels();
|
|||
// Sync lookup against the collection
|
||||
const model = models.getModel('openai', 'gpt-4o-mini')!;
|
||||
|
||||
// Define tools with TypeBox schemas for type safety and validation
|
||||
// Define tools with JSON Schema; pi-schema's builders also type the arguments
|
||||
const tools: Tool[] = [{
|
||||
name: 'get_time',
|
||||
description: 'Get the current time',
|
||||
parameters: Type.Object({
|
||||
timezone: Type.Optional(Type.String({ description: 'Optional timezone (e.g., America/New_York)' }))
|
||||
parameters: obj({
|
||||
timezone: opt(str({ description: 'Optional timezone (e.g., America/New_York)' }))
|
||||
})
|
||||
}];
|
||||
|
||||
|
|
@ -538,34 +539,35 @@ Amazon Bedrock resolves ambient AWS credentials (`AWS_PROFILE`, access key pairs
|
|||
|
||||
## Tools
|
||||
|
||||
Tools enable LLMs to interact with external systems. This library uses TypeBox schemas for type-safe tool definitions with automatic validation using TypeBox's built-in validator and value conversion utilities. TypeBox schemas can be serialized and deserialized as plain JSON, making them ideal for distributed systems.
|
||||
Tools enable LLMs to interact with external systems. A tool's `parameters` is a JSON Schema object: built with `@earendil-works/pi-schema`'s builders (which also type the arguments through its `Static`), with TypeBox, or written by hand. `validateToolCall()` and `validateToolArguments()` validate with pi-schema, coercing loosely typed arguments first (numeric strings, `"true"`, a single value for an array, `null` for an omitted optional property). Schemas are plain JSON, so they serialize with the rest of a context.
|
||||
|
||||
### Defining Tools
|
||||
|
||||
```typescript
|
||||
import { Type, type Tool, StringEnum } from '@earendil-works/pi-ai';
|
||||
import { type Tool, StringEnum } from '@earendil-works/pi-ai';
|
||||
import { arr, obj, str } from '@earendil-works/pi-schema';
|
||||
|
||||
// Define tool parameters with TypeBox
|
||||
// Define tool parameters with pi-schema's builders
|
||||
const weatherTool: Tool = {
|
||||
name: 'get_weather',
|
||||
description: 'Get current weather for a location',
|
||||
parameters: Type.Object({
|
||||
location: Type.String({ description: 'City name or coordinates' }),
|
||||
parameters: obj({
|
||||
location: str({ description: 'City name or coordinates' }),
|
||||
units: StringEnum(['celsius', 'fahrenheit'], { default: 'celsius' })
|
||||
})
|
||||
};
|
||||
|
||||
// Note: For Google API compatibility, use StringEnum helper instead of Type.Enum
|
||||
// Type.Enum generates anyOf/const patterns that Google doesn't support
|
||||
// Note: StringEnum gives { type: "string", enum: [...] }, which every provider accepts,
|
||||
// including Google's, which rejects anyOf/const patterns
|
||||
|
||||
const bookMeetingTool: Tool = {
|
||||
name: 'book_meeting',
|
||||
description: 'Schedule a meeting',
|
||||
parameters: Type.Object({
|
||||
title: Type.String({ minLength: 1 }),
|
||||
startTime: Type.String({ format: 'date-time' }),
|
||||
endTime: Type.String({ format: 'date-time' }),
|
||||
attendees: Type.Array(Type.String({ format: 'email' }), { minItems: 1 })
|
||||
parameters: obj({
|
||||
title: str({ minLength: 1 }),
|
||||
startTime: str({ format: 'date-time' }),
|
||||
endTime: str({ format: 'date-time' }),
|
||||
attendees: arr(str({ format: 'email' }), { minItems: 1 })
|
||||
})
|
||||
};
|
||||
```
|
||||
|
|
@ -578,10 +580,7 @@ Tools can opt in to provider-side constrained sampling. For JSON-schema tools, `
|
|||
const strictTool: Tool = {
|
||||
name: 'edit_file',
|
||||
description: 'Edit a file',
|
||||
parameters: Type.Object({
|
||||
path: Type.String(),
|
||||
content: Type.String()
|
||||
}, { additionalProperties: false }),
|
||||
parameters: obj({ path: str(), content: str() }), // obj() is closed: additionalProperties: false
|
||||
constrainedSampling: { type: 'json_schema', strict: 'prefer' }
|
||||
};
|
||||
```
|
||||
|
|
@ -592,9 +591,7 @@ Strict JSON-schema constrained sampling is supported for OpenAI, Anthropic, supp
|
|||
const patchTool: Tool = {
|
||||
name: 'apply_patch',
|
||||
description: 'Apply a patch',
|
||||
parameters: Type.Object({
|
||||
input: Type.String()
|
||||
}, { additionalProperties: false }),
|
||||
parameters: obj({ input: str() }),
|
||||
constrainedSampling: {
|
||||
type: 'grammar',
|
||||
variants: {
|
||||
|
|
@ -698,7 +695,7 @@ for await (const event of s) {
|
|||
|
||||
### Validating Tool Arguments
|
||||
|
||||
When implementing your own tool execution loop, use `validateToolCall` to validate arguments before passing them to your tools:
|
||||
When implementing your own tool execution loop, use `validateToolCall` to validate arguments before passing them to your tools. It coerces loosely typed arguments with pi-schema (`compile(schema, { coerce: true }).parse()`), returns the coerced arguments, and throws an error listing pi-schema's errors (`path: message`) followed by the arguments it received:
|
||||
|
||||
```typescript
|
||||
import { validateToolCall, type Tool } from '@earendil-works/pi-ai';
|
||||
|
|
@ -1726,8 +1723,8 @@ models.setProvider(openaiProvider());
|
|||
|
||||
Rules:
|
||||
|
||||
- `@earendil-works/pi-ai/models` exports the model runtime (`createModels`, `createProvider`, model helpers, and their types) without TypeBox, built-in catalogs, or SDK implementations. Other types can still use `import type` from the root.
|
||||
- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, real provider factories, or SDK implementations, but it eagerly imports TypeBox and schema validation. Unbundled Node scripts do not tree-shake its unused exports; prefer `./models`, `./providers/faux`, and specific `./utils/*` subpaths when those are all you need.
|
||||
- `@earendil-works/pi-ai/models` exports the model runtime (`createModels`, `createProvider`, model helpers, and their types) without schema validation, built-in catalogs, or SDK implementations. Other types can still use `import type` from the root.
|
||||
- `@earendil-works/pi-ai` is the core entrypoint and does not import built-in catalogs, real provider factories, or SDK implementations, but it eagerly imports schema validation (pi-schema). Unbundled Node scripts do not tree-shake its unused exports; prefer `./models`, `./providers/faux`, and specific `./utils/*` subpaths when those are all you need.
|
||||
- `@earendil-works/pi-ai/providers/<provider>` imports that provider's catalog and lazy API wrapper only.
|
||||
- `@earendil-works/pi-ai/providers/all` imports every built-in provider factory and all catalogs. Use it only when you want the full built-in set.
|
||||
- With code splitting, provider SDKs stay in lazy chunks and load on first request.
|
||||
|
|
|
|||
|
|
@ -69,14 +69,14 @@
|
|||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "0.129.0",
|
||||
"@aws-sdk/client-bedrock-runtime": "3.1127.0",
|
||||
"@earendil-works/pi-schema": "1.4.0",
|
||||
"@earendil-works/pi-telemetry": "1.4.0",
|
||||
"@google/genai": "2.21.0",
|
||||
"@smithy/node-http-handler": "4.12.1",
|
||||
"http-proxy-agent": "9.1.0",
|
||||
"https-proxy-agent": "9.1.0",
|
||||
"openai": "7.19.0",
|
||||
"partial-json": "0.1.7",
|
||||
"typebox": "1.3.27"
|
||||
"partial-json": "0.1.7"
|
||||
},
|
||||
"keywords": [
|
||||
"ai",
|
||||
|
|
@ -101,6 +101,7 @@
|
|||
"devDependencies": {
|
||||
"@types/node": "22.19.19",
|
||||
"canvas": "3.2.3",
|
||||
"typebox": "1.3.27",
|
||||
"vitest": "4.1.12"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,3 @@
|
|||
export type { Static, TSchema } from "typebox";
|
||||
export { Type } from "typebox";
|
||||
|
||||
// Core only, side-effect free: no generated catalogs, no provider factories,
|
||||
// no api-registry, no OAuth implementations, no compat. Provider factories
|
||||
// live under "@earendil-works/pi-ai/providers/*", API implementations under
|
||||
|
|
@ -42,9 +39,9 @@ export * from "./utils/json-parse.ts";
|
|||
export * from "./utils/overflow.ts";
|
||||
export * from "./utils/retry.ts";
|
||||
export * from "./utils/streaming-json.ts";
|
||||
export * from "./utils/string-enum.ts";
|
||||
export { contentText, getSystemMessageText, renderSystemMessageUpdate } from "./utils/text.ts";
|
||||
export * from "./utils/transcript.ts";
|
||||
export * from "./utils/typebox-helpers.ts";
|
||||
export { uuidv7 } from "./utils/uuid.ts";
|
||||
export * from "./utils/validation.ts";
|
||||
export { PI_AI_VERSION } from "./utils/version.ts";
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -1,4 +1,17 @@
|
|||
import { type Static, type TSchemaOptions, Type } from "typebox";
|
||||
import {
|
||||
type Annotations,
|
||||
arr,
|
||||
int,
|
||||
nul,
|
||||
num,
|
||||
oneOf,
|
||||
opt,
|
||||
type Static,
|
||||
type StaticObject,
|
||||
str,
|
||||
union,
|
||||
} from "@earendil-works/pi-schema";
|
||||
import { openObject as object } from "./schema-builders.ts";
|
||||
|
||||
export const THINKING_LEVELS = ["minimal", "low", "medium", "high", "xhigh", "max"] as const;
|
||||
export const MODEL_THINKING_LEVELS = ["off", ...THINKING_LEVELS] as const;
|
||||
|
|
@ -6,19 +19,31 @@ export const MODEL_INPUT_MODALITIES = ["text", "image"] as const;
|
|||
export const CACHE_RETENTIONS = ["none", "short", "long"] as const;
|
||||
export const MODEL_PROMPT_CACHE_RETENTIONS = ["short", "long"] as const;
|
||||
|
||||
export const ThinkingLevelSchema = Type.Enum(THINKING_LEVELS);
|
||||
export const ModelThinkingLevelSchema = Type.Enum(MODEL_THINKING_LEVELS);
|
||||
export const ModelInputModalitySchema = Type.Enum(MODEL_INPUT_MODALITIES);
|
||||
export const CacheRetentionSchema = Type.Enum(CACHE_RETENTIONS);
|
||||
export const ThinkingLevelSchema = oneOf(THINKING_LEVELS);
|
||||
export const ModelThinkingLevelSchema = oneOf(MODEL_THINKING_LEVELS);
|
||||
export const ModelInputModalitySchema = oneOf(MODEL_INPUT_MODALITIES);
|
||||
export const CacheRetentionSchema = oneOf(CACHE_RETENTIONS);
|
||||
|
||||
const ThinkingLevelMapValueSchema = Type.Union([Type.String(), Type.Null()]);
|
||||
export const ThinkingLevelMapSchema = Type.Partial(Type.Record(ModelThinkingLevelSchema, ThinkingLevelMapValueSchema), {
|
||||
description:
|
||||
"Provider value sent for each thinking level. null marks a level unsupported. Levels the model does not map use the provider's default value through high; xhigh and max are unsupported.",
|
||||
});
|
||||
const ThinkingLevelMapValueSchema = union([str(), nul()]);
|
||||
export const ThinkingLevelMapSchema = object(
|
||||
{
|
||||
off: opt(ThinkingLevelMapValueSchema),
|
||||
minimal: opt(ThinkingLevelMapValueSchema),
|
||||
low: opt(ThinkingLevelMapValueSchema),
|
||||
medium: opt(ThinkingLevelMapValueSchema),
|
||||
high: opt(ThinkingLevelMapValueSchema),
|
||||
xhigh: opt(ThinkingLevelMapValueSchema),
|
||||
max: opt(ThinkingLevelMapValueSchema),
|
||||
},
|
||||
{
|
||||
description:
|
||||
"Provider value sent for each thinking level. null marks a level unsupported. Levels the model does not map use the provider's default value through high; xhigh and max are unsupported.",
|
||||
},
|
||||
);
|
||||
|
||||
export const ModelPromptCacheSchema = Type.Partial(
|
||||
Type.Record(Type.Enum(MODEL_PROMPT_CACHE_RETENTIONS), Type.Number({ exclusiveMinimum: 0 })),
|
||||
const PromptCacheLifetimeSchema = num({ exclusiveMinimum: 0 });
|
||||
export const ModelPromptCacheSchema = object(
|
||||
{ short: opt(PromptCacheLifetimeSchema), long: opt(PromptCacheLifetimeSchema) },
|
||||
{
|
||||
description:
|
||||
"Best-effort prompt cache lifetime in seconds for the short and long retention tiers. Pi warms only tiers with a known lifetime.",
|
||||
|
|
@ -26,63 +51,54 @@ export const ModelPromptCacheSchema = Type.Partial(
|
|||
);
|
||||
|
||||
const ModelCostRatesProperties = {
|
||||
input: Type.Number({ description: "Input cost in USD per million tokens." }),
|
||||
output: Type.Number({ description: "Output cost in USD per million tokens." }),
|
||||
cacheRead: Type.Number({ description: "Cache-read cost in USD per million tokens." }),
|
||||
cacheWrite: Type.Number({ description: "Cache-write cost in USD per million tokens." }),
|
||||
input: num({ description: "Input cost in USD per million tokens." }),
|
||||
output: num({ description: "Output cost in USD per million tokens." }),
|
||||
cacheRead: num({ description: "Cache-read cost in USD per million tokens." }),
|
||||
cacheWrite: num({ description: "Cache-write cost in USD per million tokens." }),
|
||||
};
|
||||
|
||||
export const ModelCostRatesSchema = Type.Object(ModelCostRatesProperties);
|
||||
export const ModelCostTierSchema = Type.Object({
|
||||
inputTokensAbove: Type.Number({
|
||||
export const ModelCostRatesSchema = object(ModelCostRatesProperties);
|
||||
export const ModelCostTierSchema = object({
|
||||
inputTokensAbove: num({
|
||||
description: "Use this tier when total request input exceeds this token count.",
|
||||
}),
|
||||
...ModelCostRatesProperties,
|
||||
});
|
||||
export const ModelCostSchema = Type.Object({
|
||||
export const ModelCostSchema = object({
|
||||
...ModelCostRatesProperties,
|
||||
tiers: Type.Optional(
|
||||
Type.Array(ModelCostTierSchema, {
|
||||
tiers: opt(
|
||||
arr(ModelCostTierSchema, {
|
||||
description: "Request-wide pricing tiers. The highest matching input threshold applies to the full request.",
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
function modelImageResizeOptions(options?: TSchemaOptions) {
|
||||
return Type.Object(
|
||||
{
|
||||
maxWidth: Type.Optional(Type.Integer({ minimum: 1 })),
|
||||
maxHeight: Type.Optional(Type.Integer({ minimum: 1 })),
|
||||
maxBytes: Type.Optional(
|
||||
Type.Integer({ minimum: 1, description: "Maximum base64-encoded payload size in bytes." }),
|
||||
),
|
||||
jpegQuality: Type.Optional(Type.Integer({ minimum: 1, maximum: 100 })),
|
||||
},
|
||||
options,
|
||||
);
|
||||
const ModelImageResizeProperties = {
|
||||
maxWidth: opt(int({ minimum: 1 })),
|
||||
maxHeight: opt(int({ minimum: 1 })),
|
||||
maxBytes: opt(int({ minimum: 1, description: "Maximum base64-encoded payload size in bytes." })),
|
||||
jpegQuality: opt(int({ minimum: 1, maximum: 100 })),
|
||||
};
|
||||
|
||||
function modelImageResizeOptions(options?: Annotations<StaticObject<typeof ModelImageResizeProperties>>) {
|
||||
return object(ModelImageResizeProperties, options);
|
||||
}
|
||||
|
||||
export const ModelImageResizeOptionsSchema = modelImageResizeOptions();
|
||||
|
||||
export const ModelImageInputLimitsSchema = Type.Object({
|
||||
resize: Type.Optional(
|
||||
export const ModelImageInputLimitsSchema = object({
|
||||
resize: opt(
|
||||
modelImageResizeOptions({
|
||||
description: "Cache-safe resize profile applied before a new image enters conversation history.",
|
||||
}),
|
||||
),
|
||||
maxPerMessage: Type.Optional(
|
||||
Type.Integer({ minimum: 1, description: "Maximum images accepted in one provider message." }),
|
||||
),
|
||||
maxPerRequest: Type.Optional(
|
||||
Type.Integer({ minimum: 1, description: "Maximum images accepted across one provider request." }),
|
||||
),
|
||||
maxPerMessage: opt(int({ minimum: 1, description: "Maximum images accepted in one provider message." })),
|
||||
maxPerRequest: opt(int({ minimum: 1, description: "Maximum images accepted across one provider request." })),
|
||||
});
|
||||
|
||||
export const ModelInputLimitsSchema = Type.Object({
|
||||
maxRequestBytes: Type.Optional(
|
||||
Type.Integer({ minimum: 1, description: "Maximum serialized provider request size in bytes." }),
|
||||
),
|
||||
images: Type.Optional(ModelImageInputLimitsSchema),
|
||||
export const ModelInputLimitsSchema = object({
|
||||
maxRequestBytes: opt(int({ minimum: 1, description: "Maximum serialized provider request size in bytes." })),
|
||||
images: opt(ModelImageInputLimitsSchema),
|
||||
});
|
||||
|
||||
export type ThinkingLevel = Static<typeof ThinkingLevelSchema>;
|
||||
|
|
|
|||
34
packages/ai/src/providers/schema-builders.ts
Normal file
34
packages/ai/src/providers/schema-builders.ts
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
// Builders for pi-ai's own configuration schemas (model metadata, compat settings): pi-schema's builders, shaped like the
|
||||
// TypeBox schemas they replace, so models.json and the catalog validate exactly as before.
|
||||
|
||||
import {
|
||||
type Annotations,
|
||||
obj,
|
||||
type Properties,
|
||||
type Schema,
|
||||
type Static,
|
||||
type StaticObject,
|
||||
} from "@earendil-works/pi-schema";
|
||||
|
||||
/** An object that allows other properties; `additionalProperties: true` says so explicitly. */
|
||||
export function openObject<const P extends Properties>(
|
||||
properties: P,
|
||||
options: Annotations<StaticObject<P>> & { readonly additionalProperties?: true } = {},
|
||||
): Schema<StaticObject<P>> {
|
||||
const { additionalProperties, ...annotations } = options;
|
||||
const schema = obj(properties, { ...annotations, open: true });
|
||||
return additionalProperties ? { ...schema, additionalProperties } : schema;
|
||||
}
|
||||
|
||||
/** One string: `{ type: "string", const: value }`. */
|
||||
export function stringLiteral<const V extends string>(value: V, options: Annotations<V> = {}): Schema<V> {
|
||||
return { type: "string", const: value, ...options };
|
||||
}
|
||||
|
||||
/** An object with any string keys and values of `value`: `{ type: "object", patternProperties: { "^.*$": value } }`. */
|
||||
export function stringRecord<S extends Schema>(
|
||||
value: S,
|
||||
options: Annotations<Record<string, Static<S>>> = {},
|
||||
): Schema<Record<string, Static<S>>> {
|
||||
return { type: "object", patternProperties: { "^.*$": value }, ...options };
|
||||
}
|
||||
|
|
@ -742,8 +742,6 @@ export interface ClassifierResult {
|
|||
timestamp: number; // Unix timestamp in milliseconds
|
||||
}
|
||||
|
||||
import type { TSchema } from "typebox";
|
||||
|
||||
/** OpenAI grammar variants for constrained sampling. */
|
||||
export type GrammarFormat = "openai_lark" | "openai_regex";
|
||||
|
||||
|
|
@ -766,7 +764,11 @@ export type ConstrainedSamplingConfig =
|
|||
variants: GrammarVariants;
|
||||
};
|
||||
|
||||
export interface Tool<TParameters extends TSchema = TSchema> {
|
||||
/**
|
||||
* A tool the model may call. `parameters` is the JSON Schema of its arguments: pi-schema's builders (which also type
|
||||
* the arguments, see `Static` in `@earendil-works/pi-schema`), TypeBox schemas, or any JSON Schema object.
|
||||
*/
|
||||
export interface Tool<TParameters extends object = object> {
|
||||
name: string;
|
||||
description: string;
|
||||
parameters: TParameters;
|
||||
|
|
|
|||
|
|
@ -1,24 +1,24 @@
|
|||
import { type TUnsafe, Type } from "typebox";
|
||||
import type { Schema } from "@earendil-works/pi-schema";
|
||||
|
||||
/**
|
||||
* Creates a string enum schema compatible with Google's API and other providers
|
||||
* that don't support anyOf/const patterns.
|
||||
* that don't support anyOf/const patterns: `{ type: "string", enum: [...] }`.
|
||||
*
|
||||
* @example
|
||||
* const OperationSchema = StringEnum(["add", "subtract", "multiply", "divide"], {
|
||||
* description: "The operation to perform"
|
||||
* });
|
||||
*
|
||||
* type Operation = Static<typeof OperationSchema>; // "add" | "subtract" | "multiply" | "divide"
|
||||
* type Operation = Static<typeof OperationSchema>; // "add" | "subtract" | "multiply" | "divide" (Static from @earendil-works/pi-schema)
|
||||
*/
|
||||
export function StringEnum<T extends readonly string[]>(
|
||||
values: T,
|
||||
options?: { description?: string; default?: T[number] },
|
||||
): TUnsafe<T[number]> {
|
||||
return Type.Unsafe<T[number]>({
|
||||
): Schema<T[number]> {
|
||||
return {
|
||||
type: "string",
|
||||
enum: values as any,
|
||||
enum: values,
|
||||
...(options?.description && { description: options.description }),
|
||||
...(options?.default && { default: options.default }),
|
||||
});
|
||||
};
|
||||
}
|
||||
|
|
@ -133,7 +133,7 @@ export function toToolDeclaration(tool: Tool): Tool {
|
|||
* Whether two tools declare the same interface to the model.
|
||||
*
|
||||
* Both sides go through {@link toToolDeclaration} first: its JSON round-trip drops the
|
||||
* typebox symbol keys and `undefined` fields that a structural comparison would see, and
|
||||
* schema metadata keys (TypeBox's non-enumerable kind marks) and `undefined` fields that a structural comparison would see, and
|
||||
* builds both objects with the same key order, so comparing the serialized declarations
|
||||
* is exact. This avoids a deep-equal dependency in a browser-safe package.
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -1,299 +1,34 @@
|
|||
import { Compile } from "typebox/compile";
|
||||
import type { TLocalizedValidationError } from "typebox/error";
|
||||
import { Value } from "typebox/value";
|
||||
import { type CoercingValidator, compile } from "@earendil-works/pi-schema";
|
||||
import { formats } from "@earendil-works/pi-schema/formats";
|
||||
import type { Tool, ToolCall } from "../types.ts";
|
||||
|
||||
const validatorCache = new WeakMap<object, ReturnType<typeof Compile>>();
|
||||
const TYPEBOX_KIND = Symbol.for("TypeBox.Kind");
|
||||
// One compiled validator per schema object: tool schemas are long-lived, so the WeakMap drops entries with their tools.
|
||||
const validatorCache = new WeakMap<object, CoercingValidator>();
|
||||
|
||||
interface JsonSchemaObject {
|
||||
type?: string | string[];
|
||||
properties?: Record<string, JsonSchemaObject>;
|
||||
required?: string[];
|
||||
items?: JsonSchemaObject | JsonSchemaObject[];
|
||||
additionalProperties?: boolean | JsonSchemaObject;
|
||||
allOf?: JsonSchemaObject[];
|
||||
anyOf?: JsonSchemaObject[];
|
||||
oneOf?: JsonSchemaObject[];
|
||||
}
|
||||
|
||||
function getSchemaTypes(schema: JsonSchemaObject): string[] {
|
||||
if (typeof schema.type === "string") {
|
||||
return [schema.type];
|
||||
}
|
||||
if (Array.isArray(schema.type)) {
|
||||
return schema.type.filter((type): type is string => typeof type === "string");
|
||||
}
|
||||
return [];
|
||||
}
|
||||
|
||||
function matchesJsonType(value: unknown, type: string): boolean {
|
||||
switch (type) {
|
||||
case "number":
|
||||
return typeof value === "number";
|
||||
case "integer":
|
||||
return typeof value === "number" && Number.isInteger(value);
|
||||
case "boolean":
|
||||
return typeof value === "boolean";
|
||||
case "string":
|
||||
return typeof value === "string";
|
||||
case "null":
|
||||
return value === null;
|
||||
case "array":
|
||||
return Array.isArray(value);
|
||||
case "object":
|
||||
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function getSubSchemaValidator(schema: JsonSchemaObject): ReturnType<typeof Compile> | undefined {
|
||||
try {
|
||||
return getValidator(schema as Tool["parameters"]);
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
function coercePrimitiveByType(value: unknown, type: string): unknown {
|
||||
switch (type) {
|
||||
case "number": {
|
||||
if (value === null) {
|
||||
return 0;
|
||||
}
|
||||
if (typeof value === "string" && value.trim() !== "") {
|
||||
const parsed = Number(value);
|
||||
if (Number.isFinite(parsed)) {
|
||||
return parsed;
|
||||
}
|
||||
}
|
||||
if (typeof value === "boolean") {
|
||||
return value ? 1 : 0;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
case "integer": {
|
||||
if (value === null) {
|
||||
return 0;
|
||||
}
|
||||
if (typeof value === "string" && value.trim() !== "") {
|
||||
const parsed = Number(value);
|
||||
if (Number.isInteger(parsed)) {
|
||||
return parsed;
|
||||
}
|
||||
}
|
||||
if (typeof value === "boolean") {
|
||||
return value ? 1 : 0;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
case "boolean": {
|
||||
if (value === null) {
|
||||
return false;
|
||||
}
|
||||
if (typeof value === "string") {
|
||||
if (value === "true") {
|
||||
return true;
|
||||
}
|
||||
if (value === "false") {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (typeof value === "number") {
|
||||
if (value === 1) {
|
||||
return true;
|
||||
}
|
||||
if (value === 0) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return value;
|
||||
}
|
||||
case "string": {
|
||||
if (value === null) {
|
||||
return "";
|
||||
}
|
||||
if (typeof value === "number" || typeof value === "boolean") {
|
||||
return String(value);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
case "null": {
|
||||
if (value === "" || value === 0 || value === false) {
|
||||
return null;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
default:
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
function applySchemaObjectCoercion(value: Record<string, unknown>, schema: JsonSchemaObject): void {
|
||||
const properties = schema.properties;
|
||||
const definedKeys = new Set<string>(properties ? Object.keys(properties) : []);
|
||||
|
||||
if (properties) {
|
||||
for (const [key, propertySchema] of Object.entries(properties)) {
|
||||
if (!(key in value)) {
|
||||
continue;
|
||||
}
|
||||
value[key] = coerceWithJsonSchema(value[key], propertySchema);
|
||||
}
|
||||
}
|
||||
|
||||
if (schema.additionalProperties && typeof schema.additionalProperties === "object") {
|
||||
for (const [key, propertyValue] of Object.entries(value)) {
|
||||
if (definedKeys.has(key)) {
|
||||
continue;
|
||||
}
|
||||
value[key] = coerceWithJsonSchema(propertyValue, schema.additionalProperties);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function applySchemaArrayCoercion(value: unknown[], schema: JsonSchemaObject): void {
|
||||
if (Array.isArray(schema.items)) {
|
||||
for (let index = 0; index < value.length; index++) {
|
||||
const itemSchema = schema.items[index];
|
||||
if (!itemSchema) {
|
||||
continue;
|
||||
}
|
||||
value[index] = coerceWithJsonSchema(value[index], itemSchema);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (schema.items && typeof schema.items === "object") {
|
||||
for (let index = 0; index < value.length; index++) {
|
||||
value[index] = coerceWithJsonSchema(value[index], schema.items);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function coerceWithUnionSchema(value: unknown, schemas: JsonSchemaObject[]): unknown {
|
||||
for (const schema of schemas) {
|
||||
const validator = getSubSchemaValidator(schema);
|
||||
if (validator?.Check(value)) {
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
||||
for (const schema of schemas) {
|
||||
const candidate = structuredClone(value);
|
||||
const coerced = coerceWithJsonSchema(candidate, schema);
|
||||
const validator = getSubSchemaValidator(schema);
|
||||
if (validator?.Check(coerced)) {
|
||||
return coerced;
|
||||
}
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function coerceWithJsonSchema(value: unknown, schema: JsonSchemaObject): unknown {
|
||||
let nextValue = value;
|
||||
|
||||
if (Array.isArray(schema.allOf)) {
|
||||
for (const nested of schema.allOf) {
|
||||
nextValue = coerceWithJsonSchema(nextValue, nested);
|
||||
}
|
||||
}
|
||||
|
||||
if (Array.isArray(schema.anyOf)) {
|
||||
nextValue = coerceWithUnionSchema(nextValue, schema.anyOf);
|
||||
}
|
||||
|
||||
if (Array.isArray(schema.oneOf)) {
|
||||
nextValue = coerceWithUnionSchema(nextValue, schema.oneOf);
|
||||
}
|
||||
|
||||
const schemaTypes = getSchemaTypes(schema);
|
||||
const matchesUnionMember =
|
||||
schemaTypes.length > 1 && schemaTypes.some((schemaType) => matchesJsonType(nextValue, schemaType));
|
||||
if (schemaTypes.length > 0 && !matchesUnionMember) {
|
||||
for (const schemaType of schemaTypes) {
|
||||
const candidate = coercePrimitiveByType(nextValue, schemaType);
|
||||
if (candidate !== nextValue) {
|
||||
nextValue = candidate;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (
|
||||
schemaTypes.includes("object") &&
|
||||
typeof nextValue === "object" &&
|
||||
nextValue !== null &&
|
||||
!Array.isArray(nextValue)
|
||||
) {
|
||||
applySchemaObjectCoercion(nextValue as Record<string, unknown>, schema);
|
||||
}
|
||||
|
||||
if (schemaTypes.includes("array") && Array.isArray(nextValue)) {
|
||||
applySchemaArrayCoercion(nextValue, schema);
|
||||
}
|
||||
|
||||
return nextValue;
|
||||
}
|
||||
|
||||
function normalizeOptionalNulls(value: unknown, schema: JsonSchemaObject): void {
|
||||
if (Array.isArray(value)) {
|
||||
if (Array.isArray(schema.items)) {
|
||||
for (let index = 0; index < value.length; index++) {
|
||||
const itemSchema = schema.items[index];
|
||||
if (itemSchema) normalizeOptionalNulls(value[index], itemSchema);
|
||||
}
|
||||
} else if (schema.items) {
|
||||
for (const item of value) normalizeOptionalNulls(item, schema.items);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (typeof value !== "object" || value === null || !schema.properties) return;
|
||||
|
||||
const object = value as Record<string, unknown>;
|
||||
const required = new Set(schema.required ?? []);
|
||||
for (const [key, propertySchema] of Object.entries(schema.properties)) {
|
||||
if (!(key in object)) continue;
|
||||
if (
|
||||
object[key] === null &&
|
||||
!required.has(key) &&
|
||||
typeof (propertySchema as { $ref?: unknown }).$ref !== "string" &&
|
||||
getSubSchemaValidator(propertySchema)?.Check(null) === false
|
||||
) {
|
||||
delete object[key];
|
||||
} else {
|
||||
normalizeOptionalNulls(object[key], propertySchema);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function getValidator(schema: Tool["parameters"]): ReturnType<typeof Compile> {
|
||||
const key = schema as object;
|
||||
const cached = validatorCache.get(key);
|
||||
function getValidator(schema: Tool["parameters"]): CoercingValidator {
|
||||
const cached = validatorCache.get(schema);
|
||||
if (cached) {
|
||||
return cached;
|
||||
}
|
||||
const validator = Compile(schema);
|
||||
validatorCache.set(key, validator);
|
||||
// A $ref that names nothing (a dangling definition in an MCP server's schema) rejects only values at its place,
|
||||
// instead of making every call of the tool fail.
|
||||
const validator = compile(schema, { formats, coerce: true, unresolvedRefs: "reject" });
|
||||
validatorCache.set(schema, validator);
|
||||
return validator;
|
||||
}
|
||||
|
||||
function formatValidationPath(error: TLocalizedValidationError): string {
|
||||
if (error.keyword === "required") {
|
||||
const requiredProperties = (error.params as { requiredProperties?: string[] }).requiredProperties;
|
||||
const requiredProperty = requiredProperties?.[0];
|
||||
if (requiredProperty) {
|
||||
const basePath = error.instancePath.replace(/^\//, "").replace(/\//g, ".");
|
||||
return basePath ? `${basePath}.${requiredProperty}` : requiredProperty;
|
||||
}
|
||||
}
|
||||
const path = error.instancePath.replace(/^\//, "").replace(/\//g, ".");
|
||||
return path || "root";
|
||||
/** A pi-schema error path (a JSON Pointer) as the dotted path the model sees: `/edits/0/oldText` -> `edits.0.oldText`. */
|
||||
function formatValidationPath(pointer: string): string {
|
||||
if (pointer === "") return "root";
|
||||
return pointer
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.map((segment) => segment.replace(/~1/g, "/").replace(/~0/g, "~"))
|
||||
.join(".");
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds a tool by name and validates the tool call arguments against its TypeBox schema
|
||||
* Finds a tool by name and validates the tool call arguments against its JSON Schema
|
||||
* @param tools Array of tool definitions
|
||||
* @param toolCall The tool call from the LLM
|
||||
* @returns The validated arguments
|
||||
|
|
@ -308,8 +43,10 @@ export function validateToolCall(tools: Tool[], toolCall: ToolCall): any {
|
|||
}
|
||||
|
||||
/**
|
||||
* Validates tool call arguments against the tool's TypeBox schema
|
||||
* @param tool The tool definition with TypeBox schema
|
||||
* Validates tool call arguments against the tool's JSON Schema (any JSON Schema: pi-schema's builders, TypeBox, plain
|
||||
* objects). The arguments are coerced first (pi-schema's `parse()`): models often send `"42"` for a number, a single
|
||||
* value for an array, or `null` for an omitted optional property.
|
||||
* @param tool The tool definition with its JSON Schema
|
||||
* @param toolCall The tool call from the LLM
|
||||
* @returns The validated (and potentially coerced) arguments
|
||||
* @throws Error with formatted message if validation fails
|
||||
|
|
@ -318,34 +55,13 @@ export function validateToolArguments(tool: Tool, toolCall: ToolCall): any {
|
|||
if (toolCall.argumentsError) {
|
||||
throw new Error(`Invalid arguments for tool "${toolCall.name}": ${toolCall.argumentsError}`);
|
||||
}
|
||||
const args = structuredClone(toolCall.arguments);
|
||||
normalizeOptionalNulls(args, tool.parameters as JsonSchemaObject);
|
||||
Value.Convert(tool.parameters, args);
|
||||
|
||||
const validator = getValidator(tool.parameters);
|
||||
if (!Object.getOwnPropertySymbols(tool.parameters).includes(TYPEBOX_KIND)) {
|
||||
const coerced = coerceWithJsonSchema(args, tool.parameters as JsonSchemaObject);
|
||||
if (coerced !== args) {
|
||||
if (typeof args === "object" && args !== null && typeof coerced === "object" && coerced !== null) {
|
||||
for (const key of Object.keys(args)) {
|
||||
delete args[key];
|
||||
}
|
||||
Object.assign(args, coerced);
|
||||
} else {
|
||||
return validator.Check(coerced) ? coerced : args;
|
||||
}
|
||||
}
|
||||
// A copy, so tools that modify their arguments never modify the tool call in the transcript.
|
||||
const result = getValidator(tool.parameters).parse(structuredClone(toolCall.arguments));
|
||||
if (result.ok) {
|
||||
return result.value;
|
||||
}
|
||||
|
||||
if (validator.Check(args)) {
|
||||
return args;
|
||||
}
|
||||
|
||||
const errors =
|
||||
validator
|
||||
.Errors(args)
|
||||
.map((error) => ` - ${formatValidationPath(error)}: ${error.message}`)
|
||||
.join("\n") || "Unknown validation error";
|
||||
const errors = result.errors.map((error) => ` - ${formatValidationPath(error.path)}: ${error.message}`).join("\n");
|
||||
|
||||
const errorMessage = `Validation failed for tool "${toolCall.name}":\n${errors}\n\nReceived arguments:\n${JSON.stringify(toolCall.arguments, null, 2)}`;
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,4 @@
|
|||
import type { Static } from "typebox";
|
||||
import { Compile } from "typebox/compile";
|
||||
import type { Static } from "@earendil-works/pi-schema";
|
||||
import { describe, expect, expectTypeOf, it } from "vitest";
|
||||
import {
|
||||
AnthropicMessagesCompatSchema,
|
||||
|
|
@ -11,6 +10,16 @@ import {
|
|||
|
||||
type ProviderCompat = Static<typeof ProviderCompatSchema>;
|
||||
|
||||
/** The `default` annotation of each property of an object schema. */
|
||||
function defaults(schema: { readonly [keyword: string]: unknown }): Record<string, unknown> {
|
||||
const properties = schema.properties as Record<string, { default?: unknown }>;
|
||||
return Object.fromEntries(
|
||||
Object.entries(properties)
|
||||
.filter(([, property]) => property.default !== undefined)
|
||||
.map(([name, property]) => [name, property.default]),
|
||||
);
|
||||
}
|
||||
|
||||
describe("compatibility schemas", () => {
|
||||
it("preserves property types in the provider superset", () => {
|
||||
expectTypeOf<ProviderCompat["supportsStore"]>().toEqualTypeOf<boolean | undefined>();
|
||||
|
|
@ -20,21 +29,21 @@ describe("compatibility schemas", () => {
|
|||
});
|
||||
|
||||
it("preserves API-specific defaults", () => {
|
||||
const completionsDefaults = Compile(OpenAICompletionsCompatSchema).Default({});
|
||||
const completionsDefaults = defaults(OpenAICompletionsCompatSchema);
|
||||
expect(completionsDefaults).not.toHaveProperty("thinkingFormat");
|
||||
expect(completionsDefaults).not.toHaveProperty("supportsLongCacheRetention");
|
||||
expect(Compile(OpenAIResponsesCompatSchema).Default({})).toMatchObject({ supportsDeveloperRole: true });
|
||||
const anthropicDefaults = Compile(AnthropicMessagesCompatSchema).Default({});
|
||||
expect(defaults(OpenAIResponsesCompatSchema)).toMatchObject({ supportsDeveloperRole: true });
|
||||
const anthropicDefaults = defaults(AnthropicMessagesCompatSchema);
|
||||
expect(anthropicDefaults).not.toHaveProperty("sendSessionAffinityHeaders");
|
||||
expect(anthropicDefaults).toMatchObject({ supportsLongCacheRetention: true });
|
||||
expect(Compile(BedrockCompatSchema).Default({})).toMatchObject({ supportsStrictMode: false });
|
||||
expect(defaults(BedrockCompatSchema)).toMatchObject({ supportsStrictMode: false });
|
||||
});
|
||||
|
||||
it("does not assign API-specific defaults to the provider superset", () => {
|
||||
const defaults = Compile(ProviderCompatSchema).Default({});
|
||||
expect(defaults).not.toHaveProperty("supportsDeveloperRole");
|
||||
expect(defaults).not.toHaveProperty("sendSessionAffinityHeaders");
|
||||
expect(defaults).not.toHaveProperty("supportsLongCacheRetention");
|
||||
expect(defaults).not.toHaveProperty("supportsStrictMode");
|
||||
const providerDefaults = defaults(ProviderCompatSchema);
|
||||
expect(providerDefaults).not.toHaveProperty("supportsDeveloperRole");
|
||||
expect(providerDefaults).not.toHaveProperty("sendSessionAffinityHeaders");
|
||||
expect(providerDefaults).not.toHaveProperty("supportsLongCacheRetention");
|
||||
expect(providerDefaults).not.toHaveProperty("supportsStrictMode");
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { Type } from "typebox";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import {
|
||||
complete,
|
||||
|
|
@ -7,7 +8,6 @@ import {
|
|||
fauxToolCall,
|
||||
registerFauxProvider,
|
||||
stream,
|
||||
Type,
|
||||
} from "../src/compat.ts";
|
||||
import type { AssistantMessageEvent, Context } from "../src/types.ts";
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import { describe, expect, it } from "vitest";
|
|||
import { completeSimple, getModel } from "../src/compat.ts";
|
||||
import { getEnvApiKey } from "../src/env-api-keys.ts";
|
||||
import type { Api, Context, Model, StopReason, Tool, ToolCall, ToolResultMessage } from "../src/types.ts";
|
||||
import { StringEnum } from "../src/utils/typebox-helpers.ts";
|
||||
import { StringEnum } from "../src/utils/string-enum.ts";
|
||||
import { hasBedrockCredentials } from "./bedrock-utils.ts";
|
||||
|
||||
const calculatorSchema = Type.Object({
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import type { Api, Context, ImageContent, Model, StreamOptions, Tool, ToolResult
|
|||
|
||||
type StreamOptionsWithExtras = StreamOptions & Record<string, unknown>;
|
||||
|
||||
import { StringEnum } from "../src/utils/typebox-helpers.ts";
|
||||
import { StringEnum } from "../src/utils/string-enum.ts";
|
||||
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.ts";
|
||||
import { hasBedrockCredentials } from "./bedrock-utils.ts";
|
||||
import { hasCloudflareAiGatewayCredentials, hasCloudflareWorkersAICredentials } from "./cloudflare-utils.ts";
|
||||
|
|
|
|||
779
packages/ai/test/validation-probes.test.ts
Normal file
779
packages/ai/test/validation-probes.test.ts
Normal file
|
|
@ -0,0 +1,779 @@
|
|||
// The reviewers' probes of the schema switch (round 1) as regression tests: TypeBox-built and plain JSON
|
||||
// (MCP-shaped) tool schemas with arguments through validateToolArguments. Each case's outcome is "ok <coerced
|
||||
// arguments>", "REJECT <first error line>" or "THROW <message>". Cases marked as TypeBox-only record the documented
|
||||
// differences from TypeBox (see the CHANGELOG): Refine and BigInt.
|
||||
|
||||
// biome-ignore-all lint/suspicious/noTemplateCurlyInString: TypeBox template literal syntax.
|
||||
// biome-ignore-all lint/suspicious/noThenProperty: JSON Schema's `then` keyword.
|
||||
import { Type } from "typebox";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { Tool, ToolCall } from "../src/types.ts";
|
||||
import { validateToolArguments } from "../src/utils/validation.ts";
|
||||
|
||||
/** The outcome of one call, in the probes' format. */
|
||||
function outcome(name: string, schema: object, args: unknown): string {
|
||||
const tool = { name, description: "", parameters: schema } as Tool;
|
||||
const call = { type: "toolCall", id: "1", name, arguments: args } as ToolCall;
|
||||
try {
|
||||
return `ok ${JSON.stringify(validateToolArguments(tool, call), (_key, value) => (typeof value === "bigint" ? `${value}n` : value))}`;
|
||||
} catch (error) {
|
||||
const message = (error as Error).message;
|
||||
return message.startsWith("Validation failed")
|
||||
? `REJECT ${message.split("\n")[1]?.trim()}`
|
||||
: `THROW ${message.split("\n")[0]}`;
|
||||
}
|
||||
}
|
||||
|
||||
type Case = { name: string; schema: object; args: unknown[] };
|
||||
|
||||
const Node = Type.Cyclic(
|
||||
{ Node: Type.Object({ name: Type.String(), children: Type.Array(Type.Ref("Node")) }) },
|
||||
"Node",
|
||||
);
|
||||
|
||||
const cases: Case[] = [
|
||||
{
|
||||
name: "cyclic",
|
||||
schema: Type.Object({ tree: Node }),
|
||||
args: [{ tree: { name: "a", children: [{ name: "b", children: [] }] } }, { tree: { name: 1, children: [] } }],
|
||||
},
|
||||
{ name: "ref-with-$id", schema: Type.Object({ a: Type.Ref("A") }), args: [{ a: { x: "1" } }] },
|
||||
{
|
||||
name: "function",
|
||||
schema: Type.Object({ f: Type.Optional(Type.Function([], Type.Void())) }),
|
||||
args: [{}, { f: "x" }],
|
||||
},
|
||||
{
|
||||
name: "codec",
|
||||
schema: Type.Object({
|
||||
c: Type.Codec(Type.String())
|
||||
.Decode((s) => Number(s))
|
||||
.Encode((n) => String(n)),
|
||||
}),
|
||||
args: [{ c: "12" }, { c: 12 }],
|
||||
},
|
||||
{ name: "undefined-optional", schema: Type.Object({ u: Type.Optional(Type.Undefined()) }), args: [{}, { u: null }] },
|
||||
{ name: "bigint", schema: Type.Object({ n: Type.BigInt() }), args: [{ n: 1 }, { n: "12" }] },
|
||||
{
|
||||
name: "refine",
|
||||
schema: Type.Object({ n: Type.Refine(Type.Number(), (v) => v > 0) }),
|
||||
args: [{ n: 5 }, { n: -5 }],
|
||||
},
|
||||
{
|
||||
name: "template-literal",
|
||||
schema: Type.Object({ t: Type.TemplateLiteral("id-${number}") }),
|
||||
args: [{ t: "id-12" }, { t: "id-x" }, { t: 3 }],
|
||||
},
|
||||
{
|
||||
name: "record-string",
|
||||
schema: Type.Object({ r: Type.Record(Type.String(), Type.Number()) }),
|
||||
args: [{ r: { a: "1" } }, { r: { a: 1, b: "x" } }],
|
||||
},
|
||||
{
|
||||
name: "record-literal-keys",
|
||||
schema: Type.Object({ r: Type.Record(Type.Union([Type.Literal("a"), Type.Literal("b")]), Type.Number()) }),
|
||||
args: [{ r: { a: 1, b: 2 } }, { r: { a: 1 } }, { r: { a: 1, b: 2, c: 3 } }],
|
||||
},
|
||||
{
|
||||
name: "intersect",
|
||||
schema: Type.Intersect([Type.Object({ a: Type.Number() }), Type.Object({ b: Type.Boolean() })]),
|
||||
args: [
|
||||
{ a: "1", b: "true" },
|
||||
{ a: 1, b: true, c: 1 },
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "union-objects",
|
||||
schema: Type.Object({
|
||||
v: Type.Union([
|
||||
Type.Object({ k: Type.Literal("x"), n: Type.Number() }),
|
||||
Type.Object({ k: Type.Literal("y"), s: Type.String() }),
|
||||
]),
|
||||
}),
|
||||
args: [{ v: { k: "x", n: "2" } }, { v: { k: "y", s: 2 } }],
|
||||
},
|
||||
{ name: "minLength-astral", schema: Type.Object({ s: Type.String({ minLength: 2 }) }), args: [{ s: "😀" }] },
|
||||
{ name: "maxLength-astral", schema: Type.Object({ s: Type.String({ maxLength: 1 }) }), args: [{ s: "😀" }] },
|
||||
{ name: "pattern-astral", schema: Type.Object({ s: Type.String({ pattern: "^.$" }) }), args: [{ s: "😀" }] },
|
||||
{
|
||||
name: "pattern-class-dash",
|
||||
schema: Type.Object({ s: Type.String({ pattern: "^[\\w-]+$" }) }),
|
||||
args: [{ s: "a-b" }],
|
||||
},
|
||||
{
|
||||
name: "pattern-unicode-prop",
|
||||
schema: Type.Object({ s: Type.String({ pattern: "^\\p{L}+$" }) }),
|
||||
args: [{ s: "äb" }, { s: "p{L}" }],
|
||||
},
|
||||
{
|
||||
name: "multipleOf-float",
|
||||
schema: Type.Object({ n: Type.Number({ multipleOf: 0.01 }) }),
|
||||
args: [{ n: 0.07 }, { n: 1.1 }],
|
||||
},
|
||||
{
|
||||
name: "uniqueItems-keyorder",
|
||||
schema: Type.Object({ a: Type.Array(Type.Unknown(), { uniqueItems: true }) }),
|
||||
args: [
|
||||
{
|
||||
a: [
|
||||
{ x: 1, y: 2 },
|
||||
{ y: 2, x: 1 },
|
||||
],
|
||||
},
|
||||
{ a: [1, 1.0] },
|
||||
],
|
||||
},
|
||||
{ name: "const-object", schema: Type.Object({ c: Type.Literal("a") }), args: [{ c: "a" }] },
|
||||
{
|
||||
name: "formats",
|
||||
schema: Type.Object({
|
||||
e: Type.Optional(Type.String({ format: "email" })),
|
||||
u: Type.Optional(Type.String({ format: "uri" })),
|
||||
d: Type.Optional(Type.String({ format: "date-time" })),
|
||||
x: Type.Optional(Type.String({ format: "made-up" })),
|
||||
h: Type.Optional(Type.String({ format: "hostname" })),
|
||||
r: Type.Optional(Type.String({ format: "regex" })),
|
||||
i: Type.Optional(Type.String({ format: "uuid" })),
|
||||
ip: Type.Optional(Type.String({ format: "ipv4" })),
|
||||
}),
|
||||
args: [
|
||||
{ e: "a@b.co" },
|
||||
{ e: "not-an-email" },
|
||||
{ u: "relative/path" },
|
||||
{ u: "https://x.y/z" },
|
||||
{ d: "2024-01-01T00:00:00Z" },
|
||||
{ d: "2024-01-01" },
|
||||
{ x: "anything" },
|
||||
{ h: "under_score.example" },
|
||||
{ r: "[" },
|
||||
{ i: "00000000-0000-0000-0000-000000000000" },
|
||||
{ i: "not" },
|
||||
{ ip: "999.1.1.1" },
|
||||
],
|
||||
},
|
||||
{ name: "integer-big", schema: Type.Object({ n: Type.Integer() }), args: [{ n: 1e21 }, { n: 2 ** 53 + 2 }] },
|
||||
{
|
||||
name: "additional-false",
|
||||
schema: Type.Object({ a: Type.String() }, { additionalProperties: false }),
|
||||
args: [{ a: "x", b: 1 }],
|
||||
},
|
||||
{
|
||||
name: "prototype-key",
|
||||
schema: Type.Object({ constructor: Type.Optional(Type.String()), toString: Type.Optional(Type.Number()) }),
|
||||
args: [{}, { constructor: 1 }],
|
||||
},
|
||||
{
|
||||
name: "proto-key-value",
|
||||
schema: Type.Object({}, { additionalProperties: Type.Array(Type.String()) }),
|
||||
args: [JSON.parse('{"__proto__": "x", "a": "y"}')],
|
||||
},
|
||||
// Plain JSON (MCP-shaped)
|
||||
{
|
||||
name: "mcp-draft7-ref-definitions",
|
||||
schema: {
|
||||
$schema: "http://json-schema.org/draft-07/schema#",
|
||||
type: "object",
|
||||
properties: { t: { $ref: "#/definitions/T" } },
|
||||
required: ["t"],
|
||||
definitions: { T: { type: "object", properties: { path: { type: "string" } }, required: ["path"] } },
|
||||
},
|
||||
args: [{ t: { path: "a" } }, { t: { path: 42 } }],
|
||||
},
|
||||
{
|
||||
name: "mcp-ref-unresolved",
|
||||
schema: { type: "object", properties: { t: { $ref: "#/definitions/Missing" } } },
|
||||
args: [{ t: 1 }, {}],
|
||||
},
|
||||
{
|
||||
name: "mcp-ref-external",
|
||||
schema: { type: "object", properties: { t: { $ref: "https://example.com/schemas/t.json" } } },
|
||||
args: [{ t: 1 }],
|
||||
},
|
||||
{
|
||||
name: "mcp-nullable-openapi",
|
||||
schema: { type: "object", properties: { s: { type: "string", nullable: true } }, required: ["s"] },
|
||||
args: [{ s: null }, { s: "x" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-draft7-tuple",
|
||||
schema: {
|
||||
$schema: "http://json-schema.org/draft-07/schema#",
|
||||
type: "object",
|
||||
properties: { p: { type: "array", items: [{ type: "number" }, { type: "string" }], additionalItems: false } },
|
||||
},
|
||||
args: [{ p: [1, "a"] }, { p: [1, "a", 3] }, { p: ["1", 2] }],
|
||||
},
|
||||
{
|
||||
name: "mcp-draft7-ref-sibling",
|
||||
schema: {
|
||||
$schema: "http://json-schema.org/draft-07/schema#",
|
||||
type: "object",
|
||||
properties: { s: { $ref: "#/definitions/S", maxLength: 1 } },
|
||||
definitions: { S: { type: "string" } },
|
||||
},
|
||||
args: [{ s: "abc" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-draft4-exclusive-bool",
|
||||
schema: {
|
||||
$schema: "http://json-schema.org/draft-04/schema#",
|
||||
type: "object",
|
||||
properties: { n: { type: "number", minimum: 0, exclusiveMinimum: true } },
|
||||
},
|
||||
args: [{ n: 0 }, { n: 1 }],
|
||||
},
|
||||
{
|
||||
name: "mcp-id-draft4",
|
||||
schema: {
|
||||
$schema: "http://json-schema.org/draft-04/schema#",
|
||||
id: "http://x/y",
|
||||
type: "object",
|
||||
properties: { a: { type: "string" } },
|
||||
},
|
||||
args: [{ a: "x" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-bad-pattern",
|
||||
schema: { type: "object", properties: { s: { type: "string", pattern: "(?<!x)y" } } },
|
||||
args: [{ s: "y" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-invalid-regex",
|
||||
schema: { type: "object", properties: { s: { type: "string", pattern: "[" } } },
|
||||
args: [{ s: "y" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-type-integer-float-0",
|
||||
schema: { type: "object", properties: { n: { type: "integer" } } },
|
||||
args: [{ n: 1.0 }, { n: 1.5 }],
|
||||
},
|
||||
{
|
||||
name: "mcp-format-unknown",
|
||||
schema: { type: "object", properties: { s: { type: "string", format: "binary" } } },
|
||||
args: [{ s: "xx" }],
|
||||
},
|
||||
{
|
||||
name: "mcp-required-null-string",
|
||||
schema: { type: "object", properties: { q: { type: "string" } }, required: ["q"] },
|
||||
args: [{ q: null }],
|
||||
},
|
||||
{
|
||||
name: "mcp-array-json-string",
|
||||
schema: { type: "object", properties: { p: { type: "array", items: { type: "string" } } }, required: ["p"] },
|
||||
args: [{ p: '["a","b"]' }],
|
||||
},
|
||||
{
|
||||
name: "mcp-integer-float",
|
||||
schema: { type: "object", properties: { n: { type: "integer" } }, required: ["n"] },
|
||||
args: [{ n: 3.7 }, { n: "3.7" }],
|
||||
},
|
||||
{ name: "mcp-empty-schema", schema: {}, args: [{ a: 1 }] },
|
||||
{ name: "mcp-type-object-no-props", schema: { type: "object" }, args: [{ a: 1 }] },
|
||||
{
|
||||
name: "mcp-boolean-true-prop",
|
||||
schema: { type: "object", properties: { a: true, b: false } },
|
||||
args: [{ a: 1 }, { b: 1 }],
|
||||
},
|
||||
{
|
||||
name: "mcp-huge-enum",
|
||||
schema: {
|
||||
type: "object",
|
||||
properties: { e: { type: "string", enum: Array.from({ length: 5000 }, (_, i) => `v${i}`) } },
|
||||
},
|
||||
args: [{ e: "v4999" }, { e: "v5000" }],
|
||||
},
|
||||
];
|
||||
|
||||
const o = (properties: Record<string, unknown>, extra: Record<string, unknown> = {}) => ({
|
||||
type: "object",
|
||||
properties,
|
||||
...extra,
|
||||
});
|
||||
const keywordCases: [string, object, unknown[]][] = [
|
||||
[
|
||||
"oneOf-overlap",
|
||||
o({ v: { oneOf: [{ type: "string" }, { type: "string", maxLength: 5 }] } }),
|
||||
[{ v: "abc" }, { v: "abcdefgh" }],
|
||||
],
|
||||
["oneOf-number-integer", o({ v: { oneOf: [{ type: "number" }, { type: "integer" }] } }), [{ v: 2 }, { v: 2.5 }]],
|
||||
[
|
||||
"oneOf-objects-open",
|
||||
o({
|
||||
v: {
|
||||
oneOf: [
|
||||
{ type: "object", properties: { a: { type: "string" } } },
|
||||
{ type: "object", properties: { b: { type: "string" } } },
|
||||
],
|
||||
},
|
||||
}),
|
||||
[{ v: { a: "x" } }, { v: {} }],
|
||||
],
|
||||
[
|
||||
"if-then-else",
|
||||
o(
|
||||
{ kind: { type: "string" }, n: { type: "number" } },
|
||||
{ if: { properties: { kind: { const: "a" } } }, then: { required: ["n"] } },
|
||||
),
|
||||
[{ kind: "a" }, { kind: "b" }],
|
||||
],
|
||||
[
|
||||
"dependentRequired",
|
||||
o({ a: { type: "string" }, b: { type: "string" } }, { dependentRequired: { a: ["b"] } }),
|
||||
[{ a: "x" }],
|
||||
],
|
||||
[
|
||||
"dependencies-d7",
|
||||
o({ a: { type: "string" }, b: { type: "string" } }, { dependencies: { a: ["b"] } }),
|
||||
[{ a: "x" }],
|
||||
],
|
||||
["propertyNames", o({}, { propertyNames: { pattern: "^[a-z]+$" } }), [{ abc: 1 }, { ABC: 1 }]],
|
||||
["contains", o({ l: { type: "array", contains: { type: "number" } } }), [{ l: ["a"] }, { l: ["a", 1] }]],
|
||||
["not", o({ s: { not: { type: "null" } } }), [{ s: "x" }, { s: null }]],
|
||||
["minProperties", o({ m: { type: "object", minProperties: 1 } }), [{ m: {} }]],
|
||||
[
|
||||
"unevaluatedProperties",
|
||||
o({ a: { type: "string" } }, { unevaluatedProperties: false }),
|
||||
[{ a: "x" }, { a: "x", b: 1 }],
|
||||
],
|
||||
[
|
||||
"allOf-additional-false",
|
||||
{ allOf: [o({ a: { type: "string" } }, { additionalProperties: false }), o({ b: { type: "string" } })] },
|
||||
[{ a: "x" }, { a: "x", b: "y" }],
|
||||
],
|
||||
["const-object", o({ c: { const: { x: 1, y: 2 } } }), [{ c: { y: 2, x: 1 } }]],
|
||||
["enum-object", o({ e: { enum: [{ x: 1, y: 2 }] } }), [{ e: { y: 2, x: 1 } }]],
|
||||
["type-array-multi", o({ v: { type: ["string", "number"] } }), [{ v: true }, { v: null }]],
|
||||
["nullable-type-array", o({ v: { type: ["string", "null"] } }, { required: ["v"] }), [{ v: null }, { v: 5 }]],
|
||||
["exclusiveMinimum-number", o({ n: { type: "number", exclusiveMinimum: 0 } }), [{ n: 0 }, { n: "0" }]],
|
||||
["maxItems", o({ l: { type: "array", maxItems: 1 } }), [{ l: [1, 2] }, { l: "x" }]],
|
||||
[
|
||||
"discriminator-openapi",
|
||||
o({
|
||||
p: {
|
||||
oneOf: [
|
||||
o({ kind: { type: "string", enum: ["a"] }, x: { type: "number" } }, { required: ["kind"] }),
|
||||
o({ kind: { type: "string", enum: ["b"] } }, { required: ["kind"] }),
|
||||
],
|
||||
discriminator: { propertyName: "kind" },
|
||||
},
|
||||
}),
|
||||
[{ p: { kind: "a", x: "1" } }, { p: { kind: "b" } }],
|
||||
],
|
||||
[
|
||||
"ref-root-defs",
|
||||
{ $ref: "#/$defs/Root", $defs: { Root: o({ a: { type: "string" } }, { required: ["a"] }) } },
|
||||
[{ a: "x" }, { a: 1 }, {}],
|
||||
],
|
||||
[
|
||||
"ref-recursive",
|
||||
{
|
||||
$defs: { N: o({ c: { type: "array", items: { $ref: "#/$defs/N" } }, v: { type: "integer" } }) },
|
||||
...o({ root: { $ref: "#/$defs/N" } }),
|
||||
},
|
||||
[{ root: { v: "2", c: [{ v: "3", c: [] }] } }],
|
||||
],
|
||||
[
|
||||
"ref-percent",
|
||||
{ ...o({ a: { $ref: "#/$defs/a%20b" } }), $defs: { "a b": { type: "string" } } },
|
||||
[{ a: "x" }, { a: 1 }],
|
||||
],
|
||||
["ref-tilde", { ...o({ a: { $ref: "#/$defs/a~1b" } }), $defs: { "a/b": { type: "string" } } }, [{ a: "x" }]],
|
||||
[
|
||||
"defs-with-$id",
|
||||
{ ...o({ a: { $ref: "item.json" } }), $defs: { item: { $id: "item.json", type: "string" } } },
|
||||
[{ a: "x" }, { a: 2 }],
|
||||
],
|
||||
[
|
||||
"anyOf-null-default",
|
||||
o({ v: { anyOf: [{ type: "string" }, { type: "null" }], default: null } }),
|
||||
[{ v: null }, {}],
|
||||
],
|
||||
[
|
||||
"format-date-time-lower",
|
||||
o({ d: { type: "string", format: "date-time" } }),
|
||||
[{ d: "2024-01-01t00:00:00z" }, { d: "2024-01-01 00:00:00Z" }],
|
||||
],
|
||||
["format-url", o({ u: { type: "string", format: "url" } }), [{ u: "not a url" }]],
|
||||
["integer-as-float-1e0", o({ n: { type: "integer" } }), [{ n: 1e3 }]],
|
||||
["required-missing-props", { type: "object", required: ["a"] }, [{}, { a: null }]],
|
||||
["string-to-number-trim", o({ n: { type: "number" } }), [{ n: " 12 " }, { n: "0x10" }, { n: "1e3" }, { n: "" }]],
|
||||
];
|
||||
|
||||
const tree = {
|
||||
type: "object",
|
||||
properties: { t: { $ref: "#/$defs/Tree" } },
|
||||
$defs: { Tree: { anyOf: [{ type: "string" }, { type: "array", items: { $ref: "#/$defs/Tree" } }] } },
|
||||
};
|
||||
const json = {
|
||||
type: "object",
|
||||
properties: { data: { $ref: "#/$defs/Json" } },
|
||||
required: ["data"],
|
||||
$defs: {
|
||||
Json: {
|
||||
anyOf: [
|
||||
{ type: "string" },
|
||||
{ type: "number" },
|
||||
{ type: "boolean" },
|
||||
{ type: "array", items: { $ref: "#/$defs/Json" } },
|
||||
{ type: "object", additionalProperties: { $ref: "#/$defs/Json" } },
|
||||
],
|
||||
},
|
||||
},
|
||||
};
|
||||
const filter = {
|
||||
type: "object",
|
||||
properties: { where: { $ref: "#/$defs/F" } },
|
||||
$defs: {
|
||||
F: {
|
||||
oneOf: [
|
||||
{
|
||||
type: "object",
|
||||
properties: { field: { type: "string" }, eq: { type: "string" } },
|
||||
required: ["field", "eq"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
{
|
||||
type: "object",
|
||||
properties: { and: { type: "array", items: { $ref: "#/$defs/F" } } },
|
||||
required: ["and"],
|
||||
additionalProperties: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
};
|
||||
const recursionCases: [string, object, unknown][] = [
|
||||
["tree valid", tree, { t: ["a", ["b"]] }],
|
||||
["tree number", tree, { t: 5 }],
|
||||
["tree [5]", tree, { t: [5] }],
|
||||
["json valid", json, { data: { a: [1, "x", { b: true }] } }],
|
||||
["json null", json, { data: null }],
|
||||
["json nested null", json, { data: { a: [1, null] } }],
|
||||
["filter valid", filter, { where: { and: [{ field: "a", eq: "b" }] } }],
|
||||
["filter bad leaf", filter, { where: { and: [{ field: "a", eq: 1, x: 2 }] } }],
|
||||
["filter deep bad", filter, { where: { and: [{ and: [{ and: [{ and: [{ and: [{ field: 1 }] }] }] }] }] } }],
|
||||
];
|
||||
|
||||
const expected: Record<string, string> = {
|
||||
'cyclic {"tree":{"name":"a","children":[{"name":"b","children":[]}]}}':
|
||||
'ok {"tree":{"name":"a","children":[{"name":"b","children":[]}]}}',
|
||||
'cyclic {"tree":{"name":1,"children":[]}}': 'ok {"tree":{"name":"1","children":[]}}',
|
||||
'ref-with-$id {"a":{"x":"1"}}': "REJECT - a: unresolved reference A",
|
||||
"function {}": "ok {}",
|
||||
'function {"f":"x"}': "REJECT - f: expected function, got string",
|
||||
'codec {"c":"12"}': 'ok {"c":"12"}',
|
||||
'codec {"c":12}': 'ok {"c":"12"}',
|
||||
"undefined-optional {}": "ok {}",
|
||||
'undefined-optional {"u":null}': "ok {}",
|
||||
'bigint {"n":1}': "REJECT - n: expected bigint, got integer",
|
||||
'bigint {"n":"12"}': "REJECT - n: expected bigint, got string",
|
||||
'refine {"n":5}': 'ok {"n":5}',
|
||||
'refine {"n":-5}': 'ok {"n":-5}',
|
||||
'template-literal {"t":"id-12"}': 'ok {"t":"id-12"}',
|
||||
'template-literal {"t":"id-x"}': "REJECT - t: expected to match ^id--?(?:0|[1-9][0-9]*)(?:\\.[0-9]+)?$",
|
||||
'template-literal {"t":3}': "REJECT - t: expected to match ^id--?(?:0|[1-9][0-9]*)(?:\\.[0-9]+)?$",
|
||||
'record-string {"r":{"a":"1"}}': 'ok {"r":{"a":1}}',
|
||||
'record-string {"r":{"a":1,"b":"x"}}': "REJECT - r.b: expected number, got string",
|
||||
'record-literal-keys {"r":{"a":1,"b":2}}': 'ok {"r":{"a":1,"b":2}}',
|
||||
'record-literal-keys {"r":{"a":1}}': "REJECT - r.b: missing",
|
||||
'record-literal-keys {"r":{"a":1,"b":2,"c":3}}': 'ok {"r":{"a":1,"b":2,"c":3}}',
|
||||
'intersect {"a":"1","b":"true"}': 'ok {"a":1,"b":true}',
|
||||
'intersect {"a":1,"b":true,"c":1}': 'ok {"a":1,"b":true,"c":1}',
|
||||
'union-objects {"v":{"k":"x","n":"2"}}': 'ok {"v":{"k":"x","n":2}}',
|
||||
'union-objects {"v":{"k":"y","s":2}}': 'ok {"v":{"k":"y","s":"2"}}',
|
||||
'minLength-astral {"s":"😀"}': "REJECT - s: expected at least 2 characters",
|
||||
'maxLength-astral {"s":"😀"}': 'ok {"s":"😀"}',
|
||||
'pattern-astral {"s":"😀"}': 'ok {"s":"😀"}',
|
||||
'pattern-class-dash {"s":"a-b"}': 'ok {"s":"a-b"}',
|
||||
'pattern-unicode-prop {"s":"äb"}': 'ok {"s":"äb"}',
|
||||
'pattern-unicode-prop {"s":"p{L}"}': "REJECT - s: expected to match ^\\p{L}+$",
|
||||
'multipleOf-float {"n":0.07}': 'ok {"n":0.07}',
|
||||
'multipleOf-float {"n":1.1}': 'ok {"n":1.1}',
|
||||
'uniqueItems-keyorder {"a":[{"x":1,"y":2},{"y":2,"x":1}]}': "REJECT - a: expected unique items",
|
||||
'uniqueItems-keyorder {"a":[1,1]}': "REJECT - a: expected unique items",
|
||||
'const-object {"c":"a"}': 'ok {"c":"a"}',
|
||||
'formats {"e":"a@b.co"}': 'ok {"e":"a@b.co"}',
|
||||
'formats {"e":"not-an-email"}': "REJECT - e: expected format email",
|
||||
'formats {"u":"relative/path"}': "REJECT - u: expected format uri",
|
||||
'formats {"u":"https://x.y/z"}': 'ok {"u":"https://x.y/z"}',
|
||||
'formats {"d":"2024-01-01T00:00:00Z"}': 'ok {"d":"2024-01-01T00:00:00Z"}',
|
||||
'formats {"d":"2024-01-01"}': "REJECT - d: expected format date-time",
|
||||
'formats {"x":"anything"}': 'ok {"x":"anything"}',
|
||||
'formats {"h":"under_score.example"}': "REJECT - h: expected format hostname",
|
||||
'formats {"r":"["}': "REJECT - r: expected format regex",
|
||||
'formats {"i":"00000000-0000-0000-0000-000000000000"}': 'ok {"i":"00000000-0000-0000-0000-000000000000"}',
|
||||
'formats {"i":"not"}': "REJECT - i: expected format uuid",
|
||||
'formats {"ip":"999.1.1.1"}': "REJECT - ip: expected format ipv4",
|
||||
'integer-big {"n":1e+21}': 'ok {"n":1e+21}',
|
||||
'integer-big {"n":9007199254740994}': 'ok {"n":9007199254740994}',
|
||||
'additional-false {"a":"x","b":1}': "REJECT - b: unknown field",
|
||||
"prototype-key {}": "ok {}",
|
||||
'prototype-key {"constructor":1}': 'ok {"constructor":"1"}',
|
||||
'proto-key-value {"__proto__":"x","a":"y"}': 'ok {"__proto__":["x"],"a":["y"]}',
|
||||
'mcp-draft7-ref-definitions {"t":{"path":"a"}}': 'ok {"t":{"path":"a"}}',
|
||||
'mcp-draft7-ref-definitions {"t":{"path":42}}': 'ok {"t":{"path":"42"}}',
|
||||
'mcp-ref-unresolved {"t":1}': "REJECT - t: unresolved reference #/definitions/Missing",
|
||||
"mcp-ref-unresolved {}": "ok {}",
|
||||
'mcp-ref-external {"t":1}': "REJECT - t: unresolved reference https://example.com/schemas/t.json",
|
||||
'mcp-nullable-openapi {"s":null}': 'ok {"s":"null"}',
|
||||
'mcp-nullable-openapi {"s":"x"}': 'ok {"s":"x"}',
|
||||
'mcp-draft7-tuple {"p":[1,"a"]}': 'ok {"p":[1,"a"]}',
|
||||
'mcp-draft7-tuple {"p":[1,"a",3]}': "REJECT - p.2: no value is allowed here",
|
||||
'mcp-draft7-tuple {"p":["1",2]}': 'ok {"p":[1,"2"]}',
|
||||
'mcp-draft7-ref-sibling {"s":"abc"}': "REJECT - s: expected at most 1 characters",
|
||||
'mcp-draft4-exclusive-bool {"n":0}': 'ok {"n":0}',
|
||||
'mcp-draft4-exclusive-bool {"n":1}': 'ok {"n":1}',
|
||||
'mcp-id-draft4 {"a":"x"}': 'ok {"a":"x"}',
|
||||
'mcp-bad-pattern {"s":"y"}': 'ok {"s":"y"}',
|
||||
'mcp-invalid-regex {"s":"y"}': "THROW Invalid regular expression: /[/: Unterminated character class",
|
||||
'mcp-type-integer-float-0 {"n":1}': 'ok {"n":1}',
|
||||
'mcp-type-integer-float-0 {"n":1.5}': 'ok {"n":1}',
|
||||
'mcp-format-unknown {"s":"xx"}': 'ok {"s":"xx"}',
|
||||
'mcp-required-null-string {"q":null}': 'ok {"q":"null"}',
|
||||
'mcp-array-json-string {"p":"[\\"a\\",\\"b\\"]"}': 'ok {"p":["[\\"a\\",\\"b\\"]"]}',
|
||||
'mcp-integer-float {"n":3.7}': 'ok {"n":3}',
|
||||
'mcp-integer-float {"n":"3.7"}': 'ok {"n":3}',
|
||||
'mcp-empty-schema {"a":1}': 'ok {"a":1}',
|
||||
'mcp-type-object-no-props {"a":1}': 'ok {"a":1}',
|
||||
'mcp-boolean-true-prop {"a":1}': 'ok {"a":1}',
|
||||
'mcp-boolean-true-prop {"b":1}': "REJECT - b: no value is allowed here",
|
||||
'mcp-huge-enum {"e":"v4999"}': 'ok {"e":"v4999"}',
|
||||
'mcp-huge-enum {"e":"v5000"}':
|
||||
'REJECT - e: expected one of ["v0","v1","v2","v3","v4","v5","v6","v7","v8","v9","v10","v11","v12","v13","v14","v15","v16","v17","v18","v19", ... and 4980 more]',
|
||||
'oneOf-overlap {"v":"abc"}': "REJECT - v: matches more than one alternative",
|
||||
'oneOf-overlap {"v":"abcdefgh"}': 'ok {"v":"abcdefgh"}',
|
||||
'oneOf-number-integer {"v":2}': "REJECT - v: matches more than one alternative",
|
||||
'oneOf-number-integer {"v":2.5}': 'ok {"v":2.5}',
|
||||
'oneOf-objects-open {"v":{"a":"x"}}': "REJECT - v: matches more than one alternative",
|
||||
'oneOf-objects-open {"v":{}}': "REJECT - v: matches more than one alternative",
|
||||
'if-then-else {"kind":"a"}': "REJECT - n: missing",
|
||||
'if-then-else {"kind":"b"}': 'ok {"kind":"b"}',
|
||||
'dependentRequired {"a":"x"}': "REJECT - b: required when a is present",
|
||||
'dependencies-d7 {"a":"x"}': "REJECT - b: required when a is present",
|
||||
'propertyNames {"abc":1}': 'ok {"abc":1}',
|
||||
'propertyNames {"ABC":1}': "REJECT - ABC: invalid property name",
|
||||
'contains {"l":["a"]}': "REJECT - l: expected at least 1 matching item",
|
||||
'contains {"l":["a",1]}': 'ok {"l":["a",1]}',
|
||||
'not {"s":"x"}': 'ok {"s":"x"}',
|
||||
'not {"s":null}': "ok {}",
|
||||
'minProperties {"m":{}}': "REJECT - m: expected at least 1 properties",
|
||||
'unevaluatedProperties {"a":"x"}': 'ok {"a":"x"}',
|
||||
'unevaluatedProperties {"a":"x","b":1}': "REJECT - b: unknown field",
|
||||
'allOf-additional-false {"a":"x"}': 'ok {"a":"x"}',
|
||||
'allOf-additional-false {"a":"x","b":"y"}': "REJECT - b: unknown field",
|
||||
'const-object {"c":{"y":2,"x":1}}': 'ok {"c":{"y":2,"x":1}}',
|
||||
'enum-object {"e":{"y":2,"x":1}}': 'ok {"e":{"y":2,"x":1}}',
|
||||
'type-array-multi {"v":true}': 'ok {"v":"true"}',
|
||||
'type-array-multi {"v":null}': "ok {}",
|
||||
'nullable-type-array {"v":null}': 'ok {"v":null}',
|
||||
'nullable-type-array {"v":5}': 'ok {"v":"5"}',
|
||||
'exclusiveMinimum-number {"n":0}': "REJECT - n: expected > 0",
|
||||
'exclusiveMinimum-number {"n":"0"}': "REJECT - n: expected > 0",
|
||||
'maxItems {"l":[1,2]}': "REJECT - l: expected at most 1 items",
|
||||
'maxItems {"l":"x"}': 'ok {"l":["x"]}',
|
||||
'discriminator-openapi {"p":{"kind":"a","x":"1"}}': 'ok {"p":{"kind":"a","x":1}}',
|
||||
'discriminator-openapi {"p":{"kind":"b"}}': 'ok {"p":{"kind":"b"}}',
|
||||
'ref-root-defs {"a":"x"}': 'ok {"a":"x"}',
|
||||
'ref-root-defs {"a":1}': 'ok {"a":"1"}',
|
||||
"ref-root-defs {}": "REJECT - a: missing",
|
||||
'ref-recursive {"root":{"v":"2","c":[{"v":"3","c":[]}]}}': 'ok {"root":{"v":2,"c":[{"v":3,"c":[]}]}}',
|
||||
'ref-percent {"a":"x"}': 'ok {"a":"x"}',
|
||||
'ref-percent {"a":1}': 'ok {"a":"1"}',
|
||||
'ref-tilde {"a":"x"}': 'ok {"a":"x"}',
|
||||
'defs-with-$id {"a":"x"}': 'ok {"a":"x"}',
|
||||
'defs-with-$id {"a":2}': 'ok {"a":"2"}',
|
||||
'anyOf-null-default {"v":null}': 'ok {"v":null}',
|
||||
"anyOf-null-default {}": "ok {}",
|
||||
'format-date-time-lower {"d":"2024-01-01t00:00:00z"}': 'ok {"d":"2024-01-01t00:00:00z"}',
|
||||
'format-date-time-lower {"d":"2024-01-01 00:00:00Z"}': "REJECT - d: expected format date-time",
|
||||
'format-url {"u":"not a url"}': "REJECT - u: expected format url",
|
||||
'integer-as-float-1e0 {"n":1000}': 'ok {"n":1000}',
|
||||
"required-missing-props {}": "REJECT - a: missing",
|
||||
'required-missing-props {"a":null}': 'ok {"a":null}',
|
||||
'string-to-number-trim {"n":" 12 "}': 'ok {"n":12}',
|
||||
'string-to-number-trim {"n":"0x10"}': 'ok {"n":16}',
|
||||
'string-to-number-trim {"n":"1e3"}': 'ok {"n":1000}',
|
||||
'string-to-number-trim {"n":""}': 'ok {"n":0}',
|
||||
"tree valid": 'ok {"t":["a",["b"]]}',
|
||||
"tree number": 'ok {"t":"5"}',
|
||||
"tree [5]": 'ok {"t":["5"]}',
|
||||
"json valid": 'ok {"data":{"a":[1,"x",{"b":true}]}}',
|
||||
"json null": 'ok {"data":"null"}',
|
||||
"json nested null": 'ok {"data":{"a":[1,"null"]}}',
|
||||
"filter valid": 'ok {"where":{"and":[{"field":"a","eq":"b"}]}}',
|
||||
"filter bad leaf": "REJECT - where.and.0.eq: expected string, got integer",
|
||||
"filter deep bad": "REJECT - where.and.0.and.0.and.0.and.0.and.0.eq: missing",
|
||||
};
|
||||
|
||||
/** Every case with the key of its expected outcome: the name and the arguments, or the name alone for recursion. */
|
||||
const all: Array<[string, string, object, unknown]> = [
|
||||
...cases.flatMap(({ name, schema, args }) =>
|
||||
args.map((arg): [string, string, object, unknown] => [`${name} ${JSON.stringify(arg)}`, name, schema, arg]),
|
||||
),
|
||||
...keywordCases.flatMap(([name, schema, args]) =>
|
||||
args.map((arg): [string, string, object, unknown] => [`${name} ${JSON.stringify(arg)}`, name, schema, arg]),
|
||||
),
|
||||
...recursionCases.map(([name, schema, arg]): [string, string, object, unknown] => [name, name, schema, arg]),
|
||||
];
|
||||
|
||||
describe("reviewer probes", () => {
|
||||
for (const [key, name, schema, arg] of all) {
|
||||
it(key, () => {
|
||||
expect(outcome(name, schema, arg)).toBe(expected[key]);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe("reviewer probes (Sol)", () => {
|
||||
const tree = {
|
||||
type: "object",
|
||||
properties: { t: { $ref: "#/$defs/Tree" } },
|
||||
$defs: { Tree: { anyOf: [{ type: "string" }, { type: "array", items: { $ref: "#/$defs/Tree" } }] } },
|
||||
};
|
||||
const solCases: Array<[string, object, unknown, string]> = [
|
||||
// TypeBox-only: Type.BigInt and Type.Undefined have no JSON form; TypeBox converted to them.
|
||||
[
|
||||
"BigInt from a JSON string",
|
||||
Type.Object({ n: Type.BigInt() }),
|
||||
{ n: "42" },
|
||||
"REJECT - n: expected bigint, got string",
|
||||
],
|
||||
[
|
||||
"Undefined from JSON null",
|
||||
Type.Object({ n: Type.Undefined() }),
|
||||
{ n: null },
|
||||
"REJECT - n: expected undefined, got null",
|
||||
],
|
||||
// TypeBox-only: its additionalProperties also converted declared properties; pi-schema converts only undeclared ones.
|
||||
[
|
||||
"additionalProperties on declared properties",
|
||||
Type.Object({ amount: Type.Number(), label: Type.String() }, { additionalProperties: Type.Integer() }),
|
||||
{ amount: 3.5, label: "7" },
|
||||
'ok {"amount":3.5,"label":"7"}',
|
||||
],
|
||||
// TypeBox-only: property names are names, not regular expressions.
|
||||
[
|
||||
"property names are not patterns",
|
||||
Type.Object({ "a.b": Type.Integer(), axb: Type.String() }),
|
||||
{ "a.b": "4", axb: "5.8" },
|
||||
'ok {"a.b":4,"axb":"5.8"}',
|
||||
],
|
||||
// TypeBox-only: Refine runs only in TypeBox's own validator.
|
||||
[
|
||||
"Refine is not run",
|
||||
Type.Object({ mode: Type.Refine(Type.String(), (value) => value === "safe") }),
|
||||
{ mode: "unsafe" },
|
||||
'ok {"mode":"unsafe"}',
|
||||
],
|
||||
[
|
||||
"scalar-or-array that refers to itself",
|
||||
{
|
||||
type: "object",
|
||||
properties: {
|
||||
value: { anyOf: [{ type: "number" }, { type: "array", items: { $ref: "#/properties/value" } }] },
|
||||
},
|
||||
required: ["value"],
|
||||
},
|
||||
{ value: "42" },
|
||||
'ok {"value":42}',
|
||||
],
|
||||
[
|
||||
"StringEnum with an empty member from null",
|
||||
Type.Object({ mode: Type.Unsafe({ type: "string", enum: ["", "x"] }) }),
|
||||
{ mode: null },
|
||||
'ok {"mode":""}',
|
||||
],
|
||||
[
|
||||
"Cyclic with named local definitions",
|
||||
Type.Object({
|
||||
node: Type.Cyclic(
|
||||
{ Node: Type.Object({ n: Type.Number(), next: Type.Optional(Type.Ref("Node")) }) },
|
||||
"Node",
|
||||
),
|
||||
}),
|
||||
{ node: { n: 1 } },
|
||||
'ok {"node":{"n":1}}',
|
||||
],
|
||||
[
|
||||
"unpaired high surrogate, minLength 2",
|
||||
Type.Object({ s: Type.String({ minLength: 2 }) }),
|
||||
{ s: "\ud800A" },
|
||||
'ok {"s":"\\ud800A"}',
|
||||
],
|
||||
[
|
||||
"unpaired high surrogate, maxLength 1",
|
||||
Type.Object({ s: Type.String({ maxLength: 1 }) }),
|
||||
{ s: "\ud800A" },
|
||||
"REJECT - s: expected at most 1 characters",
|
||||
],
|
||||
["integer from a string", Type.Object({ n: Type.Integer() }), { n: "42" }, 'ok {"n":42}'],
|
||||
["recursive union with a value no branch converts", tree, { t: {} }, "REJECT - t: expected string, got object"],
|
||||
];
|
||||
for (const [name, schema, arg, result] of solCases) {
|
||||
it(name, () => {
|
||||
expect(outcome(name, schema, arg)).toBe(result);
|
||||
});
|
||||
}
|
||||
|
||||
it("accepts the last member of a 10,000-value enum", () => {
|
||||
const values = Array.from({ length: 10_000 }, (_, i) => `member${i}`);
|
||||
const schema = { type: "object", properties: { choice: { type: "string", enum: values } }, required: ["choice"] };
|
||||
expect(outcome("enum", schema, { choice: "member9999" })).toBe('ok {"choice":"member9999"}');
|
||||
expect(outcome("enum", schema, { choice: "x" })).toBe(
|
||||
`REJECT - choice: expected one of ${JSON.stringify(values.slice(0, 20)).slice(0, -1)}, ... and 9980 more]`,
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps the validator of a schema object, as before, also when the schema is changed later", () => {
|
||||
const parameters = Type.Object({ n: Type.Number({ minimum: 0 }) });
|
||||
expect(outcome("late", parameters, { n: "4" })).toBe('ok {"n":4}');
|
||||
Object.assign(parameters.properties.n, { minimum: 10 });
|
||||
expect(outcome("late", parameters, { n: "4" })).toBe('ok {"n":4}');
|
||||
});
|
||||
});
|
||||
|
||||
describe("reviewer probes (round 2)", () => {
|
||||
const item = { type: "object", properties: { n: { type: "integer", minimum: 1 } } };
|
||||
const nested: Case[] = [
|
||||
{ name: "declared", schema: { type: "object", properties: { entry: item } }, args: [{ entry: { n: null } }] },
|
||||
{ name: "additional", schema: { type: "object", additionalProperties: item }, args: [{ entry: { n: null } }] },
|
||||
{
|
||||
name: "pattern",
|
||||
schema: { type: "object", patternProperties: { "^entry$": item }, additionalProperties: false },
|
||||
args: [{ entry: { n: null } }],
|
||||
},
|
||||
];
|
||||
for (const { name, schema, args } of nested) {
|
||||
it(`drops an optional null inside ${name} property values`, () => {
|
||||
expect(outcome(name, schema, args[0])).toBe('ok {"entry":{}}');
|
||||
});
|
||||
}
|
||||
|
||||
for (const keyword of ["anyOf", "oneOf"]) {
|
||||
it(`converts a ${keyword} branch together with the node's sibling properties`, () => {
|
||||
const schema = {
|
||||
type: "object",
|
||||
[keyword]: [
|
||||
{ type: "object", properties: { a: { type: "array", items: { type: "integer" } } }, required: ["a"] },
|
||||
{ type: "object", properties: { b: { type: "boolean" } }, required: ["b"] },
|
||||
],
|
||||
properties: { c: { type: "integer" } },
|
||||
required: ["c"],
|
||||
};
|
||||
expect(outcome(keyword, schema, { a: "3", c: "2" })).toBe('ok {"a":[3],"c":2}');
|
||||
});
|
||||
}
|
||||
|
||||
it("rejects a deep value every branch of a recursive object union rejects, without exponential work", () => {
|
||||
const S = {
|
||||
anyOf: [
|
||||
{ type: "object", additionalProperties: { $ref: "#/$defs/S" }, required: ["a"] },
|
||||
{ type: "object", additionalProperties: { $ref: "#/$defs/S" }, required: ["b"] },
|
||||
],
|
||||
};
|
||||
const schema = { type: "object", properties: { s: { $ref: "#/$defs/S" } }, $defs: { S } };
|
||||
let value: unknown = { x: 1 };
|
||||
for (let i = 0; i < 40; i++) value = { c: value };
|
||||
expect(outcome("deep", schema, { s: value })).toBe("REJECT - s.a: missing");
|
||||
});
|
||||
});
|
||||
|
|
@ -1,5 +1,4 @@
|
|||
import { Type } from "typebox";
|
||||
import { Compile } from "typebox/compile";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { JsonValue, Tool, ToolCall } from "../src/types.ts";
|
||||
import { validateToolArguments } from "../src/utils/validation.ts";
|
||||
|
|
@ -61,7 +60,7 @@ describe("validateToolArguments", () => {
|
|||
}
|
||||
});
|
||||
|
||||
it("coerces serialized plain JSON schemas with AJV-compatible primitive rules", () => {
|
||||
it("coerces serialized plain JSON schemas like TypeBox schemas", () => {
|
||||
const passingCases: Array<{
|
||||
schema: Tool["parameters"];
|
||||
input: unknown;
|
||||
|
|
@ -75,7 +74,8 @@ describe("validateToolArguments", () => {
|
|||
{ schema: { type: "boolean" } as Tool["parameters"], input: "false", expected: false },
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: 1, expected: true },
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: 0, expected: false },
|
||||
{ schema: { type: "string" } as Tool["parameters"], input: null, expected: "" },
|
||||
// TypeBox's conversion rules come first: null is "null" for a required string.
|
||||
{ schema: { type: "string" } as Tool["parameters"], input: null, expected: "null" },
|
||||
{ schema: { type: "string" } as Tool["parameters"], input: true, expected: "true" },
|
||||
{ schema: { type: "null" } as Tool["parameters"], input: "", expected: null },
|
||||
{ schema: { type: "null" } as Tool["parameters"], input: 0, expected: null },
|
||||
|
|
@ -90,6 +90,11 @@ describe("validateToolArguments", () => {
|
|||
input: "1",
|
||||
expected: 1,
|
||||
},
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: "1", expected: true },
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: "0", expected: false },
|
||||
{ schema: { type: "null" } as Tool["parameters"], input: "null", expected: null },
|
||||
{ schema: { type: "integer" } as Tool["parameters"], input: "42.1", expected: 42 },
|
||||
{ schema: { type: "array", items: { type: "number" } } as Tool["parameters"], input: "7", expected: [7] },
|
||||
];
|
||||
|
||||
for (const testCase of passingCases) {
|
||||
|
|
@ -184,10 +189,6 @@ describe("validateToolArguments", () => {
|
|||
{ type: ["array", "null"], items: { type: "string" } } as Tool["parameters"],
|
||||
null,
|
||||
);
|
||||
// The CSP test above selects TypeBox's process-wide interpreted fallback, so exercise the generated validator explicitly.
|
||||
const generatedCheck = new Function(Compile(tool.parameters).Code())() as (value: unknown) => boolean;
|
||||
|
||||
expect(generatedCheck(toolCall.arguments)).toBe(true);
|
||||
expect(validateToolArguments(tool, toolCall)).toEqual({ value: null });
|
||||
});
|
||||
|
||||
|
|
@ -196,10 +197,10 @@ describe("validateToolArguments", () => {
|
|||
schema: Tool["parameters"];
|
||||
input: unknown;
|
||||
}> = [
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: "1" },
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: "0" },
|
||||
{ schema: { type: "null" } as Tool["parameters"], input: "null" },
|
||||
{ schema: { type: "integer" } as Tool["parameters"], input: "42.1" },
|
||||
{ schema: { type: "boolean" } as Tool["parameters"], input: "yes" },
|
||||
{ schema: { type: "number" } as Tool["parameters"], input: "abc" },
|
||||
{ schema: { type: "null" } as Tool["parameters"], input: "nil" },
|
||||
{ schema: { type: "integer" } as Tool["parameters"], input: "4x" },
|
||||
];
|
||||
|
||||
for (const testCase of failingCases) {
|
||||
|
|
@ -207,6 +208,25 @@ describe("validateToolArguments", () => {
|
|||
expect(() => validateToolArguments(tool, toolCall)).toThrow("Validation failed");
|
||||
}
|
||||
});
|
||||
it("lists pi-schema's errors, then the arguments it received", () => {
|
||||
const { tool, toolCall } = createToolCallWithPlainSchema({ type: "number" } as Tool["parameters"], "many");
|
||||
expect(() => validateToolArguments(tool, toolCall)).toThrow(
|
||||
'Validation failed for tool "echo":\n - value: expected number, got string\n\nReceived arguments:\n{\n "value": "many"\n}',
|
||||
);
|
||||
});
|
||||
|
||||
it("reports a missing required property at the property", () => {
|
||||
const { tool, toolCall } = createToolCallWithPlainSchema({ type: "string" } as Tool["parameters"], "x");
|
||||
expect(() => validateToolArguments(tool, { ...toolCall, arguments: {} })).toThrow(" - value: missing");
|
||||
});
|
||||
|
||||
it("returns a copy of the arguments, also when nothing is coerced", () => {
|
||||
const { tool, toolCall } = createToolCallWithPlainSchema({ type: "string" } as Tool["parameters"], "x");
|
||||
const args = validateToolArguments(tool, toolCall);
|
||||
expect(args).toEqual({ value: "x" });
|
||||
expect(args).not.toBe(toolCall.arguments);
|
||||
});
|
||||
|
||||
// #10897
|
||||
it("reports malformed arguments JSON instead of validating empty arguments", () => {
|
||||
const tool: Tool = {
|
||||
|
|
|
|||
|
|
@ -13,5 +13,8 @@ export default defineConfig({
|
|||
},
|
||||
resolve: {
|
||||
alias: [{ find: /^@earendil-works\/pi-telemetry$/, replacement: telemetrySrcIndex }],
|
||||
// Workspace packages without a build (pi-schema) resolve to their source.
|
||||
conditions: ["source"],
|
||||
},
|
||||
ssr: { resolve: { conditions: ["source"] } },
|
||||
});
|
||||
Loading…
Add table
Add a link
Reference in a new issue