sscoble/pi-mono
1
0
Fork 0
mirror of https://github.com/badlogic/pi-mono.git synced 2026-11-05 23:37:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

feat(codemode): run sandboxes on Cloudflare Workers via a remote Durable Object

Browse source 
Split the host into transport-independent execution logic and a VM driver, so the VM can run in a worker thread (CodemodeSandbox, unchanged), on the calling thread (InlineCodemodeSandbox), or in another isolate over RPC (RemoteCodemodeSandbox).

Add the /portable entry without Node imports and the /cloudflare entry with CodemodeSandboxDurableObject, meant to be deployed as its own Worker so scripts never block the caller's thread. Scripts that compute without awaiting a tool are bounded by an interrupt budget, now also polled while draining promise jobs. examples/cloudflare shows the setup.
This commit is contained in:
Armin Ronacher 2026-10-10 14:41:36 +02:00
parent 42a3497d03
commit 748b351fe2
26 changed files with 2505 additions and 1242 deletions
Download patch file Download diff file Expand all files Collapse all files

10
packages/codemode/CHANGELOG.md
View file

@ -2,6 +2,16 @@
## [Unreleased]
### Added
- Added `RemoteCodemodeSandbox`, which runs each script's VM in another isolate while tools run in the caller. `CodemodeSandboxDurableObject` from the new `@earendil-works/pi-codemode/cloudflare` entry is its remote end, deployed as its own Cloudflare Worker. `examples/cloudflare` shows the setup
- Added `InlineCodemodeSandbox`, which runs the VM on the calling thread for runtimes without worker threads. Its `interruptBudget` limits scripts that compute without awaiting a tool
- Added the `@earendil-works/pi-codemode/portable` entry, which has no Node imports, and `CodemodeSandboxBase`, the API all three sandboxes share
### Changed
- The VM now polls for interrupts while it drains promise jobs, so a script such as `while (true) await null` is stopped at the same points as a synchronous loop
## [1.1.0] - 2026-10-07
### Added

149
packages/codemode/README.md
View file

@ -4,7 +4,16 @@ Runs model-written JavaScript in a QuickJS VM (compiled to WebAssembly) where th
Scripts use `tools`, `ALL_TOOLS`, `text`, `image`, `exit`, `store`, and `load`, and may start with a `// @options:` line. The coding agent uses it for its built-in `codemode` tool. It has no pi dependencies and can be used on its own to expose any functions (remote APIs, MCP servers, application services) to model-written scripts.
## Usage
The package runs in two kinds of environments. Each has its own sandbox class, but they share one API, one script environment, and one result format:
| Environment | Sandbox | Import from | Where scripts run |
| --- | --- | --- | --- |
| Node and Bun | `CodemodeSandbox` | `@earendil-works/pi-codemode` | a worker thread per execution |
| Cloudflare Workers and Durable Objects | `RemoteCodemodeSandbox` | `@earendil-works/pi-codemode/portable` | a Durable Object in a separately deployed Worker |
See [Usage on Node and Bun](#usage-on-node-and-bun) and [Usage on Cloudflare Workers](#usage-on-cloudflare-workers). A third sandbox, `InlineCodemodeSandbox`, runs scripts on the calling thread for other runtimes, or on Cloudflare when blocking the caller is acceptable.
## Usage on Node and Bun
```ts
import { CodemodeSandbox } from "@earendil-works/pi-codemode";
@ -35,7 +44,7 @@ else console.error(result.error.kind, result.error.message);
await sandbox.close();
```
`code` is the body of an async function: `return` and top-level `await` work. Inside the script:
`code` is the body of an async function: `return` and top-level `await` work. Inside the script, with every sandbox:
- `tools.<name>(args)` returns a promise. Arguments and results make a JSON round trip. A tool that throws rejects with an `Error` carrying the same message. Tool names are also exposed as identifiers: characters that are not valid in identifiers become `_` (`toCodemodeIdentifier`), so `my-tool` is `tools.my_tool` as well as `tools["my-tool"]`.
- `ALL_TOOLS` lists `{ name, description }` for every tool, with `name` as the identifier.
@ -76,9 +85,9 @@ text(JSON.parse(source).name);
Supported fields are `max_output_tokens`, a token budget for the output, and `timeout_ms`, a hard deadline. The sandbox does not act on them; the caller decides. The options line is replaced by an empty line, so line numbers in stack traces still match the input. Empty input, invalid JSON, unknown fields, or an options line without code throw `CodemodeSourceError`. `CODEMODE_SOURCE_GRAMMAR` is a Lark grammar for providers that support grammar-constrained tool input. Both are also available from the lightweight `@earendil-works/pi-codemode/source` entry.
## Bundled hosts
## Bundled Node and Bun hosts
By default the sandbox loads `quickjs-wasi/quickjs.wasm` from the installed package and starts the worker file that sits next to this package's module. Neither exists on disk when the host is bundled, so pass both:
By default `CodemodeSandbox` loads `quickjs-wasi/quickjs.wasm` from the installed package and starts the worker file that sits next to this package's module. Neither exists on disk when the host is bundled, so pass both:
```ts
import { CodemodeSandbox, loadQuickJSWasm } from "@earendil-works/pi-codemode";
@ -96,6 +105,130 @@ const sandbox = new CodemodeSandbox({
additional build entrypoint and pass its relative source path as a string, for example
`"./src/codemode-worker.ts"`; Bun resolves that form from its embedded module graph.
## Usage on Cloudflare Workers
`CodemodeSandbox` needs Node or Bun. It runs each script in a worker thread and reads the QuickJS binary from disk. Cloudflare Workers have neither, and they cannot compile WebAssembly at runtime. Two other sandboxes have the same API, the same script environment, and the same `CodemodeResult`. Only where the VM runs differs:
| Sandbox | Where the VM runs | Use it for |
| --- | --- | --- |
| `CodemodeSandbox` | a worker thread | Node and Bun |
| `RemoteCodemodeSandbox` | another isolate, usually a Durable Object in a separate Worker | a Durable Object that runs an agent session |
| `InlineCodemodeSandbox` | the calling thread | anywhere else with WebAssembly, when blocking the caller is acceptable |
Import the last two from `@earendil-works/pi-codemode/portable`, which has no Node imports. On Cloudflare, `@earendil-works/pi-codemode/cloudflare` adds the deployable Durable Object and the QuickJS module compiled by Wrangler.
### Why a separate Worker
QuickJS runs synchronously, so a script that computes holds its thread until it awaits a tool. Measured on Cloudflare (October 2026), a script that computed for ten seconds delayed every other request to the session object by the full ten seconds in two placements:
- inside the session's Durable Object
- in a Dynamic Worker loaded by that object
In a Durable Object of a separately deployed Worker, the delay was about 50 ms. `RemoteCodemodeSandbox` therefore keeps the tools and call records in the session and sends only the VM to the sandbox Worker. Each tool call travels back to the session over Workers RPC.
### Deploying the sandbox Worker
A complete setup is in [`examples/cloudflare`](https://github.com/earendil-works/pi/tree/main/packages/codemode/examples/cloudflare). The sandbox Worker is one file:
```ts
// sandbox/src/index.ts
export { CodemodeSandboxDurableObject as CodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
export default {
fetch: () => new Response("Not found", { status: 404 }),
};
```
```jsonc
// sandbox/wrangler.jsonc
{
"name": "pi-codemode-sandbox",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
"workers_dev": false,
"preview_urls": false,
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["CodemodeSandbox"] }],
"limits": { "cpu_ms": 60000 }
}
```
Install `@earendil-works/pi-codemode` in the sandbox Worker's project and run `npx wrangler deploy`. Wrangler bundles QuickJS (about 640 KB) as a compiled WebAssembly module. The object has no routes and keeps no state, and it never uses its storage.
### Using it from the agent's Worker
Bind the class with `script_name`. Binding a class exported from the agent's own Worker would bring back the blocking described above.
```jsonc
// agent/wrangler.jsonc
"durable_objects": {
"bindings": [
{ "name": "CODEMODE_SANDBOX", "class_name": "CodemodeSandbox", "script_name": "pi-codemode-sandbox" }
]
}
```
Then replace the sandbox. Tools, options other than `wasm` and `workerUrl`, and results stay the same:
```ts
import type { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
import { RemoteCodemodeSandbox } from "@earendil-works/pi-codemode/portable";
interface Env {
CODEMODE_SANDBOX: DurableObjectNamespace<CodemodeSandboxDurableObject>;
}
// Before: new CodemodeSandbox({ tools, timeoutMs: 60_000 })
const sandbox = new RemoteCodemodeSandbox({
tools,
timeoutMs: 60_000,
remote: () => env.CODEMODE_SANDBOX.getByName(sessionId),
});
const result = await sandbox.execute(code, { signal, store });
```
Import the sandbox from `/portable` in the agent's Worker. Importing `/cloudflare` would also bundle QuickJS there. `remote` is called once per execution and picks the Durable Object that runs it.
- **One object per session**, `getByName(sessionId)`, keeps a session's scripts together. Two scripts of the same session take turns on one thread.
- **One object per execution**, `getByName(crypto.randomUUID())`, runs every script on its own.
Run both Workers locally with `wrangler dev -c agent/wrangler.jsonc -c sandbox/wrangler.jsonc`.
### Limits on Cloudflare
- **Interrupt budget.** While a script computes without calling a tool, no timer can run, and `Date.now()` does not advance. A script that never yields is limited only by the interrupt budget, not by `timeoutMs`. QuickJS polls an interrupt handler while it runs, and the sandbox counts the polls. Past the budget, the execution fails with `kind: "timeout"`.
- The default `DEFAULT_INTERRUPT_BUDGET` of 100,000 polls is about 2 seconds of a tight loop on an Apple M-series laptop and about 10 seconds on Cloudflare.
- Keep the budget below the Worker's `cpu_ms` limit (30 seconds by default). Hitting that limit resets the object instead of failing one script.
- **Memory.** Each VM may use up to 32 MiB by default. All VMs in one isolate share its 128 MB.
- **Changing the limits.** The Durable Object caps every request at its own limits, whatever the client asks for. To change them, subclass it in the sandbox Worker:
```ts
import { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
export class CodemodeSandbox extends CodemodeSandboxDurableObject {
protected maxInterruptBudget = 200_000;
protected maxMemoryLimitBytes = 64 * 1024 * 1024;
}
```
- **Aborts and timeouts.** `timeoutMs` and `signal` end the execution in the caller right away, with `kind: "timeout"` or `kind: "aborted"`. The remote VM stops at its next tool call. A remote VM that is computing at that moment keeps running until it calls a tool or uses up its budget.
- **Billing.** Each execution is one Durable Object request. The sandbox object is billed for wall-clock duration while an execution runs, including time spent waiting on tools.
### On the calling thread
When blocking the caller is acceptable, for example in a stateless Worker that serves one request, run the VM on the caller's thread instead. Nothing extra needs to be deployed:
```ts
import { createInlineCodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
const sandbox = createInlineCodemodeSandbox({ tools, timeoutMs: 60_000, interruptBudget: 50_000 });
const result = await sandbox.execute(code);
```
`createInlineCodemodeSandbox()` passes the QuickJS module Wrangler compiled to `InlineCodemodeSandbox`. With another bundler, import `InlineCodemodeSandbox` from `/portable` and pass your own compiled module as `wasm`.
- **Blocking.** A script that computes blocks the caller until it awaits a tool or uses up its `interruptBudget`.
- **Stack.** QuickJS's stack guard is reduced to `INLINE_MAX_STACK_SIZE`, so that it trips before the caller's smaller stack runs out.
## Declarations for the model
Tools and globals can carry `description`, `inputSchema`, and `outputSchema` (JSON Schema). `renderDeclarations()` turns them into TypeScript declarations for a model-facing tool description:
@ -167,14 +300,16 @@ const codemodeTool: AgentTool = {
| kind | meaning |
| --------- | --------------------------------------------------------------------------- |
| `script` | the script threw or failed to parse; `stack` points at `codemode.js:<line>` |
| `timeout` | the deadline expired; the worker was terminated |
| `aborted` | `options.signal` fired or `close()` was called; the worker was terminated |
| `sandbox` | the worker or VM failed, for example a wasm trap or a missing worker file |
| `timeout` | the deadline expired or the interrupt budget ran out; the VM was stopped |
| `aborted` | `options.signal` fired or `close()` was called; the VM was stopped |
| `sandbox` | the VM or its transport failed, for example a wasm trap, a missing worker file, or a failed remote |
`result.output` holds the text and image items in the order the script produced them, also for failed executions. The host keeps all of it until the script ends, so output is limited to `MAX_OUTPUT_CHARS` (16 Mi) characters of text and base64 image data and `MAX_OUTPUT_ITEMS` (100000) items. Past either limit the script fails with a `RangeError`, even if it catches the error. `result.calls` lists every tool call with `status: "ok" | "error" | "cancelled"`. A call that is still running when the script returns (not awaited) is aborted through the tool's `signal` and reported as `cancelled`.
## How it works
This section describes `CodemodeSandbox`. The other sandboxes share the same host side, prelude, and VM, and replace only the worker thread. See [Usage on Cloudflare Workers](#usage-on-cloudflare-workers).
Each `execute()` starts a worker thread (about 20 ms including VM creation) that instantiates a fresh QuickJS VM from the compiled wasm module. The VM is a separate wasm instance with its own linear memory. Its only imports are a WASI shim (clock, random, and stdout/stderr writes, which the worker discards) and one host-call entry point, so the script cannot reach the host except through the functions the worker registers.
The worker evaluates a prelude inside the VM that holds the single host bridge in a closure and builds `tools`, `console`, and globals on top of it. The script is compiled as an async function body.

31
packages/codemode/examples/cloudflare/README.md Normal file
View file

@ -0,0 +1,31 @@
# pi-codemode on Cloudflare
This example has two Workers:
- **`sandbox/`** deploys `CodemodeSandboxDurableObject` as its own Worker, `pi-codemode-sandbox`.
Scripts run there, in a separate isolate.
- **`agent/`** has a `Session` Durable Object that owns the tools. It runs scripts through
`RemoteCodemodeSandbox`, which keeps the tools and results in `Session` while the sandbox Worker
runs the VM.
The [README of the package](../../README.md#usage-on-cloudflare-workers) explains why the
sandbox must be a separate Worker, and which limits apply.
## Try it locally
Install `@earendil-works/pi-codemode` and `wrangler` in each directory, then start both Workers in
one process:
```sh
npx wrangler dev -c agent/wrangler.jsonc -c sandbox/wrangler.jsonc
curl -X POST http://localhost:8787 --data 'const n = await tools.count({ add: 2 }); text("n=" + n); return n'
```
## Deploy
Deploy the sandbox first, because the agent's binding refers to it by name:
```sh
(cd sandbox && npx wrangler deploy)
(cd agent && npx wrangler deploy)
```

50
packages/codemode/examples/cloudflare/agent/src/index.ts Normal file
View file

@ -0,0 +1,50 @@
// A Worker whose Durable Object runs codemode the way an agent session would: tools run here, in the
// session object, and scripts run in the separately deployed pi-codemode-sandbox Worker.
//
// curl -X POST http://localhost:8787 --data 'const n = await tools.count({ add: 2 }); text("n=" + n); return n'
import type { CodemodeSandboxDurableObject } from "@earendil-works/pi-codemode/cloudflare";
// The portable entry has no QuickJS module: the WebAssembly lives only in the sandbox Worker.
import { type CodemodeTool, RemoteCodemodeSandbox } from "@earendil-works/pi-codemode/portable";
import { DurableObject } from "cloudflare:workers";
interface Env {
SESSION: DurableObjectNamespace<Session>;
CODEMODE_SANDBOX: DurableObjectNamespace<CodemodeSandboxDurableObject>;
}
export class Session extends DurableObject<Env> {
private counter = 0;
async runCodemode(code: string) {
const tools: CodemodeTool[] = [
{
name: "count",
description: "Adds to a counter kept in the session and returns the new value",
execute: (args) => {
this.counter += (args as { add: number }).add;
return this.counter;
},
},
];
// Before: new CodemodeSandbox({ tools, timeoutMs: 60_000 })
const sandbox = new RemoteCodemodeSandbox({
tools,
timeoutMs: 60_000,
// One sandbox instance per session keeps a session's scripts together and apart from others.
remote: () => this.env.CODEMODE_SANDBOX.getByName(this.ctx.id.toString()),
});
try {
return await sandbox.execute(code);
} finally {
await sandbox.close();
}
}
}
export default {
async fetch(request, env) {
if (request.method !== "POST") return new Response("POST a script\n", { status: 405 });
const result = await env.SESSION.getByName("demo").runCodemode(await request.text());
return Response.json(result);
},
} satisfies ExportedHandler<Env>;

15
packages/codemode/examples/cloudflare/agent/wrangler.jsonc Normal file
View file

@ -0,0 +1,15 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
"name": "pi-codemode-agent-example",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
"durable_objects": {
"bindings": [
{ "name": "SESSION", "class_name": "Session" },
// The sandbox class lives in its own Worker. `script_name` is what puts scripts in a different
// isolate; binding a class exported from this Worker would run them on the session's thread.
{ "name": "CODEMODE_SANDBOX", "class_name": "CodemodeSandbox", "script_name": "pi-codemode-sandbox" }
]
},
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["Session"] }]
}

7
packages/codemode/examples/cloudflare/sandbox/src/index.ts Normal file
View file

@ -0,0 +1,7 @@
// The whole sandbox Worker: one Durable Object class that runs codemode scripts for callers bound to
// it. Only Workers with a binding can reach the class; HTTP requests get a 404.
export { CodemodeSandboxDurableObject as CodemodeSandbox } from "@earendil-works/pi-codemode/cloudflare";
export default {
fetch: () => new Response("Not found", { status: 404 }),
};

13
packages/codemode/examples/cloudflare/sandbox/wrangler.jsonc Normal file
View file

@ -0,0 +1,13 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
"name": "pi-codemode-sandbox",
"main": "src/index.ts",
"compatibility_date": "2026-04-22",
// Reached only through Durable Object bindings, never over HTTP.
"workers_dev": false,
"preview_urls": false,
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["CodemodeSandbox"] }],
// Optional. The default is 30 seconds of CPU per request; one execution is one request. Keep it above
// the time the interrupt budget allows, so scripts fail with a timeout instead of resetting the object.
"limits": { "cpu_ms": 60000 }
}

14
packages/codemode/package.json
View file

@ -21,6 +21,16 @@
"types": "./dist/source.d.ts",
"import": "./dist/source.js"
},
"./portable": {
"source": "./src/portable.ts",
"types": "./dist/portable.d.ts",
"import": "./dist/portable.js"
},
"./cloudflare": {
"source": "./src/cloudflare.ts",
"types": "./dist/cloudflare.d.ts",
"import": "./dist/cloudflare.js"
},
"./worker": {
"source": "./src/runtime/worker.ts",
"types": "./dist/runtime/worker.d.ts",
@ -42,7 +52,9 @@
"codemode",
"sandbox",
"quickjs",
"wasm"
"wasm",
"cloudflare",
"durable-objects"
],
"author": "Earendil Works",
"license": "MIT",

19
packages/codemode/src/cloudflare-modules.d.ts vendored Normal file
View file

@ -0,0 +1,19 @@
// Modules that only exist when cloudflare.ts is built by Wrangler. Consumers get the real
// declarations from @cloudflare/workers-types or `wrangler types`; these keep this repository's
// type check free of a Workers type dependency and are not published.
declare module "cloudflare:workers" {
export abstract class DurableObject<Env = unknown> {
protected ctx: unknown;
protected env: Env;
constructor(ctx: unknown, env: Env);
}
}
// A pattern, not the exact specifier: coding-agent declares "quickjs-wasi/quickjs.wasm" as a path
// for Bun, and an exact declaration takes precedence where both are visible. cloudflare.ts casts the
// import to the compiled module Wrangler actually produces.
declare module "*/quickjs.wasm" {
const module: unknown;
export default module;
}

75
packages/codemode/src/cloudflare.ts Normal file
View file

@ -0,0 +1,75 @@
/**
* Cloudflare Workers entry, for Workers built with Wrangler.
*
* Wrangler's default `CompiledWasm` rule turns the `.wasm` import below into a
* `WebAssembly.Module`; Workers cannot compile WebAssembly from bytes at runtime. Other bundlers
* do not support the import: use `@earendil-works/pi-codemode/portable` with your own module there.
*
* Two ways to run codemode on Cloudflare:
* - `RemoteCodemodeSandbox` with a {@link CodemodeSandboxDurableObject} deployed as its own
* Worker: scripts run in a separate isolate and never block the caller. Use this from a Durable
* Object that serves a session.
* - {@link createInlineCodemodeSandbox}: scripts run on the caller's thread. Simplest; a script
* that computes blocks the caller until it awaits a tool or uses up its interrupt budget.
*/
import { DurableObject } from "cloudflare:workers";
import quickjsWasmImport from "quickjs-wasi/quickjs.wasm";
import {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
import { type CodemodeRemoteExchange, type CodemodeRemoteRequest, serveCodemodeRemote } from "./runtime/remote.ts";
import type { CodemodeWasmModule } from "./wasm.ts";
export * from "./portable.ts";
// Wrangler's `CompiledWasm` rule makes this a compiled module; the declaration cannot say so (see
// cloudflare-modules.d.ts).
const quickjsWasm = quickjsWasmImport as unknown as CodemodeWasmModule;
/** Default VM memory limit of {@link CodemodeSandboxDurableObject}. Isolates have 128 MB in total. */
export const DEFAULT_REMOTE_MEMORY_LIMIT_BYTES = 32 * 1024 * 1024;
/** `quickjs-wasi/quickjs.wasm`, compiled by Wrangler. */
export const cloudflareQuickJSWasm: CodemodeWasmModule = quickjsWasm;
/** `InlineCodemodeSandbox` with the QuickJS module Wrangler bundled. */
export function createInlineCodemodeSandbox(options: Omit<InlineCodemodeSandboxOptions, "wasm"> = {}) {
return new InlineCodemodeSandbox({ ...options, wasm: quickjsWasm });
}
/**
* The remote end of `RemoteCodemodeSandbox`: a Durable Object whose one RPC method runs a script's
* VM and calls back into the client for every tool call.
*
* Deploy it as its own Worker and bind it from the Worker that runs your agent with `script_name`.
* A separate Worker is what keeps scripts off the agent's thread: a class in the same Worker can
* share an isolate, and therefore a thread, with the objects that call it.
*
* The object keeps no state between executions and never touches storage. Executions that reach
* the same instance share its thread: they interleave while they wait on tools, and a script that
* computes delays the others. Name instances per session (`getByName(sessionId)`) to keep one
* session's scripts together, or per execution for full independence.
*
* Limits apply to every request, whatever the client asks for. Override the fields in a subclass:
*
* export class CodemodeSandbox extends CodemodeSandboxDurableObject {
* protected maxInterruptBudget = 200_000;
* }
*/
export class CodemodeSandboxDurableObject<Env = unknown> extends DurableObject<Env> {
/** Interrupt polls per execution; the client may only ask for less. See `InlineCodemodeSandboxOptions`. */
protected maxInterruptBudget: number = DEFAULT_INTERRUPT_BUDGET;
/** VM memory per execution; the client may only ask for less. */
protected maxMemoryLimitBytes: number = DEFAULT_REMOTE_MEMORY_LIMIT_BYTES;
/** Called by `RemoteCodemodeSandbox` over RPC. Resolves when the script has settled or was stopped. */
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange): Promise<void> {
return serveCodemodeRemote(request, exchange, {
wasm: quickjsWasm,
maxInterruptBudget: this.maxInterruptBudget,
maxMemoryLimitBytes: this.maxMemoryLimitBytes,
});
}
}

18
packages/codemode/src/index.ts
View file

@ -10,13 +10,30 @@ export {
schemaToType,
} from "./declarations.ts";
export { toCodemodeIdentifier } from "./identifier.ts";
export { CodemodeSandboxBase } from "./runtime/execution.ts";
export { CodemodeSandbox } from "./runtime/host.ts";
export {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
export {
MAX_OUTPUT_CHARS,
MAX_OUTPUT_ITEMS,
MAX_STORE_TOTAL_CHARS,
MAX_STORE_VALUE_CHARS,
} from "./runtime/prelude-source.ts";
export {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteMessage,
type CodemodeRemoteReply,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
RemoteCodemodeSandbox,
type RemoteCodemodeSandboxOptions,
serveCodemodeRemote,
} from "./runtime/remote.ts";
export {
CODEMODE_OPTIONS_PREFIX,
CODEMODE_SOURCE_GRAMMAR,
@ -34,6 +51,7 @@ export type {
CodemodeJsonSchema,
CodemodeOutputItem,
CodemodeResult,
CodemodeSandboxBaseOptions,
CodemodeSandboxOptions,
CodemodeStoreWrites,
CodemodeTool,

46
packages/codemode/src/portable.ts Normal file
View file

@ -0,0 +1,46 @@
/**
* Sandboxes that need no Node built-ins: `InlineCodemodeSandbox` runs the VM on the calling thread
* and `RemoteCodemodeSandbox` runs it in another isolate. Neither imports `node:*` modules, so this
* entry works in Cloudflare Workers, Durable Objects, and browsers. `CodemodeSandbox`, which uses
* worker threads, stays in the main entry.
*/
export { toCodemodeIdentifier } from "./identifier.ts";
export { CodemodeSandboxBase } from "./runtime/execution.ts";
export {
DEFAULT_INTERRUPT_BUDGET,
InlineCodemodeSandbox,
type InlineCodemodeSandboxOptions,
} from "./runtime/inline.ts";
export {
MAX_OUTPUT_CHARS,
MAX_OUTPUT_ITEMS,
MAX_STORE_TOTAL_CHARS,
MAX_STORE_VALUE_CHARS,
} from "./runtime/prelude-source.ts";
export type { HostToWorkerMessage, VmStartData, WorkerToHostMessage } from "./runtime/protocol.ts";
export {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteMessage,
type CodemodeRemoteReply,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
RemoteCodemodeSandbox,
type RemoteCodemodeSandboxOptions,
serveCodemodeRemote,
} from "./runtime/remote.ts";
export type {
CodemodeCall,
CodemodeCallStatus,
CodemodeError,
CodemodeErrorKind,
CodemodeExecuteOptions,
CodemodeJsonSchema,
CodemodeOutputItem,
CodemodeResult,
CodemodeSandboxBaseOptions,
CodemodeStoreWrites,
CodemodeTool,
CodemodeToolContext,
} from "./types.ts";
export type { CodemodeWasmModule } from "./wasm.ts";

436
packages/codemode/src/runtime/execution.ts Normal file
View file

@ -0,0 +1,436 @@
/**
* Host side of an execution, shared by every place a VM can run.
*
* The host owns everything the script must not influence: the tool table, call records, output,
* the deadline, the caller's abort signal, and the decoding of what the VM reports. The VM itself
* runs somewhere a {@link VmLauncher} decides: a worker thread (`CodemodeSandbox`), the calling
* thread (`InlineCodemodeSandbox`), or another isolate reached over RPC (`RemoteCodemodeSandbox`).
* The two sides only exchange the messages in `protocol.ts`, so the launchers differ in transport
* and termination, not in behavior.
*
* Lifecycle of one execution:
* 1. The constructor arms the deadline and abort listener, then asks the launcher for a channel.
* 2. Messages from the VM arrive through {@link VmEvents.message}; tool calls run here and their
* results go back through {@link VmChannel.post}. Replies produced before the launcher has
* returned its channel are queued and flushed when it arrives.
* 3. The first terminal event wins: `done`, `crash`, a launcher failure, the deadline, or an abort.
* `finish()` aborts pending tool calls, stops the channel, and resolves the result once the
* channel reports it stopped. Later events are ignored.
*
* The launcher, not this class, is responsible for actually stopping a running VM: a worker is
* terminated, an in-thread VM is interrupted and disposed, and a remote VM is told to stop at its
* next exchange.
*/
import { toCodemodeIdentifier } from "../identifier.ts";
import type {
CodemodeCall,
CodemodeCallStatus,
CodemodeError,
CodemodeExecuteOptions,
CodemodeOutputItem,
CodemodeResult,
CodemodeSandboxBaseOptions,
CodemodeStoreWrites,
CodemodeTool,
} from "../types.ts";
import {
type HostToWorkerMessage,
isWorkerToHostMessage,
type VmStartData,
type WorkerToHostMessage,
} from "./protocol.ts";
const DEFAULT_TIMEOUT_MS = 300_000;
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
const RESERVED_GLOBALS: ReadonlySet<string> = new Set([
"tools",
"ALL_TOOLS",
"console",
"text",
"image",
"exit",
"globalThis",
"store",
"load",
]);
/** Callbacks a launcher uses to report what its VM does. */
export interface VmEvents {
/** A message from the VM. Validated here, so launchers can pass through anything they receive. */
message(message: unknown): void;
/** The VM failed or stopped outside the bridge, for example a worker exit or a remote error. */
failure(error: CodemodeError): void;
}
/** The host's handle on a running VM. */
export interface VmChannel {
post(message: HostToWorkerMessage): void;
/** Stops the VM. Called once, after the execution has its result. */
stop(): Promise<void>;
}
/**
* Starts a VM for one execution. Rejections become `sandbox` errors with the rejection's message.
* The VM may start reporting messages before the returned promise settles.
*/
export type VmLauncher = (data: VmStartData, events: VmEvents) => VmChannel | Promise<VmChannel>;
export function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function serializeStore(store: Readonly<Record<string, unknown>> | undefined): Record<string, string> {
const serialized: Record<string, string> = {};
for (const [key, value] of Object.entries(store ?? {})) {
const json = JSON.stringify(value);
if (json !== undefined) serialized[key] = json;
}
return serialized;
}
/**
* The prelude that serializes values for the host runs in the same VM as the script, so a script
* that patches built-ins (for example `Array.prototype.toJSON`) can make it send malformed data.
* Payloads that do not decode throw this error, and the execution fails as a sandbox error.
*/
class BridgeError extends Error {}
function parseBridgeJson(json: string, what: string): unknown {
try {
return JSON.parse(json);
} catch {
throw new BridgeError(`${what} is not valid JSON`);
}
}
function parseStoreWrites(json: string): CodemodeStoreWrites {
const entries = parseBridgeJson(json, "store writes");
if (!Array.isArray(entries)) throw new BridgeError("store writes are not an array");
const writes: CodemodeStoreWrites = { set: {}, delete: [] };
for (const entry of entries) {
if (
!Array.isArray(entry) ||
typeof entry[0] !== "string" ||
(entry.length === 2 ? typeof entry[1] !== "string" : entry.length !== 1)
) {
throw new BridgeError("store writes contain a malformed entry");
}
const [key, value] = entry as [string, string?];
if (value === undefined) writes.delete.push(key);
else writes.set[key] = parseBridgeJson(value, `store value for ${JSON.stringify(key)}`);
}
return writes;
}
function parseScriptError(json: string): CodemodeError {
const parsed = parseBridgeJson(json, "script error");
if (typeof parsed !== "object" || parsed === null) throw new BridgeError("script error is not an object");
const { name, message, stack } = parsed as Record<string, unknown>;
if (
typeof message !== "string" ||
(name !== undefined && typeof name !== "string") ||
(stack !== undefined && typeof stack !== "string")
) {
throw new BridgeError("script error is malformed");
}
return { kind: "script", name, message, stack };
}
interface PendingCall {
record: CodemodeCall | undefined;
startedAt: number;
controller: AbortController;
}
interface ExecutionOptions {
code: string;
tools: ReadonlyMap<string, CodemodeTool>;
globals: ReadonlyMap<string, CodemodeTool>;
timeoutMs: number;
signal: AbortSignal | undefined;
memoryLimitBytes: number | undefined;
store: Record<string, string>;
launch: VmLauncher;
}
/** One script run in its own VM. */
class Execution {
readonly promise: Promise<CodemodeResult>;
private resolveResult!: (result: CodemodeResult) => void;
private channel: VmChannel | undefined;
/** Replies produced before the launcher returned the channel. */
private unsent: HostToWorkerMessage[] = [];
private readonly tools: ReadonlyMap<string, CodemodeTool>;
private readonly globals: ReadonlyMap<string, CodemodeTool>;
private readonly signal: AbortSignal | undefined;
private readonly timer: ReturnType<typeof setTimeout> | undefined;
private readonly output: CodemodeOutputItem[] = [];
private readonly calls: CodemodeCall[] = [];
private readonly pending = new Map<number, PendingCall>();
private finished = false;
constructor(options: ExecutionOptions) {
this.promise = new Promise<CodemodeResult>((resolve) => {
this.resolveResult = resolve;
});
this.tools = options.tools;
this.globals = options.globals;
this.signal = options.signal;
if (Number.isFinite(options.timeoutMs)) {
this.timer = setTimeout(() => {
this.finish({ kind: "timeout", message: `Execution timed out after ${options.timeoutMs} ms` });
}, options.timeoutMs);
}
if (options.signal) {
if (options.signal.aborted) {
this.onAbort();
} else {
options.signal.addEventListener("abort", this.onAbort, { once: true });
}
}
if (this.finished) return;
const data: VmStartData = {
code: options.code,
tools: [...options.tools.values()].map((tool) => ({
name: tool.name,
jsName: toCodemodeIdentifier(tool.name),
description: tool.description ?? "",
})),
globals: [...options.globals.values()].map((global) => ({
name: global.name,
spread: global.spread === true,
})),
memoryLimitBytes: options.memoryLimitBytes,
store: options.store,
};
const events: VmEvents = {
message: (message) => this.receive(message),
failure: (error) => this.finish(error),
};
new Promise<VmChannel>((resolve) => resolve(options.launch(data, events))).then(
(channel) => {
if (this.finished) {
void channel.stop().catch(() => undefined);
return;
}
this.channel = channel;
for (const message of this.unsent.splice(0)) channel.post(message);
},
(error: unknown) => {
this.finish({ kind: "sandbox", message: errorMessage(error) });
},
);
}
abort(message: string): Promise<CodemodeResult> {
this.finish({ kind: "aborted", message });
return this.promise;
}
private readonly onAbort = (): void => {
const reason: unknown = this.signal?.reason;
this.finish({ kind: "aborted", message: reason instanceof Error ? reason.message : "Execution aborted" });
};
private post(message: HostToWorkerMessage): void {
if (this.channel) this.channel.post(message);
else this.unsent.push(message);
}
/** An exception here would escape into the launcher's transport and leave the execution unsettled. */
private receive(message: unknown): void {
try {
this.handleMessage(message);
} catch (error) {
this.finish({
kind: "sandbox",
message:
error instanceof BridgeError
? `Sandbox bridge broken: ${error.message}. The script may have modified built-ins such as a prototype's toJSON.`
: `Sandbox host failed: ${errorMessage(error)}`,
});
}
}
private handleMessage(message: unknown): void {
if (this.finished) return;
if (!isWorkerToHostMessage(message)) throw new BridgeError("unknown message from the worker");
switch (message.type) {
case "output":
this.output.push(message.item);
break;
case "call":
if (this.pending.has(message.id)) throw new BridgeError(`duplicate call id ${message.id}`);
void this.handleCall(message);
break;
case "done":
this.handleDone(message);
break;
case "crash":
this.finish({ kind: "sandbox", message: message.message });
break;
}
}
private handleDone(message: Extract<WorkerToHostMessage, { type: "done" }>): void {
// Decode everything before finish(), which must not throw once it starts.
if (!message.ok) {
this.finish(parseScriptError(message.error));
return;
}
const value = message.value === undefined ? undefined : parseBridgeJson(message.value, "return value");
this.finish(undefined, value, parseStoreWrites(message.writes));
}
private async handleCall(message: Extract<WorkerToHostMessage, { type: "call" }>): Promise<void> {
const { id, name } = message;
const isTool = message.target === "tool";
const record: CodemodeCall | undefined = isTool ? { name, status: "cancelled", durationMs: 0 } : undefined;
if (record) this.calls.push(record);
const pending: PendingCall = { record, startedAt: performance.now(), controller: new AbortController() };
this.pending.set(id, pending);
let status: CodemodeCallStatus;
let reply: HostToWorkerMessage;
try {
const tool = (isTool ? this.tools : this.globals).get(name);
if (!tool) throw new Error(`Unknown ${isTool ? "tool" : "global"} "${name}"`);
const args: unknown = message.args === undefined ? undefined : JSON.parse(message.args);
const value = await tool.execute(args, { signal: pending.controller.signal });
reply = { type: "result", id, ok: true, payload: value === undefined ? undefined : JSON.stringify(value) };
status = "ok";
} catch (error) {
reply = { type: "result", id, ok: false, payload: errorMessage(error) };
status = "error";
}
// Already cancelled by finish(): the record keeps "cancelled" and the
// VM is gone or going.
if (!this.pending.delete(id)) return;
if (record) {
record.status = status;
record.durationMs = performance.now() - pending.startedAt;
}
this.post(reply);
}
private finish(error: CodemodeError | undefined, value?: unknown, writes?: CodemodeStoreWrites): void {
if (this.finished) return;
this.finished = true;
clearTimeout(this.timer);
this.signal?.removeEventListener("abort", this.onAbort);
const now = performance.now();
for (const pending of this.pending.values()) {
if (pending.record) pending.record.durationMs = now - pending.startedAt;
pending.controller.abort();
}
this.pending.clear();
this.unsent = [];
const result: CodemodeResult = error
? { ok: false, error, output: this.output, calls: this.calls }
: {
ok: true,
value,
output: this.output,
calls: this.calls,
storeWrites: writes ?? { set: {}, delete: [] },
};
if (!this.channel) {
this.resolveResult(result);
return;
}
this.channel
.stop()
.catch(() => undefined)
.then(() => this.resolveResult(result));
}
}
/**
* Tool table, defaults, and in-flight executions; the behavior every sandbox shares. Subclasses
* only choose where the VM runs by passing a {@link VmLauncher}.
*
* The script sees `tools.<name>(args)` for every registered tool, `ALL_TOOLS`, the output helpers
* `text`, `image`, `exit`, and `console.*`, `store`/`load`, and the configured globals; nothing
* else (no timers, `fetch`, `process`, `require`, modules). Each `execute()` gets its own VM; the
* sandbox only holds the tool table and defaults. `close()` aborts in-flight executions.
*/
export class CodemodeSandboxBase {
private readonly toolsByName = new Map<string, CodemodeTool>();
private readonly globalsByName = new Map<string, CodemodeTool>();
private readonly timeoutMs: number;
private readonly memoryLimitBytes: number | undefined;
private readonly launch: VmLauncher;
private readonly running = new Set<Execution>();
private closed = false;
constructor(options: CodemodeSandboxBaseOptions, launch: VmLauncher) {
this.timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
this.memoryLimitBytes = options.memoryLimitBytes;
this.launch = launch;
for (const tool of options.tools ?? []) this.registerTool(tool);
const namespaces = new Set<string>();
for (const global of options.globals ?? []) {
const parts = global.name.split(".");
if (parts.length > 2 || !parts.every((part) => IDENTIFIER.test(part)) || RESERVED_GLOBALS.has(parts[0])) {
throw new Error(`Invalid global name "${global.name}"`);
}
if (this.globalsByName.has(global.name)) throw new Error(`Global "${global.name}" is already registered`);
if (parts.length === 2) namespaces.add(parts[0]);
this.globalsByName.set(global.name, global);
}
for (const name of namespaces) {
if (this.globalsByName.has(name)) throw new Error(`Global "${name}" conflicts with the namespace "${name}"`);
}
}
/** Throws if a tool with the same name is already registered. */
registerTool(tool: CodemodeTool): void {
if (this.toolsByName.has(tool.name)) throw new Error(`Tool "${tool.name}" is already registered`);
this.toolsByName.set(tool.name, tool);
}
unregisterTool(name: string): boolean {
return this.toolsByName.delete(name);
}
get tools(): CodemodeTool[] {
return [...this.toolsByName.values()];
}
get globals(): CodemodeTool[] {
return [...this.globalsByName.values()];
}
/**
* `code` is an async function body: `return` and top-level `await` work.
* Never rejects for script failures; those come back as `{ ok: false }`.
* The script can use `store(key, value)` and `load(key)` on `options.store`.
*/
execute(code: string, options: CodemodeExecuteOptions = {}): Promise<CodemodeResult> {
if (this.closed) return Promise.reject(new Error("Sandbox is closed"));
const execution = new Execution({
code,
tools: new Map(this.toolsByName),
globals: this.globalsByName,
timeoutMs: options.timeoutMs ?? this.timeoutMs,
signal: options.signal,
memoryLimitBytes: this.memoryLimitBytes,
store: serializeStore(options.store),
launch: this.launch,
});
this.running.add(execution);
return execution.promise.finally(() => this.running.delete(execution));
}
/** Aborts in-flight executions (they resolve with `kind: "aborted"`) and rejects new ones. */
async close(): Promise<void> {
this.closed = true;
await Promise.all([...this.running].map((execution) => execution.abort("Sandbox closed")));
}
}

424
packages/codemode/src/runtime/host.ts
View file

@ -1,416 +1,70 @@
import { Worker } from "node:worker_threads";
import { toCodemodeIdentifier } from "../identifier.ts";
import type {
CodemodeCall,
CodemodeCallStatus,
CodemodeError,
CodemodeExecuteOptions,
CodemodeOutputItem,
CodemodeResult,
CodemodeSandboxOptions,
CodemodeStoreWrites,
CodemodeTool,
} from "../types.ts";
import type { CodemodeSandboxOptions } from "../types.ts";
import { type CodemodeWasmModule, loadQuickJSWasm } from "../wasm.ts";
import {
type HostToWorkerMessage,
isWorkerToHostMessage,
type WorkerData,
type WorkerToHostMessage,
} from "./protocol.ts";
const DEFAULT_TIMEOUT_MS = 300_000;
const IDENTIFIER = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
const RESERVED_GLOBALS: ReadonlySet<string> = new Set([
"tools",
"ALL_TOOLS",
"console",
"text",
"image",
"exit",
"globalThis",
"store",
"load",
]);
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function serializeStore(store: Readonly<Record<string, unknown>> | undefined): Record<string, string> {
const serialized: Record<string, string> = {};
for (const [key, value] of Object.entries(store ?? {})) {
const json = JSON.stringify(value);
if (json !== undefined) serialized[key] = json;
}
return serialized;
}
/**
* The prelude that serializes values for the host runs in the same VM as the script, so a script
* that patches built-ins (for example `Array.prototype.toJSON`) can make it send malformed data.
* Payloads that do not decode throw this error, and the execution fails as a sandbox error.
*/
class BridgeError extends Error {}
function parseBridgeJson(json: string, what: string): unknown {
try {
return JSON.parse(json);
} catch {
throw new BridgeError(`${what} is not valid JSON`);
}
}
function parseStoreWrites(json: string): CodemodeStoreWrites {
const entries = parseBridgeJson(json, "store writes");
if (!Array.isArray(entries)) throw new BridgeError("store writes are not an array");
const writes: CodemodeStoreWrites = { set: {}, delete: [] };
for (const entry of entries) {
if (
!Array.isArray(entry) ||
typeof entry[0] !== "string" ||
(entry.length === 2 ? typeof entry[1] !== "string" : entry.length !== 1)
) {
throw new BridgeError("store writes contain a malformed entry");
}
const [key, value] = entry as [string, string?];
if (value === undefined) writes.delete.push(key);
else writes.set[key] = parseBridgeJson(value, `store value for ${JSON.stringify(key)}`);
}
return writes;
}
function parseScriptError(json: string): CodemodeError {
const parsed = parseBridgeJson(json, "script error");
if (typeof parsed !== "object" || parsed === null) throw new BridgeError("script error is not an object");
const { name, message, stack } = parsed as Record<string, unknown>;
if (
typeof message !== "string" ||
(name !== undefined && typeof name !== "string") ||
(stack !== undefined && typeof stack !== "string")
) {
throw new BridgeError("script error is malformed");
}
return { kind: "script", name, message, stack };
}
import { CodemodeSandboxBase, errorMessage, type VmLauncher } from "./execution.ts";
import type { WorkerData } from "./protocol.ts";
function defaultWorkerUrl(): URL {
// `.ts` when running from source (tests, tsx), `.js` from the published dist.
return new URL(import.meta.url.endsWith(".ts") ? "./worker.ts" : "./worker.js", import.meta.url);
}
interface PendingCall {
record: CodemodeCall | undefined;
startedAt: number;
controller: AbortController;
}
interface ExecutionOptions {
code: string;
tools: ReadonlyMap<string, CodemodeTool>;
globals: ReadonlyMap<string, CodemodeTool>;
timeoutMs: number;
signal: AbortSignal | undefined;
memoryLimitBytes: number | undefined;
store: Record<string, string>;
wasm: Promise<CodemodeWasmModule>;
workerUrl: string | URL;
}
/**
* One script run in its own worker and QuickJS VM. A fresh worker per run keeps
* termination simple: a runaway script, including one that only spins the
* microtask queue, is killed with `terminate()` and cannot poison a later run.
* Each execution gets its own worker thread and QuickJS VM. A fresh worker per run keeps
* termination simple: a runaway script, including one that only spins the microtask queue, is
* killed with `terminate()` and cannot poison a later run. Before terminating, the host sets a
* shared interrupt flag that the VM polls, because Bun's `terminate()` cannot stop a thread that
* is spinning in wasm.
*/
class Execution {
readonly promise: Promise<CodemodeResult>;
private resolveResult!: (result: CodemodeResult) => void;
private worker: Worker | undefined;
private readonly interrupt = new SharedArrayBuffer(4);
private readonly tools: ReadonlyMap<string, CodemodeTool>;
private readonly globals: ReadonlyMap<string, CodemodeTool>;
private readonly signal: AbortSignal | undefined;
private readonly timer: NodeJS.Timeout | undefined;
private readonly output: CodemodeOutputItem[] = [];
private readonly calls: CodemodeCall[] = [];
private readonly pending = new Map<number, PendingCall>();
private finished = false;
constructor(options: ExecutionOptions) {
this.promise = new Promise<CodemodeResult>((resolve) => {
this.resolveResult = resolve;
});
this.tools = options.tools;
this.globals = options.globals;
this.signal = options.signal;
if (Number.isFinite(options.timeoutMs)) {
this.timer = setTimeout(() => {
this.finish({ kind: "timeout", message: `Execution timed out after ${options.timeoutMs} ms` });
}, options.timeoutMs);
function workerLauncher(
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule> | undefined,
workerUrl: string | URL,
): VmLauncher {
return async (data, events) => {
let module: CodemodeWasmModule;
try {
module = await (wasm ?? loadQuickJSWasm());
} catch (error) {
throw new Error(`Failed to load QuickJS: ${errorMessage(error)}`);
}
if (options.signal) {
if (options.signal.aborted) {
this.onAbort();
} else {
options.signal.addEventListener("abort", this.onAbort, { once: true });
}
}
options.wasm.then(
(wasm) => this.start(options, wasm),
(error: unknown) => {
this.finish({ kind: "sandbox", message: `Failed to load QuickJS: ${errorMessage(error)}` });
},
);
}
abort(message: string): Promise<CodemodeResult> {
this.finish({ kind: "aborted", message });
return this.promise;
}
private start(options: ExecutionOptions, wasm: CodemodeWasmModule): void {
if (this.finished) return;
const workerData: WorkerData = {
code: options.code,
tools: [...options.tools.values()].map((tool) => ({
name: tool.name,
jsName: toCodemodeIdentifier(tool.name),
description: tool.description ?? "",
})),
globals: [...options.globals.values()].map((global) => ({
name: global.name,
spread: global.spread === true,
})),
wasm,
memoryLimitBytes: options.memoryLimitBytes,
store: options.store,
interrupt: this.interrupt,
};
const interrupt = new SharedArrayBuffer(4);
const workerData: WorkerData = { ...data, wasm: module, interrupt };
let worker: Worker;
try {
worker = new Worker(options.workerUrl, { workerData });
worker = new Worker(workerUrl, { workerData });
} catch (error) {
this.finish({ kind: "sandbox", message: `Failed to start worker: ${errorMessage(error)}` });
return;
throw new Error(`Failed to start worker: ${errorMessage(error)}`);
}
this.worker = worker;
worker.on("message", (message: unknown) => {
// An exception here would be uncaught in the host process and leave the execution unsettled.
try {
this.handleMessage(message);
} catch (error) {
this.finish({
kind: "sandbox",
message:
error instanceof BridgeError
? `Sandbox bridge broken: ${error.message}. The script may have modified built-ins such as a prototype's toJSON.`
: `Sandbox host failed: ${errorMessage(error)}`,
});
}
});
worker.on("message", (message: unknown) => events.message(message));
worker.on("error", (error: unknown) => {
this.finish({
events.failure({
kind: "sandbox",
name: error instanceof Error ? error.name : undefined,
message: errorMessage(error),
});
});
worker.on("exit", (code) => {
this.finish({ kind: "sandbox", message: `Worker exited with code ${code} before the script settled` });
events.failure({ kind: "sandbox", message: `Worker exited with code ${code} before the script settled` });
});
}
private readonly onAbort = (): void => {
const reason: unknown = this.signal?.reason;
this.finish({ kind: "aborted", message: reason instanceof Error ? reason.message : "Execution aborted" });
return {
post: (message) => worker.postMessage(message),
stop: async () => {
Atomics.store(new Int32Array(interrupt), 0, 1);
await worker.terminate().catch(() => undefined);
},
};
};
private post(message: HostToWorkerMessage): void {
this.worker?.postMessage(message);
}
private handleMessage(message: unknown): void {
if (this.finished) return;
if (!isWorkerToHostMessage(message)) throw new BridgeError("unknown message from the worker");
switch (message.type) {
case "output":
this.output.push(message.item);
break;
case "call":
if (this.pending.has(message.id)) throw new BridgeError(`duplicate call id ${message.id}`);
void this.handleCall(message);
break;
case "done":
this.handleDone(message);
break;
case "crash":
this.finish({ kind: "sandbox", message: message.message });
break;
}
}
private handleDone(message: Extract<WorkerToHostMessage, { type: "done" }>): void {
// Decode everything before finish(), which must not throw once it starts.
if (!message.ok) {
this.finish(parseScriptError(message.error));
return;
}
const value = message.value === undefined ? undefined : parseBridgeJson(message.value, "return value");
this.finish(undefined, value, parseStoreWrites(message.writes));
}
private async handleCall(message: Extract<WorkerToHostMessage, { type: "call" }>): Promise<void> {
const { id, name } = message;
const isTool = message.target === "tool";
const record: CodemodeCall | undefined = isTool ? { name, status: "cancelled", durationMs: 0 } : undefined;
if (record) this.calls.push(record);
const pending: PendingCall = { record, startedAt: performance.now(), controller: new AbortController() };
this.pending.set(id, pending);
let status: CodemodeCallStatus;
let reply: HostToWorkerMessage;
try {
const tool = (isTool ? this.tools : this.globals).get(name);
if (!tool) throw new Error(`Unknown ${isTool ? "tool" : "global"} "${name}"`);
const args: unknown = message.args === undefined ? undefined : JSON.parse(message.args);
const value = await tool.execute(args, { signal: pending.controller.signal });
reply = { type: "result", id, ok: true, payload: value === undefined ? undefined : JSON.stringify(value) };
status = "ok";
} catch (error) {
reply = { type: "result", id, ok: false, payload: errorMessage(error) };
status = "error";
}
// Already cancelled by finish(): the record keeps "cancelled" and the
// worker is gone or going.
if (!this.pending.delete(id)) return;
if (record) {
record.status = status;
record.durationMs = performance.now() - pending.startedAt;
}
this.post(reply);
}
private finish(error: CodemodeError | undefined, value?: unknown, writes?: CodemodeStoreWrites): void {
if (this.finished) return;
this.finished = true;
clearTimeout(this.timer);
this.signal?.removeEventListener("abort", this.onAbort);
const now = performance.now();
for (const pending of this.pending.values()) {
if (pending.record) pending.record.durationMs = now - pending.startedAt;
pending.controller.abort();
}
this.pending.clear();
const result: CodemodeResult = error
? { ok: false, error, output: this.output, calls: this.calls }
: {
ok: true,
value,
output: this.output,
calls: this.calls,
storeWrites: writes ?? { set: {}, delete: [] },
};
if (!this.worker) {
this.resolveResult(result);
return;
}
Atomics.store(new Int32Array(this.interrupt), 0, 1);
this.worker
.terminate()
.catch(() => undefined)
.then(() => this.resolveResult(result));
}
}
/**
* Runs JavaScript in a QuickJS VM (a separate wasm instance) inside a worker
* thread. The script sees `tools.<name>(args)` for every registered tool, `ALL_TOOLS`,
* the output helpers `text`, `image`, `exit`, and `console.*`, `store`/`load`, and the
* configured globals; nothing else (no timers, `fetch`, `process`, `require`, modules).
*
* Each `execute()` gets its own worker and VM; the sandbox only holds the tool
* table and defaults. `close()` aborts in-flight executions.
* Runs JavaScript in a QuickJS VM (a separate wasm instance) inside a worker thread. The worker
* keeps script execution off the host thread: QuickJS runs synchronously, so a spinning script on
* the host thread would block its event loop. Requires `node:worker_threads` (Node or Bun); see
* `InlineCodemodeSandbox` and `RemoteCodemodeSandbox` in `@earendil-works/pi-codemode/portable`
* for other runtimes.
*/
export class CodemodeSandbox {
private readonly toolsByName = new Map<string, CodemodeTool>();
private readonly globalsByName = new Map<string, CodemodeTool>();
private readonly timeoutMs: number;
private readonly memoryLimitBytes: number | undefined;
private readonly wasm: CodemodeWasmModule | Promise<CodemodeWasmModule> | undefined;
private readonly workerUrl: string | URL;
private readonly running = new Set<Execution>();
private closed = false;
export class CodemodeSandbox extends CodemodeSandboxBase {
constructor(options: CodemodeSandboxOptions = {}) {
this.timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
this.memoryLimitBytes = options.memoryLimitBytes;
this.wasm = options.wasm;
this.workerUrl = options.workerUrl ?? defaultWorkerUrl();
for (const tool of options.tools ?? []) this.registerTool(tool);
const namespaces = new Set<string>();
for (const global of options.globals ?? []) {
const parts = global.name.split(".");
if (parts.length > 2 || !parts.every((part) => IDENTIFIER.test(part)) || RESERVED_GLOBALS.has(parts[0])) {
throw new Error(`Invalid global name "${global.name}"`);
}
if (this.globalsByName.has(global.name)) throw new Error(`Global "${global.name}" is already registered`);
if (parts.length === 2) namespaces.add(parts[0]);
this.globalsByName.set(global.name, global);
}
for (const name of namespaces) {
if (this.globalsByName.has(name)) throw new Error(`Global "${name}" conflicts with the namespace "${name}"`);
}
}
/** Throws if a tool with the same name is already registered. */
registerTool(tool: CodemodeTool): void {
if (this.toolsByName.has(tool.name)) throw new Error(`Tool "${tool.name}" is already registered`);
this.toolsByName.set(tool.name, tool);
}
unregisterTool(name: string): boolean {
return this.toolsByName.delete(name);
}
get tools(): CodemodeTool[] {
return [...this.toolsByName.values()];
}
get globals(): CodemodeTool[] {
return [...this.globalsByName.values()];
}
/**
* `code` is an async function body: `return` and top-level `await` work.
* Never rejects for script failures; those come back as `{ ok: false }`.
* The script can use `store(key, value)` and `load(key)` on `options.store`.
*/
execute(code: string, options: CodemodeExecuteOptions = {}): Promise<CodemodeResult> {
if (this.closed) return Promise.reject(new Error("Sandbox is closed"));
const execution = new Execution({
code,
tools: new Map(this.toolsByName),
globals: this.globalsByName,
timeoutMs: options.timeoutMs ?? this.timeoutMs,
signal: options.signal,
memoryLimitBytes: this.memoryLimitBytes,
store: serializeStore(options.store),
wasm: this.wasm === undefined ? loadQuickJSWasm() : Promise.resolve(this.wasm),
workerUrl: this.workerUrl,
});
this.running.add(execution);
return execution.promise.finally(() => this.running.delete(execution));
}
/** Aborts in-flight executions (they resolve with `kind: "aborted"`) and rejects new ones. */
async close(): Promise<void> {
this.closed = true;
await Promise.all([...this.running].map((execution) => execution.abort("Sandbox closed")));
super(options, workerLauncher(options.wasm, options.workerUrl ?? defaultWorkerUrl()));
}
}

125
packages/codemode/src/runtime/inline.ts Normal file
View file

@ -0,0 +1,125 @@
/**
* Runs the VM on the calling thread, for runtimes without worker threads (Cloudflare Workers and
* Durable Objects, browsers) and for the server side of `RemoteCodemodeSandbox`.
*
* Problem: QuickJS runs synchronously. While a script computes, the host's event loop does not
* run, so neither the deadline timer nor an abort signal can fire. On Workers even `Date.now()` is
* frozen during synchronous execution, so the VM cannot check a clock either. Example: with a
* worker thread, `while (true) {}` is stopped by the host's 200 ms timer; on the calling thread
* that timer never gets a chance to run.
*
* Solution: an interrupt budget. QuickJS polls its interrupt handler periodically while a script
* runs; this launcher counts the polls across the whole execution and interrupts the script once
* the count passes the budget. The execution then fails with `kind: "timeout"`. The deadline and
* abort signal still work whenever the script waits on a tool, since the event loop runs then.
*
* The rate of polls depends on the code and the machine. Measured with a tight loop: about 50,000
* per second of CPU on an Apple M-series laptop and about 10,000 per second on Cloudflare Durable
* Objects (October 2026). The default of 100,000 allows roughly 2 to 10 seconds of uninterrupted
* computation; keep it well below the platform's own CPU limit, because hitting that limit resets
* a Durable Object instead of failing one script.
*
* Once the VM reports `done` or `crash`, the result is decided, so the VM is interrupted at the
* next poll instead of running on until the host stops it. Example: a script that keeps printing in
* a `for (;;)` loop after its output limit was reached would otherwise run until the budget ran out
* and be reported as a timeout.
*
* Stack: the VM runs on the caller's stack, which is smaller than a worker thread's (about 1 MB
* on Node's main thread), so QuickJS's stack guard is set to {@link INLINE_MAX_STACK_SIZE}.
*
* Reentrancy: VM callbacks run inside QuickJS. Messages from the VM are delivered to the host in
* a microtask, after the VM has returned, so host code (tool calls, `finish()`, disposal) never
* runs inside a VM callback and never re-enters the VM.
*/
import type { CodemodeSandboxBaseOptions } from "../types.ts";
import type { CodemodeWasmModule } from "../wasm.ts";
import { CodemodeSandboxBase, errorMessage, type VmLauncher } from "./execution.ts";
import { startVm } from "./vm.ts";
export const DEFAULT_INTERRUPT_BUDGET = 100_000;
/** Allows recursion about 1,000 calls deep and trips well before a 1 MB host stack overflows. */
export const INLINE_MAX_STACK_SIZE = 256 * 1024;
export interface InlineCodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Compiled `quickjs-wasi/quickjs.wasm`. On Cloudflare, import it with Wrangler's `CompiledWasm`
* rule (`@earendil-works/pi-codemode/cloudflare` does this); on Node, `loadQuickJSWasm()`.
*/
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>;
/**
* Interrupt polls an execution may use before it fails with `kind: "timeout"`. The only limit on
* a script that computes without awaiting a tool. Default: {@link DEFAULT_INTERRUPT_BUDGET}.
*/
interruptBudget?: number;
}
export function inlineLauncher(
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>,
interruptBudget: number = DEFAULT_INTERRUPT_BUDGET,
): VmLauncher {
return async (data, events) => {
let module: CodemodeWasmModule;
try {
module = await wasm;
} catch (error) {
throw new Error(`Failed to load QuickJS: ${errorMessage(error)}`);
}
let stopped = false;
let settled = false;
let polls = 0;
let exhausted = false;
const session = await startVm({
data,
wasm: module,
maxStackSize: INLINE_MAX_STACK_SIZE,
shouldInterrupt: () => {
if (stopped || settled || exhausted) return true;
if (++polls <= interruptBudget) return false;
exhausted = true;
return true;
},
post: (message) => {
const terminal = message.type === "done" || message.type === "crash";
// Only the first terminal message counts: a crash caused by interrupting a settled VM is noise.
if (terminal && settled) return;
if (terminal) settled = true;
queueMicrotask(() => {
if (stopped) return;
// The interrupt surfaces as a crash or a failed done; report what caused it instead.
if (exhausted && (message.type === "done" || message.type === "crash")) {
events.failure({
kind: "timeout",
message: `Execution exceeded its interrupt budget of ${interruptBudget}`,
});
return;
}
events.message(message);
});
},
});
return {
post: (message) => {
if (!stopped) session.deliver(message);
},
stop: async () => {
if (stopped) return;
stopped = true;
session.dispose();
},
};
};
}
/**
* Runs each execution's QuickJS VM on the calling thread. Works in any runtime with WebAssembly
* and microtasks; needs no worker threads, files, or Node built-ins. A script that computes
* blocks the caller's thread until it awaits a tool or uses up its interrupt budget, so prefer
* `CodemodeSandbox` where worker threads exist, and `RemoteCodemodeSandbox` to keep scripts off a
* thread that must stay responsive (such as a Durable Object that serves a session).
*/
export class InlineCodemodeSandbox extends CodemodeSandboxBase {
constructor(options: InlineCodemodeSandboxOptions) {
super(options, inlineLauncher(options.wasm, options.interruptBudget));
}
}

13
packages/codemode/src/runtime/protocol.ts
View file

@ -7,16 +7,23 @@ import type { CodemodeWasmModule } from "../wasm.ts";
* out of the QuickJS VM as strings and never builds structured values itself.
*/
export interface WorkerData {
/**
* Everything a VM needs to run one script, independent of where it runs. Plain data, so it can
* cross a `postMessage` or a Workers RPC call unchanged.
*/
export interface VmStartData {
code: string;
/** `jsName` is the identifier the script uses; `description` is listed in `ALL_TOOLS`. */
tools: { name: string; jsName: string; description: string }[];
globals: { name: string; spread: boolean }[];
/** Compiled `quickjs-wasi` module. Structured clone shares the compiled code with the worker. */
wasm: CodemodeWasmModule;
memoryLimitBytes: number | undefined;
/** Snapshot for `load()`: key to JSON text. */
store: Record<string, string>;
}
export interface WorkerData extends VmStartData {
/** Compiled `quickjs-wasi` module. Structured clone shares the compiled code with the worker. */
wasm: CodemodeWasmModule;
/**
* One Int32 the host sets to non-zero before terminating the worker. The VM's interrupt handler
* polls it, because Bun's `worker.terminate()` cannot stop a thread that is spinning in wasm.

272
packages/codemode/src/runtime/remote.ts Normal file
View file

@ -0,0 +1,272 @@
/**
* Runs the VM in another isolate, reached over an RPC boundary such as a Cloudflare Durable Object
* stub, while tools keep running in the caller.
*
* Problem: a VM on the calling thread blocks that thread while the script computes. In a Durable
* Object that serves an agent session, a CPU-heavy script would stall every other request to the
* session (including the user's abort) for seconds. Measured on Cloudflare (October 2026), a
* script that computed for 10 seconds delayed the session's other requests by 10 seconds when it
* ran in the session object or in a Dynamic Worker, but by about 50 ms when it ran in a Durable
* Object of a separately deployed Worker.
*
* Solution: the caller keeps the host role (`RemoteCodemodeSandbox`: tools, call records,
* deadline, abort) and the remote side only runs the VM (`serveCodemodeRemote`, behind
* `CodemodeSandboxDurableObject` in `@earendil-works/pi-codemode/cloudflare`).
*
* Protocol, one RPC call per execution:
*
* client remote
* remote.execute(request, exchange) ───────▶ starts the VM (in-thread, with interrupt budget)
* ◀─────── exchange([...outputs, call]) one call per tool call
* runs the tool, replies with result ──────▶ VM continues
* ◀─────── exchange([...outputs, done]) terminal, reply ignored
* execute() resolves ◀────────────────────── VM disposed
*
* `exchange` is a function the client passes in; over Workers RPC it becomes a stub that calls back
* into the client. Output is buffered on the remote side and sent with the next call or the
* terminal message, so a script that prints in a loop does not make one RPC per line. Each
* `exchange` carries at most one call, so concurrent tool calls are concurrent RPCs. Calls on one
* stub are delivered in order, which keeps output ordered before the call or result that follows.
*
* Termination: the client cannot interrupt a remote VM that is computing. When the client finishes
* first (deadline, abort, `close()`), it answers every pending and later `exchange` with
* `{ type: "stop" }`, and the remote stops the VM at that point. A script that computes without
* calling a tool runs until its interrupt budget is used up. The remote reports that as a
* `failure` message, which the client turns into the same `kind: "timeout"` error an in-thread
* VM produces.
*
* Failure: if the RPC call rejects (the remote isolate crashed, hit its CPU limit, or was
* redeployed) or returns without a terminal message, the execution fails with `kind: "sandbox"`.
*/
import type { CodemodeError, CodemodeSandboxBaseOptions } from "../types.ts";
import type { CodemodeWasmModule } from "../wasm.ts";
import { CodemodeSandboxBase, errorMessage, type VmChannel, type VmLauncher } from "./execution.ts";
import { DEFAULT_INTERRUPT_BUDGET, inlineLauncher } from "./inline.ts";
import {
type HostToWorkerMessage,
isWorkerToHostMessage,
type VmStartData,
type WorkerToHostMessage,
} from "./protocol.ts";
export interface CodemodeRemoteRequest {
data: VmStartData;
/** Requested interrupt budget. The remote may lower it, see {@link CodemodeServeOptions}. */
interruptBudget?: number;
}
/** What the remote sends to the client: VM messages, or a failure outside the VM bridge. */
export type CodemodeRemoteMessage = WorkerToHostMessage | { type: "failure"; error: CodemodeError };
/**
* The client's answer to an exchange: the result of the call the exchange carried, `null` when it
* carried no call, or `stop` once the client no longer wants the execution.
*/
export type CodemodeRemoteReply = HostToWorkerMessage | { type: "stop" } | null;
export type CodemodeRemoteExchange = (messages: CodemodeRemoteMessage[]) => Promise<CodemodeRemoteReply>;
/**
* The remote end, usually a Durable Object stub for `CodemodeSandboxDurableObject`. Anything with
* this method works, for example an in-process object in tests.
*/
export interface CodemodeRemote {
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange): Promise<void>;
}
export interface RemoteCodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Returns the remote that runs one execution, called once per `execute()`. For a Durable Object,
* pick the instance here, for example `() => env.CODEMODE_SANDBOX.getByName(sessionId)`.
*/
remote: () => CodemodeRemote;
/** Interrupt budget requested from the remote. Default: the remote's default. */
interruptBudget?: number;
}
/** Output buffered on the remote before it is flushed without waiting for a call. */
const FLUSH_ITEMS = 1_000;
const FLUSH_CHARS = 1 << 20;
export interface CodemodeServeOptions {
wasm: CodemodeWasmModule | Promise<CodemodeWasmModule>;
/** Upper bound for a request's interrupt budget, and the budget when a request names none. */
maxInterruptBudget?: number;
/** Upper bound for a request's VM memory, and the limit when a request names none. */
maxMemoryLimitBytes?: number;
}
/**
* Remote side of {@link RemoteCodemodeSandbox}: runs one execution's VM on this thread and
* forwards everything it reports through `exchange`. Resolves once the VM has stopped and the
* terminal message has been delivered, or the client asked to stop. Never rejects.
*/
export function serveCodemodeRemote(
request: CodemodeRemoteRequest,
exchange: CodemodeRemoteExchange,
options: CodemodeServeOptions,
): Promise<void> {
const maxBudget = options.maxInterruptBudget ?? DEFAULT_INTERRUPT_BUDGET;
const budget = Math.min(request.interruptBudget ?? maxBudget, maxBudget);
const requestedMemory = request.data.memoryLimitBytes;
const maxMemory = options.maxMemoryLimitBytes;
const memoryLimitBytes =
requestedMemory === undefined
? maxMemory
: maxMemory === undefined
? requestedMemory
: Math.min(requestedMemory, maxMemory);
const data: VmStartData = { ...request.data, memoryLimitBytes };
return new Promise<void>((resolve) => {
let channel: VmChannel | undefined;
let ended = false;
let buffered: CodemodeRemoteMessage[] = [];
let bufferedChars = 0;
/** Replies that arrived before the launcher returned the channel. */
const unsent: HostToWorkerMessage[] = [];
const takeBuffer = (): CodemodeRemoteMessage[] => {
const batch = buffered;
buffered = [];
bufferedChars = 0;
return batch;
};
const end = (terminal: CodemodeRemoteMessage | undefined): void => {
if (ended) return;
ended = true;
const batch = takeBuffer();
const delivered = terminal ? exchange([...batch, terminal]).catch(() => null) : Promise.resolve();
void delivered
.then(() => channel?.stop())
.catch(() => undefined)
.then(() => resolve());
};
const reply = (message: CodemodeRemoteReply): void => {
if (ended || message === null) return;
if (message.type === "stop") {
end(undefined);
return;
}
if (channel) channel.post(message);
else unsent.push(message);
};
const onMessage = (message: unknown): void => {
if (ended) return;
const typed = message as WorkerToHostMessage;
switch (typed.type) {
case "output": {
buffered.push(typed);
bufferedChars += typed.item.type === "text" ? typed.item.text.length : typed.item.data.length;
if (buffered.length >= FLUSH_ITEMS || bufferedChars >= FLUSH_CHARS) {
exchange(takeBuffer()).then(reply, () => end(undefined));
}
break;
}
case "call":
exchange([...takeBuffer(), typed]).then(reply, () => end(undefined));
break;
default:
end(typed);
}
};
const launch = inlineLauncher(options.wasm, budget);
Promise.resolve(
launch(data, {
message: onMessage,
failure: (error) => end({ type: "failure", error }),
}),
).then(
(started) => {
channel = started;
if (ended) {
void started.stop().catch(() => undefined);
return;
}
for (const message of unsent.splice(0)) started.post(message);
},
(error: unknown) => end({ type: "failure", error: { kind: "sandbox", message: errorMessage(error) } }),
);
});
}
function remoteLauncher(remote: () => CodemodeRemote, interruptBudget: number | undefined): VmLauncher {
return (data, events) => {
let stopped = false;
let settled = false;
const waiting = new Map<number, (reply: CodemodeRemoteReply) => void>();
const exchange: CodemodeRemoteExchange = (messages) =>
new Promise<CodemodeRemoteReply>((resolve) => {
if (stopped || !Array.isArray(messages)) {
resolve({ type: "stop" });
return;
}
// Register the waiter first: the host may answer a call synchronously, for example when the
// tool throws before returning a promise.
const call = messages.find((message) => isWorkerToHostMessage(message) && message.type === "call");
if (call?.type === "call") waiting.set(call.id, resolve);
else resolve(null);
for (const message of messages) {
if (typeof message === "object" && message !== null && message.type === "failure") {
settled = true;
events.failure(message.error);
continue;
}
// Malformed messages still go to the host, which fails the execution as a broken bridge.
events.message(message);
if (isWorkerToHostMessage(message) && (message.type === "done" || message.type === "crash"))
settled = true;
}
});
let target: CodemodeRemote;
try {
target = remote();
} catch (error) {
throw new Error(`Failed to reach the remote sandbox: ${errorMessage(error)}`);
}
const request: CodemodeRemoteRequest = interruptBudget === undefined ? { data } : { data, interruptBudget };
Promise.resolve()
.then(() => target.execute(request, exchange))
.then(
() => {
if (!settled && !stopped) {
events.failure({ kind: "sandbox", message: "Remote sandbox returned before the script settled" });
}
},
(error: unknown) => {
if (!stopped)
events.failure({ kind: "sandbox", message: `Remote sandbox failed: ${errorMessage(error)}` });
},
);
return {
post: (message) => {
const resolve = waiting.get(message.id);
if (!resolve) return;
waiting.delete(message.id);
resolve(message);
},
stop: async () => {
stopped = true;
for (const resolve of waiting.values()) resolve({ type: "stop" });
waiting.clear();
},
};
};
}
/**
* Runs each execution's VM on a remote, usually a Cloudflare Durable Object deployed with
* `CodemodeSandboxDurableObject`, while tools run here. Same API and results as
* `CodemodeSandbox`, so it can replace it. Differences: a script that computes without calling a
* tool is limited by the remote's interrupt budget rather than `timeoutMs`, and after a deadline or
* abort the remote VM keeps computing until it next calls a tool or uses up that budget.
*/
export class RemoteCodemodeSandbox extends CodemodeSandboxBase {
constructor(options: RemoteCodemodeSandboxOptions) {
super(options, remoteLauncher(options.remote, options.interruptBudget));
}
}

210
packages/codemode/src/runtime/vm.ts Normal file
View file

@ -0,0 +1,210 @@
/**
* Drives one QuickJS VM through one script, independent of the transport that connects it to the
* host. The worker thread, the in-thread launcher, and the remote server all run the VM through
* this module, so the bridge between prelude and host exists once.
*
* Contract with the caller:
* - `post` receives every message for the host, synchronously from inside VM callbacks. The
* caller must not re-enter the VM from `post` (no `deliver`, no `dispose`); a worker posts to
* another thread, and the in-thread launcher defers delivery to a microtask.
* - `deliver` hands the VM a tool result and runs the script until it waits again. It must only be
* called while the VM is idle, which holds for any caller that never re-enters from `post`.
* - `shouldInterrupt` is polled by QuickJS while the script runs, and here between batches of
* promise jobs. Returning true throws an uncatchable error in the script; the VM then reports
* `crash` or a failed `done`. The job batches matter for a script such as `while (true) await
* null`: every iteration is a separate job that executes only a few opcodes, so QuickJS's own
* polling, which counts opcodes, would let it run for a long time between polls.
* - The VM never decides on its own when it is finished: after `done` or `crash` it stays alive
* until the caller disposes it (or terminates the worker that hosts it).
*
* Failures after the VM exists are reported as a `crash` message; `startVm` only rejects when the
* VM cannot be created at all.
*/
import { JSException, type JSValueHandle, MAX_STACK_SIZE, QuickJS } from "quickjs-wasi";
import type { CodemodeWasmModule } from "../wasm.ts";
import { PRELUDE_SOURCE } from "./prelude-source.ts";
import type { HostToWorkerMessage, VmStartData, WorkerToHostMessage } from "./protocol.ts";
export interface VmSession {
deliver(message: HostToWorkerMessage): void;
/** Releases the VM's wasm instance. The session must not be used afterwards. */
dispose(): void;
}
export interface StartVmOptions {
data: VmStartData;
wasm: CodemodeWasmModule;
/**
* QuickJS's guard on its own stack, which turns deep recursion into a catchable `RangeError`.
* The wasm frames also use the host thread's native stack, so the guard must trip before that
* stack runs out: a host overflow fails the whole execution instead. Default: `MAX_STACK_SIZE`
* (512 KiB), which fits the 4 MB stack of a Node worker thread but not a 1 MB main thread.
*/
maxStackSize?: number;
post(message: WorkerToHostMessage): void;
shouldInterrupt(): boolean;
}
/**
* QuickJS writes engine diagnostics to fd 1 and 2, which the default shim
* forwards to the host's stdout and stderr. That output belongs to the host
* application (for example a TUI), so it is discarded. Reporting every byte as
* written keeps libc from retrying.
*/
function discardOutput(memory: { readonly buffer: ArrayBufferLike }) {
return {
fd_write(_fd: number, iovsPtr: number, iovsLen: number, nwrittenPtr: number): number {
const view = new DataView(memory.buffer);
let written = 0;
for (let i = 0; i < iovsLen; i++) {
written += view.getUint32(iovsPtr + i * 8 + 4, true);
}
view.setUint32(nwrittenPtr, written, true);
return 0;
},
};
}
function describeException(error: JSException): string {
const head = error.message ? `${error.name}: ${error.message}` : error.name;
const stack = error.stack?.trimEnd();
return JSON.stringify({ name: error.name, message: error.message, stack: stack ? `${head}\n${stack}` : head });
}
/** Promise jobs run between two `shouldInterrupt` polls while draining the job queue. */
const JOBS_PER_POLL = 50;
function crashMessage(error: unknown): WorkerToHostMessage {
return { type: "crash", message: error instanceof Error ? `${error.name}: ${error.message}` : String(error) };
}
export async function startVm(options: StartVmOptions): Promise<VmSession> {
const { data, post } = options;
const vm = await QuickJS.create({
wasm: options.wasm,
memoryLimit: data.memoryLimitBytes,
// Without a guard, deep recursion overflows the wasm stack and traps instead of throwing a
// catchable RangeError.
maxStackSize: options.maxStackSize ?? MAX_STACK_SIZE,
interruptHandler: options.shouldInterrupt,
wasi: discardOutput,
});
let disposed = false;
const session: VmSession = {
deliver(message) {
if (disposed) return;
try {
vm.withScope(() => {
vm.callFunction(
settle,
api,
vm.newNumber(message.id),
message.ok ? vm.true : vm.false,
message.payload === undefined ? vm.undefined : vm.newString(message.payload),
);
});
drain();
} catch (error) {
post(crashMessage(error));
}
},
dispose() {
if (disposed) return;
disposed = true;
vm.dispose();
},
};
let api: JSValueHandle;
let settle: JSValueHandle;
let stalled: JSValueHandle;
// `executePendingJobs()` runs the queue to empty without a chance to interrupt, so jobs run one at a
// time through the (internal, but public) exports; quickjs-wasi is pinned to an exact version.
const exports = vm._getExports();
/** Run queued jobs, then fail a script that waits on nothing that can ever resume it. */
const drain = () => {
let jobs = 0;
while (exports.qjs_is_job_pending()) {
// A job only fails for uncatchable errors, such as an interrupt.
if (exports.qjs_execute_pending_job() < 0) throw new Error("Promise job failed");
if (++jobs % JOBS_PER_POLL === 0 && options.shouldInterrupt()) throw new Error("Execution interrupted");
}
vm.callFunction(stalled, api).dispose();
};
try {
// Called from the prelude with primitives only.
const bridge = vm.newFunction("bridge", (kind, a, b, c) => {
switch (kind.toString()) {
case "call":
case "global":
post({
type: "call",
id: a.toNumber(),
target: kind.toString() === "call" ? "tool" : "global",
name: b.toString(),
args: c === undefined || c.isUndefined ? undefined : c.toString(),
});
break;
case "output":
post({
type: "output",
item:
a.toString() === "image"
? { type: "image", data: b.toString(), mimeType: c.toString() }
: a.toString() === "console"
? { type: "text", text: b.toString(), console: true }
: { type: "text", text: b.toString() },
});
break;
case "done":
if (a.toBoolean()) {
post({
type: "done",
ok: true,
value: b === undefined || b.isUndefined ? undefined : b.toString(),
writes: c.toString(),
});
} else {
post({ type: "done", ok: false, error: b.toString() });
}
break;
}
return vm.undefined;
});
// These handles live as long as the VM, which the caller disposes as a whole.
api = vm.withScope((scope) =>
scope.escape(
vm.callFunction(
vm.evalCode(PRELUDE_SOURCE, "codemode-prelude.js"),
vm.undefined,
bridge,
vm.newString(JSON.stringify(data.tools)),
vm.newString(JSON.stringify(data.globals)),
vm.newString(JSON.stringify(data.store)),
),
),
);
settle = api.getProp("settle");
const run = api.getProp("run");
stalled = api.getProp("stalled");
// The prefix shares the first line with the script so reported line numbers
// match the script as written.
let fn: JSValueHandle;
try {
fn = vm.evalCode(`(async (tools, console) => {${data.code}\n})`, "codemode.js");
} catch (error) {
if (!(error instanceof JSException)) throw error;
post({ type: "done", ok: false, error: describeException(error) });
return session;
}
vm.callFunction(run, api, fn).dispose();
fn.dispose();
drain();
} catch (error) {
post(crashMessage(error));
}
return session;
}

142
packages/codemode/src/runtime/worker.ts
View file

@ -11,151 +11,29 @@
* or embedded-module string specifier as `workerUrl`.
*/
import { parentPort, workerData } from "node:worker_threads";
import { JSException, type JSValueHandle, MAX_STACK_SIZE, QuickJS } from "quickjs-wasi";
import { PRELUDE_SOURCE } from "./prelude-source.ts";
import { isHostToWorkerMessage, type WorkerData, type WorkerToHostMessage } from "./protocol.ts";
import { startVm } from "./vm.ts";
function post(message: WorkerToHostMessage): void {
parentPort?.postMessage(message);
}
function crash(error: unknown): void {
post({ type: "crash", message: error instanceof Error ? `${error.name}: ${error.message}` : String(error) });
}
/**
* QuickJS writes engine diagnostics to fd 1 and 2, which the default shim
* forwards to the host's stdout and stderr. That output belongs to the host
* application (for example a TUI), so it is discarded. Reporting every byte as
* written keeps libc from retrying.
*/
function discardOutput(memory: { readonly buffer: ArrayBufferLike }) {
return {
fd_write(_fd: number, iovsPtr: number, iovsLen: number, nwrittenPtr: number): number {
const view = new DataView(memory.buffer);
let written = 0;
for (let i = 0; i < iovsLen; i++) {
written += view.getUint32(iovsPtr + i * 8 + 4, true);
}
view.setUint32(nwrittenPtr, written, true);
return 0;
},
};
}
function describeException(error: JSException): string {
const head = error.message ? `${error.name}: ${error.message}` : error.name;
const stack = error.stack?.trimEnd();
return JSON.stringify({ name: error.name, message: error.message, stack: stack ? `${head}\n${stack}` : head });
}
async function main(data: WorkerData): Promise<void> {
const interrupt = new Int32Array(data.interrupt);
const vm = await QuickJS.create({
// The VM lives until the host terminates the worker, so the session is never disposed.
const session = await startVm({
data,
wasm: data.wasm,
memoryLimit: data.memoryLimitBytes,
// Without a guard, deep recursion overflows the wasm stack and traps instead of throwing a
// catchable RangeError.
maxStackSize: MAX_STACK_SIZE,
interruptHandler: () => Atomics.load(interrupt, 0) !== 0,
wasi: discardOutput,
post,
shouldInterrupt: () => Atomics.load(interrupt, 0) !== 0,
});
// Called from the prelude with primitives only.
const bridge = vm.newFunction("bridge", (kind, a, b, c) => {
switch (kind.toString()) {
case "call":
case "global":
post({
type: "call",
id: a.toNumber(),
target: kind.toString() === "call" ? "tool" : "global",
name: b.toString(),
args: c === undefined || c.isUndefined ? undefined : c.toString(),
});
break;
case "output":
post({
type: "output",
item:
a.toString() === "image"
? { type: "image", data: b.toString(), mimeType: c.toString() }
: a.toString() === "console"
? { type: "text", text: b.toString(), console: true }
: { type: "text", text: b.toString() },
});
break;
case "done":
if (a.toBoolean()) {
post({
type: "done",
ok: true,
value: b === undefined || b.isUndefined ? undefined : b.toString(),
writes: c.toString(),
});
} else {
post({ type: "done", ok: false, error: b.toString() });
}
break;
}
return vm.undefined;
});
// The VM lives until the host terminates the worker, so these handles are never disposed.
const api = vm.withScope((scope) =>
scope.escape(
vm.callFunction(
vm.evalCode(PRELUDE_SOURCE, "codemode-prelude.js"),
vm.undefined,
bridge,
vm.newString(JSON.stringify(data.tools)),
vm.newString(JSON.stringify(data.globals)),
vm.newString(JSON.stringify(data.store)),
),
),
);
const settle = api.getProp("settle");
const run = api.getProp("run");
const stalled = api.getProp("stalled");
/** Run queued jobs, then fail a script that waits on nothing that can ever resume it. */
const drain = () => {
vm.executePendingJobs();
vm.callFunction(stalled, api).dispose();
};
parentPort?.on("message", (message: unknown) => {
if (!isHostToWorkerMessage(message)) return;
try {
vm.withScope(() => {
vm.callFunction(
settle,
api,
vm.newNumber(message.id),
message.ok ? vm.true : vm.false,
message.payload === undefined ? vm.undefined : vm.newString(message.payload),
);
});
drain();
} catch (error) {
crash(error);
}
if (isHostToWorkerMessage(message)) session.deliver(message);
});
// The prefix shares the first line with the script so reported line numbers
// match the script as written.
let fn: JSValueHandle;
try {
fn = vm.evalCode(`(async (tools, console) => {${data.code}\n})`, "codemode.js");
} catch (error) {
if (!(error instanceof JSException)) throw error;
post({ type: "done", ok: false, error: describeException(error) });
return;
}
vm.callFunction(run, api, fn).dispose();
fn.dispose();
drain();
}
if (parentPort) {
main(workerData as WorkerData).catch(crash);
main(workerData as WorkerData).catch((error: unknown) => {
post({ type: "crash", message: error instanceof Error ? `${error.name}: ${error.message}` : String(error) });
});
}

19
packages/codemode/src/types.ts
View file

@ -59,11 +59,17 @@ export interface CodemodeCall {
export type CodemodeErrorKind =
/** The script threw or failed to parse. `name` and `stack` come from the script's error. */
| "script"
/** The overall deadline expired. The worker was terminated. */
/**
* The overall deadline expired, or a VM on the calling thread used up its interrupt budget. The VM
* was stopped.
*/
| "timeout"
/** The caller's signal fired or the sandbox was closed. The worker was terminated. */
/** The caller's signal fired or the sandbox was closed. The VM was stopped. */
| "aborted"
/** The worker or VM failed outside the script's control (for example a wasm trap or a missing worker file). */
/**
* The VM or its transport failed outside the script's control (for example a wasm trap, a missing
* worker file, or a remote sandbox that failed).
*/
| "sandbox";
export interface CodemodeError {
@ -91,7 +97,8 @@ export type CodemodeResult =
}
| { ok: false; error: CodemodeError; output: CodemodeOutputItem[]; calls: CodemodeCall[] };
export interface CodemodeSandboxOptions {
/** Options every sandbox accepts, wherever its VM runs. */
export interface CodemodeSandboxBaseOptions {
tools?: CodemodeTool[];
/**
* Functions exposed as top-level identifiers instead of on `tools`, for host helpers such as
@ -111,6 +118,10 @@ export interface CodemodeSandboxOptions {
* `InternalError: out of memory`. Default: no limit beyond wasm32's 4 GiB address space.
*/
memoryLimitBytes?: number;
}
/** Options of `CodemodeSandbox`, which runs each VM in a Node or Bun worker thread. */
export interface CodemodeSandboxOptions extends CodemodeSandboxBaseOptions {
/**
* Compiled `quickjs-wasi/quickjs.wasm`, usually from {@link loadQuickJSWasm}. Default:
* `loadQuickJSWasm()`, the file in the installed `quickjs-wasi` package. Pass it when that file

159
packages/codemode/test/portable.test.ts Normal file
View file

@ -0,0 +1,159 @@
import { describe, expect, it } from "vitest";
import {
type CodemodeRemote,
type CodemodeRemoteExchange,
type CodemodeRemoteRequest,
type CodemodeServeOptions,
InlineCodemodeSandbox,
loadQuickJSWasm,
RemoteCodemodeSandbox,
serveCodemodeRemote,
} from "../src/index.ts";
const wasm = loadQuickJSWasm();
/** An in-process remote that records what crosses the boundary, cloned as Workers RPC would. */
function recordingRemote(options: Partial<CodemodeServeOptions> = {}) {
const exchanges: number[] = [];
const served: Promise<void>[] = [];
const remote: CodemodeRemote = {
execute(request: CodemodeRemoteRequest, exchange: CodemodeRemoteExchange) {
const done = serveCodemodeRemote(
structuredClone(request),
async (messages) => {
exchanges.push(messages.length);
return structuredClone(await exchange(structuredClone(messages)));
},
{ wasm, ...options },
);
served.push(done);
return done;
},
};
return { remote, exchanges, served };
}
describe("InlineCodemodeSandbox", () => {
it("fails a script that computes past its interrupt budget as a timeout", async () => {
const sandbox = new InlineCodemodeSandbox({ wasm, interruptBudget: 1_000 });
for (const code of ["while (true) {}", "while (true) await null"]) {
const result = await sandbox.execute(code, { timeoutMs: Number.POSITIVE_INFINITY });
expect(result).toMatchObject({
ok: false,
error: { kind: "timeout", message: "Execution exceeded its interrupt budget of 1000" },
});
}
});
it("counts the budget across waits on tools", async () => {
const sandbox = new InlineCodemodeSandbox({
wasm,
interruptBudget: 1_000,
tools: [{ name: "tick", execute: () => 1 }],
});
const result = await sandbox.execute("for (;;) { for (let i = 0; i < 1e5; i++); await tools.tick(); }");
expect(result).toMatchObject({ ok: false, error: { kind: "timeout" } });
expect(result.calls.length).toBeGreaterThan(1);
});
});
describe("RemoteCodemodeSandbox", () => {
it("runs tools in the caller and the script on the remote", async () => {
const { remote, served } = recordingRemote();
const seen: unknown[] = [];
const sandbox = new RemoteCodemodeSandbox({
remote: () => remote,
tools: [
{
name: "add",
execute: (args) => {
seen.push(args);
return (args as { a: number }).a + 1;
},
},
],
});
const result = await sandbox.execute("text('hi'); return await tools.add({ a: 1 })", { store: { k: 1 } });
expect(result).toMatchObject({ ok: true, value: 2, output: [{ type: "text", text: "hi" }] });
expect(seen).toEqual([{ a: 1 }]);
expect(result.calls).toMatchObject([{ name: "add", status: "ok" }]);
await Promise.all(served);
});
it("batches output instead of making one exchange per line", async () => {
const { remote, exchanges } = recordingRemote();
const sandbox = new RemoteCodemodeSandbox({ remote: () => remote });
const result = await sandbox.execute("for (let i = 0; i < 5000; i++) console.log(i); return 'ok'");
expect(result).toMatchObject({ ok: true, value: "ok" });
expect(result.output).toHaveLength(5000);
expect(result.output[4999]).toEqual({ type: "text", text: "4999", console: true });
expect(exchanges.length).toBeLessThanOrEqual(6);
});
it("caps the interrupt budget and memory at the remote's limits", async () => {
const { remote } = recordingRemote({ maxInterruptBudget: 1_000, maxMemoryLimitBytes: 8 * 1024 * 1024 });
const sandbox = new RemoteCodemodeSandbox({ remote: () => remote, interruptBudget: 1e9 });
expect(await sandbox.execute("while (true) {}")).toMatchObject({
ok: false,
error: { kind: "timeout", message: "Execution exceeded its interrupt budget of 1000" },
});
const memory = await sandbox.execute(
"const parts = []; for (;;) parts.push('x'.repeat(1 << 20) + parts.length);",
);
expect(memory).toMatchObject({ ok: false });
expect(JSON.stringify(memory)).toContain("out of memory");
});
it("tells the remote to stop when the caller aborts", async () => {
const { remote, served } = recordingRemote();
let started: () => void = () => {};
const call = new Promise<void>((resolve) => {
started = resolve;
});
const sandbox = new RemoteCodemodeSandbox({
remote: () => remote,
tools: [
{
name: "hang",
execute: () => {
started();
return new Promise(() => {});
},
},
],
});
const controller = new AbortController();
const pending = sandbox.execute("await tools.hang(); return 'never'", { signal: controller.signal });
await call;
controller.abort();
expect(await pending).toMatchObject({ ok: false, error: { kind: "aborted" } });
// The remote's pending exchange is answered with "stop", so its VM is disposed and execute() returns.
await Promise.all(served);
});
it("reports a failing or vanishing remote as a sandbox error", async () => {
const failing = new RemoteCodemodeSandbox({
remote: () => ({ execute: () => Promise.reject(new Error("object reset")) }),
});
expect(await failing.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Remote sandbox failed: object reset" },
});
const silent = new RemoteCodemodeSandbox({ remote: () => ({ execute: () => Promise.resolve() }) });
expect(await silent.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Remote sandbox returned before the script settled" },
});
const unreachable = new RemoteCodemodeSandbox({
remote: () => {
throw new Error("no binding");
},
});
expect(await unreachable.execute("return 1")).toMatchObject({
ok: false,
error: { kind: "sandbox", message: "Failed to reach the remote sandbox: no binding" },
});
});
});

1472
packages/codemode/test/sandbox.test.ts
View file

File diff suppressed because it is too large Load diff

2
packages/codemode/tsconfig.build.json
View file

@ -5,5 +5,7 @@
"rootDir": "./src"
},
"include": ["src/**/*.ts"],
// Ambient declarations the build needs; `exclude` below does not apply to `files`.
"files": ["src/cloudflare-modules.d.ts"],
"exclude": ["node_modules", "dist", "**/*.d.ts", "src/**/*.d.ts"]
}

14
scripts/check-entry-graphs.mjs
View file

@ -41,6 +41,20 @@ const BUDGETS = {
},
"./utils/*": { maxFiles: 3, forbid: ["providers/", "api/", "index.ts"] },
},
"packages/codemode": {
// Node and Bun users never load the Cloudflare entry, which imports the `cloudflare:workers` built-in.
".": { maxFiles: 12, forbid: ["codemode/src/cloudflare.ts"] },
// Runtimes without Node built-ins (Workers, browsers) import these; neither may reach the worker-thread host
// or the file-based wasm loader.
"./portable": {
maxFiles: 8,
forbid: ["codemode/src/index.ts", "runtime/host.ts", "runtime/worker.ts", "codemode/src/wasm.ts"],
},
"./cloudflare": {
maxFiles: 9,
forbid: ["codemode/src/index.ts", "runtime/host.ts", "runtime/worker.ts", "codemode/src/wasm.ts"],
},
},
"packages/durable": {
".": {
// Tool argument validation reaches TypeBox; provider-session creation reaches pi-ai's lean UUID utility; the

10
scripts/check-runtime-deps.mjs
View file

@ -35,7 +35,15 @@ function checkSource(source, manifest) {
function checkSpecifier(node) {
if (!node || !(isStringLiteral(node) || isNoSubstitutionTemplateLiteral(node))) return;
const specifier = node.text;
if (specifier.startsWith(".") || specifier.startsWith("/") || isBuiltin(specifier)) return;
// `cloudflare:` modules are built into the Workers runtime, like `node:` modules are into Node.
if (
specifier.startsWith(".") ||
specifier.startsWith("/") ||
specifier.startsWith("cloudflare:") ||
isBuiltin(specifier)
) {
return;
}
const name = specifier.split("/").slice(0, specifier.startsWith("@") ? 2 : 1).join("/");
if (declared.has(name)) return;
const { line } = source.getLineAndCharacterOfPosition(node.getStart(source));

2
tsconfig.json
View file

@ -13,7 +13,9 @@
"@earendil-works/pi-telemetry": ["./packages/telemetry/src/index.ts"],
"@earendil-works/pi-telemetry/testing": ["./packages/telemetry/src/testing/index.ts"],
"@earendil-works/pi-codemode": ["./packages/codemode/src/index.ts"],
"@earendil-works/pi-codemode/cloudflare": ["./packages/codemode/src/cloudflare.ts"],
"@earendil-works/pi-codemode/declarations": ["./packages/codemode/src/declarations.ts"],
"@earendil-works/pi-codemode/portable": ["./packages/codemode/src/portable.ts"],
"@earendil-works/pi-codemode/source": ["./packages/codemode/src/source.ts"],
"@earendil-works/pi-codemode/worker": ["./packages/codemode/src/runtime/worker.ts"],
"@earendil-works/pi-mcp": ["./packages/mcp/src/index.ts"],