feat(ai): lazy OAuth flow and Bedrock loaders for code-split Node bundles

registerLazyOAuthFlows (@earendil-works/pi-ai/lazy-oauth) and registerLazyBedrockProvider
(@earendil-works/pi-ai/lazy-bedrock-provider, built on setBedrockProviderLoader) load the
flows and Bedrock through literal dynamic imports, so a splitting bundler keeps them inside
the bundle as lazy chunks. The variable-specifier default is unchanged.
This commit is contained in:
Mario Zechner 2026-11-05 01:27:21 +01:00
parent 5be85f3be3
commit da95cd79c6
6 changed files with 73 additions and 5 deletions

View file

@ -6,6 +6,7 @@
- Anthropic requests with a `sessionId` (and `cacheRetention` not `"none"`) now use message threads (`message-threads-2026-08-12` beta): the server continues from the previous reply and pi sends only the new messages instead of the whole conversation. When anything earlier in the request changed, or the server can't continue the thread, the full request is sent. Token counts and prompt caching are unchanged. On by default only for the `anthropic` provider on `api.anthropic.com`; set `compat.supportsMessageThreads` to change that, for example for gateways that forward the `thread` field. Call `cleanupSessionResources(sessionId)` when a session ends to free the per-session state ([#11545](https://github.com/earendil-works/pi/pull/11545))
- Added `StreamOptions.disableContinuation`, which keeps a request from using or replacing server-side continuation state such as Anthropic message threads. Set it on requests that repeat an earlier one, such as cache warming ([#11545](https://github.com/earendil-works/pi/pull/11545))
- Added `@earendil-works/pi-ai/lazy-oauth` (`registerLazyOAuthFlows()`) and `@earendil-works/pi-ai/lazy-bedrock-provider` (`registerLazyBedrockProvider()`, built on the new `setBedrockProviderLoader()`) for Node bundles that split at dynamic imports: the OAuth login flows and Bedrock load through literal dynamic imports, so the bundler keeps them inside the bundle as chunks that load on first use, instead of through the variable-specifier imports that load pi-ai's own files at runtime. Nothing changes without calling them.
### Fixed

View file

@ -1771,6 +1771,18 @@ setBedrockProviderModule(bedrockProviderModule);
That explicit override bundles the AWS SDK. Without it, Bedrock's opaque runtime import expects the package's Bedrock implementation file to be available at runtime.
Node bundles that split at dynamic imports (esbuild `splitting: true`) can keep Bedrock and the OAuth login flows inside the bundle but lazy: register loaders with literal dynamic imports, which the bundler turns into chunks that load on first use:
```typescript
import { registerLazyBedrockProvider } from '@earendil-works/pi-ai/lazy-bedrock-provider';
import { registerLazyOAuthFlows } from '@earendil-works/pi-ai/lazy-oauth';
registerLazyBedrockProvider();
registerLazyOAuthFlows();
```
Both modules are Node-only; never import them into a browser bundle.
### Provider-Scoped Environment Overrides
Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, `PI_CACHE_RETENTION`, and `HTTP_PROXY`/`HTTPS_PROXY`.

View file

@ -46,6 +46,14 @@
"./bun-oauth": {
"types": "./dist/bun-oauth.d.ts",
"import": "./dist/bun-oauth.js"
},
"./lazy-bedrock-provider": {
"types": "./dist/lazy-bedrock-provider.d.ts",
"import": "./dist/lazy-bedrock-provider.js"
},
"./lazy-oauth": {
"types": "./dist/lazy-oauth.d.ts",
"import": "./dist/lazy-oauth.js"
}
},
"bin": {

View file

@ -14,7 +14,7 @@ const importNodeOnlyApi = (specifier: string): Promise<unknown> => {
return import(new URL(runtimeSpecifier, import.meta.url).href);
};
let bedrockModuleOverride: ProviderStreams | undefined;
let bedrockModuleLoader: (() => Promise<ProviderStreams>) | undefined;
/**
* Overrides the dynamically imported bedrock implementation. Used by the Bun
@ -22,11 +22,22 @@ let bedrockModuleOverride: ProviderStreams | undefined;
* build registers a statically imported module instead.
*/
export function setBedrockProviderModule(module: ProviderStreams): void {
bedrockModuleOverride = module;
bedrockModuleLoader = async () => module;
}
/**
* Overrides how the bedrock implementation is imported, loaded on the first
* bedrock request. Used by Node bundles that split at dynamic imports (see
* `@earendil-works/pi-ai/lazy-bedrock-provider`): the loader's literal
* `import()` lets the bundler keep the implementation in a lazy chunk.
*/
export function setBedrockProviderLoader(load: () => Promise<ProviderStreams>): void {
bedrockModuleLoader = load;
}
export const bedrockConverseStreamApi = (): ProviderStreams =>
lazyApi(
async () =>
bedrockModuleOverride ?? ((await importNodeOnlyApi("./bedrock-converse-stream.ts")) as ProviderStreams),
lazyApi(async () =>
bedrockModuleLoader
? await bedrockModuleLoader()
: ((await importNodeOnlyApi("./bedrock-converse-stream.ts")) as ProviderStreams),
);

View file

@ -0,0 +1,13 @@
import { setBedrockProviderLoader } from "./api/bedrock-converse-stream.lazy.ts";
/**
* Registers a bedrock loader with a literal dynamic import, for Node bundles
* that split at dynamic imports (esbuild `splitting`, Chord's worker base):
* the implementation and the AWS SDK land in a chunk loaded on the first
* bedrock request, inside the bundle. Without it, pi-ai imports bedrock
* through a variable specifier that bundlers cannot follow, so a bundle loads
* it from pi-ai's own files. Browser bundles must not import this module.
*/
export function registerLazyBedrockProvider(): void {
setBedrockProviderLoader(() => import("./api/bedrock-converse-stream.ts"));
}

View file

@ -0,0 +1,23 @@
import { registerBundledOAuthFlowLoaders } from "./auth/oauth/load.ts";
/**
* Registers OAuth flow loaders with literal dynamic imports, for Node bundles
* that split at dynamic imports (esbuild `splitting`, Chord's worker base):
* each flow lands in a chunk loaded on its first use, inside the bundle.
* Without it, pi-ai imports the flows through a variable specifier that
* bundlers cannot follow, so a bundle loads them from pi-ai's own files.
* Browser bundles must not import this module (the flows are Node-only).
*/
export function registerLazyOAuthFlows(): void {
registerBundledOAuthFlowLoaders({
anthropic: async () => (await import("./auth/oauth/anthropic.ts")).anthropicOAuth,
openaiCodex: async () => (await import("./auth/oauth/openai-codex.ts")).openaiCodexOAuth,
openaiChatGPT: async () => (await import("./auth/oauth/openai-chatgpt.ts")).openaiChatGPTOAuth,
githubCopilot: async () => (await import("./auth/oauth/github-copilot.ts")).githubCopilotOAuth,
openrouter: async () => (await import("./auth/oauth/openrouter.ts")).openRouterOAuth,
kimiCoding: async () => (await import("./auth/oauth/kimi-coding.ts")).kimiCodingOAuth,
meta: async () => (await import("./auth/oauth/meta.ts")).metaOAuth,
xai: async () => (await import("./auth/oauth/xai.ts")).xaiOAuth,
radius: async (options) => (await import("./auth/oauth/radius.ts")).createRadiusOAuth(options),
});
}